H. S. Teoh wrote:
> On Fri, Oct 26, 2012 at 08:08:36AM +0200, Robik wrote:
> > On Friday, 26 October 2012 at 01:35:43 UTC, Adam D. Ruppe wrote:
> > >On Thursday, 25 October 2012 at 22:27:52 UTC, Jens Mueller wrote:
> > >>>5. setting the contents of the title bar
> > >>The title bar of what?
> > >
> > >Here's how you do it on xterm:
> > >
> > >writefln("\033]0;%s\007", title);
> > >
> > 
> > Yeah, the problem is it does not work in all terminals.
> 
> The correct solution is to examine the TERM environment variable to find out what kind of terminal it is, and then use escape sequences specific to that terminal.

Or use a library that does this? There is terminfo. Don't know whether
this is supported on all Posix platforms.
I think the terminal's capabilities are stored in some files on the
system.

Jens

On Fri, Oct 26, 2012 at 04:46:26PM +0200, Jens Mueller wrote:
> H. S. Teoh wrote:
> > On Fri, Oct 26, 2012 at 08:08:36AM +0200, Robik wrote:
> > > On Friday, 26 October 2012 at 01:35:43 UTC, Adam D. Ruppe wrote:
[...]
> > > >writefln("\033]0;%s\007", title);
> > > >
> > > 
> > > Yeah, the problem is it does not work in all terminals.
> > 
> > The correct solution is to examine the TERM environment variable to find out what kind of terminal it is, and then use escape sequences specific to that terminal.
> 
> Or use a library that does this? There is terminfo. Don't know whether
> this is supported on all Posix platforms.
> I think the terminal's capabilities are stored in some files on the
> system.
[...]

Well, if you're going to use a library, terminfo is the way to go. All modern Posix systems should have it. As for terminal description files, don't rely on that, because it varies wildly from system to system. Your best bet is to either recognize known values of $TERM, or use a library like terminfo that takes care of all the dirty details for you. It does get very messy once you get down to the nitty gritty of how different *nixes handle terminal capabilities.


T

-- 
People tell me that I'm paranoid, but they're just out to get me.

I'm slapping together a minimal termcap reader for D now.

Here we go, some more basic functions:

http://arsdnet.net/dcode/terminal.d

The unix stuff is more implemented than the windows.


Let's walk through main and I'll discuss why I'm doing things the way I am. I'm throwing this out just to show one possible way this can be done and to explain why I like this way. Feel free to take some or all of my code if you want.


	auto terminal = Terminal(ConsoleOutputType.cellular);

First, the terminal is a separate type, a struct, rather than just magic functions. This way it can use the struct destructor to reset things to normal.

The argument here is either linear or cellular - do you want line based output or do you want to control the whole screen as a grid of cells?

If you use cellular, it will use the alternative screen buffer on unix terminals. IIRC Windows works basically the same way. This is what we want for a TUI app.

The escape sequences to make this happen is based on the envrionment variable TERMCAP, which is set by gnu screen, xterm, and most other emulators.

It is *not* set by the linux console or rxvt on my box, so this isn't perfect yet, but it is a start. If it can't find a capability in there, it simply ignores those method calls, meaning it should be backward compatible (once fully implemented - I haven't done this on title yet.)

	terminal.setTitle("Basic I/O");

All changes are done with methods. This is how Windows does it and is cleaner anyway.

	auto input = terminal.captureInput(ConsoleInputFlags.raw);

To get real time input, you use a special method which returns an input struct. Which could probably be an input range actually but isn't now.

Anyway the reason is you have to change the terminal mode in linux to get real time input. Otherwise it will be line buffered. Again, a struct lets us reset the mode automatically in the destructor. The flags let you control automatic echo.

	terminal.color(Color.green | Bright, Color.black);

I insist that you set foreground and background colors together. This ensure you get readable text.

The colors are defined with a platform independent color enum and you can flag in bright to get brighter colors.


BTW I'm thinking about calling it LowContrast instead of bright... then your color definitions can always be the same regardless of bg, and it automatically flips the bright bit if needed. But since I'm forcing you to specify background here too it isn't a big deal with now.


	int centerX = terminal.width / 2;
	int centerY = terminal.height / 2;

I prefer width and height to rows and columns. I think more in terms of gui here.

	while(true) {
		if(input.kbhit()) {

Since we have real time capture, can poll for new input instead of blocking.

			auto c = input.getch();

Get our character... (BTW this function still needs a lot of work)

			if(c == 'q' || c == 'Q')
				break;
			terminal.moveTo(centerX, centerY);
			terminal.writef("%c", c);
			terminal.flush();

And output the stuff, centering the text. All output methods are attached to the terminal to ensure the proper api stuff is handled.

		usleep(10000);

This is just there so we don't eat too much cpu waiting on input.

On Fri, Oct 26, 2012 at 10:06:38AM +0200, Jens Mueller wrote:
> H. S. Teoh wrote:
> > On Fri, Oct 26, 2012 at 12:27:38AM +0200, Jens Mueller wrote:
> > > Walter Bright wrote:
> > [...]
[...]
> > > > 1. getting mouse input
> > > 
> > > Anybody an idea how to this on Linux?
> > 
> > You can only do this in terminals that support it. XTerm, I believe, has escape sequences that you can send to turn on mouse tracking. Those will show up as special escape sequences on stdin, which will have to be intercepted and removed from the program's normal input stream. Other modern terminals probably have their own way of doing mouse tracking.  AFAIK, there isn't any standard for this, so you'll have to stick with terminal-specific code.
> > 
> > (Which is why I recommended earlier that this module must be modular so that support for specific terminals can be plugged in easily -- for the initial stab, something very simple such as vt100 support may be good enough, as long as it's easy to add support for new terminals.)
> 
> So there exists no portable library abstracting mouse input?  If that's the case then being modular is the only option. How would you design it? I don't plan to implement this. But leaving the door open for others should be possible.

Hmm. I googled around a bit, and found that the only portable libraries for abstracting terminal capabilities appear to be curses and its derivatives like ncurses.  So if we're going to reengineer a D console library that doesn't depend on ncurses, we'll have to get our hands dirty with interpreting $TERM and parsing terminal capabilities. :-(

As for design, I think a very simple and easily extensible design would be something along these lines:

Have a generic Terminal base class that contains all the API functions for various terminal capabilities, like interacting with the mouse, setting color, moving the cursor, etc. These functions are stubbed to throw an UnsupportedTerminalCapability (or something like that) exception when they are called. Then each supported terminal type will derive from Terminal, and override those functions that are supported for that terminal type. The module initialization code will determine at runtime which of these subclasses to instantiate.

Then build an additional layer on top, with higher-level API functions that eventually call the object's methods to perform various terminal functions (e.g., writeln can be extended to support color by interpreting color escape sequences that ultimately result in calling some underlying method in Terminal).

When the module is extended to handle new capabilities, we just add new methods to Terminal, stubbed to throw UnsupportedTerminalCapability. When we add support for new terminal types, we just create a new subclass of Terminal that implements the new methods.

I think for maintainability, Terminal should not be directly accessed by the user, so that its methods can be kept concise and at the correct abstraction level for interacting with low-level terminal functions. The module should provide a higher-level API with underlying calls to these functions, say by extending writeln, implementing screen buffering, etc..


[...]
> > > > 7. getting no-echo raw input
> > > 
> > > Tried this but couldn't make it work yet. This is useful for passwords prompts, right?
> > 
> > Not just that, but also for fully-interactive programs that want to capture every keystroke immediately (instead of waiting for the user to hit Enter). Like games and stuff. Or menu-driven systems where the user can navigate between menus and items without needing to hit Enter each time.
> > 
> > For Unix terminals, you need to send certain escape sequences (specific to the terminal) to enable what is called 'raw' mode or 'cbreak' mode.  (Googling for 'cbreak' should give you useful references.) This will cause the terminal to immediately transmit keystrokes, instead of buffering them until the user hits Enter.
> > 
> > Also, make sure that there is a way to turn off this mode after the program is finished, otherwise the terminal may become unusable when it returns to the shell prompt. :)
> 
> Thanks for the pointers.
> So these are then two things. First noecho and the other one is raw
> input.

Yeah, noecho is useful for password input; raw input is needed for interactive apps like games or menu-driven programs. Both should be supported.


[...]
> > The important stuff are cursor positioning, box drawing, incremental updates, cbreak mode, etc..
> 
> Can you say something about incremental updates? Any pointers?
[...]

The idea behind incremental updates is this: if I write a string "ABCDEF" at position (10,10), then I write "ABDCEF" at the same position, the console library should know to only replace "CD" with "DC" the second time round, instead of redrawing the entire string. Or, on larger scale, if my app draws an almost-identical copy of the current screen, the library should know to only redraw the "diff" between the previous state of the screen and the new one.

This is commonly implemented by buffering the current state of the screen in the library, and marking parts of the buffer "dirty" when they are changed. Then when there is a pause (say the app is waiting for input) or the screen needs to scroll, etc., the library updates only the parts of the screen that correspond with the "dirty" buffers, and clears the dirty flag.

The basic motivation is that the terminal may be connected to a remote machine via a slow or congested network, so it's faster to keep track of things locally and only send "diffs" to the remote end. Waiting for a pause also allows you to group several updates into a single network packet, instead of sending one packet per character, which is very slow.

(Even if the terminal is local, it can be faster to do things this way, because it minimizes writes to video RAM, which is slower than writing to main memory. Or in the case of X11 terminal emulators, it saves the cost of redrawing the same pixels over and over.)


T

-- 
People demand freedom of speech to make up for the freedom of thought which they avoid. -- Soren Aabye Kierkegaard (1813-1855)

On Fri, Oct 26, 2012 at 01:56:28PM +0200, Tobias Pankrath wrote:
> On Tuesday, 23 October 2012 at 22:47:40 UTC, Walter Bright wrote:
[...]
> >A more comprehensive module that included:
> >
> >1. getting mouse input
> >2. getting size of the console
> >3. moving the cursor around
> >4. drawing boxes in the console window
> >5. setting the contents of the title bar
> >6. supporting cut/paste
> >7. getting no-echo raw input
> >8. setting the size of the cursor
> >
> >would be a definite candidate. I.e. a module that can support building a text mode screen app (like a text editor).
> 
> This would look like a full blown TUI-Toolkit and we should model the API after successfull GUI-Frameworks like Qt, i.e. provide a event loop, use a Signal/Slot mechanism etc.

If we implement an event loop, I think it should be optional. Many apps only need to do simple things like allow editing operations on the current input line (support backspace, insert, delete, moving cursor left/right, etc.). Having a full-blown event loop is overkill.

OTOH, having the *option* of using an event loop makes it easier to write things like network-based apps, where input from many different directions can be handled asynchronously. It also makes it possible to "skin" an app to work with both GUI and TUI if the underlying code is pretty much the same, except for different low-level calls at the bottom. :) So I think it's a good thing to have.  Just make it optional, not mandatory.


> That would be a real improvement over nCurses. What do you think?
[...]

Anything that improves on ncurses is welcome by me. Although ncurses does what it does very well, the API is a poorly-designed patchwork of functions that overlap too much in some areas, and not enough in others. (Try UTF-8 processing on ncurses sometime. Or maybe, _don't_, because it leads to pain and suffering.) Having a well-designed, consistent API would be a major plus.


T

-- 
You have to expect the unexpected. -- RL

On Friday, 26 October 2012 at 18:05:09 UTC, H. S. Teoh wrote:
> If we implement an event loop, I think it should be optional.

I think this is another benefit of capturing the input with a special type and method.

auto input = terminal.captureInput(ConsoleInputFlags.raw | ConsoleInputFlags.mouse);
while(true) {
   InputEvent = input.nextEvent();
   // blah blah blah
}


Then you can loop on it and get all kinds of data, or you can use the more plain read/write functions.

This is more or less how it works on Windows. (Really, the people who say text programming on Windows sucks always confuse me. It's a pretty decent design, lightyears better than the garbage you have to put up with on Linux.)

http://msdn.microsoft.com/en-us/library/windows/desktop/ms684961%28v=vs.85%29.aspx

for the fancier events or

http://msdn.microsoft.com/en-us/library/windows/desktop/ms684958%28v=vs.85%29.aspx

if all you care about is keyboard input.

On Friday, 26 October 2012 at 18:58:20 UTC, Adam D. Ruppe wrote:
> I think this is another benefit of capturing the input with a special type and method.

I've implemented the basic events for linux now:

http://arsdnet.net/dcode/terminal.d

Still need to check more of the input sequences, but it correctly detects mouse events (when requested) and has set the groundwork for the rest. Just a matter of actually making it happen yet.

It uses $TERM and sometimes $TERMCAP to check for a requested feature before enabling it to keep out trash output.

The loop looks like this right now:

        loop: while(true) {
                auto event = input.nextEvent();

                terminal.writef("%s\n", event.type);
                final switch(event.type) {
                        case InputEvent.Type.CharacterEvent:
                                auto ev = event.get!(InputEvent.Type.CharacterEvent);
                                terminal.writef("\t%s\n", ev);
                                if(ev.character == 'Q')
                                        break loop;
                        break;
                        case InputEvent.Type.NonCharacterKeyEvent:
                                terminal.writef("\t%s\n", event.get!(InputEvent.Type.NonCharacterKeyEvent));
                        break;
                        case InputEvent.Type.PasteEvent:
                                terminal.writef("\t%s\n", event.get!(InputEvent.Type.PasteEvent));
                        break;
                        case InputEvent.Type.MouseEvent:
                                terminal.writef("\t%s\n", event.get!(InputEvent.Type.MouseEvent));
                        break;
                }
         }

It now can translate most PC keyboard input sequences into char or non-char key events, including requesting UTF-8 input for chars:

http://arsdnet.net/dcode/terminal.d


We could just about start writing real apps with this now. Biggest problem left is it doesn't actually scan the termcap file - it only looks for a TERMCAP environment variable. This means many keys are ignored on some terminals.

Should be a fairly easy fix I just haven't gotten around to it yet.


Then finish the Windows support side of it and we have a fairly functional, totally standalone, little text library here.



BTW this is actually kinda off topic for ColorD since I'm going more fancy - if you just want to add color to stdout, my code has probably gone too far.

On Sun, Oct 28, 2012 at 04:49:03AM +0100, Adam D. Ruppe wrote:
> It now can translate most PC keyboard input sequences into char or non-char key events, including requesting UTF-8 input for chars:
> 
> http://arsdnet.net/dcode/terminal.d
> 
> We could just about start writing real apps with this now. Biggest problem left is it doesn't actually scan the termcap file - it only looks for a TERMCAP environment variable. This means many keys are ignored on some terminals.

This is too cool! You should polish it up and submit a Phobos entry for it. I would use it!


> Should be a fairly easy fix I just haven't gotten around to it yet.
> 
> Then finish the Windows support side of it and we have a fairly functional, totally standalone, little text library here.

Yeah, I would vote for it as a Phobos entry.


> BTW this is actually kinda off topic for ColorD since I'm going more fancy - if you just want to add color to stdout, my code has probably gone too far.

I don't think it's too far, I think it's just about right for the beginnings of a console module in Phobos. If you'd put it on github, I might submit a pull request to add some ddocs, then with added Windows support, it should be just about ready for a Phobos entry.


T

-- 
The diminished 7th chord is the most flexible and fear-instilling chord. Use it often, use it unsparingly, to subdue your listeners into submission!