February 23, 2012
On 23 February 2012 19:41, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:
> On Thu, Feb 23, 2012 at 06:02:30PM +1300, James Miller wrote: [...]
>> And I've been playing with trying to write my own terminal emulator. I actually kinda have one working, in C. I just need to wrap all the C code in functions and do all the interesting stuff in D.
> [...]
>> But I have plans for it, including some cool graphical stuff. I'm using clutter to render everything, so I can do all sorts of things with it, but to start with, I'd like to be able to display pictures inline, so you can do `cat my_pic.jpg` and rather throwing masses of garbage at you, it renders the picture, I might have to write my own cat to do so however...
> [...]
>
> Now *that* is an excellent idea!
>
> Perhaps there's a way of detecting JPEG or PNG output, say some kind of magic number detection ala /usr/bin/file. Buffer the first few bytes at output at the start of new lines, and if it looks like a JPEG file, buffer the rest of the output instead of displaying it immediately, and if after reading more input it does appear to be a valid JPEG file, then display it as an image.  Otherwise flush the buffer and display it as text.
>
> To prevent problems with false positives causing output not to appear, you could use a timer: if the data type can't be determined within, say, 1/20 of a second, then skip the detection and display the output as text. Presumably if you do cat my_pic.jpg more than just the first few bytes will come out in the first 1/20 of a second, so it will pretty much reliably detect image data.
>
> Or you can do it the easy way by having a dedicated escape sequence that switches to image mode, and write a wrapper for cat that outputs this sequence when it detects image data in its output. The danger though is that you might get stuck in image mode if cat crashes before the image data is completely spooled, in that case it'll be like the ole DOS days when you got stuck in VGA mode when it has dropped back to the DOS prompt, and typing only produces line noise on the screen. :P
>
>
> T
>
> --
> This is a tpyo.

I was thinking about both of those options and came to pretty much the same conclusions. What would be cool would be to add in new STD* file descriptors (ala STDIN, STDOUT, etc) that are dedicated to presentation (also means that you can output things to the terminal while running a pipe), however that would also require shell support (to set the correct FDs on the child process) and I'm not writing a terminal /and/ shell (yet).

Otherwise, I think having the terminal take note of what is going on in the shell would solve the stuck mode problem, (which i encountered accidentally by putting my terminal into alt_charset mode) just have it notice if the current process is a shell or not and basically say "Shells can't be in image mode". It would probably require grabbing /etc/shells.

The other advantage of using an escape sequence is that assuming that it can read fast enough, then it could easily be extended to all sorts of things, video for example. Displays a small video in the terminal, so you can check that the video is what you actually want, don't replace an actual movie player, that would be silly. you could also have things like syntax highlighting for supported files.

As you can probably tell, I am less than enamored by the current terminal/shell line-up we have, I just think that there is so much more we could do with modern tech, and it doesn't need to even be incompatible with older systems!
February 23, 2012
On Fri, Feb 24, 2012 at 12:38:52AM +1300, James Miller wrote:
> On 23 February 2012 19:41, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:
[...]
> > Perhaps there's a way of detecting JPEG or PNG output, say some kind of magic number detection ala /usr/bin/file. Buffer the first few bytes at output at the start of new lines, and if it looks like a JPEG file, buffer the rest of the output instead of displaying it immediately, and if after reading more input it does appear to be a valid JPEG file, then display it as an image.  Otherwise flush the buffer and display it as text.
[...]
> > Or you can do it the easy way by having a dedicated escape sequence that switches to image mode, and write a wrapper for cat that outputs this sequence when it detects image data in its output.
[...]
> I was thinking about both of those options and came to pretty much the same conclusions. What would be cool would be to add in new STD* file descriptors (ala STDIN, STDOUT, etc) that are dedicated to presentation (also means that you can output things to the terminal while running a pipe), however that would also require shell support (to set the correct FDs on the child process) and I'm not writing a terminal /and/ shell (yet).

Well, currently you *can* output to terminal in a pipe by writing to stderr, which usually points to the terminal even if stdout has been redirected. Of course, the user can redirect stderr too, but in that case he doesn't *want* any terminal output, so there's no point fighting it.


> Otherwise, I think having the terminal take note of what is going on in the shell would solve the stuck mode problem, (which i encountered accidentally by putting my terminal into alt_charset mode) just have it notice if the current process is a shell or not and basically say "Shells can't be in image mode". It would probably require grabbing /etc/shells.

The problem is that the terminal doesn't know who's sending output to it. It's just reading one end of a pipe, and anything could be at the other end, and there could be an arbitrary number of stuff in between.

I'm not sure I know any easy and reliable way to solve this.


> The other advantage of using an escape sequence is that assuming that it can read fast enough, then it could easily be extended to all sorts of things, video for example. Displays a small video in the terminal, so you can check that the video is what you actually want, don't replace an actual movie player, that would be silly. you could also have things like syntax highlighting for supported files.

Actually, if the terminal supported video output, you could actually have the movie player play inside the terminal. :P Or am I just being silly?


> As you can probably tell, I am less than enamored by the current terminal/shell line-up we have, I just think that there is so much more we could do with modern tech, and it doesn't need to even be incompatible with older systems!

True.

Taking the idea of an in-terminal video player further, what about a general escape sequence for "application-specific output"? This can then be used along with file type (data type) detection to multiplex between different apps, like image display, movie player, sound player, etc.. The apps will have access to a given region on the terminal screen (perhaps passed as a subwindow on the underlying windowing system), and can do whatever they want with it.

This also means you don't need ncurses anymore. The app can get direct access to the terminal as a graphical drawable, and can do much more than displaying a bunch of characters in a grid.


T

-- 
It is not the employer who pays the wages. Employers only handle the money. It is the customer who pays the wages. -- Henry Ford
February 23, 2012
On Thursday, 23 February 2012 at 21:17:35 UTC, H. S. Teoh wrote:
> Taking the idea of an in-terminal video player further, what about a general escape sequence for "application-specific output"?

<html>.....


seriously, once you take the in-terminal stuff too far, you
have a beast of a program that does everything; you end up
with something like a web browser or nested X server, and
you've lose the original beauty of the terminal.


This is the path I started on, and ended up with what I called
the D Windowing System - which is still on my todo list, but
has been for a long time and will surely be for a long time
more.

But, I started with a terminal that can optionally output
images. Then wanted improved character display and figured
if it does images, it can be a bitmap drawable.

Then wanted multiple channels... and at this point, you have
a poor man's X.


Realizing this, I went full into it and shifted gears to a higher
level X kind of thing, where you make an app based on widgets on
a grid, which would work in text mode or graphical mode. But, I
didn't get too far before work took over my time.




But, on a terminal, I just don't think images are a fit there.
Might as well just make something like a gui image viewer app
and keep a more traditional terminal.

Though, I'd prefer to be more like vga text mode than VT100
in terms of capabilities.
February 23, 2012
On Thu, Feb 23, 2012 at 10:34:24PM +0100, Adam D. Ruppe wrote:
> On Thursday, 23 February 2012 at 21:17:35 UTC, H. S. Teoh wrote:
> >Taking the idea of an in-terminal video player further, what about a general escape sequence for "application-specific output"?
> 
> <html>.....

That gives you the funny situation where trying to edit a HTML file will interpret the tags instead of displaying them. If your editor is unaware of that, then you have the even funnier situation where editing a tag (i.e. temporarily breaking HTML syntax) will cause it to come out as text, and once you have a fully formed tag, then it magically gets rendered again. A twisted form of WYSIWYG. :P


> seriously, once you take the in-terminal stuff too far, you have a beast of a program that does everything; you end up with something like a web browser or nested X server, and you've lose the original beauty of the terminal.

No, the idea is *not* to have the terminal do everything, but allow other programs to do it.


> This is the path I started on, and ended up with what I called the D Windowing System - which is still on my todo list, but has been for a long time and will surely be for a long time more.

lol... I suppose.


> But, I started with a terminal that can optionally output images. Then wanted improved character display and figured if it does images, it can be a bitmap drawable.
> 
> Then wanted multiple channels... and at this point, you have a poor man's X.

True.


> Realizing this, I went full into it and shifted gears to a higher level X kind of thing, where you make an app based on widgets on a grid, which would work in text mode or graphical mode. But, I didn't get too far before work took over my time.

Actually, I've thought of that before. It's something I'd love to have.

The main complaint with writing a real X11 app is that *everything* is graphical, so to do something as simple as print a string requires an inordinate amount of code. You need to create a subwindow or subdrawable, load a font (with all that living nightmare that font resolution is), calculate font proportions, spacing, line wrapping, and do all of the above in an event-based model. At the end of the day you've reinvented a GUI toolkit with a text widget.


[...]
> But, on a terminal, I just don't think images are a fit there.  Might as well just make something like a gui image viewer app and keep a more traditional terminal.
> 
> Though, I'd prefer to be more like vga text mode than VT100 in terms of capabilities.

The way I see it, it's a sort of "VGA-enabled text mode" terminal, where you can intermix graphics on what is basically a character grid. Not a full-fledged windowing system embedded in xterm.

(I mean, with graphics capabilities the application *could* implement a windowing system, but that's the app's problem, not the terminal's.)


T

-- 
One reason that few people are aware there are programs running the internet is that they never crash in any significant way: the free software underlying the internet is reliable to the point of invisibility. -- Glyn Moody, from the article "Giving it all away"
February 24, 2012
On 24 February 2012 12:03, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:
> On Thu, Feb 23, 2012 at 10:34:24PM +0100, Adam D. Ruppe wrote:
>> On Thursday, 23 February 2012 at 21:17:35 UTC, H. S. Teoh wrote:
>> >Taking the idea of an in-terminal video player further, what about a general escape sequence for "application-specific output"?
>>
>> <html>.....
>
> That gives you the funny situation where trying to edit a HTML file will interpret the tags instead of displaying them. If your editor is unaware of that, then you have the even funnier situation where editing a tag (i.e. temporarily breaking HTML syntax) will cause it to come out as text, and once you have a fully formed tag, then it magically gets rendered again. A twisted form of WYSIWYG. :P
>
>
>> seriously, once you take the in-terminal stuff too far, you have a beast of a program that does everything; you end up with something like a web browser or nested X server, and you've lose the original beauty of the terminal.
>
> No, the idea is *not* to have the terminal do everything, but allow other programs to do it.
>
>
>> This is the path I started on, and ended up with what I called the D Windowing System - which is still on my todo list, but has been for a long time and will surely be for a long time more.
>
> lol... I suppose.
>
>
>> But, I started with a terminal that can optionally output images. Then wanted improved character display and figured if it does images, it can be a bitmap drawable.
>>
>> Then wanted multiple channels... and at this point, you have a poor man's X.
>
> True.
>
>
>> Realizing this, I went full into it and shifted gears to a higher level X kind of thing, where you make an app based on widgets on a grid, which would work in text mode or graphical mode. But, I didn't get too far before work took over my time.
>
> Actually, I've thought of that before. It's something I'd love to have.
>
> The main complaint with writing a real X11 app is that *everything* is graphical, so to do something as simple as print a string requires an inordinate amount of code. You need to create a subwindow or subdrawable, load a font (with all that living nightmare that font resolution is), calculate font proportions, spacing, line wrapping, and do all of the above in an event-based model. At the end of the day you've reinvented a GUI toolkit with a text widget.
>
>
> [...]
>> But, on a terminal, I just don't think images are a fit there.  Might as well just make something like a gui image viewer app and keep a more traditional terminal.
>>
>> Though, I'd prefer to be more like vga text mode than VT100 in terms of capabilities.
>
> The way I see it, it's a sort of "VGA-enabled text mode" terminal, where you can intermix graphics on what is basically a character grid. Not a full-fledged windowing system embedded in xterm.
>
> (I mean, with graphics capabilities the application *could* implement a windowing system, but that's the app's problem, not the terminal's.)
>
>
> T
>
> --
> One reason that few people are aware there are programs running the internet is that they never crash in any significant way: the free software underlying the internet is reliable to the point of invisibility. -- Glyn Moody, from the article "Giving it all away"

I have tried to keep the scope of what the graphics capabilities of the terminal are down to a minimum, so all the drawing would still be in terms of the terminal. I was thinking that you put it into graphics mode, say how many columns and rows you want, and the length of the data  (so as to not accidentally leave image mode because the data contains the escape sequence for leaving image mode) It would start drawing in a rectangle from cursor location, and leave the cursor in the column after the last. I haven't decided what would happen if you didn't have enough columns (rows would just scroll), or how you'd deal with images that don't fit into character block pixel sizes.

So you could implement a fully graphical system inside it, but it would be a lot of work, and you wouldn't have the advantage of any windowing features, and everything would still be cursor-based anyway. Basically the idea is to make it easy to implement features that make sense in a terminal, but make it difficult to do anything outside that scope. So I probably wouldn't have a subwindow, since that instantly provides way too much functionality, also you could output pictures fine simply by decoding the pictures into some "raw" format (preferably something that can be mapped straight to a pixbuf) and outputting them, video could work by just doing the same thing, but moving the cursor back to the original starting point over-and-over. Trying to make a graphical app would be a more of a challenge if you want anything remotely complicated, and you'd end up just using X anyway.

I guess the idea is that being able to, for example, display pictures in your otherwise-terminal-based mail app is useful, being able to preview files without opening a new window (useful for people with tiling managers, like me) is useful. But having something that tries to be a window manager, or a proper app, is not useful. By having a layer of indirection, and the performance penalty that holds, trying to maintain terminal context, and making it unidirectional, it essentially forces apps to not abuse the capability, its not feasible to do anything other than just display an image.

There are a few other idea I have, but aren't even close to being properly formed yet, however the simple-to-use graphics capabilities described above is something that I think has real merit. I've seen a few similar things in the past, but I don't think they tried to be a terminal emulator in their own right, so I couldn't run vim or screen in them.
February 24, 2012
On Thursday, 23 February 2012 at 19:10:03 UTC, H. S. Teoh wrote:
> Nice!!! So you can write the same code for both the "shell library" and the actual shell itself.

Yeah. This is similar to the technique I used in my web.d
thing, though web.d's is a lot more complex. (It also handles
sub objects, argument name and default values, and some more.)

I just find it super useful.

> The echo command has a bug, it's not supposed to output 'echo' as part of its output. :)

Yeah, I messed up argv.

Here's an updated version:
http://arsdnet.net/dcode/dshell.d

added basic formatting to the data interface, lazy printing
(try "cat" and type in stdin) and bug fixes.


I just might put it up on the github over the weekend.

> It's extremely cool. Definitely orders of magnitude better than writing the equivalent thing in C.

Hell, this kind of thing is easier in D than in a lot
of dynamic languages! imo anyway. eval is sometimes nice
but  having the type information available to do a smart
wrapper is very nice.


February 24, 2012
On Fri, Feb 24, 2012 at 05:19:44PM +0100, Adam D. Ruppe wrote:
> On Thursday, 23 February 2012 at 19:10:03 UTC, H. S. Teoh wrote:
[...]
> >The echo command has a bug, it's not supposed to output 'echo' as part of its output. :)
> 
> Yeah, I messed up argv.
> 
> Here's an updated version: http://arsdnet.net/dcode/dshell.d
> 
> added basic formatting to the data interface, lazy printing (try "cat" and type in stdin) and bug fixes.

OK I'll take a look at it.


[...]
> >It's extremely cool. Definitely orders of magnitude better than writing the equivalent thing in C.
> 
> Hell, this kind of thing is easier in D than in a lot
> of dynamic languages! imo anyway. eval is sometimes nice
> but  having the type information available to do a smart
> wrapper is very nice.
[...]

Well, eval is just runtime mixin. From the program's POV, it's an opaque string that might do just about *anything*. Which is not very helpful since you can't really do too much with it except to run it or not. Type introspection OTOH allows you do more intelligent things based on what types are there.


T

-- 
WINDOWS = Will Install Needless Data On Whole System -- CompuMan
February 24, 2012
On Fri, Feb 24, 2012 at 02:32:38PM +1300, James Miller wrote: [...]
> I have tried to keep the scope of what the graphics capabilities of the terminal are down to a minimum, so all the drawing would still be in terms of the terminal. I was thinking that you put it into graphics mode, say how many columns and rows you want, and the length of the data  (so as to not accidentally leave image mode because the data contains the escape sequence for leaving image mode) It would start drawing in a rectangle from cursor location, and leave the cursor in the column after the last. I haven't decided what would happen if you didn't have enough columns (rows would just scroll), or how you'd deal with images that don't fit into character block pixel sizes.

You could just pad the unused space with blank pixels.


> So you could implement a fully graphical system inside it, but it would be a lot of work, and you wouldn't have the advantage of any windowing features, and everything would still be cursor-based anyway.

Right, 'cos the point isn't to implement a windowing system, but just to add graphical display capabilities to the terminal.


> Basically the idea is to make it easy to implement features that make sense in a terminal, but make it difficult to do anything outside that scope. So I probably wouldn't have a subwindow, since that instantly provides way too much functionality, also you could output pictures fine simply by decoding the pictures into some "raw" format (preferably something that can be mapped straight to a pixbuf) and outputting them, video could work by just doing the same thing, but moving the cursor back to the original starting point over-and-over.

Yeah that's pretty good basic functionality IMHO. Quite often I've wanted to write up some throwaway character-based games with graphical icons, and something like this would be perfectly suited. Usually these little games are just quick-n-dirty write-once-and-forget deals, so I really don't want to be dealing with the complexities of using X11 or SDL, but just to be able to say, character code X maps to image Y, draw this at character position (43,21). Job done.


[...]
> I guess the idea is that being able to, for example, display pictures in your otherwise-terminal-based mail app is useful, being able to preview files without opening a new window (useful for people with tiling managers, like me) is useful. But having something that tries to be a window manager, or a proper app, is not useful. By having a layer of indirection, and the performance penalty that holds, trying to maintain terminal context, and making it unidirectional, it essentially forces apps to not abuse the capability, its not feasible to do anything other than just display an image.

Agreed.


> There are a few other idea I have, but aren't even close to being properly formed yet, however the simple-to-use graphics capabilities described above is something that I think has real merit. I've seen a few similar things in the past, but I don't think they tried to be a terminal emulator in their own right, so I couldn't run vim or screen in them.

I think this has merit. It lets you do quick-n-dirty apps with simple graphical icons without needing to invest the effort into a full-fledged GUI app complete with a full windowing toolkit and configurable fonts. Sometimes just an icon or two in the midst of some text really is just good enough.


T

-- 
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next. -- (Stolen from the net)
February 24, 2012
On Friday, 24 February 2012 at 17:54:47 UTC, H. S. Teoh wrote:
> You could just pad the unused space with blank pixels.

whoa, make a tile based system, like the NES.

There's a private use area in unicode, 16 bits of space I think.

You could define a tile set to use that, or a simple bitmap format,
but it isn't enough room to do on demand tiles I think.


Imagine this though, your terminal lets the app define new characters,
in color. The protocol might be an escape sequence then you go
ahead and write out an image using whatever format (xpm or base64
png - something you can easily enough represent as text.)


Then, you overwrite a private use character. You can then write
that character to display the custom glyph.

Now, you treat it just like any other character for writing,
scrolling, read back, etc. Mixes right in with text.



Now, you won't be able to just "cat image.png", but you could
write a cat that does it. Define a tileset for a line, write
it out, repeat and let it scroll to show the whole image.

65535 * what's a character size, like 13x9, something like that?

You could write out a nice, big image without worrying about tile
reuse.


I think I like this..., it'd fit pretty well with the existing
terminal setup and add the new functions.
February 24, 2012
On Fri, Feb 24, 2012 at 07:19:33PM +0100, Adam D. Ruppe wrote:
> On Friday, 24 February 2012 at 17:54:47 UTC, H. S. Teoh wrote:
> >You could just pad the unused space with blank pixels.
> 
> whoa, make a tile based system, like the NES.
> 
> There's a private use area in unicode, 16 bits of space I think.

Heh, I was thinking of exactly the same thing, the unicode PUA. Great minds think alike.  :)

The PUA is actually 17 bits of space, U+F0000 -> U+10FFFF. There's also a smaller stretch in the BMP, from U+E000 -> U+F8FF, which might be useful if you want to stick with 16-bit characters.


> You could define a tile set to use that, or a simple bitmap format, but it isn't enough room to do on demand tiles I think.
>
> Imagine this though, your terminal lets the app define new characters, in color. The protocol might be an escape sequence then you go ahead and write out an image using whatever format (xpm or base64 png - something you can easily enough represent as text.)
> 
> Then, you overwrite a private use character. You can then write that character to display the custom glyph.
> 
> Now, you treat it just like any other character for writing, scrolling, read back, etc. Mixes right in with text.

I like it. Reminds me of those old DOS games that used to do that, except they have only 8 bits of space, so only trivial things could be done. :)


> Now, you won't be able to just "cat image.png", but you could write a cat that does it. Define a tileset for a line, write it out, repeat and let it scroll to show the whole image.
[...]

But why constrict yourself needlessly? Unicode already has double-width characters, like pretty much the entire CJK block. Some terminals/apps already handle that (e.g. rxvt/vim on a CJK file).

Nothing stops you from assigning an entire bitmap to a private use character. Call it multiwidth-multiheight. Now you have lots more room in the character encoding for reusable bitmaps. :P

Or if multiheight is too messy to implement, at least multiwidth will allow you to represent an entire bitmap row as a single PUA character. So a large PNG will just be a linear sequence of PUA characters representing rows in the image.


T

-- 
No! I'm not in denial!