On 10/21/2012 06:55 PM, Jens Mueller wrote:
> Chad J wrote:
>> On 10/21/2012 06:11 PM, Jens Mueller wrote:
>>> Chad J wrote:
>>>> On 10/21/2012 05:01 PM, Jens Mueller wrote:
>>>>
>>>> It seems to have a hard ncurses/termcap/etc dependency.
>>>
>>> Yes. I think you cannot make it portable without. Please proof me wrong
>>> and I'll fix this.
>>>
>>
>> Well, traditionally it's done with automake/autoconf.  You'd end up
>> with preprocessor defines that tell you whether the lib has been
>> statically linked or not.  This isn't available here because Phobos
>> doesn't use these as a build system and I hope it never does.
>
> I mean to detect if your terminal is ANSI compatible without adding
> another dependency.
> It's easy to provide different version(...) to support different modes.
> One could do something like:
> 1. Check at run time if ncurses etc. are available.
>    * If they are use them.
>    * Otherwise fall back to ANSI codes or throw an Exception.
>
> What do you think?
>

I completely agree.

The difficulty I encountered is actually /doing/ the runtime detection.  Does Phobos have a way to dynamically link .so files yet?

If yes, then we could search the obvious places ("/lib/libncurses.so.5" matches on my machine right now) and link to any files found.

Since color is probably never necessary for program correctness, I think it is acceptable to ignore color formatting and produce plain text when detection fails.  It would make sense to make this configurable though: the Terminal object could have a .throwOnDetectionFailure flag that can be set if people want to be a bit more hardcore about it.

>>>> I'll admit when I started trying to work on doing this thing, I
>>>> never got anything onto the screen.  What stopped me was that I
>>>> couldn't figure out how to detect ncurses/termcap/etc.  I was going
>>>> to shoot for Phobos inclusion and making Phobos always link with
>>>> ncurses seems like a bad idea.
>>>
>>> Dependence on Phobos is bad. If you can detect whether a terminal is
>>> ANSI compatible then this mode should be default. But I don't know how
>>> to detect this.
>>>
>>
>> Wrong direction on the dependency.  I wouldn't expect Terminal
>> coloring/detection to rely on Phobos.  I'd expect it to be one of
>> the lower-level modules built into Phobos.
>
> I mean it's bad to have Phobos depend on ncurses.
> Though one can go with loading at run time.
>

Yes.

>>>> Ultimately I expect it to work with writeln or writefln to make it
>>>> discoverable and easy to work with.
>>>
>>> One could try this. At least for Linux. You just have to add the
>>> appropriate escape sequences. But this won't work on Windows.
>>>
>>
>> I remember having a plan for this.  See below.
>>
>>>> Back then I did design a format spec for introducing colors into
>>>> format strings:
>>>> www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html
>>>
>>> I doubt that the Phobos maintainers will accept this. This is very
>>> invasive.
>>
>> Hmmm, depends what is meant by invasive.
>>
>> I feel it's the only way to have discoverable and concise syntax.
>> I'd be pretty disappointed if they didn't, regardless of who submits
>> the pull request.
>>
>> I remember it being possible in Phobos to determine the destination
>> of the format operation.  If the destination is a string in memory,
>> then no color formatting would be applied.  If the destination is a
>> Linux terminal of some kind, then some ncurses terminal info would
>> be looked up (possible a cached lookup) and escape sequences
>> generated based on that.  If the destination is a Windows terminal,
>> then these approaches can be considered:
>> (1) Split the formatted text up on the color format boundaries.
>> Send the slices into the stream one by one, calling the necessary
>> WinAPI color formatting functions inbetween.  I think this might not
>> have been possible with Phobos' architecture.
>> (2) Insert ANSI escape sequences into the text.  The I/O code for
>> Windows would then have to intercept these and convert them into the
>> appropriate WinAPI calls.  I think this was possible, and even
>> distinguishable from the case of writing to a string in memory.
>>
>> If the invasiveness worry comes from the possibility of dumping
>> escape sequences into non-terminal destinations, then I hope the
>> above wall of text can alleviate that concern.
>
> Checking whether something is a terminal can be done using isatty on the
> file handle. I think this will work.
> But it is invasive because you want to add it to the formatting spec. Is
> this the usual way it is done? I don't know how it is done in Python
> or other languages.
>

Is it relevant?  I posit that having format syntax is simply better than not.  This is on an objective basis.

There is no weakness to this.  The only shred of a counterargument I can think of is that it makes the format strings more difficult to learn. Other than that, it is possible to detect the destination of the formatter, so color codes will never end up in places where they shouldn't.  A conservative approach to this should handle most desires and never interfere with all the people with no interest in color.

On the upshot are the things I've mentioned:
- A format specifier is potentially more discoverable.
- A format specifier is more concise.  This keeps your lines from wrapping.  They are probably too long already.
- To cement the previous point: nesting requires a few extra characters with a format specifier, rather than a couple extra /lines/ for extra function calls.
- Calls to stateful console functions allow people to write bugs like saving console state and then forgetting to restore it (or throwing an exception and neglecting to restore from within a scope guard).  Format specifiers do not have this problem.
- etc (I'm sure I'm forgetting one or two.)

These are the reasons why my ideal language has color formatting built into its I/O routines.

>>> I added writecf, writec, etc. with additional arguments.
>>> writec(Color.red, "some text")
>>> or
>>> writecf(Color.red, "%s", "some text")
>>> This is fine I think. But better options may be worth investigating.
>>>
>>> Jens
>>
>> I really think this should be in Phobos.  If it doesn't go into
>> Phobos, then people will write crappy terminal apps with no color.
>> If it does go into Phobos, then the better devs will see the
>> opportunity and use it.  Something 3rd party is much less
>> discoverable and won't have nearly as much impact.  The use case is
>> almost all CLI apps, so it's not like an uncommon corner-case or
>> something.
>
> True. When I asked there was less/no interest. I think we should just
> join and make one module that shows up top ten when googled for "D
> terminal/console color".
>
>> I run a Gentoo system where things are configured to use color
>> output wherever possible.  The portage devs went through all of the
>> necessary contortions to get Python to output colored text, somehow.
>> I feel the end result is indispensable.  Color is an extremely
>> useful tool for making sure that the user doesn't overlook important
>> bits while scanning text.  Outside of Gentoo, I find this most
>> notable in grep: uncolored grep output is just awful, but the
>> coloring makes it possible to easily identify why the regular
>> expression behaved the way it did.
>>
>> I look forward to a better CLI ecosystem where highly reliable D
>> programs are written quickly and write beautiful colored output ;)
>
> It is. I'd like you to join and get this done.
> And you're right we should have Phobos integration in mind. Maybe they
> will add it. But in the mean time we can have a separate module.
>
> Jens

I think this is very reasonable.  Sounds like a plan!

I'll probably make pull requests when I allocate time to work on this.

Would implementing the format spec make sense?  It might give me a good excuse to write an snprintf alternative in D that can be used at compile-time and won't introduce nasty link-time dependencies.

On Sun, 21 Oct 2012 17:32:41 -0500, Chad J <chadjoan@__spam.is.bad__gmail.com.is.bad__gmail.com.com> wrote:

> On 10/21/2012 06:11 PM, Jens Mueller wrote:
>> Chad J wrote:
>>> On 10/21/2012 05:01 PM, Jens Mueller wrote:
>>>
>>> It seems to have a hard ncurses/termcap/etc dependency.
>>
>> Yes. I think you cannot make it portable without. Please proof me wrong
>> and I'll fix this.
>>
>
> Well, traditionally it's done with automake/autoconf.  You'd end up with preprocessor defines that tell you whether the lib has been statically linked or not.  This isn't available here because Phobos doesn't use these as a build system and I hope it never does.
>
>>> I'll admit when I started trying to work on doing this thing, I
>>> never got anything onto the screen.  What stopped me was that I
>>> couldn't figure out how to detect ncurses/termcap/etc.  I was going
>>> to shoot for Phobos inclusion and making Phobos always link with
>>> ncurses seems like a bad idea.
>>
>> Dependence on Phobos is bad. If you can detect whether a terminal is
>> ANSI compatible then this mode should be default. But I don't know how
>> to detect this.
>>
>
> Wrong direction on the dependency.  I wouldn't expect Terminal coloring/detection to rely on Phobos.  I'd expect it to be one of the lower-level modules built into Phobos.
>
>>> Ultimately I expect it to work with writeln or writefln to make it
>>> discoverable and easy to work with.
>>
>> One could try this. At least for Linux. You just have to add the
>> appropriate escape sequences. But this won't work on Windows.
>>
>
> I remember having a plan for this.  See below.
>
>>> Back then I did design a format spec for introducing colors into
>>> format strings:
>>> www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html
>>
>> I doubt that the Phobos maintainers will accept this. This is very
>> invasive.
>
> Hmmm, depends what is meant by invasive.
>
> I feel it's the only way to have discoverable and concise syntax.  I'd be pretty disappointed if they didn't, regardless of who submits the pull request.
>
> I remember it being possible in Phobos to determine the destination of the format operation.  If the destination is a string in memory, then no color formatting would be applied.  If the destination is a Linux terminal of some kind, then some ncurses terminal info would be looked up (possible a cached lookup) and escape sequences generated based on that.  If the destination is a Windows terminal, then these approaches can be considered:
> (1) Split the formatted text up on the color format boundaries.  Send the slices into the stream one by one, calling the necessary WinAPI color formatting functions inbetween.  I think this might not have been possible with Phobos' architecture.
> (2) Insert ANSI escape sequences into the text.  The I/O code for Windows would then have to intercept these and convert them into the appropriate WinAPI calls.  I think this was possible, and even distinguishable from the case of writing to a string in memory.
>
> If the invasiveness worry comes from the possibility of dumping escape sequences into non-terminal destinations, then I hope the above wall of text can alleviate that concern.
>
>> I added writecf, writec, etc. with additional arguments.
>> writec(Color.red, "some text")
>> or
>> writecf(Color.red, "%s", "some text")
>> This is fine I think. But better options may be worth investigating.
>>
>> Jens
>
> I really think this should be in Phobos.  If it doesn't go into Phobos, then people will write crappy terminal apps with no color.  If it does go into Phobos, then the better devs will see the opportunity and use it.  Something 3rd party is much less discoverable and won't have nearly as much impact.  The use case is almost all CLI apps, so it's not like an uncommon corner-case or something.
>
> I run a Gentoo system where things are configured to use color output wherever possible.  The portage devs went through all of the necessary contortions to get Python to output colored text, somehow.  I feel the end result is indispensable.  Color is an extremely useful tool for making sure that the user doesn't overlook important bits while scanning text.  Outside of Gentoo, I find this most notable in grep: uncolored grep output is just awful, but the coloring makes it possible to easily identify why the regular expression behaved the way it did.
>
> I look forward to a better CLI ecosystem where highly reliable D programs are written quickly and write beautiful colored output ;)

Actually, IIRC, You can version it to link with ncurses or pdcurses if available.
The interface to pdcurses and ncurses is the same, or simply similar enough that it works.

I agree color is an important bit to not forget.
-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

Chad J wrote:
> On 10/21/2012 06:55 PM, Jens Mueller wrote:
> >Chad J wrote:
> >>On 10/21/2012 06:11 PM, Jens Mueller wrote:
> >>>Chad J wrote:
> >>>>On 10/21/2012 05:01 PM, Jens Mueller wrote:
> >>>>
> >>>>It seems to have a hard ncurses/termcap/etc dependency.
> >>>
> >>>Yes. I think you cannot make it portable without. Please proof me wrong and I'll fix this.
> >>>
> >>
> >>Well, traditionally it's done with automake/autoconf.  You'd end up with preprocessor defines that tell you whether the lib has been statically linked or not.  This isn't available here because Phobos doesn't use these as a build system and I hope it never does.
> >
> >I mean to detect if your terminal is ANSI compatible without adding
> >another dependency.
> >It's easy to provide different version(...) to support different modes.
> >One could do something like:
> >1. Check at run time if ncurses etc. are available.
> >   * If they are use them.
> >   * Otherwise fall back to ANSI codes or throw an Exception.
> >
> >What do you think?
> >
> 
> I completely agree.
> 
> The difficulty I encountered is actually /doing/ the runtime detection.  Does Phobos have a way to dynamically link .so files yet?

It is possible since very long time to link dynamically using dmd.

> If yes, then we could search the obvious places ("/lib/libncurses.so.5" matches on my machine right now) and link to any files found.

For that purpose in particular I have written ddl.
https://github.com/jkm/ddl
http://jkm.github.com/ddl/ddl.html
Just haven't found the time to integrate with the terminal stuff.

> Since color is probably never necessary for program correctness, I think it is acceptable to ignore color formatting and produce plain text when detection fails.  It would make sense to make this configurable though: the Terminal object could have a .throwOnDetectionFailure flag that can be set if people want to be a bit more hardcore about it.

Sounds useful.

> >>>>Ultimately I expect it to work with writeln or writefln to make it discoverable and easy to work with.
> >>>
> >>>One could try this. At least for Linux. You just have to add the appropriate escape sequences. But this won't work on Windows.
> >>>
> >>
> >>I remember having a plan for this.  See below.
> >>
> >>>>Back then I did design a format spec for introducing colors into format strings: www.chadjoan.com/d/dmd.2.058/html/d/phobos/std_format.html
> >>>
> >>>I doubt that the Phobos maintainers will accept this. This is very invasive.
> >>
> >>Hmmm, depends what is meant by invasive.
> >>
> >>I feel it's the only way to have discoverable and concise syntax. I'd be pretty disappointed if they didn't, regardless of who submits the pull request.
> >>
> >>I remember it being possible in Phobos to determine the destination
> >>of the format operation.  If the destination is a string in memory,
> >>then no color formatting would be applied.  If the destination is a
> >>Linux terminal of some kind, then some ncurses terminal info would
> >>be looked up (possible a cached lookup) and escape sequences
> >>generated based on that.  If the destination is a Windows terminal,
> >>then these approaches can be considered:
> >>(1) Split the formatted text up on the color format boundaries.
> >>Send the slices into the stream one by one, calling the necessary
> >>WinAPI color formatting functions inbetween.  I think this might not
> >>have been possible with Phobos' architecture.
> >>(2) Insert ANSI escape sequences into the text.  The I/O code for
> >>Windows would then have to intercept these and convert them into the
> >>appropriate WinAPI calls.  I think this was possible, and even
> >>distinguishable from the case of writing to a string in memory.
> >>
> >>If the invasiveness worry comes from the possibility of dumping escape sequences into non-terminal destinations, then I hope the above wall of text can alleviate that concern.
> >
> >Checking whether something is a terminal can be done using isatty on the
> >file handle. I think this will work.
> >But it is invasive because you want to add it to the formatting spec. Is
> >this the usual way it is done? I don't know how it is done in Python
> >or other languages.
> >
> 
> Is it relevant?  I posit that having format syntax is simply better than not.  This is on an objective basis.

Maybe. Don't know. But in any case this is a add on. On top of the basic API. It is a Phobos integration thing.

> There is no weakness to this.  The only shred of a counterargument I can think of is that it makes the format strings more difficult to learn. Other than that, it is possible to detect the destination of the formatter, so color codes will never end up in places where they shouldn't.  A conservative approach to this should handle most desires and never interfere with all the people with no interest in color.
> 
> On the upshot are the things I've mentioned:
> - A format specifier is potentially more discoverable.
> - A format specifier is more concise.  This keeps your lines from
> wrapping.  They are probably too long already.

Do you consider this
writecf(Color.red, "something %s", "here")
concise as well?

> - To cement the previous point: nesting requires a few extra characters with a format specifier, rather than a couple extra /lines/ for extra function calls.

Don't understand this point. Can you give an example?

> - Calls to stateful console functions allow people to write bugs like saving console state and then forgetting to restore it (or throwing an exception and neglecting to restore from within a scope guard).  Format specifiers do not have this problem.

The same holds for
writecf(Color.red, "something %s", "here")

> - etc (I'm sure I'm forgetting one or two.)
> 
> These are the reasons why my ideal language has color formatting built into its I/O routines.

Just wanted to point out that instead of that you can add writec*
functions. I think the only thing is that these are less discoverable
but they also work without formatting, e.g.
writec(Color.red, "my text");

> >>I run a Gentoo system where things are configured to use color output wherever possible.  The portage devs went through all of the necessary contortions to get Python to output colored text, somehow. I feel the end result is indispensable.  Color is an extremely useful tool for making sure that the user doesn't overlook important bits while scanning text.  Outside of Gentoo, I find this most notable in grep: uncolored grep output is just awful, but the coloring makes it possible to easily identify why the regular expression behaved the way it did.
> >>
> >>I look forward to a better CLI ecosystem where highly reliable D programs are written quickly and write beautiful colored output ;)
> >
> >It is. I'd like you to join and get this done.
> >And you're right we should have Phobos integration in mind. Maybe they
> >will add it. But in the mean time we can have a separate module.
> >
> >Jens
> 
> I think this is very reasonable.  Sounds like a plan!
> 
> I'll probably make pull requests when I allocate time to work on this.

That would be nice.
Let's coordinate with Robik. I think having Robik's ANSI stuff as a fall
back would be nice. And we need to decide on a repository.

> Would implementing the format spec make sense?  It might give me a good excuse to write an snprintf alternative in D that can be used at compile-time and won't introduce nasty link-time dependencies.

This is probably interesting for Phobos. But I'm not the one to make a
decision. The core Phobos developers should decide.
Hopefully somebody is reading this.

Jens

This could be improved, the example you give
>         writeln(Colors.Red, "Foo Bar", Colors.Reset);

imply that if you forget to put Colors.Reset at the end of writeln then all the following call will be made in Colors.Red which I don't like, IMHO "writeln(Colors.Red, "Foo Bar");" is better and at the end of the writeln your settings are resetted.
If you want to change the current settings, you have to use a different function call..

BR,
renoX

On Sunday, 21 October 2012 at 22:32:35 UTC, Walter Bright wrote:
> On 10/21/2012 12:28 PM, Robik wrote:
> > Simple example:
> >
> > import std.stdio, colord;
> > void main()
> > {
> >      setConsoleColors(Fg.red, Bg.blue);
> >      writeln("Red text on blue background.");
> >      resetConsoleColors(); // Bring back initial state
> > }
>
> Need a method to get the current state, and reset the current state. Otherwise, nested calls to the console functions will screw up the state.
>
> I.e.:
>
>     auto save = getConsoleState();
>     setConsoleColors(Fg.red, Bg.blue);
>     writeln("Red text on blue background.");
>     setConsoleState(save); // Bring back initial state
>
> Or better:
>
>     auto save = getConsoleState();
>     scope (exit) setConsoleState(save);
>     setConsoleColors(Fg.red, Bg.blue);
>     writeln("Red text on blue background.");

On Windows, setting color to initial sets console colors to ones that were set before launch of the program. On Posix it sets default (ANSI remove formatting).
I will try to check if it is possible to get current colors on Posix.

On 10/22/12 9:47 AM, Jens Mueller wrote:
> This is probably interesting for Phobos. But I'm not the one to make a
> decision. The core Phobos developers should decide.
> Hopefully somebody is reading this.

Off the top of my head something that is specific for only certain systems (Unixen in this case) is decidedly of less Phobos interest. We could, nevertheless, put such functionality in system-specific modules.

Andrei

On Sunday, 21 October 2012 at 19:28:21 UTC, Robik wrote:
> Hello,
>
> I would like to introduce ColorD, small library that allows to simply manipulate console output colors, both on Windows and Posix operating systems. It also supports font styles such as underline and strikethrough(Posix feature only).
>
>
> Simple example:
>
> import std.stdio, colord;
> void main()
> {
>     setConsoleColors(Fg.red, Bg.blue);
>     writeln("Red text on blue background.");
>     resetConsoleColors(); // Bring back initial state
> }
>
>
> Feedback welcome.
>
> GitHub: https://github.com/robik/ColorD
>
> Regards.

This is very much related to the ycurses and dcurses projects, and I strongly suggest you work with people behind those projects and come up with a nice/flexible/robust "console" API/package for D.

Andrei Alexandrescu wrote:
> On 10/22/12 9:47 AM, Jens Mueller wrote:
> >This is probably interesting for Phobos. But I'm not the one to make a
> >decision. The core Phobos developers should decide.
> >Hopefully somebody is reading this.
> 
> Off the top of my head something that is specific for only certain systems (Unixen in this case) is decidedly of less Phobos interest. We could, nevertheless, put such functionality in system-specific modules.

It also works on Windows. Even already implemented. The situation is similar to the file abstraction which has a different API in the Unix world and the Windows world.

Jens

Robik wrote:
> On Sunday, 21 October 2012 at 22:32:35 UTC, Walter Bright wrote:
> >On 10/21/2012 12:28 PM, Robik wrote:
> >> Simple example:
> >>
> >> import std.stdio, colord;
> >> void main()
> >> {
> >>      setConsoleColors(Fg.red, Bg.blue);
> >>      writeln("Red text on blue background.");
> >>      resetConsoleColors(); // Bring back initial state
> >> }
> >
> >Need a method to get the current state, and reset the current state. Otherwise, nested calls to the console functions will screw up the state.
> >
> >I.e.:
> >
> >    auto save = getConsoleState();
> >    setConsoleColors(Fg.red, Bg.blue);
> >    writeln("Red text on blue background.");
> >    setConsoleState(save); // Bring back initial state
> >
> >Or better:
> >
> >    auto save = getConsoleState();
> >    scope (exit) setConsoleState(save);
> >    setConsoleColors(Fg.red, Bg.blue);
> >    writeln("Red text on blue background.");
> 
> On Windows, setting color to initial sets console colors to ones
> that were set before launch of the program. On Posix it sets default
> (ANSI remove formatting).
> I will try to check if it is possible to get current colors on
> Posix.

Please look for this. But my research concluded that there is no such way on Posix.

Jens

On Monday, 22 October 2012 at 12:30:38 UTC, Jens Mueller wrote:
> But my research concluded that there is no such way on Posix.

Yeah, if there was a way to get current colors, you wouldn't have to set environment variables for programs like vi to know if the background is light or dark.

It's a pity the unix designers didn't change the definition slightly of the text palette way back when so you didn't have to know anyway...