On Saturday, 24 March 2018 at 16:11:18 UTC, Andrei Alexandrescu wrote:
> Anyhow. Right now the order of processing is the same as the lexical order in which flags are passed to getopt. There may be use cases for which that's the more desirable way to go about things, so if you author a PR to change the order you'd need to build an argument on why command-line order is better. FWIW the traditional POSIX doctrine makes behavior of flags independent of their order, which would imply the current choice is more natural.

Several of the TSV tools I built rely on command-line order. There is an enhancement request here: https://issues.dlang.org/show_bug.cgi?id=16539.

A few of the tools use a paradigm where the user is entering a series instructions on the command line, and there are times when the user entered order matters. Two general cases:

* Display/output order - The tool produces delimited output, and the user wants to control the order. The order of command line options determines the order.

* Short-circuiting - tsv-filter in particular allows numeric tests like less-than, but also allow the user to short-circuit the test by testing if the data contains a valid number prior to making the numeric test. This is done by evaluating the command line arguments in left-to-right order.

Short-circuiting is supported the Unix `find` utility.

I have used this approach for CLI tools I've written in other languages. Perl's Getopt::Long processes args in command-line, so it supports this.

I considered submitting a PR to getopt to change this, but decided against it. The approach used looks like it is central to the design, and changing it in a backward compatible way would be a meaningful undertaking. Instead I wrote a cover to getopt that processes arguments in command-line order. It is here: https://github.com/eBay/tsv-utils-dlang/blob/master/common/src/getopt_inorder.d. It handles most of what std.getopt handles.

The TSV utilities documentation should help illustrate these cases. tsv-filter use short circuiting: https://github.com/eBay/tsv-utils-dlang/blob/master/docs/ToolReference.md#tsv-filter-reference. Look for "Short-circuiting expressions" toward the bottom of the section.

tsv-summarize obeys the command-line order for output/display. See: https://github.com/eBay/tsv-utils-dlang/blob/master/docs/ToolReference.md#tsv-summarize-reference.

There's one other general limitation I encountered with the current compile-time approach to command-line argument processing. I couldn't find a clean way to allow it to be extended in a plug-in manner.

In particular, the original goal for the tsv-summarize tool was to allow users to create custom operators. The tool has a fair number of built-in operators, like median, sum, min, max, etc. Each of these operators has a getopt arg invoking it, eg. '--median', '--sum', etc. However, it is common for people to have custom analysis needs, so allowing extension of the set would be quite useful.

The code is setup to allow this. People would clone the repo, write their own operator, placed in a separate file they maintain, and rebuild. However, I couldn't figure out a clean way to allow additions to command line argument set. There may be a reasonable way and I just couldn't find it, but my current thinking is that I need to write my own command line argument handler to support this idea.

I think handling command line argument processing at run-time would make this simpler, at the cost loosing some compile-time validation.

--Jon

On 3/24/18 12:59 PM, H. S. Teoh wrote:
> On Sat, Mar 24, 2018 at 12:11:18PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> [...]
>> Anyhow. Right now the order of processing is the same as the lexical
>> order in which flags are passed to getopt. There may be use cases for
>> which that's the more desirable way to go about things, so if you
>> author a PR to change the order you'd need to build an argument on why
>> command-line order is better. FWIW the traditional POSIX doctrine
>> makes behavior of flags independent of their order, which would imply
>> the current choice is more natural.
> 
> So what about making this configurable?

That'd be great. I'm thinking something like an option std.getopt.config.commandLineOrder. Must be first option specified right after arguments. Sounds good?

On Sat, Mar 24, 2018 at 05:24:28PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> On 3/24/18 12:59 PM, H. S. Teoh wrote:
> > On Sat, Mar 24, 2018 at 12:11:18PM -0400, Andrei Alexandrescu via Digitalmars-d wrote: [...]
> > > Anyhow. Right now the order of processing is the same as the lexical order in which flags are passed to getopt. There may be use cases for which that's the more desirable way to go about things, so if you author a PR to change the order you'd need to build an argument on why command-line order is better. FWIW the traditional POSIX doctrine makes behavior of flags independent of their order, which would imply the current choice is more natural.
> > 
> > So what about making this configurable?
> 
> That'd be great. I'm thinking something like an option std.getopt.config.commandLineOrder. Must be first option specified right after arguments. Sounds good?

Great!

Not so sure how easy it is to implement while supporting everything else, though, given the current structure of the code.


T

-- 
Time flies like an arrow. Fruit flies like a banana.

On Saturday, March 24, 2018 09:59:44 H. S. Teoh via Digitalmars-d wrote:
> And given the defensiveness surrounding std.getopt, my conclusion can only be: dump std.getopt, roll my own.  It's sad, since in general Phobos design tends to be superior to its C++ counterparts.  But we then have warts like std.getopt that people refuse to acknowledge is a problem.  So be it.

I think that there are at least a couple alternatives to std.getopt on code.dlang.org if you want alternatives. Personally, the only complaints I've had with std.getopt is that bundling isn't the default and that it's not always easy to figure out whether an argument has been set or not. But at least the bundling can be configured, and getopt can probably be improved to work with Nullable so that it'll be easier to figure out whether an argument has been set.

As for defensiveness, I'm not quite sure what you're referring to. The main point was that given how often getopt gets called in a program, improving its Big-O complexity isn't worth it, but there have been a number of improvements to getopt over the years, so it's not like we're not allowed to improve it. It's just that improving its Big-O complexity is kind of pointless. In any case, as Andrei said, if a new option can be added to fix your use case, then that shouldn't be a problem, though I have no clue how much of a pain that will be to implement, particularly since std.getopt isn't exactly simple.

- Jonathan M Davis

On Sat, Mar 24, 2018 at 10:30:31PM -0600, Jonathan M Davis via Digitalmars-d wrote:
> On Saturday, March 24, 2018 09:59:44 H. S. Teoh via Digitalmars-d wrote:
> > And given the defensiveness surrounding std.getopt, my conclusion can only be: dump std.getopt, roll my own.  It's sad, since in general Phobos design tends to be superior to its C++ counterparts. But we then have warts like std.getopt that people refuse to acknowledge is a problem.  So be it.
[...]
> As for defensiveness, I'm not quite sure what you're referring to. The main point was that given how often getopt gets called in a program, improving its Big-O complexity isn't worth it, but there have been a number of improvements to getopt over the years, so it's not like we're not allowed to improve it. It's just that improving its Big-O complexity is kind of pointless. In any case, as Andrei said, if a new option can be added to fix your use case, then that shouldn't be a problem, though I have no clue how much of a pain that will be to implement, particularly since std.getopt isn't exactly simple.
[...]

OK, the part about defensiveness may be just my overreaction. I apologize.  But yeah, I glanced at the code, and don't see any easy way to implement what Andrei agreed with. It's just too much work for something I could just write for myself in a much shorter time. I guess I'll just log an enhancement request in bugzilla and leave it at that.


T

-- 
It always amuses me that Windows has a Safe Mode during bootup. Does that mean that Windows is normally unsafe?

On Sunday, 25 March 2018 at 04:30:31 UTC, Jonathan M Davis wrote:
> On Saturday, March 24, 2018 09:59:44 H. S. Teoh via Digitalmars-d wrote:
>> And given the defensiveness surrounding std.getopt, my conclusion can only be: dump std.getopt, roll my own.  It's sad, since in general Phobos design tends to be superior to

Yeah I have "dumb XYZ, roll my own" experience often too.
As there are already many big libraries like `arsd` or `ae` out there, I don't think I'm the only one with these feeling.
I wonder if someone ever tries to fork/reboot Phobos with all the goodies, but without the legacy cruft like auto-decoding and similar friends whose breaking changes can't be made.

>> its C++ counterparts.  But we then have warts like std.getopt that people refuse to acknowledge is a problem.  So be it.
>
> I think that there are at least a couple alternatives to std.getopt on code.dlang.org if you want alternatives.

Yes, two good ones are:

https://blog.thecybershadow.net/2014/08/05/ae-utils-funopt
http://code.dlang.org/packages/darg

> Personally, the only complaints I've had with std.getopt is

Hehe I like many things about std.getopt, but it's not perfect either.
A few examples:

- I often just want to map CLI arguments to a config object where using UDAs would more natural and less boilerplate

    struct Config {
       @option("c|compiler")
       string compiler;
    }

Now with the rejected/postponeed __traits(documentation) the ddoc help text could be automatically read and put into the auto-generated CLI help.

- I don't like to manually check for .helpWanted

Imho I constantly find myself doing this:

    if (helpInformation.helpWanted)
    {
`DDoc wrapper
All unknown options are passed to the compiler.
./ddoc <file>...
`.defaultGetoptPrinter(helpInformation.options);
        return 1;
    }

I would have preferred this being the default behavior or at least the default behavior if a help text string is explicitly provided e.g. like:

    getopt(`My program
./program ...`, args, ...);

or maybe with sth. like `.withHelp("")`

- setting shared configs doesn't work

I know, I could use a TLS config or use an atomicOp or synchronized assignment to set it, but often casting is easier and that's rather ugly:

https://github.com/dlang/dlang.org/blob/master/dspec_tester.d#L101

- in theory getopt should be @safe and use ref

It just does a bit of string manipulation, but it looks like we have to wait until DIP1000 for this:

https://github.com/dlang/phobos/pull/6281

Also similarly to std.stdio.read or std.format.formattedRead there's no need to use pointers, D's @safe ref would have worked too. Now, it looks like this change can't be made anymore as it would be a breaking one due to ambiguities.

- it would be really cool to support generating zsh/bash completions files

This is the last point on my list as it's not really a limitation of std.getopt and GetoptResult should be enough for this, but it looks like no one bothered enough to write a zshGetoptPrinter so far.

> getopt can probably be improved to work with Nullable so that it'll be easier to figure out whether an argument has been set.

Yes, supporting Nullable would really cool!

On Sunday, 25 March 2018 at 06:58:50 UTC, Seb wrote:
>> I think that there are at least a couple alternatives to std.getopt on code.dlang.org if you want alternatives.
>
> Yes, two good ones are:
>
> https://blog.thecybershadow.net/2014/08/05/ae-utils-funopt

funopt is based on getopt underneath, so this issue still applies to it, sorry!

Well, funopt translates options to function arguments, so there's no way to specify a delegate anyway, but at least the performance aspect applies.

On 3/23/2018 10:55 PM, Chris Katko wrote:
> Last question though, is there any kind of list of features, and minor features and fixes that can or need to be done? Perhaps it already exists,
And here it is:

https://issues.dlang.org/

On Sunday, 25 March 2018 at 09:27:31 UTC, Walter Bright wrote:
> On 3/23/2018 10:55 PM, Chris Katko wrote:
>> Last question though, is there any kind of list of features, and minor features and fixes that can or need to be done? Perhaps it already exists,
> And here it is:
>
> https://issues.dlang.org/

Not a very comprehensive list. Virtually all of those issues have no comment on them. If it's a feature request you might as assume it requires a DIP cause there's no reason to otherwise waste your time implementing the feature. Oddly enough there's almost no other way to get anyone's attention of whether a feature requires a DIP or not unless there's a pull request for the feature. So if you want a feature you almost have to risk wasting your time implementing it. It's not a very good system, but someone throws up some stats about how many issues get solved/pull requests get created per month and they conclude that it's working fine.

On Sat, Mar 24, 2018 at 10:05:36PM -0700, H. S. Teoh via Digitalmars-d wrote: [...]
> OK, the part about defensiveness may be just my overreaction. I apologize.  But yeah, I glanced at the code, and don't see any easy way to implement what Andrei agreed with. It's just too much work for something I could just write for myself in a much shorter time. I guess I'll just log an enhancement request in bugzilla and leave it at that.
[...]

Turns out, there's already been an issue for this, filed 2 years ago:

https://issues.dlang.org/show_bug.cgi?id=16539


T

-- 
People say I'm arrogant, and I'm proud of it.