On Tue, 2019-11-19 at 10:38 +0000, Joseph Rushton Wakeling via Digitalmars-d- announce wrote:
> On Monday, 18 November 2019 at 20:48:53 UTC, bachmeier wrote:
> > IMO this is one of the most important parts of the first five minutes with the language. Someone has installed the compiler, and now they want to test it out. If they have a bad experience with Dub, they will not continue with the language. A package manager, including the choice of format, is something you have to get right. Rust understands this.
> 
> Fair point.  But that isn't something that has to be decided at the _start_ of a rewrite: better to focus on wanted behaviour, and then derive the ideal config format from that.

I'd argue that from a socio-technical perspective it is an important factor that should be an integral part of developing any user facing tool. The functionality of the tool is important, but how that functionality is presented to a user new to the tool and a user proficient with the tool should be equally important.

As an example of how not to do things, perhaps I can suggest earlier forms of Git. It took nigh on a decade to simplify things to get to the overcomplicated CLI we have today with Git.

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On Tue, 2019-11-19 at 10:32 +0000, Joseph Rushton Wakeling via Digitalmars-d- announce wrote:
> […]
> 
> I'm simply concerned that if we don't put enough scrutiny on the app features and behaviour, we run the risk of simply reproducing some of the problematic design decisions of the existing tool.
> 
> As an example -- try running `dub test --build=release`. Intuition suggests that this is testing a release build.  But actually it strips out your unittests, because `--build=release` determines not only the optimization flags, but the complete set of DFLAGS used -- and `-unittest` isn't among them.
> 
> I ran `dub test && dub test --build=release` as a matter of habit for some time before discovering the latter wasn't actually testing anything.  And while I now know that I _can_ define a custom build to get what I want, that's clumsy and hard(er) to discover, and annoying to have to repeat for every project I create.
> 
> That's the kind of usability and design concern we really ought to be revisiting in any rewrite.

How can I not agree with this being a critical bug in Dub!

:-)

-- 
Russel.
===========================================
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk

On 11/18/19 1:19 PM, Tobias Pankrath wrote:
> On Monday, 18 November 2019 at 19:54:38 UTC, Russel Winder wrote:
> 
>> Probably yes. Though Cargo has taken many different decisions to Dub and mostly I think Cargo took better decisions.
> 
> Could you elaborate a bit, please? I am not familiar with Cargo though.

See also Russel's previous post on this issue [1]:

> Go and Rust emphasised using Git, Mercurial, and Breezy repositories as packages from the outset. Go chose not to add a central repository, Rust chose to add one. Rust's choice was the correct one for supporting developers. In hindsight, Go has had problems with packages from the outset based on the initial workspace model. Slowly over the decade Go is finding ways forward. Rust got it right from the beginning, once they had stripped down the standard library and emphasised use of the central repository – if only Phobos could be stripped right back to the absolute necessary and everything else provided via the central repository. Obviously not all is good in Rust-land, just as with Python and PyPI, the central repository is not curated, and this leads to horrible messes. Ceylon got this more right, but it is a language few have heard of and even fewer use.
> 
> Dub does not allow for use of Git, Mercurial, or Breezy repositories only the uncurated (and therefore potentially problematic) central repository. OK so you can do Git checkouts as a separate activity and use local filestore references, but this is not feasible for packages in the central repository.
> 
> Dub builds packages to a location outside the project being worked on. Cargo pulls sources to such a central non-project place and then compiles into a project specific location. Dub tries to store all compiled version out of project in the same area as the sources. Does this matter? Isn't Dub making things easier to share? Sort of, sort of, and no. Dub stores all compilations, but hides them and presents only the last compilation easily. This makes things hard to work with for non-Dub tooling. Cargo makes it a lot easier, at the expense of lack of sharing but always compiling everything into the project area.
> 
> Cargo uses TOML, Dub uses SDL (or if you are masochistic JSON). 'nuff said.
> QED.
> 
> Dub seems to have become the de facto, and indeed de jure, standard for D build, and yet somehow Cargo is just assumed by all Rust programmers whereas Dub is a source of contention and ill-will in the D community.
> 
> Dub's biggest problem is that there are many in the D community who will not use it – and they are vocal about it. The solution is not censorship, the solution is for Dub to evolve very rapidly so that these vocal people have the rug pulled from under them, i.e. their complaints become invalid.

[1]: https://forum.dlang.org/post/mailman.543.1571300816.8294.digitalmars-d@puremagic.com (scroll toward the end of the post)

On 11/19/19 3:15 AM, Robert Schadek wrote:
> On Monday, 18 November 2019 at 23:08:13 UTC, Laurent Tréguier wrote:
> 
>> I don't understand why this would apply to JSON specifically. Whatever the language is, the config files will be hand-written; spelling errors are pretty universal, and anything we write is prone to mistakes to some extent
> 
> dud already tells you if you mistyped a key.
> 
> Adding new file formats is "trivial" with dud current code base.
> 
> This https://github.com/symmetryinvestments/dud/blob/master/pkgdescription/source/dud/pkgdescription/package.d is the data structure used to mirror the file content.
> 
> This https://github.com/symmetryinvestments/dud/blob/master/pkgdescription/source/dud/pkgdescription/json.d is the json reader.
> 
> Currently, I have no plans to add any other file format.
> But PR's are always welcome.
> The decision on json and sdl has been made a long time ago, for better
> or for worse.
> Please don't turn this thread into bike-shedding.
> Additionally, dub/dud is already on the way to add cli functions to manipulate
> the config file.
> A look at adding dependencies with "dub add" already shows that.

And I would complain that the fact json exists as a file format already screws up dub add -- using dub add removes ALL comments in an SDL file, and rewrites the file in the order it sees fit.

result: I don't use dub add any more.

-Steve

On 11/19/19 11:30 AM, Steven Schveighoffer wrote:
> 
> And I would complain that the fact json exists as a file format already screws up dub add -- using dub add removes ALL comments in an SDL file, and rewrites the file in the order it sees fit.
> 
> result: I don't use dub add any more.

Oops, that's probably actually my fault, not dub's. Those are current limitations of the sdl lib, SDLang-D.

1. The (pull) parser doesn't currently emit comments (and consequently the DOM doesn't support them either, not that it matters for dub which uses the pull parser directly). IIRC, the comments are currently being ignored in the lexer. Shouldn't be TOO terribly hard to fix, though. Modify the lexer to emit them as a new token type, and adjust the parser's grammar logic to accept them and to emit them as a new pull parser event. (PR's always welcome!)

2. IIRC, the lib's only method of generating sdl involves using the DOM, and TBH, I don't remember offhand just how friendly/unfriendly it is to maintaining a specific ordering. I'd have to look into that. Ideally, there should probably be a range-based reverse-pull-parser that just takes user-generated pull parser events and spits out an sdl doc.

On 11/19/19 3:29 AM, Robert Schadek wrote:
> On Monday, 18 November 2019 at 16:31:09 UTC, Nick Sabalausky (Abscissa) wrote:
>> As has been discussed elsewhere a few months ago, dependency resolution should be outsourced to an established SAT <https://en.wikipedia.org/wiki/Boolean_satisfiability_problem> solving lib, to avoid re-inventing a notoriously difficult wheel. This is what all the serious package managers have been moving towards, after invariably hitting problems (much like dub) trying to roll-their-own.
> 
> OT: By saying "all the __serious__" you effectively ended this part of the thread.
> You basically say, that whatever I do, solve P=NP for instance, dud will
> never be a __serious__ package manager because it does not use an
> existing SAT solver.
> That's just bad arguing.

Dude, it's just casual speech, not formal logic. A colloquialism, not a formal claim of "For all X, X always necessarily implies Y".

> The thing I want from dud, is to tell me what dependency chain let to conflicts
> and this I find hard to extract from current SAT solvers.
> Most I have worked with just told me: "This solution works" "NO"
> Also the debug experience is not really great most of the time.

Ok, now that I wasn't aware of.

On Monday, 18 November 2019 at 12:59:25 UTC, Joseph Rushton Wakeling wrote:
>
> Cool :-)  Since I have also been experiencing a fair bit of production-use DUB pain in the last year, I really appreciate your taking action on this.
>
> A few things that would be good to understand up front:
>
>   * what are the particular issues with DUB that you want to address?
>
>       - making the codebase cleaner and more functional is obviously
>         nice but is at most a means to an end -- what's the real end
>         you have in mind?

My reason for making it cleaner is, because I assume this will give
me a build tool I can fix when broken. And hopefully it has less
bugs to begin with because its cleaner.

>
>       - I would imagine getting dependency resolution really right
>         would be top of the list -- it would be good to aim to fix
>         issues like https://github.com/dlang/dub/issues/1732

That is one thing yes.

>
>       - I would personally really appreciate it if you would make it
>         a design goal to better separate concerns in terms of what
>         DFLAGS are used and why (for example, the fact that right now
>         `dub test --build=release` will not actually run unittests,
>         as `--build=release` drops the `-unittest` flag)
>
>   * are there any particular known issues with DUB that this definitely
>     will NOT address?

I'm not sure yet.

>
>   * are there any key _breaking_ changes you have in mind?
>
>   * where does this stand w.r.t. some of the proposals to break DUB apart
>     into more cleanly separated components, e.g. determining compatible
>     dependencies, downloading dependencies, building or running tests ... ?

It is build as a library first. The CLI is just using the library
constructs.

>
> Some concrete feedback on the project as it stands:
>
>   * the tickboxes of compatible commands are nice, but it would be good to
>     have a more contextualized roadmap, in particular outlining the design
>     concerns for core data structures and functionality
>
>       - this should probably be in issues rather than the README, but
>         it needs to be somewhere, otherwise it's hard for anyone outside
>         to identify things they can do

True, I'll work on that.

>
>       - it might be nice to use a GitHub project to manage things so that
>         us outside folks can identify better what's being worked on and
>         what's blocked by what

I already started that, somewhat.

>
>   * I don't mind breaking changes in either the config format or the command
>     line API if it gets us to a nicer tool, so please embrace the opportunity
>     where appropriate
>
>       - I can imagine this might be the reason why the aim is to support
>         a "tasteful subset" of DUB's features: it means that others can
>         be re-implemented in an incompatible but better way
>
>       - auto-conversion mechanisms may be preferable to outright support
>         for old formats and commands
>
>   * I really recommend trying to start writing clean, clear commit messages
>     from the very start -- think of this as another form of code documentation
>     that communicates to all interested readers the intent and considerations
>     behind any particular change to the codebase.  This makes it much easier
>     for outsiders to get a clear understanding of the project and get into the
>     habit of contribution
>
>       - right now, pretty much all the commit messages read like spontaneous
>         notes where even YOU won't remember the whys or wherefores in a few
>         months' time

I know, I'll try to do better

>
>       - long term it saves a LOT of time when trying to look back and understand
>         "Why did we do things that way?" -- particularly useful when trying to
>         fix some subtle design bug
>
>   * for the same reasons, really try to provide good documentation and comments
>     for all code from the start -- this really makes it much easier for anyone
>     interested to grasp the major design concerns and get contributing

Here is disagree, to a degree I consider comments a code smell.
If I have to write them, I failed to convey the information
needed to understand the code in the code.

>
>   * these concerns are going to be strongest for the key data structures and
>     algorithms -- please make sure these are REALLY documented well, from the
>     very start
>
> Hope all of that's helpful, and best wishes for taking this forward -- I will try to help as I can, but time right now is very constrained ... ;-)
>
> Thanks & best wishes,
>
>      -- Joe

Thank you

On Tuesday, 19 November 2019 at 16:30:26 UTC, Steven Schveighoffer wrote:
> And I would complain that the fact json exists as a file format already screws up dub add -- using dub add removes ALL comments in an SDL file, and rewrites the file in the order it sees fit.
>
> result: I don't use dub add any more.
>
> -Steve

I haven't implemented dud add yet, but it should be as easy as
parsing the file checking if the dep is already in.
If it is not the case add a new line to the end of the file.
dub.sdl files have no order after all.

On Tuesday, 19 November 2019 at 17:13:49 UTC, Nick Sabalausky (Abscissa) wrote:
> On 11/19/19 11:30 AM, Steven Schveighoffer wrote:
>> 
>> And I would complain that the fact json exists as a file format already screws up dub add -- using dub add removes ALL comments in an SDL file, and rewrites the file in the order it sees fit.
>> 
>> result: I don't use dub add any more.
>
> Oops, that's probably actually my fault, not dub's. Those are current limitations of the sdl lib, SDLang-D.

dud has its own sdlang parser, I wanted to have no dependencies other than
phobos and SDLang-D was not pure nor @safe, mostly because of
static Key[5] keywords; in Lexer.lexIdentKeyword.

On Wednesday, 20 November 2019 at 11:40:19 UTC, Robert Schadek wrote:

> Here is disagree, to a degree I consider comments a code smell.
> If I have to write them, I failed to convey the information
> needed to understand the code in the code.

You think this is a code smell: https://dlang.org/phobos/std_array.html ?

--
/Jacob Carlborg