So while we have quite a large list of potential GSoC projects [1], I think that many of these ideas are still a bit too abstract for potential students.

Hence, I will try to introduce this new [gsoc] series aimed at (1) discussing potential ideas in depth and (2) maybe finding new ideas. I encourage everyone (students or just curious D enthusiasts) to start more of these [gsoc] threads, s.t. our potential students can learn more about the proposed projects (students: please don't be shy).

The first item on my list is D's package manager DUB.

My personal favorite "missing feature" in DUB is a direct git support. It would be really awesome if we would implement a "GitRegistry" for packages, s.t. one can directly depend on git repositories:

dependency "my-package" version="git://.../my-branch#my-commit"

There's also a DEP about this feature [2] (though the proposed format there isn't set in stone). DUB's architecture is already fairly modular, so as a start students could look into creating a GitRegistry supplier [3]. There's also some previous work [4] which can of course be used for inspiration too. While initially, the `git` CLI binary can be used, it's not guaranteed that the user will have this installed (especially on Windows), so one would either need to use e.g. libgit2 (see [5] for a D binding) or implement Git's protocol itself (see e.g. [6]).

@community: what features do you miss the most from DUB?

Disclaimer: The span of the GSoC is three months of full time work and proposals from students should take this into consideration. The DUB project is a bit special as some individual ideas might take less than three months of work, but in this case one could just combine two or three DUB features/ideas into one project.

[1] https://wiki.dlang.org/GSOC_2019_Ideas
[2] https://github.com/dlang/dub/wiki/DEP6
[3] https://github.com/dlang/dub/blob/master/source/dub/packagesuppliers/packagesupplier.d
[4] https://github.com/dlang/dub/pull/1403
[5] https://github.com/s-ludwig/dlibgit
[6] https://github.com/git/git/blob/master/Documentation/technical/http-protocol.txt

On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
> So while we have quite a large list of potential GSoC projects [1], I think that many of these ideas are still a bit too abstract for potential students.
>
> [...]

Yes beeing able to specify git and also http dependencies would be great.

Also there is the cross compilation support missing (https://github.com/dlang/dub/pull/1541).

Another topic is watch command for dub, this would be also really great.

Kind regards
Andre

On 2/28/19 10:50 AM, Seb wrote:
> 
> @community: what features do you miss the most from DUB?
> 

Oh my...where to start...?

- Need a clear, obvious, dead-simple, 100% reliable way to tell DUB to keep its grubby hands out of my project's compilation. Tell it "Run this command to compile, run this other command to generate docs, etc., here's where you can find the resulting output, and otherwise, ***stay out of it!!***". Both the sdl/json config and the launched process should support appropriate envvars so I can cleanly pass-through CLI information like "Make docs instead of do build" or "use LDC" to my scripts without complicating them too much.

- Need to be able to tell DUB "I want *THIS* compiler flag included/omitted", WITHOUT triggering a stupid big ugly warning, or having to manually translate the CLI flag into DUB-speak. Both GDC and LDC come with compatibility wrappers for DMD's CLI, so an additional abstraction layer here is neither necessary nor welcome.

- Compile times that don't make a mockery of DMD.

- DUB's default stdout output needs to STFU by default.

- DUB and the compilers themselves should be bona-fide dependencies just like anything else.

- Needs good, solid support for non-source dependencies, system dependencies, and tools that the build itself depends on, not just libs your source depends on.

- This following workflow needs to be a BASIC STANDARD FULLY-ROBUST use-case with the same priority and importance as running "dub" to build a project: I have a non-DUB project and I want to include a library (say, Vibe.D) via DUB. I have my CLI command(s) to invoke the compiler, and all I have to do to use Vibe.d (or whatever) is insert this as part of my compiler invocation:

dmd blah blah `dub include vibe-d:~0.8.3` blah blah

And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed working, correct and ready to go.

At one point, I did a lot of work to make that happen, and I made some progress, but it was just so contrary to the way dub wanted to work I finally gave up.

- There's probably more I'm forgetting at the moment...

On Thu, Feb 28, 2019 at 12:45:56PM -0500, Nick Sabalausky (Abscissa) via Digitalmars-d wrote:
> On 2/28/19 10:50 AM, Seb wrote:
> > @community: what features do you miss the most from DUB?
[...]
> Oh my...where to start...?
> 
> - Need a clear, obvious, dead-simple, 100% reliable way to tell DUB to keep its grubby hands out of my project's compilation. Tell it "Run this command to compile, run this other command to generate docs, etc., here's where you can find the resulting output, and otherwise, ***stay out of it!!***". Both the sdl/json config and the launched process should support appropriate envvars so I can cleanly pass-through CLI information like "Make docs instead of do build" or "use LDC" to my scripts without complicating them too much.

+1.


> - Need to be able to tell DUB "I want *THIS* compiler flag included/omitted", WITHOUT triggering a stupid big ugly warning, or having to manually translate the CLI flag into DUB-speak. Both GDC and LDC come with compatibility wrappers for DMD's CLI, so an additional abstraction layer here is neither necessary nor welcome.

+1.


> - Compile times that don't make a mockery of DMD.

+100.  I think dub has improved in this area the last time I checked, but still, there's room for improvement.


> - DUB's default stdout output needs to STFU by default.

+1. Though to be fair, dub is already fairly quiet by defaut, compared to some other programs that tend to get run during builds.


> - DUB and the compilers themselves should be bona-fide dependencies just like anything else.

+100.  I don't have a problem with special handling of certain types of dependencies, but the user needs to be empowered to write the same functionality without hacking dub's source code.  Empowerment before convenience is my motto.


> - Needs good, solid support for non-source dependencies, system dependencies, and tools that the build itself depends on, not just libs your source depends on.

A generic dependency resolution system that the user can directly access would address this point.


> - This following workflow needs to be a BASIC STANDARD FULLY-ROBUST use-case with the same priority and importance as running "dub" to build a project: I have a non-DUB project and I want to include a library (say, Vibe.D) via DUB. I have my CLI command(s) to invoke the compiler, and all I have to do to use Vibe.d (or whatever) is insert this as part of my compiler invocation:
> 
> dmd blah blah `dub include vibe-d:~0.8.3` blah blah
> 
> And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed working, correct and ready to go.
> 
> At one point, I did a lot of work to make that happen, and I made some progress, but it was just so contrary to the way dub wanted to work I finally gave up.
[...]

I tried doing a non-dub project with dub dependencies before, and ended up with so much frustration that I threw in the towel and resorted to the hack of creating an empty dummy dub project in a subdirectory, whose sole purpose was to declare dub dependencies. I'd only run dub from the subdirectory when one or more dependencies need to be updated, but the actual compilation / linking is done by a different, sane build system that pulls in the objects dub built.

The fact that such hacks are needed just to make dub work the way I want it to, is a sign that something is fundamentally broken with its design. Again, it's an issue of user empowerment, or lack thereof: in theory it should be a tool that provides pluggable functionalities that the user can integrate into whatever else he may be using.  Unfortunately, its current design assumes the universe revolves around it, and requires everything else to orbit around how it works. When that doesn't happen it either works poorly, requiring contorted workarounds, or doesn't work at all, needing to boxed off into its own sub-universe in which it's the center of revolution before it is willing to play nice.  It's a hammer that wields the user, rather than the other way round.  Most of the above points are consequences of this fundamental point.


T

-- 
No! I'm not in denial!

On 2/28/19 8:38 PM, H. S. Teoh wrote:
> 
> It's a hammer
> that wields the user, rather than the other way round.  Most of the
> above points are consequences of this fundamental point.

+1, Definitely true...plus, I like this metaphor.

Though, to be fair, that description applies to most software these days. But that's a whole other rant...

On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
> @community: what features do you miss the most from DUB?


- SOUNDNESS: dub.selections.json doesn't encode deps for all comfigurations. The design isn't sound.

- configurations-based dependencies, or OS-based dependencies: if that is possible at all

- remove optional dependencies, removedub fetch/remove (it should never matter what is "installed"), Optional dependencies can be done with a dynamic loader anyway, i na way that is SOUND,

- remove SDLang support ("a stratagic mistake"), we are paying for this since it was introduced, with CONFUSION and having to double the recommendations on recipe files ("if you use JSON, use this... if you use SDLang, blah...")

- remove `dub describe` (no IDE really relies on it AFAIK), do not check the network by default which makes `dub describe` unusable in the first place

- build faster

- choose to keep one: overrides, add-local or add-path... too many way to have user-defined modifications

- colors, with way easier error messages! DUB IS THE PRIMARY INTERFACE TO D.

- remove `dub search`, you can as well type code.dlang.org in the browser, and it will be a better experience

There are too many features that scare people in DUB!

On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:

> - remove SDLang support ("a stratagic mistake"), we are paying for this since it was introduced, with CONFUSION and having to double the recommendations on recipe files ("if you use JSON, use this... if you use SDLang, blah...")

I'll stake my flag on that hill! Remove JSON if anything :-) It's ugly, it's noisy, and it doesn't allow comments. SDLang is much cleaner and human-friendly.

On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
> On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:
>
>> - remove SDLang support ("a stratagic mistake"), we are paying for this since it was introduced, with CONFUSION and having to double the recommendations on recipe files ("if you use JSON, use this... if you use SDLang, blah...")
>
> I'll stake my flag on that hill! Remove JSON if anything :-) It's ugly, it's noisy, and it doesn't allow comments. SDLang is much cleaner and human-friendly.

I agree. I started using json and then I switched to sdl. I had some difficulties but now I vote for sdl and I don't like the idea to switch back to json format.

On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
> On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:
>
>> - remove SDLang support ("a stratagic mistake"), we are paying for this since it was introduced, with CONFUSION and having to double the recommendations on recipe files ("if you use JSON, use this... if you use SDLang, blah...")
>
> I'll stake my flag on that hill! Remove JSON if anything :-) It's ugly, it's noisy, and it doesn't allow comments. SDLang is much cleaner and human-friendly.

No. I like json more instead of sdl. My whole build process is build upon json so it makes it easy to work with everything else that isn't dub related.

And there are json versions that allow comments, just switch to json5.

https://json5.org/

On Friday, 1 March 2019 at 11:11:56 UTC, bauss wrote:

> No. I like json more instead of sdl. My whole build process is build upon json so it makes it easy to work with everything else that isn't dub related.
>
> And there are json versions that allow comments, just switch to json5.
>
> https://json5.org/

Well, if they would make it less ugly and noisy, then it would be useful!

This shows why we should continue to support both.