Hi,

The history of std.(experimental.)logger and the latest thread about gui functionality inclusion into Phobos made me think about how to solve the problem of adding new modules.

I came up with the idea (maybe not new) to create a additional library(along druntime and Phobos) delivered with dmd package and named "Mars" (Deimos is unfortunately already taken ).

The library itself would be driven by community (not individual library developer) in order to be... the standard.

The process would be something similar to that other committees use
(e.g http://www.iec.ch/standardsdev/how/processes/development/) where before the standard is approved it goes through a draft stage. Still a draft can be used to create a working product (e.g. some Wi-Fi solutions based on IEEE 802.11 drafts)

Possible initial prerequisites:
- User awareness about the usage consequences
- Library placed at https://github.com/D-Programming-Language/
- Only well recognized community members have pull rights
- design decision made on the best known sw engineering patterns used in D
- New module should be functional with D/Phobos standards applied
- API and implementation allowed to change any time in order to make a progress
- no external dependencies beside OS services
- "draft" as the root module name e.g. "module draft.gui"

Advantages:
- community driven process which ensures the lowest level of controversy
- fast path for modules like GUI to be standardized

Disadvantages:
- additional effort for the sw release process

Ready to be destroyed ;)

Piotrek

On Saturday, 31 January 2015 at 14:06:20 UTC, Piotrek wrote:
> and named "Mars" (Deimos is unfortunately already taken ).

BTW, Mars is the name of the language, before people started calling it D (which stuck).

On Saturday, 31 January 2015 at 14:33:35 UTC, Vladimir Panteleev wrote:
> On Saturday, 31 January 2015 at 14:06:20 UTC, Piotrek wrote:
>> and named "Mars" (Deimos is unfortunately already taken ).
>
> BTW, Mars is the name of the language, before people started calling it D (which stuck).

I know that, but the naming is the least important aspect of the proposal.
We can refarm the unused "Mars" name or choose another one, maybe "Curiosity", or anything suitable ;)

Piotrek

This has the advantage over existing situation where you have the official library where things need to go through exacting and time consuming process and then dub.  Within dub every project is at the same level and it is not obvious which projects are the ones to use, and there is not necessarily coherence in the different ways things are done, and the only one motivated to work on supplementary features like documentation is the project author.  So you have very useful, well written libraries that have no docs.

On Saturday, 31 January 2015 at 18:11:48 UTC, Laeeth Isharc wrote:
> This has the advantage over existing situation where you have the official library where things need to go through exacting and time consuming process and then dub.  Within dub every project is at the same level and it is not obvious which projects are the ones to use, and there is not necessarily coherence in the different ways things are done, and the only one motivated to work on supplementary features like documentation is the project author.  So you have very useful, well written libraries that have no docs.

What i hate about dub is that no every library is ready to use. Some require additional steps/resources in order to get them to work. Dub does not have any indication of that because the library authors dont usually include that information.

What is good about this "extra community driven standard library" is that everything will be included in one place and will be at the ready when it is needed.

On Saturday, 31 January 2015 at 18:45:13 UTC, Israel wrote:
> On Saturday, 31 January 2015 at 18:11:48 UTC, Laeeth Isharc wrote:
>> This has the advantage over existing situation where you have the official library where things need to go through exacting and time consuming process and then dub.  Within dub every project is at the same level and it is not obvious which projects are the ones to use, and there is not necessarily coherence in the different ways things are done, and the only one motivated to work on supplementary features like documentation is the project author.  So you have very useful, well written libraries that have no docs.
>
> What i hate about dub is that no every library is ready to use. Some require additional steps/resources in order to get them to work. Dub does not have any indication of that because the library authors dont usually include that information.
>
> What is good about this "extra community driven standard library" is that everything will be included in one place and will be at the ready when it is needed.

Well, I certainly do appreciate that all these libraries exist and, as D matures, better documentation a and a general tidying up will surely be part of that process, as has already begun with phobos.

There is something of a hacker disdain for docs - they are not fun to write (for most people), and one thinks that a few pointers should be enough for someone to figure it out.  And that's true, for someone like the author of the code.  But it is hard to imagine what it is like not to have the skillset and experience one has.  And so it is a very different experience for the new user coming from python than what he experienced when he learnt python, even though if you have the basic ability D need not be intrinsically harder.

The division of labour makes sense - the people who like and are good at writing docs are not necessarily those who like and are good at writing code.  But its harder to recruit helpers for a personal project than a shared library.

On Saturday, 31 January 2015 at 19:03:55 UTC, Laeeth Isharc wrote:
> On Saturday, 31 January 2015 at 18:45:13 UTC, Israel wrote:
>
> the people who like and are good at writing docs are not necessarily those who like and are good at writing code.

This is exactly how i feel.

On Saturday, 31 January 2015 at 14:06:20 UTC, Piotrek wrote:
> Hi,
>
> The history of std.(experimental.)logger and the latest thread about gui functionality inclusion into Phobos made me think about how to solve the problem of adding new modules.
>
> I came up with the idea (maybe not new) to create a additional library(along druntime and Phobos) delivered with dmd package and named "Mars" (Deimos is unfortunately already taken ).
>
> The library itself would be driven by community (not individual library developer) in order to be... the standard.
>
> The process would be something similar to that other committees use
> (e.g http://www.iec.ch/standardsdev/how/processes/development/) where before the standard is approved it goes through a draft stage. Still a draft can be used to create a working product (e.g. some Wi-Fi solutions based on IEEE 802.11 drafts)
>
> Possible initial prerequisites:
> - User awareness about the usage consequences
> - Library placed at https://github.com/D-Programming-Language/
> - Only well recognized community members have pull rights
> - design decision made on the best known sw engineering patterns used in D
> - New module should be functional with D/Phobos standards applied
> - API and implementation allowed to change any time in order to make a progress
> - no external dependencies beside OS services
> - "draft" as the root module name e.g. "module draft.gui"
>
> Advantages:
> - community driven process which ensures the lowest level of controversy
> - fast path for modules like GUI to be standardized
>
> Disadvantages:
> - additional effort for the sw release process
>
> Ready to be destroyed ;)
>
> Piotrek

The most important thing about a standard library is decisiveness in the leadership about what *kinds* of things should be in it. When it's been made clear that a given task is worthy of inclusion in the standard library, there will be enough incentive for contributors to put in the time and effort required to meet its high standards.

But there are so many tasks that aren't quite up to that bar, either because they serve too specific a niche, or because there are two or more good, but conflicting ways to implement solutions, and no good way to decide between them.

For any such task that doesn't meet those high standards, the library's development is best left to the community. But there is a place for the official recognition of existing libraries which have reached a level of quality that can be safely recommended. This is what is currently missing.

There needs to be a page for this. The existing Wiki page:

http://wiki.dlang.org/Libraries_and_Frameworks

... suffers from exactly the same problem as code.dlang.org. It contains everything in existence.

This is more of a "second-party" library system, for which idea I am indebted to this article: http://www.jfbillingsley.com/blog/?p=206

Suggestions for implementation?

On Saturday, 31 January 2015 at 14:06:20 UTC, Piotrek wrote:
>...
+1, basically boost for D. I heavily agree that the libraries should complement Phobos and never be orthogonal to phobos lest we have another tangos/phobos situation.

I like the idea of having an additional library that we would ship alongside Phobos with every release. There of course some obvious pros and cons for having 'Mars' (or whatever is called) as a DUB packages vs included in the standard library:

Pros for being a DUB package:
+ faster release cycles (push a new version any time).
+ semantic versioning (instead of dmd 2.0xx).
+ can depend on other DUB packages (doesn't need to include everything it uses beside Phobos, ofc).
+ can have different configurations / build types, be split in DUB subpackages and all the other nice things that the DUB package format provides.
+ can include other binaries (eg. C libs).
+ etc...
Cons:
- Probably not so well tested, because it won't get as wildly used as Phobos.
- Third-party with zero guarantees. (You have to trust the author(s) that it is not malicious or anything like that).
- Not everybody is comfortable with having DUB as a dependency on their project.
- Can be a bit of an inconvenience if you only want to write a small script.
- DUB doesn't have first class support in the major IDEs, AFAIK (for people that prefer them over the terminal). Actually I thing it's extremely simple to use on the command-ling and I have no problem with Mono-D or "dub generate visuald", but it's not like the NuGet support in VS for .NET projects (AFAIR you can now have language plugins/analyzers based on Roslyn (linting, DSLs, etc.) which are NuGet packages that you download from their gallery).
- etc...

Pros being included alongside Phobos:
+ Better testing because more people can/will use it
+ Potentially better code, because of code review during pull requests and generally high standards for new stuff (like with std.experimental.logger).
+ More stable, because people may care more for backwards comparability (though the points is that this will not be guaranteed).
+ People new to the language will feel more comfortable using 'standard' libraries.
+ etc...
Cons:
- Extremely slow release cycle.
- Hard to get new stuff (controversial like GUI) in.
- Not being able to have external dependencies than druntime and Phobos (like bindings for C libraries)
- etc...

With all that said I still think that it is a good idea to have another library shipping alongside Phobos with every DMD release (or GDC, LDC, etc.).
I think a good compromise would be the following:

1. Include DUB with DMD. We don't need a stable DUB as a library API to just use it to get other packages.
2. Make 'Mars' a DUB package and use semantic versioning to tag new releases.
3. Move it to github.com/D-Programming-Language/.
4. Include last known 'well working' with every DMD release. (Of course other implementations are free to decide whether to include it). Or we can have some post-installation script which would call DUB.
5. Afterwards if a new version of 'Mars' is released users can just do a 'dub upgrade' to update the one that's already included, or wait for a new official release.

I really do thing that this would be a step in the right direction. For example Microsoft has been been using their package manager NuGet "as a .NET Framework release vehicle" quite successfully for the last couple of years. Here[1] you can see that they are providing quite large and heavily used parts of .NET.

Another good idea is to separate Phobos from DMD and also put it on DUB. As you can see[2] many of the integral parts of.NET are provided as packages and people have no problem using them as such (you can see by the large download numbers).


[1]: http://blogs.msdn.com/b/dotnet/p/nugetpackages.aspx
[2]: https://www.nuget.org/packages