In the "Implementing Half Floats in D" thread, we seemed to have reached a consensus on two important points:
(a) Phobos should have a broad scope (rather than being small like the C standard library).
(b) The current flat structure of Phobos (every module in the root) does not scale to hundreds of modules.

It's not quite unanimous on (a), but seems to be close enough.

Together, (a) and (b) mean we need a plan. The sooner we can do it, the less painful it will be. But, on the other hand, it's something that we really don't want to get wrong.

Personal taste plays a huge role in this (Practically any structure can work, but we're looking for an arrangement which is intuitive and aesthetically pleasing to as many people as possible). It'll be most productive to stick to uncontroversial facts as long as possible.

It would be great if people with extensive experience in other languages could give a brief description of the structure of the corresponding libraries. (Also include pseudo-standard libraries, for example boost in C++).

1. How many total modules are there? How old is the library? What is the recent growth rate of the library (Percentage increase in size per year, over whatever time interval is reasonable for that library)?
2. How deep is the hierarchy? How many top-level branches are there?
3. How has the library dealt with obsolete functions and modules? Eg, are there entire top-level branches which are obsolete? How many stupid names are there (eg, "std.algorithm2") which were forced by the original design becoming obsolete?
4. How much of the library is successful/convincing? To make it a bit less subjective: how much of the library is commonly ignored, in favour of using third-party libraries instead? Which areas are failures in this way (eg, GUI, database, ...)? Which areas are the most often praised?

Maybe such a survey already exists, but I haven't been able to find an existing one.

On Wednesday, February 06, 2013 08:56:26 Don wrote:
> In the "Implementing Half Floats in D" thread, we seemed to have
> reached a consensus on two important points:
> (a) Phobos should have a broad scope (rather than being small
> like the C standard library).
> (b) The current flat structure of Phobos (every module in the
> root) does not scale to hundreds of modules.
> 
> It's not quite unanimous on (a), but seems to be close enough.
> 
> Together, (a) and (b) mean we need a plan. The sooner we can do
> it, the less painful it will be. But, on the other hand, it's
> something that we really don't want to get wrong.
> 
> Personal taste plays a huge role in this (Practically any structure can work, but we're looking for an arrangement which is intuitive and aesthetically pleasing to as many people as possible). It'll be most productive to stick to uncontroversial facts as long as possible.

I agree that we should look at minimizing how many top-level modules we add in the future, but I don't think that it's generally worth trying to rearrange the modules that we already have. So, adding std.halffloat would be bad, but I'm not sure that it's worth doing something like moving std.uri into std.net. Even if we did, we'd be forced to leave std.uri around for quite some time (if not permanently, given Walter's attitude about such things), which negates the gain to some exent. You can also argue endlessly about what should be where in the hierarchy, which is needless bikeshedding. So, if we do any moving, we need to be sure that that's what we want to do and be done with it, and we shouldn't be moving new modules around. They need to go into the right spot when they're added, not be rearranged later.

> It would be great if people with extensive experience in other languages could give a brief description of the structure of the corresponding libraries. (Also include pseudo-standard libraries, for example boost in C++).

Java and C# would probably be good places to look, but I haven't dealt with either of them recently. It's easy enough to look at the docs though. They're well-organized. However, it should be kept in mind that both of those languages have the downside of being forced to have one public class per file (at least Java does - I don't remember for sure if that's true for C# or not), which can be good but is often overkill, which means that we wouldn't necessarily want to lay stuff out in quite the same way as they do, even if we agreed that they do a good job of it.

- Jonathan M Davis

On 2013-02-06 08:56, Don wrote:
> In the "Implementing Half Floats in D" thread, we seemed to have reached
> a consensus on two important points:
> (a) Phobos should have a broad scope (rather than being small like the C
> standard library).
> (b) The current flat structure of Phobos (every module in the root) does
> not scale to hundreds of modules.
>
> It's not quite unanimous on (a), but seems to be close enough.
>
> Together, (a) and (b) mean we need a plan. The sooner we can do it, the
> less painful it will be. But, on the other hand, it's something that we
> really don't want to get wrong.

+1

This is long overdue.

-- 
/Jacob Carlborg

On 2013-02-06 08:56, Don wrote:

> It would be great if people with extensive experience in other languages
> could give a brief description of the structure of the corresponding
> libraries. (Also include pseudo-standard libraries, for example boost in
> C++).
>
> 1. How many total modules are there? How old is the library? What is the
> recent growth rate of the library (Percentage increase in size per year,
> over whatever time interval is reasonable for that library)?

Tango has 365 modules according to the documentation for D1 (including the object module).

> 2. How deep is the hierarchy? How many top-level branches are there?

These are spread over 9 packages in the top level "tango" package.

It seems to have at most four levels of packages, including the top level "tango" package.

> 3. How has the library dealt with obsolete functions and modules? Eg,
> are there entire top-level branches which are obsolete? How many stupid
> names are there (eg, "std.algorithm2") which were forced by the original
> design becoming obsolete?

There are no obsolete top level packages. As far as I know there are no stupid names. At one time they did remove the "collection" package and replaced it with a "container" package. It's not really a stupid name. There can be other cases like this where the optimal name is not used but there are no names like "std.algorithm2".

-- 
/Jacob Carlborg

On Wednesday, 6 February 2013 at 08:16:38 UTC, Jonathan M Davis wrote:
> On Wednesday, February 06, 2013 08:56:26 Don wrote:
>> In the "Implementing Half Floats in D" thread, we seemed to have
>> reached a consensus on two important points:
>> (a) Phobos should have a broad scope (rather than being small
>> like the C standard library).
>> (b) The current flat structure of Phobos (every module in the
>> root) does not scale to hundreds of modules.
>> 
>> It's not quite unanimous on (a), but seems to be close enough.
>> 
>> Together, (a) and (b) mean we need a plan. The sooner we can do
>> it, the less painful it will be. But, on the other hand, it's
>> something that we really don't want to get wrong.
>> 
>> Personal taste plays a huge role in this (Practically any
>> structure can work, but we're looking for an arrangement which is
>> intuitive and aesthetically pleasing to as many people as
>> possible). It'll be most productive to stick to uncontroversial
>> facts as long as possible.

> You can also argue endlessly about what should be where in
> the hierarchy, which is needless bikeshedding.

Yes, exactly. That's why I want to defer passing judgement on anything until we have a reasonable set of data. Just because a design works well for a particular language doesn't mean it would also be good for D, but it's still good to look at.

I think it's more constructive to say "that design was tried in Ruby, but they abandoned it" than to say "I personally don't like it". OTOH, if three different languages have totally different structures, and all have been widely praised, then it's clear it's a personal preference issue.

And if we can say, "all nine libraries we've looked at did it this way", then the argument to do the same thing is very strong.

On Wednesday, 6 February 2013 at 10:15:28 UTC, Don wrote:
> On Wednesday, 6 February 2013 at 08:16:38 UTC, Jonathan M Davis wrote:
>> On Wednesday, February 06, 2013 08:56:26 Don wrote:
>>> In the "Implementing Half Floats in D" thread, we seemed to have
>>> reached a consensus on two important points:
>>> (a) Phobos should have a broad scope (rather than being small
>>> like the C standard library).
>>> (b) The current flat structure of Phobos (every module in the
>>> root) does not scale to hundreds of modules.
>>> 
>>> It's not quite unanimous on (a), but seems to be close enough.
>>> 
>>> Together, (a) and (b) mean we need a plan. The sooner we can do
>>> it, the less painful it will be. But, on the other hand, it's
>>> something that we really don't want to get wrong.
>>> 
>>> Personal taste plays a huge role in this (Practically any
>>> structure can work, but we're looking for an arrangement which is
>>> intuitive and aesthetically pleasing to as many people as
>>> possible). It'll be most productive to stick to uncontroversial
>>> facts as long as possible.
>
>> You can also argue endlessly about what should be where in
>> the hierarchy, which is needless bikeshedding.
>
> Yes, exactly. That's why I want to defer passing judgement on anything until we have a reasonable set of data. Just because a design works well for a particular language doesn't mean it would also be good for D, but it's still good to look at.
>
> I think it's more constructive to say "that design was tried in Ruby, but they abandoned it" than to say "I personally don't like it". OTOH, if three different languages have totally different structures, and all have been widely praised, then it's clear it's a personal preference issue.
>
> And if we can say, "all nine libraries we've looked at did it this way", then the argument to do the same thing is very strong.

I don't know if this is related or not (I'd think it is), but if we were to re-think phobos' module organization, we may also want to think about how we want to deal with changing entire modules at once.

I'm mostly thinking about things like "random2.d", "xmk2.d" or whatnot.

Or it could be un-related. Just saying we might want to take this into account.

On Wednesday, February 06, 2013 11:28:25 monarch_dodra wrote:
> I don't know if this is related or not (I'd think it is), but if we were to re-think phobos' module organization, we may also want to think about how we want to deal with changing entire modules at once.
> 
> I'm mostly thinking about things like "random2.d", "xmk2.d" or whatnot.

I believe that that's what we decided on previously. It just hasn't actually been done yet, because no one has implemented such a module since the decision was made.

- Jonathan M Davis

On Wednesday, 6 February 2013 at 07:56:26 UTC, Don wrote:
>
> It would be great if people with extensive experience in other languages could give a brief description of the structure of the corresponding libraries. (Also include pseudo-standard libraries, for example boost in C++).
>

I know very well the BCL in C#, but I don't think is possible to map existing D modules to the same hierarchy, because D is lacking the ideea of namespace. More than that, the way things are organized in BCL doesn't match the D phobos rules, only the System namespace in BCL contains more than 300 different types with thousands of functions. I don't think anybody will like a system.d file containing that number of elements.

The class hierarchy in main BCL assembly (mscorlib.dll) is organized as follows (important classes - almost everything in C# is a class) from D point of view:
- system - basic types, general purpose functions;
- system.collections - object ranges;
- system.collections.generic - generic ranges;
- system.diagnostics - debug helpers;
- system.globalization - locale support;
- system.io - streams, files, readers, writers
- system.reflection - runtime traits;
- system.resources - Windows resource management;
- system.runtime.interopservices - WinAPI;
- system.runtime.serialization - runtime serialization;
- system.security - general purpose security functions;
- system.security.cryptography - encryption algorithms;
- system.text - text encoding/decoding support
- system.threading - threading support
- system.threading.tasks - task based threading, asynchrony

Another important assembly (lib will be the immediate equivalent in D) is the system.dll assembly containing following important namespaces:
- system - general purpose types;
- system.codedom - compiler tools. somehow equivalent to ctfe in D;
- system.collections.concurrent - shared collections and algorithms
- system.componentmodel - particular cases of interfaces, events, classes
- system.configuration - user settings, reading registry, ini files or various forms of persiostent data;
- system.diagnostics - profiling, performance counters, unit testing;
- system.compression - well known compression algorithms and associated streams;
- system.io.ports - general purpose port interaction functions;
- system.net - various communication protocols
- system.security - various authentication systems
- system.text.regularexpressions - regex;

This is not a comprehensive list, there are more assemblies in BCL(eg: system.core.dll assembly contains also some important classes like linq extensions or system.xml.dll which contains all xml related stuff), and the namespaces above were shortened (eg: in BCL C# has system.net.mail for SMTP/IMAP or system.net.mime namespace for mime encoding).

Regarding code organization, C# compiler is not dependent of the file system, there are no files or packages, you can code any number of classes in the same file or you can even code a class in multiple files (partial classes).

On Wednesday, 6 February 2013 at 10:51:19 UTC, Jonathan M Davis wrote:
> On Wednesday, February 06, 2013 11:28:25 monarch_dodra wrote:
>> I don't know if this is related or not (I'd think it is), but if
>> we were to re-think phobos' module organization, we may also want
>> to think about how we want to deal with changing entire modules
>> at once.
>> 
>> I'm mostly thinking about things like "random2.d", "xmk2.d" or
>> whatnot.
>
> I believe that that's what we decided on previously. It just hasn't actually
> been done yet, because no one has implemented such a module since the decision
> was made.
>

Andrei made a proposal to allow transforming a module into a package. It seems to me like the way to go for this.

This is even possible to manage several versions of the lib using version that way. And different module can even import their own. Sound like a better idea than messing up with names.

On Wednesday, February 06, 2013 12:06:18 deadalnix wrote:
> Andrei made a proposal to allow transforming a module into a package. It seems to me like the way to go for this.

Yes, but that's a different issue. It's one thing to take std.algorithm or std.datetime and turn them into packages while still allowing you to import std.algorithm or std.datetime as before. It's quite another to completely replace a module with another module. To do that, you need a new module. Anything else will break code. The question then is what to name the new module.

> This is even possible to manage several versions of the lib using version that way. And different module can even import their own. Sound like a better idea than messing up with names.

You could replace the entire module in-place, but doing so will break code, and Walter in particular is very much against that in general. Whatever we did would require allowing people to transition from one to the other and we might even have to leave the old one around permanently (which I don't like, but I'm sure that Walter would favor). Simply changing it with a new version of Phobos wouldn't cut it.

- Jonathan M Davis