On Wednesday, 20 August 2014 at 09:40:37 UTC, ketmar via Digitalmars-d-announce wrote:
> imagine mybiglib, mybiglib.internal, mybiglib.other, and each of them
> not single module, but package itself. so we have module
> mybiglib.other.thingy which wants to use functions from
> module mybiglib.internal.wisdom, which is part of mybiglib.internal
> package.

Do we need a hierarchy of internals, is the problem this big? Why mybiglib.wisdom is not good?

On Wednesday, 20 August 2014 at 14:25:59 UTC, Kagamin wrote:
> If some utility is internal to std.package1 an used in the entire std.package1, shouldn't it sit in std.package1? Why push it to subpackage?
>
> std.package1.module1
> std.package1.module2
> std.package1.internal <- package1's internals go here
> std.package1.subpackage.module1
> std.package1.subpackage.module2

It may semantically belong to subpackage but still needs to be available to package1, something not uncommon in templated code (subpackage is generic implementation, package1 is specialization that still needs access to non-public functions).

Also you seem to imply that "internal" is always small tightly coupled module which is hardly true for any higher level functionality. So in practice it will be more like this:

std.package1.module1
std.package1.module2
std.package1.internal1
std.package1.internal2
...
std.package1.internal20

Not really encouraging..

What you propose is effectively prohibiting to use packages to organize your code and requiring to design your module hierarchy based on desired protection relations, not other way around. I think it is conceptually wrong approach and unnecessarily restrictive compared to overall D design principles (no single "true" way of doing things)

On Wednesday, 20 August 2014 at 14:33:53 UTC, Kagamin wrote:
> On Wednesday, 20 August 2014 at 09:40:37 UTC, ketmar via Digitalmars-d-announce wrote:
>> imagine mybiglib, mybiglib.internal, mybiglib.other, and each of them
>> not single module, but package itself. so we have module
>> mybiglib.other.thingy which wants to use functions from
>> module mybiglib.internal.wisdom, which is part of mybiglib.internal
>> package.
>
> Do we need a hierarchy of internals, is the problem this big? Why mybiglib.wisdom is not good?

As I see on the realistic example of datetime, which is BIG, we only need to split it into a flat set of files without an overly deep package hierarchy.

On Wednesday, 20 August 2014 at 14:36:59 UTC, Kagamin wrote:
> As I see on the realistic example of datetime, which is BIG, we only need to split it into a flat set of files without an overly deep package hierarchy.

We _may_ split it into flat set files (solving only part of the problem) but it is desirable to have a deeper package hierarchy. Package hierarchy is not just an encapsulation tool, it is also a great way to simplify navigation and finding needed modules - something that Phobos is current terrible at exactly because of flat hierarchies.

On Wed, Aug 20, 2014 at 4:33 PM, Kagamin via Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> wrote:

> Do we need a hierarchy of internals, is the problem this big? Why mybiglib.wisdom is not good?
>

It gets really unwieldy when you want to put a whole bunch of things into one project and give control of different packages to different teams. Yes you could use multiple libraries in a lot of cases but if you're wanting it in a single setup / package (like phobos), especially one that is internal to another package you need sub-packages.

Examples that would use this are:
- kitchen sink libraries like phobos
- frameworks such as vibe.d
- libraries with lots of internal code  such as Ogre3d if it were made in D

Well that seems to be good reasons to me.

On Wed, 20 Aug 2014 14:33:52 +0000
Kagamin via Digitalmars-d-announce
<digitalmars-d-announce@puremagic.com> wrote:

> Do we need a hierarchy of internals, is the problem this big? Why mybiglib.wisdom is not good?
ah, why we need such things as subdirectories at all? CP/M was fine without concept of subdirectories!

On Wednesday, 20 August 2014 at 14:35:31 UTC, Dicebot wrote:
> It may semantically belong to subpackage but still needs to be available to package1, something not uncommon in templated code (subpackage is generic implementation, package1 is specialization that still needs access to non-public functions).

If generic implementation is designed to be customized, it probably means, it should be widely available for customization, like xml document and base64 encoder. Also it's probably a logical error if base type is less visible than the derived type.

> What you propose is effectively prohibiting to use packages to organize your code and requiring to design your module hierarchy based on desired protection relations, not other way around. I think it is conceptually wrong approach and unnecessarily restrictive compared to overall D design principles (no single "true" way of doing things)

It's ok for packages to exchange public interface, but internals? If a subpackage has internals, they are for its usage, and consumable functionality should be provided through public interface.

On Wednesday, 20 August 2014 at 15:13:06 UTC, Kagamin wrote:
> On Wednesday, 20 August 2014 at 14:35:31 UTC, Dicebot wrote:
>> It may semantically belong to subpackage but still needs to be available to package1, something not uncommon in templated code (subpackage is generic implementation, package1 is specialization that still needs access to non-public functions).
>
> If generic implementation is designed to be customized, it probably means, it should be widely available for customization, like xml document and base64 encoder. Also it's probably a logical error if base type is less visible than the derived type.

This is just your design preference and hardly a good "one size fits them all" decision. I also don't speak about inheritance but about composition - customization may be implemented via unsafe field exposure and unsuitable for general public usage but useful for simplifying internal implementation maintenance. We should not force users into single good package structure based on certain design beliefs.

>> What you propose is effectively prohibiting to use packages to organize your code and requiring to design your module hierarchy based on desired protection relations, not other way around. I think it is conceptually wrong approach and unnecessarily restrictive compared to overall D design principles (no single "true" way of doing things)
>
> It's ok for packages to exchange public interface, but internals? If a subpackage has internals, they are for its usage, and consumable functionality should be provided through public interface.

Same here. As library size grows you completely lose the distinction between "public" and "internal", certain parts of the library may become hidden from other parts and public but available for others. It is pretty much a necessity to keep up with maintenance when there is a large team working on it simultaneously. Right now people mostly rely on convention, I have seem quite some comments like "should have been private but needs to be accessed from module X, don't ever touch it" (public-but-undocumented functions of Phobos sometimes fall into the same category).

Telling people that they design applications in a wrong way and need to change their habits is a good approach to alienate them against the language.

On 8/20/14, 7:49 AM, ketmar via Digitalmars-d-announce wrote:
> On Wed, 20 Aug 2014 14:33:52 +0000
> Kagamin via Digitalmars-d-announce
> <digitalmars-d-announce@puremagic.com> wrote:
>
>> Do we need a hierarchy of internals, is the problem this big? Why
>> mybiglib.wisdom is not good?
> ah, why we need such things as subdirectories at all? CP/M was fine
> without concept of subdirectories!

No need to demean the question. It is valid. -- Andrei

On Wednesday, 20 August 2014 at 17:19:58 UTC, Andrei Alexandrescu wrote:
> On 8/20/14, 7:49 AM, ketmar via Digitalmars-d-announce wrote:
>> On Wed, 20 Aug 2014 14:33:52 +0000
>> Kagamin via Digitalmars-d-announce
>> <digitalmars-d-announce@puremagic.com> wrote:
>>
>>> Do we need a hierarchy of internals, is the problem this big? Why
>>> mybiglib.wisdom is not good?
>> ah, why we need such things as subdirectories at all? CP/M was fine
>> without concept of subdirectories!
>
> No need to demean the question. It is valid. -- Andrei

Originally flat Phobos hierarchy was considered "good enough". Now we can see that such approach doesn't scale well - features are often missed because of non-intuitive module placement, compile times suffer because of many cross-module dependencies (flat hierarchy encourages big modules).

Is there any reason to think that same logic applied to sub-packages will scale any better as Phobos size grows? I doubt so. In fact I feel it is already beyond the size where it is convenient and only reason why even smaller deeply nested modules are not an option is exactly because it is too hard to keep both existing protection attribute relation and provide easy to navigate hierarchy at the same time.

The fact that some of people who have actually tried to use package.d support this language change is pretty good anecdotal evidence that there is a problem to be solved.