On Saturday, 23 August 2014 at 10:04:40 UTC, Dmitry Olshansky wrote:
>> If the user relies that a symbol is found in
>> std.internal.regex.backtracking, moving it will break the code too. So
>> what's the difference?
>
> I don't see the difference. In any case user should see public aliases and never know where they lead to. Dicebot - any objections?

Well difference is that "internal" substring in the fully qualified name that is much more likely to tell user he is better to not touch it. However, original Kagamin proposal of embedding it into module names themselves does address that if you are ok with resulting uglyness.

On Saturday, 23 August 2014 at 11:12:34 UTC, Kagamin wrote:
> On Saturday, 23 August 2014 at 11:04:35 UTC, Jacob Carlborg wrote:
>> That's what we're trying to fix with this change.
>
> The proposal is to make internals visible to super packages, I only wonder whether internals visible to some package should be hidden from its subpackages.

It addresses both - explicitly supplied package identifiers imply access to all subpackages. Current implementation is very striaghtforward and naive in that regard.

On Saturday, 23 August 2014 at 11:08:31 UTC, Kagamin wrote:
> On Saturday, 23 August 2014 at 10:04:40 UTC, Dmitry Olshansky wrote:
>> For what its worth I'd _prefer_ std.regex.internal as in my opinion internals got to be well contained. But as discussed, `package` doesn't work this way.
>
> It can be a philosophical matter, but in my experience grouping by functionality is genuine, and access is an afterthought, so grouping by access doesn't work in the long term, as code naturally (spontaneously) groups by functionality.

That does contradict your statement that any stuff used in parent packages must go to up the hierarchy which is exactly grouping by access :)

I can probably give a more practical example I have just recently encountered when re-designing one of our internal library packages. It was a "serialization" package and sub-packages defined different (de)serialization models, some of those defining special wrapper types for resulting data that enforce certain semantics specific to that serialization model via the type system. It is not surprising that most internal details of such wrappers were kept package protected as exposing those to user could have easily violated all assumptions (but was needed to implement (de)serialization efficiently).

It has become more complicated when "meta-serializers" have been added to parent package - templated stuff that took any of sub-package implementations and added some functionality (like versioning support) on top. Being generic thing it reside in higher level "serialization" package but to be implemented efficiently it needs access to those internal wrapper fields. Moving wrapper modules to parent package is not an option here because those are closely related to specific serialization model. Exposing stuff as public is not an option because anyone not deeply familiar with package implementation can easily break all type system guarantees that way. Moving "meta-serializers" to sub-packages is quite a code duplication.

Right now I keep that stuff public and say in docs "please don't use this" which is hardly a good solution.

On Saturday, 23 August 2014 at 09:00:30 UTC, Kagamin wrote:
> What is difficult to find? With flat structure you have all files right before your eyes. If you need std.datetime.systime module, you open std/datetime/systime.d file - that's the reason of needlessly restricting code structure with modules as if one size fits all.

It is the same reasoning as with deep filesystem hierarchies and, well, any data hierarchies - once the element (module / file) count becomes bigger than ~dozen you only really notice things you know to look for. Contrary to that deeply nested categorized hierarchies are easy to casually to search through if you don't know exact module name - just iteratively pick whatever package fits the theme until you find what you want.

I remember coding a bit in C#/.NET platform ages ago - it was totally possible to find relevant modules without even looking in docs, just using auto-complete through suggested package names for import. It was really positive experience for a newbie I was. At the same time a lot of people have no idea how many cool things Phobos actually has.

On Sun, 24 Aug 2014 02:39:39 +0000
Dicebot via Digitalmars-d-announce
<digitalmars-d-announce@puremagic.com> wrote:

> It is the same reasoning as with deep filesystem hierarchies and, well, any data hierarchies
i already told that, but without any effects.

On Sunday, 24 August 2014 at 02:22:41 UTC, Dicebot wrote:
> Well difference is that "internal" substring in the fully qualified name that is much more likely to tell user he is better to not touch it. However, original Kagamin proposal of embedding it into module names themselves does address that if you are ok with resulting uglyness.

Both ways it's a convention, and I don't see, why such convention should exist in the first place, member protection has nothing to do with module name, which reflects module's functionality.

On Sunday, 24 August 2014 at 02:34:01 UTC, Dicebot wrote:
>> It can be a philosophical matter, but in my experience grouping by functionality is genuine, and access is an afterthought, so grouping by access doesn't work in the long term, as code naturally (spontaneously) groups by functionality.
>
> That does contradict your statement that any stuff used in parent packages must go to up the hierarchy which is exactly grouping by access :)

Access there means protection, usage reflects functionality.

> I can probably give a more practical example I have just recently encountered when re-designing one of our internal library packages. It was a "serialization" package and sub-packages defined different (de)serialization models, some of those defining special wrapper types for resulting data that enforce certain semantics specific to that serialization model via the type system. It is not surprising that most internal details of such wrappers were kept package protected as exposing those to user could have easily violated all assumptions (but was needed to implement (de)serialization efficiently).
>
> It has become more complicated when "meta-serializers" have been added to parent package - templated stuff that took any of sub-package implementations and added some functionality (like versioning support) on top. Being generic thing it reside in higher level "serialization" package but to be implemented efficiently it needs access to those internal wrapper fields. Moving wrapper modules to parent package is not an option here because those are closely related to specific serialization model. Exposing stuff as public is not an option because anyone not deeply familiar with package implementation can easily break all type system guarantees that way. Moving "meta-serializers" to sub-packages is quite a code duplication.
>
> Right now I keep that stuff public and say in docs "please don't use this" which is hardly a good solution.

1. Making wrapper protected will preclude writing new serializer.
2. Using wrapper methods can be meaningless without serializer.
3. Serializer may just not expose wrapper, then user will have no way to access it.
4. .net has quite a lot of things like http://msdn.microsoft.com/en-us/library/system.runtime.compilerservices.configuredtaskawaitable%28v=vs.110%29.aspx and nothing explodes even though .net programmers are believed to be really stupid and evil. It's a virtue of Stackoverflow Driven Development.

On Sunday, 24 August 2014 at 02:39:40 UTC, Dicebot wrote:
> On Saturday, 23 August 2014 at 09:00:30 UTC, Kagamin wrote:
>> What is difficult to find? With flat structure you have all files right before your eyes. If you need std.datetime.systime module, you open std/datetime/systime.d file - that's the reason of needlessly restricting code structure with modules as if one size fits all.
>
> It is the same reasoning as with deep filesystem hierarchies and, well, any data hierarchies - once the element (module / file) count becomes bigger than ~dozen you only really notice things you know to look for. Contrary to that deeply nested categorized hierarchies are easy to casually to search through if you don't know exact module name - just iteratively pick whatever package fits the theme until you find what you want.

I'm afraid, hierarchies make things harder to find. If you don't know what is where, flat organization will present you with everything readily available right before your eyes. With deep hierarchy you're left with abstract or poorly chosen categories at every hierarchy level, so effectively you have to walk the entire hierarchy, which is much more tedious than scroll a flat list of modules viewing ten modules per scroll. Badly named list of modules (like what we have now in phobos) scales well up to 100, well-named list is much more manageable: if you need xml, you already know it's near the end of the list - it's easy to find even among 1000 files - you don't ever need to scroll entire list. If it's not there, where do you go? There's no obvious answer. So even shallow hierarchy is more troublesome than a flat list of 1000 modules. I don't believe hierarchy will magically solve navigation problems just because it has folders.

> I remember coding a bit in C#/.NET platform ages ago - it was totally possible to find relevant modules without even looking in docs, just using auto-complete through suggested package names for import. It was really positive experience for a newbie I was. At the same time a lot of people have no idea how many cool things Phobos actually has.

And System.Xml is not System.Text.Xml, System.Net is not System.IO.Net - where is deep hierarchy? If you want System.Xml, you type `s.x` and it doesn't matter, how many namespaces are there, you're presented with System.Xml. If it's not there, where to find it, in System.Text, System.Formats, System.Parsers, System.Lang, System.Markup, System.Sgml?

On Sunday, 24 August 2014 at 14:15:25 UTC, Kagamin wrote:
> On Sunday, 24 August 2014 at 02:22:41 UTC, Dicebot wrote:
>> Well difference is that "internal" substring in the fully qualified name that is much more likely to tell user he is better to not touch it. However, original Kagamin proposal of embedding it into module names themselves does address that if you are ok with resulting uglyness.
>
> Both ways it's a convention, and I don't see, why such convention should exist in the first place, member protection has nothing to do with module name, which reflects module's functionality.

Because we have no other means of enforcing this protection - this is why my PR event exists.

> On Sunday, 24 August 2014 at 02:34:01 UTC, Dicebot wrote:
>>> It can be a philosophical matter, but in my experience grouping by functionality is genuine, and access is an afterthought, so grouping by access doesn't work in the long term, as code naturally (spontaneously) groups by functionality.
>>
>> That does contradict your statement that any stuff used in parent packages must go to up the hierarchy which is exactly grouping by access :)
>
> Access there means protection, usage reflects functionality.

And how that makes your statement less contradictive?

>> I can probably give a more practical example <...>
>> code duplication.
>
> 1. Making wrapper protected will preclude writing new serializer.
> 2. Using wrapper methods can be meaningless without serializer.
> 3. Serializer may just not expose wrapper, then user will have no way to access it.
> 4. .net has quite a lot of things like http://msdn.microsoft.com/en-us/library/system.runtime.compilerservices.configuredtaskawaitable%28v=vs.110%29.aspx and nothing explodes even though .net programmers are believed to be really stupid and evil. It's a virtue of Stackoverflow Driven Development.

I am afraid I don't really understand what your point here is - that my design is wrong and must be banned or that package concept is flawed and must be reworked? Neither is really an acceptable argument.

There is a simple and strict requirement - no one must be able to extend serializers but few developers that routinely maintain that package (and all sub-packages). Never in user code. This idea is not a subject to reconsideration - if language says such design is illegal I am going to call that language bullshit.

> I'm afraid, hierarchies make things harder to find. If you don't know what is where, flat organization will present you with everything readily available right before your eyes. With deep hierarchy you're left with abstract or poorly chosen categories at every hierarchy level, so effectively you have to walk the entire hierarchy, which is much more tedious than scroll a flat list of modules viewing ten modules per scroll. Badly named list of modules (like what we have now in phobos) scales well up to 100, well-named list is much more manageable: if you need xml, you already know it's near the end of the list - it's easy to find even among 1000 files - you don't ever need to scroll entire list. If it's not there, where do you go? There's no obvious answer. So even shallow hierarchy is more troublesome than a flat list of 1000 modules. I don't believe hierarchy will magically solve navigation problems just because it has folders.

You are speaking about _finding_ things. I am speaking about _discovering_ things. See the difference? There is no way you can quickly tell "aha, so this data formats are supported out of the box" when looking at plain flat list of 1000 files. With solid hierarchy it becomes trivial.

> And System.Xml is not System.Text.Xml, System.Net is not System.IO.Net - where is deep hierarchy? If you want System.Xml, you type `s.x` and it doesn't matter, how many namespaces are there, you're presented with System.Xml. If it's not there, where to find it, in System.Text, System.Formats, System.Parsers, System.Lang, System.Markup, System.Sgml?

You pick few that make most sense to you and ignore the rest. If routinely needed module is not exposed via obvious path it is an issue worth filing in the bug tracker. More rarely used / obscure modules can have more surprising paths. I don't see a problem here. Again, searching for a known package is never a problem both ways - you can just, well, run the search query. Discoverability of uknowns is the key.

On Tuesday, 19 August 2014 at 17:08:25 UTC, Walter Bright wrote:
> On 8/19/2014 7:01 AM, Dicebot wrote:
> > Walter, now that release is out can you please state your
> opinion about
> > https://github.com/D-Programming-Language/dmd/pull/3651 ? It
> is blocking Phobos
> > module split and decoupling.
>
> I keep thinking there's gotta be a way to do this without language changes.

In the meanwhile I have updated
https://github.com/D-Programming-Language/dmd/pull/3651 to
address latest Walter comments and be based on recent master

..and it has just been merged! ^_^

Thanks Walter!

On Tue, 26 Aug 2014 16:46:19 +0000
Dicebot via Digitalmars-d-announce
<digitalmars-d-announce@puremagic.com> wrote:

> ..and it has just been merged! ^_^
WOW! that's great!

> Thanks Walter!
second that!