On Wednesday, June 19, 2013 08:51:38 TommiT wrote:
> On Wednesday, 19 June 2013 at 06:47:05 UTC, Jonathan M Davis
> 
> wrote:
> > Then just put whatever is supposed to be shared between them in
> > the package
> > that they were both in originally. package stuff is not part of
> > the public API,
> > so when you're refactoring package code, you're refactoring
> > code that you own
> > entirely, so it shouldn't be an issue if you have to rearrange
> > it a bit,
> > especially since it's all localized to a single package by
> > definition. It's
> > only when something is part of a public API that refactoring
> > becomes a
> > problem.
> > 
> > - Jonathan M Davis
> 
> I can't put a member function into a different file if it's supposed to have access to private data of its enclosing class/struct.

Then you can put the class in a different file. But even if you can't, I don't think that it's worth it to complicate the package attribute any further. Even if we lose something in the process, we really don't lose much, and since currently, package only applies to modules directly in the same package and not nested packages, you'd still be gaining over what we currently have.

- Jonathan M Davis

On Wednesday, 19 June 2013 at 07:12:30 UTC, Jonathan M Davis wrote:
> On Wednesday, June 19, 2013 08:51:38 TommiT wrote:
>> I can't put a member function into a different file if it's
>> supposed to have access to private data of its enclosing
>> class/struct.
>
> Then you can put the class in a different file. But even if
> you can't, I don't think that it's worth it to complicate the
> package attribute any further.

Let's now assume that we change the language specification so that 'package' access specifier allows access also to modules in nested packages. I'll show how difficult it can be to refactor your code in order to break it up into a package:

The file structure:
-------------------
/my_lib/mod_x.d
/my_lib/mod_y.d


File: /my_lib/mod_x.d
----------------------
module my_lib.mod_x;

import my_lib.mod_y;

void fun()
{
    my_lib.mod_y.S1 s1;
    s1.detail();
}


File: /my_lib/mod_y.d
----------------------
module my_lib.mod_y;

struct S1
{
    public void api()
    {
        S2 s2;
        s2.detail();
    }

    package void detail() { }
}

struct S2
{
    public void api() { }

    package void detail() { }
}

################################################################
Now, I want to break up both mod_x.d and mod_y.d into packages. I'll start by doing it naively and end up with this:

The file structure:
-------------------
/my_lib/mod_x/submod.d
/my_lib/mod_x/package.d
/my_lib/mod_y/submod_s1.d
/my_lib/mod_y/submod_s2.d
/my_lib/mod_y/package.d


File: /my_lib/mod_x/submod.d
-----------------------------
module my_lib.mod_x.submod;

import my_lib.mod_y;

void fun()
{
    my_lib.mod_y.S1 s1;
    s1.detail();
}


File: /my_lib/mod_x/package.d
------------------------------
module my_lib.mod_x;

public import my_lib.mod_x.submod;


File: /my_lib/mod_y/submod_s1.d
--------------------------------
module my_lib.mod_y.submod_s1;

import my_lib.mod_y.submod_s2;

struct S1
{
    public void api()
    {
        S2 s2;
        s2.detail();
    }

    package void detail() { }
}


File: /my_lib/mod_y/submod_s2.d
--------------------------------
module my_lib.mod_y.submod_s2;

struct S2
{
    public void api() { }

    package void detail() { }
}


File: /my_lib/mod_y/package.d
------------------------------
module my_lib.mod_y;

public import my_lib.mod_y.submod_s1;
public import my_lib.mod_y.submod_s2;

################################################################
At this point I notice this doesn't work because my_lib.mod_x.submod.fun tries to call my_lib.mod_y.S1.detail which is marked package and now in different package. In order to fix this without changing the public API, I'll create a new file called shared_s1.d, move the struct S1 there, and public import shared_s1.d back into the S1's original submod_s1.d file:

The file structure:
-------------------
/my_lib/mod_x/submod.d
/my_lib/mod_x/package.d
/my_lib/mod_y/submod_s1.d
/my_lib/mod_y/submod_s2.d
/my_lib/mod_y/package.d
/my_lib/shared_s1.d


File: /my_lib/shared_s1.d
--------------------------
module my_lib.shared_s1;

import my_lib.mod_y.submod_s2;

struct S1
{
    public void api()
    {
        S2 s2;
        s2.detail();
    }

    package void detail() { }
}


File: /my_lib/mod_y/submod_s1.d
--------------------------------
module my_lib.mod_y.submod_s1;

public import my_lib.shared_s1;


(other files stay the same)
################################################################
Now my_lib.mod_x.submod.fun has access rights and is able to call my_lib.mod_y.S1.detail because it's now an alias to my_lib.shared_s1.S1.detail. But then we notice that it doesn't work, because by moving the definition of S1 one folder up, we made it so that my_lib.shared_s1.S1.api isn't able to call my_lib.mod_y.submod_s2.S2.detail. So, we must do the same trick for the struct S2 now. Notice how viral this effect is. We end up with:

The file structure:
-------------------
/my_lib/mod_x/submod.d
/my_lib/mod_x/package.d
/my_lib/mod_y/submod_s1.d
/my_lib/mod_y/submod_s2.d
/my_lib/mod_y/package.d
/my_lib/shared_s1.d
/my_lib/shared_s2.d


File: /my_lib/shared_s2.d
--------------------------
module my_lib.shared_s2;

struct S2
{
    public void api() { }

    package void detail() { }
}


File: /my_lib/mod_y/submod_s2.d
--------------------------------
module my_lib.mod_y.submod_s2;

public import my_lib.shared_s2;


################################################################
And now it all works. But what did we learn from all this:

Your suggestion makes breaking modules into packages easier to understand but harder to do.

My suggestion makes breaking modules into packages harder to understand but easier to do.


On Wednesday, 19 June 2013 at 07:12:30 UTC, Jonathan M Davis wrote:
> Even if we lose something in the process, we really don't lose
> much, and since currently, package only applies to modules
> directly in the same package and not nested packages, you'd
> still be gaining over what we currently have.

I don't want 'package' symbols to give away access to nested packages.

On Wednesday, 19 June 2013 at 09:55:32 UTC, TommiT wrote:
> Your suggestion makes breaking modules into packages easier to understand but harder to do.

...and not to mention, just plain ugly. I mean look at how those files that are part of a package end up being in different folders and you have those dummy (shared) files which the end user is not supposed to import.

On Wednesday, 19 June 2013 at 07:12:30 UTC, Jonathan M Davis wrote:
> [..] But even if you can't, I don't think that it's worth it to
> complicate the package attribute any further. [..]

I'd like to quote the TDPL foreword by Walter:
"To the best of my knowledge, D offers an unprecedentedly adroit integration of several powerful programming paradigms: imperative, object-oriented, functional, and meta.
At first blush, it would appear that such a language could not be simple. And indeed, D is not a simple language. But I’d argue that is the wrong way to view a language. A more useful view is, what do programming solutions in that language look like? Are D programs complicated and obtuse, or simple and elegant?"
  -Walter Bright

Thus, we shouldn't strive so much to make the language simple (by having the simplest possible definition for the 'package' keyword), but rather, we should try to make programs, which use modules broken into packages, look simple, elegant and easy to understand.

On Wednesday, June 19, 2013 13:13:54 TommiT wrote:
> On Wednesday, 19 June 2013 at 07:12:30 UTC, Jonathan M Davis
> 
> wrote:
> > [..] But even if you can't, I don't think that it's worth it to complicate the package attribute any further. [..]
> 
> I'd like to quote the TDPL foreword by Walter:
> "To the best of my knowledge, D offers an unprecedentedly adroit
> integration of several powerful programming paradigms:
> imperative, object-oriented, functional, and meta.
> At first blush, it would appear that such a language could not be
> simple. And indeed, D is not a simple language. But I’d argue
> that is the wrong way to view a language. A more useful view is,
> what do programming solutions in that language look like? Are D
> programs complicated and obtuse, or simple and elegant?"
> -Walter Bright
> 
> Thus, we shouldn't strive so much to make the language simple (by having the simplest possible definition for the 'package' keyword), but rather, we should try to make programs, which use modules broken into packages, look simple, elegant and easy to understand.

Sure, but there's a limit to how much complexity is reasonable, and every feature has to pull its own weight. And while I agree that there are cases where refactoring packages may cause complications with the package modifier specfically giving nested packages access to their parent packages' package stuff but not do anything like what you're suggesting, I don't agree that it's a big enough deal to merit complicating the package access level any further. If it's a problem, then maybe the code just plain needs to be redesigned rather than simply shuffled around. If we were really looking for full power with rearranging stuff like that, we'd have friend functions and classes, and D specifically elected _not_ to go for that level of complication. And overall, it's worked great for us thus far.

- Jonathan M Davis

On Wednesday, 19 June 2013 at 20:05:56 UTC, Jonathan M Davis
wrote:
> On Wednesday, June 19, 2013 13:13:54 TommiT wrote:
>> On Wednesday, 19 June 2013 at 07:12:30 UTC, Jonathan M Davis
>> 
>> wrote:
>> > [..] But even if you can't, I don't think that it's worth it to
>> > complicate the package attribute any further. [..]
>> 
>> I'd like to quote the TDPL foreword by Walter:
>> "To the best of my knowledge, D offers an unprecedentedly adroit
>> integration of several powerful programming paradigms:
>> imperative, object-oriented, functional, and meta.
>> At first blush, it would appear that such a language could not be
>> simple. And indeed, D is not a simple language. But I’d argue
>> that is the wrong way to view a language. A more useful view is,
>> what do programming solutions in that language look like? Are D
>> programs complicated and obtuse, or simple and elegant?"
>> -Walter Bright
>> 
>> Thus, we shouldn't strive so much to make the language simple (by
>> having the simplest possible definition for the 'package'
>> keyword), but rather, we should try to make programs, which use
>> modules broken into packages, look simple, elegant and easy to
>> understand.
>
> Sure, but there's a limit to how much complexity is reasonable, and every
> feature has to pull its own weight. And while I agree that there are cases
> where refactoring packages may cause complications with the package modifier
> specfically giving nested packages access to their parent packages' package
> stuff but not do anything like what you're suggesting, I don't agree that it's
> a big enough deal to merit complicating the package access level any further.
> If it's a problem, then maybe the code just plain needs to be redesigned
> rather than simply shuffled around. If we were really looking for full power
> with rearranging stuff like that, we'd have friend functions and classes, and D
> specifically elected _not_ to go for that level of complication. And overall,
> it's worked great for us thus far.
>
> - Jonathan M Davis

No, we are not hoping to rearrange stuff like that. We are _forced_
to rearrange stuff like that if we go with your suggestion. With
my suggestion, what I presented as the first attempt at breaking
modules into packages would just work. (in case you actually read my post)

And no, we don't want friend function, I never said that.

Also, I'm just curious why do you keep saying "we don't want to
complicate the package access specifier any further"? Because
isn't the current specification of the package access specifier
the simplest possible that it could ever be? "Everything under
the same folder has access to symbols labeled package". It takes
just 11 words to define it. You're talking about it like it's
already somehow complicated.

On Thursday, June 20, 2013 00:21:45 TommiT wrote:
> Also, I'm just curious why do you keep saying "we don't want to complicate the package access specifier any further"? Because isn't the current specification of the package access specifier the simplest possible that it could ever be? "Everything under the same folder has access to symbols labeled package". It takes just 11 words to define it. You're talking about it like it's already somehow complicated.

No, it's not complicated, but the lanugage as a whole is complicated, and any new feature that's added to it increases its complexity. As such, it needs to pull its weight, and I really don't believe that that's the case here. I just don't think that complicating the package access modifier any further is worth the gain. There is some gain, but I think that it's ultimately quite small, and I'd much prefer that access modifiers stay simple. Obviously, you're entitled to think that the extra complexity is worth it, but I don't agree.

- Jonathan M Davis

On Wednesday, 19 June 2013 at 22:40:47 UTC, Jonathan M Davis wrote:
> On Thursday, June 20, 2013 00:21:45 TommiT wrote:
>> Also, I'm just curious why do you keep saying "we don't want to
>> complicate the package access specifier any further"? Because
>> isn't the current specification of the package access specifier
>> the simplest possible that it could ever be? "Everything under
>> the same folder has access to symbols labeled package". It takes
>> just 11 words to define it. You're talking about it like it's
>> already somehow complicated.
>
> No, it's not complicated, but the lanugage as a whole is complicated, and any
> new feature that's added to it increases its complexity. As such, it needs to
> pull its weight, and I really don't believe that that's the case here. I just
> don't think that complicating the package access modifier any further is worth
> the gain. There is some gain, but I think that it's ultimately quite small,
> and I'd much prefer that access modifiers stay simple. Obviously, you're
> entitled to think that the extra complexity is worth it, but I don't agree.
>
> - Jonathan M Davis

You're willing to add extra complexity and inconvenience to programming in D just in order to keep the language specification simple. I don't think it's a good trade-off. Learning the complex details of language is a one-time cost that all programmers must pay when they start with the language. Whereas complexity and inconvenience in actually programming with the language is a running cost and may be a source of bugs as well. The running cost should clearly over-weight the one-time cost here.

On Thursday, June 20, 2013 05:39:30 TommiT wrote:
> On Wednesday, 19 June 2013 at 22:40:47 UTC, Jonathan M Davis
> 
> wrote:
> > On Thursday, June 20, 2013 00:21:45 TommiT wrote:
> >> Also, I'm just curious why do you keep saying "we don't want to
> >> complicate the package access specifier any further"? Because
> >> isn't the current specification of the package access specifier
> >> the simplest possible that it could ever be? "Everything under
> >> the same folder has access to symbols labeled package". It
> >> takes
> >> just 11 words to define it. You're talking about it like it's
> >> already somehow complicated.
> > 
> > No, it's not complicated, but the lanugage as a whole is
> > complicated, and any
> > new feature that's added to it increases its complexity. As
> > such, it needs to
> > pull its weight, and I really don't believe that that's the
> > case here. I just
> > don't think that complicating the package access modifier any
> > further is worth
> > the gain. There is some gain, but I think that it's ultimately
> > quite small,
> > and I'd much prefer that access modifiers stay simple.
> > Obviously, you're
> > entitled to think that the extra complexity is worth it, but I
> > don't agree.
> > 
> > - Jonathan M Davis
> 
> You're willing to add extra complexity and inconvenience to programming in D just in order to keep the language specification simple. I don't think it's a good trade-off. Learning the complex details of language is a one-time cost that all programmers must pay when they start with the language. Whereas complexity and inconvenience in actually programming with the language is a running cost and may be a source of bugs as well. The running cost should clearly over-weight the one-time cost here.

I don't think that this adds much complication to programs, because I don't think that the sorts of problems that you're worried about when refactoring code are likely to be common, and they can be gotten around by reworking how your code is designed. Worst case, you just make the functionality public but undocumented. And while that may not be ideal, I really don't think that it will happen enough to be worth the complication to the language required to avoid it.

And consider the fact that even with your suggested changes, there are going to be ways that you can't refactor your code and still keep it using the same API and/or the same access level. We can pretty much always find more ways to refactor code that won't work with the access specifiers unless you introduce something like friend which allows you to explicitly specify who can access who rather than using general groups like public and private. And you have to draw the line somewhere.

I think that making it so that the package access specifier lets nested packages access it in addition to the exact package is a good balance between simplicity and power.

However, if you want to try and convince Walter, feel free. But from what he's said in the past, I suspect that he leans more towards getting rid of package entirely than thinking that it's worth making it more complicated in the manner that you're looking for.

- Jonathan M Davis

This is how I see the logic of this new feature you're adding:

When you have a folder named "mod" which contains a file named package.d, the mod folder becomes conceptually a file with same name and the extension "d". I.e. the folder conceptually becomes a module, which you can then import to other modules. All the code contained within all the files contained within the mod folder becomes conceptually the content of this imaginary mod.d file. Therefore it is only logical that all the code within this imaginary mod.d file would have access to 'package'-labeled symbols that are in files which are located under the same folder under which this imaginary mod.d file is.