I updated DIP22 - Private symbol visibility with my findings from implementing the 'Hide module members' pull request.

https://github.com/D-Programming-Language/dmd/pull/739

I don't have much time to cope with the topic but any feedback is welcome.

I don't have anything really substantial to say except that this looks good to me and i look forward to it being implemented/pulled.

On 12/20/2013 09:48 PM, Martin Nowak wrote:
> I updated DIP22 - Private symbol visibility with my findings from
> implementing the 'Hide module members' pull request.
>
> https://github.com/D-Programming-Language/dmd/pull/739
>
> I don't have much time to cope with the topic but any feedback is welcome.

Looks mostly good to me, but the following snippets that both seem reasonable unfortunately contradict each other.

"private is an encapsulation tool. If it is not intended to be used by "outsiders", it should not interfere with them at all."

"* The least protected symbol determines the visibility for an overload set.
    After overload resolution an access check will be performed.
    Thereby overload resolution is independent of look-up origin."

Adding a private symbol to an overload set can break 3rd party code with these rules. Since private symbols are usually excluded from di files I don't think this design is viable. I think it would be better to disallow overload sets with mixed protection.

On Saturday, 21 December 2013 at 17:41:32 UTC, Timon Gehr wrote:
> Adding a private symbol to an overload set can break 3rd party code with these rules.

Yes, this is understood, but it seems like a good tradeoff when compared to the alternative where overload resolution depends on look-up origin. The latter could easily break generic code and simple refactorings could introduce subtle bugs.

> Since private symbols are usually excluded from di files I don't think this design is viable.

This, I don't have a good answer for, except that we have to put the private overload in the .di file as well.

> I think it would be better to disallow overload sets with mixed protection.

Tried that, doesn't work. The most common counterexample is mixing private/public constructors.

On Saturday, 21 December 2013 at 19:29:28 UTC, Martin Nowak wrote:
> Yes, this is understood, but it seems like a good tradeoff when compared to the alternative where overload resolution depends on look-up origin. The latter could easily break generic code and simple refactorings could introduce subtle bugs.

Still don't understand how intended and expected behavior can be called "bug".

(I don't like updated DIP at all and will be against anything that puts private symbols into overload sets)

On 12/21/2013 08:29 PM, Martin Nowak wrote:
> On Saturday, 21 December 2013 at 17:41:32 UTC, Timon Gehr wrote:
>> Adding a private symbol to an overload set can break 3rd party code
>> with these rules.
>
> Yes, this is understood, but it seems like a good tradeoff when compared
> to the alternative where overload resolution depends on look-up origin.
> The latter could easily break generic code and simple refactorings could
> introduce subtle bugs.
> ...

I think both options are similarly undesirable.

>> Since private symbols are usually excluded from di files I don't think
>> this design is viable.
>
> This, I don't have a good answer for, except that we have to put the
> private overload in the .di file as well.
>
>> I think it would be better to disallow overload sets with mixed
>> protection.
>
> Tried that, doesn't work. The most common counterexample is mixing
> private/public constructors.

You have a point there. :o)

On 12/21/2013 09:18 PM, Timon Gehr wrote:
> On 12/21/2013 08:29 PM, Martin Nowak wrote:
>> On Saturday, 21 December 2013 at 17:41:32 UTC, Timon Gehr wrote:
>>> Adding a private symbol to an overload set can break 3rd party code
>>> with these rules.
>>
>> Yes, this is understood, but it seems like a good tradeoff when compared
>> to the alternative where overload resolution depends on look-up origin.
>> The latter could easily break generic code and simple refactorings could
>> introduce subtle bugs.
>> ...
>
> I think both options are similarly undesirable.

Mixed protection in overload sets are pretty rare.
So we just need to find an acceptable solution to deal
with them without breaking valid use-cases.
I can hardly come up with any use-case but the constructor example though.

On 12/21/2013 08:54 PM, Dicebot wrote:
> On Saturday, 21 December 2013 at 19:29:28 UTC, Martin Nowak wrote:
> Still don't understand how intended and expected behavior can be called
> "bug".
>
If I make a refactoring and move a function from module A to B, but the function calls an overload set in A it should not have a different overload resolution.
Similar case, if I pass an overload set via alias to a traits template in module B, the template should follow the same overload resolution.
It's very important that we preserve these properties.

> (I don't like updated DIP at all and will be against anything that puts
> private symbols into overload sets)

I don't like them either, but they have some use-cases.

On Sat, Dec 21, 2013 at 10:09:44PM +0100, Martin Nowak wrote:
> On 12/21/2013 09:18 PM, Timon Gehr wrote:
> >On 12/21/2013 08:29 PM, Martin Nowak wrote:
> >>On Saturday, 21 December 2013 at 17:41:32 UTC, Timon Gehr wrote:
> >>>Adding a private symbol to an overload set can break 3rd party code with these rules.
> >>
> >>Yes, this is understood, but it seems like a good tradeoff when
> >>compared to the alternative where overload resolution depends on
> >>look-up origin.  The latter could easily break generic code and
> >>simple refactorings could introduce subtle bugs.
> >>...
> >
> >I think both options are similarly undesirable.
> 
> Mixed protection in overload sets are pretty rare.
> So we just need to find an acceptable solution to deal
> with them without breaking valid use-cases.
> I can hardly come up with any use-case but the constructor example
> though.

I was pretty upset when I encountered the following situation:

	---std/regex.d---
	...
	private struct Stack { ... }
	...

	---mymodule.d---
	...
	/* public */ struct Stack { ... }
	...

	---main.d---
	import std.regex;
	import mymodule;

	Stack s; // compile error: 'Stack' is ambiguous

I find it unacceptable that introducing a *private* symbol to std.regex will break existing code (std.regex.Stack wasn't there in earlier versions of Phobos). Since std.regex.Stack is private, why would it even be in *any* overload set when compiling main.d??! This is leaky abstraction at its best: changing the implementation of a module without changing its API can randomly break user code due to newly-introduced private symbols clashing with user-defined symbols.


T

-- 
Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at it. -- Pete Bleackley

On 12/21/2013 10:14 PM, Martin Nowak wrote:
> On 12/21/2013 08:54 PM, Dicebot wrote:
>> On Saturday, 21 December 2013 at 19:29:28 UTC, Martin Nowak wrote:
>> Still don't understand how intended and expected behavior can be called
>> "bug".
>>
> If I make a refactoring and move a function from module A to B, but the
> function calls an overload set in A it should not have a different
> overload resolution.
> Similar case, if I pass an overload set via alias to a traits template
> in module B, the template should follow the same overload resolution.

Well, you pass an alias. Are you saying that for alias template arguments, the scope of the template declaration applies for accessibility checking? I.e. we cannot pass private symbols as alias arguments? I think this is an unreasonably high price to pay.

Note that if we actually allow this, having any kind of differing behaviour when accessing an overload set from different modules is a consistency issue, since the same template with an alias parameter might be instantiated from both modules with the same overload set.

> It's very important that we preserve these properties.
>
>> (I don't like updated DIP at all and will be against anything that puts
>> private symbols into overload sets)
>
> I don't like them either, but they have some use-cases.

Basically just the constructor thing though?
Note that constructors cannot be passed by alias due to a syntax restriction... :o)