On 18/08/2018 1:29 AM, vit wrote:
> On Friday, 17 August 2018 at 13:02:20 UTC, Jonathan M Davis wrote:
>> On Friday, August 17, 2018 6:30:26 AM MDT Atila Neves via Digitalmars-d wrote:
>>> On Friday, 17 August 2018 at 11:37:54 UTC, rikki cattermole wrote:
>>> > On 17/08/2018 11:33 PM, Atila Neves wrote:
>>> >> . Good luck figuring out why your template functions aren't >> 
>>> @safe and coaxing the compiler to tell you why it's >> inferring the attributes it is. Bonus points if it's a >> Phobos function so you can't slap `@safe` on its definition.
>>> >
>>> > Sure you can. It's a template so it'll be initialized as its > 
>>> required.
>>> >
>>> > I've done that to fix a bug in druntime before and in my own > 
>>> libraries.
>>>
>>> It's not easy though. You have to either be building your own phobos or sudo editing files in /usr/include. You can, but it's a pain. Then there's finding out exactly where in a chain of 10 template functions that it became @system...
>>
>> The reality of the matter is that you tend to have to be very motivated to make your code @safe if you're doing much with templates (the same with other attributes such as pure). And sadly, it can quickly get to the point that it just feels like it's not worth it in spite of the fact that there is definitely value to be had in using them.
>>
>> I wonder what it would take to make it so that the compiler could actually tell you where in the function call chain it first becomes @system.
>>
>> - Jonathan M Davis
> 
> Does -dip1000 affect alias template parameters and closures?
> example:
> 
> import std.experimental.all;
> 
> 
> void main()@safe @nogc{
>      int i = 2;
> 
>      const x = iota(0, 10)
>          .filter!((x)scope => x == i)
>          .map!((x)scope => x * x)
>          .front;
> 
>      assert(x == 4);
> }
> 
> onlineapp.d(4): Error: function `D main` is @nogc yet allocates closures with the GC
> onlineapp.d(8):        onlineapp.main.__lambda1 closes over variable i at onlineapp.d(5)

No it shouldn't. You used i there which automatically requires a heap allocation for a delegate. Hence it uses GC.

On Fri, Aug 17, 2018 at 01:50:32AM -0600, Jonathan M Davis via Digitalmars-d wrote: [...]
> Honestly, the reality of the matter is that @safe is probably always going to be somewhat broken, because it's implemented via blacklisting rather than whitelisting. Instead of @safe only allowing stuff that's been proven to be @safe, it disallows stuff that a programmer decided was @system.

Sigh:

	https://issues.dlang.org/show_bug.cgi?id=12941

This was reported 4 years ago, but was unfortunately closed as invalid.

It will continue to be a problem as long as @safe is implemented via blacklisting, because every single time there's a new language feature, there's a chance that a loophole is introduced into @safe. And that's not counting the combinatorial explosion of existing language features that might lead to @safe loopholes, that we simply haven't thought of yet.  It's like allowing anyone to enter your house freely except those few people whom you've explicitly named. You can't possibly expect *not* to get robbed that way.


> The bug you ran into is a pretty glaring one that arguably should have been fixed ages ago, but given how hard it is to prove what is and isn't @safe, there are bound to be corner cases which have been missed. As we find them, they'll be fixed, but who knows how many are left or whether we'll ever actually get them all.
[...]

And that is exactly why the whole implementation of @safe is currently rather laughable. By blacklisting rather than whitelisting, we basically open the door wide open to loopholes -- anything that we haven't thought of yet could potentially be a @safe-breaking combination, and we wouldn't know until somebody discovers and reports it.

Sadly, it seems there is little interest in reimplementing @safe to use whitelisting instead of blacklisting.


T

-- 
A program should be written to model the concepts of the task it performs rather than the physical world or a process because this maximizes the potential for it to be applied to tasks that are conceptually similar and, more important, to tasks that have not yet been conceived. -- Michael B. Allen

On 8/17/18 7:26 AM, H. S. Teoh wrote:
> It will continue to be a problem as long as @safe is implemented via
> blacklisting, because every single time there's a new language feature,
> there's a chance that a loophole is introduced into @safe. And that's
> not counting the combinatorial explosion of existing language features
> that might lead to @safe loopholes, that we simply haven't thought of
> yet.  It's like allowing anyone to enter your house freely except those
> few people whom you've explicitly named. You can't possibly expect *not*
> to get robbed that way.

We should tag bugs like these as #safecracking

On Friday, 17 August 2018 at 14:26:07 UTC, H. S. Teoh wrote:
> On Fri, Aug 17, 2018 at 01:50:32AM -0600, Jonathan M Davis via Digitalmars-d wrote: [...]
>> Honestly, the reality of the matter is that @safe is probably always going to be somewhat broken, because it's implemented via blacklisting rather than whitelisting. Instead of @safe only allowing stuff that's been proven to be @safe, it disallows stuff that a programmer decided was @system.
>
> Sigh:
>
> 	https://issues.dlang.org/show_bug.cgi?id=12941
>
> This was reported 4 years ago, but was unfortunately closed as invalid.
>
> It will continue to be a problem as long as @safe is implemented via blacklisting, because every single time there's a new language feature, there's a chance that a loophole is introduced into @safe. And that's not counting the combinatorial explosion of existing language features that might lead to @safe loopholes, that we simply haven't thought of yet.  It's like allowing anyone to enter your house freely except those few people whom you've explicitly named. You can't possibly expect *not* to get robbed that way.

I knew there was something fundamentally wrong with @safe, but I could never put my finger on it.  Now that you and Jonathan mention this, it becomes clear.

This makes me exceptionally sad.  D is great in so many ways, but then this taints the pool.   I asked if D was ever going to be @safe by default at DConf (https://youtu.be/HvqsUO77FGI?t=13242), but it didn't elicit a very positive answer.

It seems D is backtracking in some ways (@nogc, -betterC), trying to evolve it into something it wasn't originally envisioned to be, and now we have another one to add to the list.

>> The bug you ran into is a pretty glaring one that arguably should have been fixed ages ago, but given how hard it is to prove what is and isn't @safe, there are bound to be corner cases which have been missed. As we find them, they'll be fixed, but who knows how many are left or whether we'll ever actually get them all.
> [...]
>
> And that is exactly why the whole implementation of @safe is currently rather laughable. By blacklisting rather than whitelisting, we basically open the door wide open to loopholes -- anything that we haven't thought of yet could potentially be a @safe-breaking combination, and we wouldn't know until somebody discovers and reports it.
>
> Sadly, it seems there is little interest in reimplementing @safe to use whitelisting instead of blacklisting.

I think there is probably some interest, though maybe not from the ones with the position or ability to make it happen.  A DIP might be the way forward, but it seems like quite a difficult task to turn it right-side-up at this point.  I actually started writing a DIP for this about a year ago, but I need to pick my battles.

Mike

On Friday, 17 August 2018 at 15:27:22 UTC, Mike Franklin wrote:
> I actually started writing a DIP for this about a year ago, but I need to pick my battles.
>
> Mike

Is it on github?

Alex

On Friday, 17 August 2018 at 14:26:07 UTC, H. S. Teoh wrote:
> [...]
>
> And that is exactly why the whole implementation of @safe is currently rather laughable. By blacklisting rather than whitelisting, we basically open the door wide open to loopholes -- anything that we haven't thought of yet could potentially be a @safe-breaking combination, and we wouldn't know until somebody discovers and reports it.
>
> Sadly, it seems there is little interest in reimplementing @safe to use whitelisting instead of blacklisting.
>
>
> T

Fundamentally, I see it as a good idea. Walter has talked about how important memory safety is for D. People thinking their @safe code is safe is a big problem when that turns out to not be the case. Imagine the black eye D would have if a company was hacked because of something like this?

IMO, the problem is that you can't just replace @safe as it is now. You could introduce something like @whitelist or @safewhitelist and begin implementing it, but it would probably be some time before it could replace @safe. Like when @whitelist is only breaking unsafe code.

On 8/17/18 1:26 PM, jmh530 wrote:
> On Friday, 17 August 2018 at 14:26:07 UTC, H. S. Teoh wrote:
>> [...]
>>
>> And that is exactly why the whole implementation of @safe is currently rather laughable. By blacklisting rather than whitelisting, we basically open the door wide open to loopholes -- anything that we haven't thought of yet could potentially be a @safe-breaking combination, and we wouldn't know until somebody discovers and reports it.
>>
>> Sadly, it seems there is little interest in reimplementing @safe to use whitelisting instead of blacklisting.
>>
>>
>> T
> 
> Fundamentally, I see it as a good idea. Walter has talked about how important memory safety is for D. People thinking their @safe code is safe is a big problem when that turns out to not be the case. Imagine the black eye D would have if a company was hacked because of something like this?

This will always be a possibility thanks to @trusted.

> IMO, the problem is that you can't just replace @safe as it is now. You could introduce something like @whitelist or @safewhitelist and begin implementing it, but it would probably be some time before it could replace @safe. Like when @whitelist is only breaking unsafe code.

I have to say, I don't see how this all helps.

In theory, black-listing and white-listing will get you to the same position. Mechanisms to get or use pointers aren't really being added to the language any more, so the set of "things" to "list" either black or white is finite.

In this thread, we are talking about something that should have been black-listed LONG ago, but was not because it was "too useful" (i.e. would break too much code). If @safe code was white-listed, nobody would use it until it was finished, so it would be theoretical anyway.

Nobody wants a feature that is @safe, but not useful.

However, a bigger problem is that we have a bug that is "fixed" (slicing static arrays) but only if you use a feature that doesn't work correctly (dip1000). Why? I think the bug should be reopened until dip1000 is the default, or it simply gets fixed (i.e. without requiring dip1000).

-Steve

On Fri, Aug 17, 2018 at 05:26:48PM +0000, jmh530 via Digitalmars-d wrote:
> On Friday, 17 August 2018 at 14:26:07 UTC, H. S. Teoh wrote:
> > [...]
> > And that is exactly why the whole implementation of @safe is
> > currently rather laughable. By blacklisting rather than
> > whitelisting, we basically open the door wide open to loopholes --
> > anything that we haven't thought of yet could potentially be a
> > @safe-breaking combination, and we wouldn't know until somebody
> > discovers and reports it.
[...]
> Fundamentally, I see it as a good idea. Walter has talked about how important memory safety is for D. People thinking their @safe code is safe is a big problem when that turns out to not be the case. Imagine the black eye D would have if a company was hacked because of something like this?

Indeed.


> IMO, the problem is that you can't just replace @safe as it is now. You could introduce something like @whitelist or @safewhitelist and begin implementing it, but it would probably be some time before it could replace @safe. Like when @whitelist is only breaking unsafe code.

The way I envision it is that the implementation would begin as a separate topic branch in the dmd repo, and gradually brought up to the point where it begins passing the testsuite.  The good thing is that nowadays with our CI system put in place, any breakages caused in major D projects like vibe.d would be detected by the autotester, so we can immediately fill in any obvious missing whitelist items.

Then we'd merge it into master but controlled by a command-line switch, say -dipxxxx if we put it under a DIP, and put out for 1 or 2 releases. People who depend on @safe can then opt in to try it out, and report any bugs (missing whitelist items) that they find.  We fix those bugs until no more bugs are reported.  Then we make -dipxxxx the default, perhaps with a reverse switch for reverting to the old @safe implementation just in case some projects out there depend on it but haven't gotten around to reporting bugs in the new implementation or whatever.  After another release or so, we finally remove the switch and delete the old implementation.

Then going forward, we will have eliminated @safe loopholes pretty much completely (the only exception being codegen bugs), and the only likely bugs are missing items in the whitelist.  Which should be much more manageable and much less disastrous -- the worst that can happen is that somebody tried to do something legal and got rejected by a big fat compiler error.  As opposed to today's situation, where somebody tries something illegal and the compiler fails to detect it, and nobody even know there's a problem until it becomes an exploitable bug that causes a security problem.  Given today's security climate, this would be utterly disastrous for D's reputation.


T

-- 
It is of the new things that men tire --- of fashions and proposals and improvements and change. It is the old things that startle and intoxicate. It is the old things that are young. -- G.K. Chesterton

On Friday, 17 August 2018 at 17:59:45 UTC, H. S. Teoh wrote:

> As opposed to today's situation, where somebody tries something illegal and the compiler fails to detect it, and nobody even know there's a problem until it becomes an exploitable bug that causes a security problem.  Given today's security climate, this would be utterly disastrous for D's reputation.

This is a good example of D needing to evolve or fizzle out. I don't see evidence that the community has yet figured out how to evolve the language. If it had, these problems would not be around for so many years.

On Friday, 17 August 2018 at 16:00:26 UTC, 12345swordy wrote:
> On Friday, 17 August 2018 at 15:27:22 UTC, Mike Franklin wrote:
>> I actually started writing a DIP for this about a year ago, but I need to pick my battles.
>>
>> Mike
>
> Is it on github?
>
> Alex

No,but here are some notes I found in my files.

@safe by default
Before Transition

    Write a program that parses D code and explicitly adds @system if no function is not decorated with any safety attribute
    Add a transitional -explicit-safety compiler switch that emits a warning if a safety attribute is not explicitly specified.
    Update all dlang repositories and maybe others with explicit attributes using aforementioned program.
        Turn on -explicit-safety for all CIs so any new PRs so explicit safety is added to all new functions.
    Add transitional @system-by-default compiler switch for backward compatibility
        This will be especially useful to keep our CIs running with legacy code
        This will also be useful for programs with dub dependencies that need to continue running
    Add -@system-main compiler switch for backward compatibility
    Add -@safe-main and -@safe-by-default compiler switch for forward-looking D projects
    Update all of D's tutorials, examples, etc… with explicit safety attributes.

Transition

    Add changelog entry warning that @safe-by-default is coming and how they can use the aforementioned tools eliminate disruption
        Run this change log for 1 year (4 releases)?

After Transition

    Remove @safe attributes in D's example code.
    Remove @safe attributes from phobos/druntime/dmd etc..
    Turn off -explicit-safety for those libraries that have updated their code.
    Deprecate @safe-main and @safe-by-default compiler switch through the normal deprecation process.
    -explicit-safety, @system-main and @system-by-default remain for an indeterminate amount of time. They should eventually be deprecated, but only after they have outlived their usefulness.

Difficulties

    The difficulty will be with printed material (books, etc…) that will become out-of-date. Recommendation is to add information to their published errata, or create a new edition.

Mike