March 16, 2017
On Thursday, 16 March 2017 at 17:47:34 UTC, H. S. Teoh wrote:
> Actually, the bug still exists even if you explicitly slice it:
>
> 	string x = func()[]; // still compiles, but shouldn't

I don't call that a bug, once it is explicitly done it means the programmer realized something is up and decided they are OK with it. Perhaps you want to pass it to a function that you know isn't going to hold the reference beyond the calling scope.

> For some reason, slicing a static array return value is somehow OK, while slicing a local variable is rejected.  Seems like the compiler is missing escaping ref checks for return values?

It's the uniqueness thing, see my last email (I probably was typing it at the same time you were typing this...)

This isn't an escape per se, `string x` is still a local variable.

immutable(char)[32] buffer;
string s = buffer[0 .. 16]; // sane and really useful optimization... just be careful not to escape it

Walter wants to expand the escape check so it automatically issues an error if you aren't careful enough, but the status quo is still usable - such code is not necessarily wrong, banning entirely it is a step backward, and programmers coming up the C tradition are used to watching lifetimes like that.
March 16, 2017
On Thu, Mar 16, 2017 at 06:09:52PM +0000, Adam D. Ruppe via Digitalmars-d-learn wrote:
> On Thursday, 16 March 2017 at 17:47:34 UTC, H. S. Teoh wrote:
> > Actually, the bug still exists even if you explicitly slice it:
> > 
> > 	string x = func()[]; // still compiles, but shouldn't
> 
> I don't call that a bug, once it is explicitly done it means the programmer realized something is up and decided they are OK with it. Perhaps you want to pass it to a function that you know isn't going to hold the reference beyond the calling scope.
> 
> > For some reason, slicing a static array return value is somehow OK, while slicing a local variable is rejected.  Seems like the compiler is missing escaping ref checks for return values?
> 
> It's the uniqueness thing, see my last email (I probably was typing it at the same time you were typing this...)
> 
> This isn't an escape per se, `string x` is still a local variable.
> 
> immutable(char)[32] buffer;
> string s = buffer[0 .. 16]; // sane and really useful optimization... just
> be careful not to escape it
> 
> Walter wants to expand the escape check so it automatically issues an error if you aren't careful enough, but the status quo is still usable - such code is not necessarily wrong, banning entirely it is a step backward, and programmers coming up the C tradition are used to watching lifetimes like that.

Ah, you're right.  And as somebody indicated in the bug comments, compiling with -dip1000 correctly rejects the `return s;` line as trying to return a reference to something that's going out of scope.

But in that case, wouldn't that mean implicit slicing isn't to blame here? (Not that I'm arguing for implicit slicing -- I think it needs to go, too, having been bitten by it before -- but this particular case wouldn't constitute as evidence against it.)


T

-- 
Meat: euphemism for dead animal. -- Flora
March 16, 2017
On Thursday, 16 March 2017 at 18:07:09 UTC, Petar Kirov [ZombineDev] wrote:
> On Thursday, 16 March 2017 at 17:51:41 UTC, Adam D. Ruppe wrote:
>> On Thursday, 16 March 2017 at 17:40:51 UTC, H. S. Teoh wrote:
>>> Seems like the real cause of bug 17261 is implicit slicing.
>>
>> Yes, sorry, it took me 10 mins to type it up (I like to double check the current behavior before posting) but I think we both see it the same way now.
>
> Guys, seriously this bug has been fixed for quite some time:
> https://issues.dlang.org/show_bug.cgi?id=8838
> https://issues.dlang.org/show_bug.cgi?id=12625
> https://github.com/dlang/dmd/pull/5972
>
> Why don't you use -dip1000???
>
> /home/zombinedev/dlang/dmd-2.073.2/linux/bin64/../../src/phobos/std/stdio.d(3439): Error: template std.stdio.File.LockingTextWriter.put cannot deduce function from argument types !()(string), candidates are:
> /home/zombinedev/dlang/dmd-2.073.2/linux/bin64/../../src/phobos/std/stdio.d(2643):        std.stdio.File.LockingTextWriter.put(A)(A writeme) if (is(ElementType!A : const(dchar)) && isInputRange!A && !isInfinite!A)
> /home/zombinedev/dlang/dmd-2.073.2/linux/bin64/../../src/phobos/std/stdio.d(2672):        std.stdio.File.LockingTextWriter.put(C)(C c) if (is(C : const(dchar)))
> scope_test.d(5): Error: template instance std.stdio.writeln!string error instantiating
> scope_test.d(11): Error: scope variable ans may not be returned
>
> The only issue is that not enough people (except Walter & Andrei) are interested in pushing -dip1000 as the default. Druntime already compiles with -dip1000, so Phobos is the next frontier.

See also https://github.com/dlang/dmd/pulls?utf8=%E2%9C%93&q=is%3Apr%20is%3Aclosed%20scope%20author%3AWalterBright%20
March 16, 2017
On Thursday, 16 March 2017 at 18:10:43 UTC, H. S. Teoh wrote:
> But in that case, wouldn't that mean implicit slicing isn't to blame here? (Not that I'm arguing for implicit slicing -- I think it needs to go, too, having been bitten by it before -- but this particular case wouldn't constitute as evidence against it.)

The implicit slicing is still bug prone, just dip1000 catches it before it snowballs. That's a good thing, dip1000 puts an additional wall between the termites and the wood, but the termites do remain.
March 16, 2017
On Thursday, 16 March 2017 at 18:07:09 UTC, Petar Kirov [ZombineDev] wrote:
> Why don't you use -dip1000???

Because it isn't the default. But even if it was, people would ask "why is this giving me an error?" and the same explanation would need to be given. Certainly, a compile error is better than runtime corruption, but the underlying issue ought to be fixed anyway.

Phobos could have been written to avoid this problem too, there's a few solutions that work, but right now, using the standard library in a way people expect to work will be silently disastrous.

It is a good study of 3 language features, two of which I think are somewhat brilliant... and the third which is a big pain in a lot of ways for no real benefit. So I want to kill that design flaw.

March 16, 2017
On Thu, Mar 16, 2017 at 06:41:40PM +0000, Adam D. Ruppe via Digitalmars-d-learn wrote:
> On Thursday, 16 March 2017 at 18:10:43 UTC, H. S. Teoh wrote:
> > But in that case, wouldn't that mean implicit slicing isn't to blame here? (Not that I'm arguing for implicit slicing -- I think it needs to go, too, having been bitten by it before -- but this particular case wouldn't constitute as evidence against it.)
> 
> The implicit slicing is still bug prone, just dip1000 catches it before it snowballs. That's a good thing, dip1000 puts an additional wall between the termites and the wood, but the termites do remain.

Actually, https://issues.dlang.org/show_bug.cgi?id=12625 shows that implicit slicing is still a problem:

	char[16] func() { ... }
	void gunk() {
		string s = func(); // implicit slice

		... // do some stuff that uses the stack

		// since func's return value is a temporary, it has gone
		// out of scope here, and there is no guarantee it
		// hasn't already been overwritten by this point, right
		// inside gunk's body!  So s may already have corrupted
		// data.
	}

The problem is that func's return value is an rvalue, so slicing it means we're already escaping a reference to data that's going out of scope (by the end of the expression). It's no different from trying to take the address of an rvalue.

Seems this isn't caught by -dip1000, though from what I understand of DIP 1000, this *should* have been rejected.


T

-- 
People who are more than casually interested in computers should have at least some idea of what the underlying hardware is like. Otherwise the programs they write will be pretty weird. -- D. Knuth
March 16, 2017
On Thursday, 16 March 2017 at 19:00:08 UTC, H. S. Teoh wrote:
> Actually, https://issues.dlang.org/show_bug.cgi?id=12625 shows that implicit slicing is still a problem:

Oh yikes, I figured it would still be copied onto the local stack even though it is an rvalue.

But you know, the fact that so many of us are still surprised by just what is going on here tells me it is iffy anyway. Complicated rules can be fine when they bring compelling benefits, but this has a lot of cost and no serious benefit - just slice explicitly when you want to!

> Seems this isn't caught by -dip1000, though from what I understand of DIP 1000, this *should* have been rejected.

yea perhaps there is a bug there too.
March 16, 2017
On Thu, Mar 16, 2017 at 07:17:41PM +0000, Adam D. Ruppe via Digitalmars-d-learn wrote:
> On Thursday, 16 March 2017 at 19:00:08 UTC, H. S. Teoh wrote:
> > Actually, https://issues.dlang.org/show_bug.cgi?id=12625 shows that implicit slicing is still a problem:
> 
> Oh yikes, I figured it would still be copied onto the local stack even though it is an rvalue.
[...]

Well, yes, it would be copied to the local stack once func() returns, but the problem is, for how long.  It may be that the current implementation in dmd keeps that stack location allocated until the function returns, but it would be equally valid to reuse that stack location after the expression is over, because by definition, rvalues in the expression would have gone out of scope.


T

-- 
Computers are like a jungle: they have monitor lizards, rams, mice, c-moss, binary trees... and bugs.
March 16, 2017
On Thursday, 16 March 2017 at 18:51:45 UTC, Adam D. Ruppe wrote:
> Phobos could have been written to avoid this problem too, there's a few solutions that work, but right now, using the standard library in a way people expect to work will be silently disastrous.

Yes, and as an outsider I find this, well, disconcerting. No complaint from the compiler about assignment to string of the result of a library function call should have produced a string with the obvious semantics.

Having read this thread I have formed a conclusion.

Implicitly slicing rvalue arrays is too much like implicitly taking the address of an rvalue.

There's an explicit postfix operator [] to do that, and if there was no implicit slicing, I'd at least know where slicing is occurring and I wouldn't use the slice of a temporary beyond its lifetime.

Now a function with a slice parameter could not be called with an rvalue array parameter without putting an explicit slice operator in at the point of call.

But writing a function like that is just a way to regard rvalue arrays of different sizes based upon the same type as being the same type, when they are not. They are distinct types. And so a template could take care of that minor syntactic problem if so desired, with one instantiation for each rvalue array type (i.e. size), with a ref parameter to avoid copying.

I see every reason to remove implicit slicing of rvalue arrays.

Trying to keep it available sometimes is a complex endeavor, and the rules will be lengthy, and consequently have more complications to explain to people joining use of D, and for what? There's almost nothing to gain. This would be a mistake. D is already very large.

If I didn't know what my general confidence level in D was for other reasons, this incident could well have driven me away. The standard library compiled completely unexpectedly insane and unsafe semantics when I just called a simple-looking function. This sort of thing is undoubtedly bringing D into disrepute with some people here and there, people just trying it out to solve a problem.



March 16, 2017
On Thursday, March 16, 2017 20:42:21 Carl Sturtivant via Digitalmars-d-learn wrote:
> Implicitly slicing rvalue arrays is too much like implicitly taking the address of an rvalue.

Well, that's just it. That's _exactly_ what's happening. It's just that it ends up in a struct with a length member along with it. You have something like

struct Array(T)
{
    size_t length;
    T* ptr;
}

instead of

T* ptr;

In both cases, ptr refers to the same address, and each case is exactly as @safe as the other - as in, not at all.

Unfortunately, for some reason, slicing static arrays has historically been considered @safe, whereas taking an address of a local variable is considered @systeme even though they're doing _exactly_ the same thing except that the slicing ends up with a struct with length instead of just with a pointer. Fortunately, the @safety improvements that Walter has been working on should finally fix that.

> I see every reason to remove implicit slicing of rvalue arrays.

Honestly, I think that it was a big mistake to have implicit slicing of static arrays in the language at all. Unfortunately, last time I tried to convince Walter of that, he seemed to think that the @safety improvements to the compiler were going to fix the problem, and they will help, but I'm of the opinion that slicing of static arrays should always be explicit. It's too easy to miss what's going on otherwise, even if it isn't an @safety problem.

In any case, the @safety fixes that are underway should eventually at least flag this as @system if not result in it being outright illegal. And I think that there's a good case to make any variant of this issue be illegal if it's guaranteed that you're going to end up with a pointer to invalid memory.

- Jonathan M Davis