On 24 September 2014 04:18, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 9/23/2014 4:21 AM, Manu via Digitalmars-d wrote:
>>
>> On 23 September 2014 19:48, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>> Manu, once again your posts have the message embedded twice in them,
>>> making
>>> for very large posts. What's happening is the posts have the text in
>>> plain
>>> text, then the text again in HTML.
>>>
>>> Please configure your news editor to only produce the plain text
>>> messages.
>>> The HTML versions are ignored and uselessly consume bandwidth and disk
>>> space.
>>
>>
>> I use Gmail. It was configured... but it seems that it reverts to html mode whenever I attach an image in any email I write, and then it remembers the setting :/
>>
>
> Well, that message was right!

I changed the setting back ;)

On 9/23/2014 4:19 AM, Manu via Digitalmars-d wrote:
>> BTW, ref (as you know) is part of the type in C++. However, I can vouch for
>> it being a special case everywhere in C++, and is a horrifying quagmire of
>> strange edge cases. That's why it's not part of the type in D.
>
> I've never had any problems with ref in C++. D presents the horrible quagmire
> of edge cases in my experience, some of which I've presented, but I've had
> many more issues in the past. Can you give me some examples of the problems
> in C++ you set out to avoid?

Q: Given a T&, and type deduction, when do you get the T and when do you get the T& ?

A: It's different for every situation. Nobody can remember or enumerate the cases. Maybe Scott Meyers.

On 24 September 2014 17:50, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 9/23/2014 4:19 AM, Manu via Digitalmars-d wrote:
>>>
>>> BTW, ref (as you know) is part of the type in C++. However, I can vouch
>>> for
>>> it being a special case everywhere in C++, and is a horrifying quagmire
>>> of
>>> strange edge cases. That's why it's not part of the type in D.
>>
>>
>> I've never had any problems with ref in C++. D presents the horrible
>> quagmire
>> of edge cases in my experience, some of which I've presented, but I've had
>> many more issues in the past. Can you give me some examples of the
>> problems
>> in C++ you set out to avoid?
>
>
> Q: Given a T&, and type deduction, when do you get the T and when do you get the T& ?
>
> A: It's different for every situation. Nobody can remember or enumerate the cases. Maybe Scott Meyers.

Scott Myers presented on this in extraordinary detail. By my recollection (without referring to his talk), it was different in *one* case, not every case. It's a binary situation, so it can't possibly be different in 'every case', there are only 2 cases, and one of them is the common case.

I suggest if the idea were explored in D, we would see if it works where type deduction always gives what you expect (ie, gives ref(T) for ref(T)). Unlike C++, we have an extensive suite of tools like PointerTarget!T, Unqual!T, isPointer!T and friends; we can use those tools explicitly in the places where we want T from deduction yielding ref(T). I suspect we can avoid the C++ deduction hack given the presence of these tools.

And if it proves that the C++ edge case is necessary (I don't think it will), then so be it. I've never had problems with it, and always found it intuitive in 20 years, whereas D's current setup is immeasurably more complex than C++'s edge case, and causes me the greatest source of pain in the language.

On Saturday, 13 September 2014 at 01:49:05 UTC, Manu via Digitalmars-d wrote:
> I'm not convinced this is a good change.
> It sounds like you're just trading one problem with another more sinister
> problem...
>

Ok, I thought about it some more. I'm still not convinced completely, but I'm warming up to the idea of making scope a type modifier. However, Ivan's objections need to be addressed.

Maybe let's start with a list of problems of the type modifier way. So far:

* What is ElementType!(ByLineImpl!(char, "\n")) in the example from the wiki page [1]?
* Should Unqual!T strip `scope`?

Anything else? How can we solve these problems?

Another argument against storage class is a syntactical ambiguity in conncection with methods:

    struct S {
        scope!this int* foo() scope;
    }

It's ambiguous whether any given scope keyword applies to the return value or `this`. One could argue though that this is a consequence of the general function attribute problem and should preferably be fixed there. (`ref` doesn't have this problem because it cannot apply to `this`.)

[1] http://wiki.dlang.org/User:Schuetzm/scope#scope.21.28const_....29

29.09.2014 18:17, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" пишет:
> * What is ElementType!(ByLineImpl!(char, "\n")) in the example from the
> wiki page [1]?

OK, I think I have an idea, but it's not overly elegant.

First of all, I would like to note that the same issue exists with, for example, findSubstring function (from "scope with owners" section), which does not have a definite return type (so it's unclear what ReturnType!findSubstring should be).

Now, an idea that I have is that bare scope should be just a syntactic sugar for self-owned scope; i.e., `scope(int*) x;` would be just a short way of writing `scope!x(int*) x;`. This would imply that every scoped variable has its own unique type, which is probably not that terrible, considering that they can't be assigned freely to each other either way. This is also somewhat more natural, because it means that there is no such thing as just "scoped" variable, it is always connected to a particular lifetime.

Then, the following:
---
	string findSubstring(scope(string) haystack,
		             scope(string) needle)
---
would be equivalent to
---
	string findSubstring(scope!haystack(string) haystack,
		             scope!needle(string) needle)
---
so each argument is self-owned and all ownership information is lost (but function is still callable with scoped arguments, because scope narrowing is allowed).

To preserve ownership information, which is encoded in an arguments' types, template is needed:
---
	scope!haystackOwner(string)
	findSubstring(alias haystackOwner)
		            (scope!haystackOwner(string) haystack,
		             scope(string) needle)
---
So, findSubstring *still* doesn't have a definite return type, but now it's because it is a function template, not a function.

As for passing unscoped strings to findSubstring, I see two alternatives:
1) Declare that all unscoped references are implicitly convertible to scope!GC or something like that (and this conversion is used in such cases). This one is probably better.
2) Require a separate overload for an unscoped string.

Now, to address ElementType problem, I would like to reconsider `this` lifetime. Since
---
	struct S {
		void f() {}
	}
---
is more or less equivalent to
---
	// not a valid D code
	struct S {}
	void f(ref S this) {}
---
(not strictly equivalent, of course, but the general idea is like that),
`this` inside a method body would behave like a ref parameter and have approximately the same lifetime as normal parameters.

Then this code:
---
	@property scope!(const this)(Char[]) front() const
---
would become illegal, since the return value is declared as having function-local scope. However, this code:
---
	@property scope!(const this)(Char[])
		front(alias thisOwner)() const scope!thisOwner
---
would work just fine, because it explicitly propagates current object's scope (the return type seems the same, but `this` itself now has a different type). The downside is that `front` is now callable only for scoped variables (it seems inappropriate to convert structs to scope!GC).

Now, ByLineImpl!(char, "\n") wouldn't be an input range, because front would not be defined in an unscoped case, and `scope ByLineImpl!(char, "\n")` is not a type (because bare scope would be purely a syntactic sugar in declarations), so not a range.

However, with this declaration:
---
	scope(ByLineImpl!(char, "\n")) x = ...;
---
typeof(x) would be scope!x(ByLineImpl!(char, "\n")), and ElementType!(typeof(x)) would be scope!(const x)(char[]).

As I have said in the beginning, not too elegant, but I think it may work. As a bonus, this is a little bit more straightforward and requires less special-casing from the compiler side: has only one scope type instead of two and no magic tricks with scoped return values. The only problems that I can see right away are that the code now is a little verbose, and that 'front' is restricted to scoped variables in a new version.

Any thoughts?

On Friday, 3 October 2014 at 19:08:10 UTC, Ivan Timokhin wrote:
> 29.09.2014 18:17, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" пишет:
>> * What is ElementType!(ByLineImpl!(char, "\n")) in the example from the
>> wiki page [1]?
>
> OK, I think I have an idea, but it's not overly elegant.
>
> First of all, I would like to note that the same issue exists with, for example, findSubstring function (from "scope with owners" section), which does not have a definite return type (so it's unclear what ReturnType!findSubstring should be).
>

Indeed. It's everywhere that an owner depends on another function argument (including `this`), but AFAICS nowhere else.

> Now, an idea that I have is that bare scope should be just a syntactic sugar for self-owned scope; i.e., `scope(int*) x;` would be just a short way of writing `scope!x(int*) x;`. This would imply that every scoped variable has its own unique type, which is probably not that terrible, considering that they can't be assigned freely to each other either way. This is also somewhat more natural, because it means that there is no such thing as just "scoped" variable, it is always connected to a particular lifetime.

I've already suggested this as an implementation detail.

>
> Then, the following:
> ---
> 	string findSubstring(scope(string) haystack,
> 		             scope(string) needle)
> ---
> would be equivalent to
> ---
> 	string findSubstring(scope!haystack(string) haystack,
> 		             scope!needle(string) needle)
> ---
> so each argument is self-owned and all ownership information is lost (but function is still callable with scoped arguments, because scope narrowing is allowed).
>
> To preserve ownership information, which is encoded in an arguments' types, template is needed:
> ---
> 	scope!haystackOwner(string)
> 	findSubstring(alias haystackOwner)
> 		            (scope!haystackOwner(string) haystack,
> 		             scope(string) needle)
> ---
> So, findSubstring *still* doesn't have a definite return type, but now it's because it is a function template, not a function.

This of course has the unfortunate side effect of incredible template bloat: For any distinct passed argument (which in practice means for almost every call), we'd get a new template instance. IMO this is not acceptable.

>
> As for passing unscoped strings to findSubstring, I see two alternatives:
> 1) Declare that all unscoped references are implicitly convertible to scope!GC or something like that (and this conversion is used in such cases). This one is probably better.

OTOH it would preclude automatic demoting of GC allocations to stack allocations. And it would marry us to the GC is "default" allocation strategy, which we might want to move away from (probably in favor of generic allocators changeable at any point in time).

We also need to allow borrowing of temporaries if we want to have rvalue references. And this runs into problems if we are dealing with scoped non-references, for which I'm seeing more use cases now than just file descriptors (in particular reference counting). In short, I'd like to avoid it.

> 2) Require a separate overload for an unscoped string.

Ugly, of course, and I believe it's not necessary.

>
> Now, to address ElementType problem, I would like to reconsider `this` lifetime. Since
> ---
> 	struct S {
> 		void f() {}
> 	}
> ---
> is more or less equivalent to
> ---
> 	// not a valid D code
> 	struct S {}
> 	void f(ref S this) {}
> ---
> (not strictly equivalent, of course, but the general idea is like that),
> `this` inside a method body would behave like a ref parameter and have approximately the same lifetime as normal parameters.
>
> Then this code:
> ---
> 	@property scope!(const this)(Char[]) front() const
> ---
> would become illegal, since the return value is declared as having function-local scope. However, this code:
> ---
> 	@property scope!(const this)(Char[])
> 		front(alias thisOwner)() const scope!thisOwner
> ---
> would work just fine, because it explicitly propagates current object's scope (the return type seems the same, but `this` itself now has a different type). The downside is that `front` is now callable only for scoped variables (it seems inappropriate to convert structs to scope!GC).

And of course, again, the template bloat :-( This time, a new instance of `front` for every instance of the range.

>
> Now, ByLineImpl!(char, "\n") wouldn't be an input range, because front would not be defined in an unscoped case, and `scope ByLineImpl!(char, "\n")` is not a type (because bare scope would be purely a syntactic sugar in declarations), so not a range.
>
> However, with this declaration:
> ---
> 	scope(ByLineImpl!(char, "\n")) x = ...;
> ---
> typeof(x) would be scope!x(ByLineImpl!(char, "\n")), and ElementType!(typeof(x)) would be scope!(const x)(char[]).
>
> As I have said in the beginning, not too elegant, but I think it may work. As a bonus, this is a little bit more straightforward and requires less special-casing from the compiler side: has only one scope type instead of two and no magic tricks with scoped return values. The only problems that I can see right away are that the code now is a little verbose, and that 'front' is restricted to scoped variables in a new version.
>
> Any thoughts?

I think the key is in separating the scope attribute and the owner. The former needs to be part of the type, the latter doesn't. In this vein, it's probably a good idea to restrict the `scope!owner` syntax to function signatures, where it may only refer to other parameters and `this`. The use cases for it elsewhere are very marginal (if they exist at all).

This naturally makes the return type of `findSubstring()` just `scope(string)`; declaring a variable of it simply works:

    typeof(findSubstring("", "")) s = findSubstring("Hello, world", "world");

is equivalent to:

    scope(string) s = findSubstring("Hello, world", "world");

This is a valid assignment, and owner propagation would even take care of preserving the owners (though only on declaration, but that's natural because the owners are only known at the call site):

    string haystack, needle;
    scope(string) s = findSubstring(haystack, needle);
    // type of `s` is scope(string), owner is `haystack`

In other words, the type part of `scope!a(T)` is just `scope(T)`, the owner is not part of the type and tracked separately.

Let's look at isInputRange:

    template isInputRange(R)
    {
        enum bool isInputRange = is(typeof(
            (inout int = 0)
            {
                R r = R.init;     // can define a range object
                if (r.empty) {}   // can test for empty
                r.popFront();     // can invoke popFront()
                auto h = r.front; // can get the front of the range
            }));
    }

    auto byline = stdin.byLine();

`isInputRange!(typeof(byline))` expands to:

    scope(ByLineImpl...) r = scope(ByLineImpl...).init;
    if (r.empty) {}
    r.popFront();
    auto h = r.front;

The first line is valid, the last line too, because `scope` is now part of the type and will of course be deduced by `auto`.

So this solves the template bloat problem, along with making ElementType usable. That still leaves us with the problem of forwarding and wrappers.

    auto trace(alias func, Args...)(Args args) {
        writeln("Calling " ~ func.stringof ~ "(" ~ args.stringof ~ ")");
        return func(args);
    }

    auto s = trace!findSubstring(haystack, needle);

Here, `Args` becomes `(scope(string), scope(string))`. This cannot work, of course, because it drops the owners. How can we a) preserve the owners on the parameters, and b) propagate the result's owner back outward?

@Manu: Under the condition that we'd find a solution for "perfect forwarding", or imperfect if you like ;-), preferrably one also applicable to `ref`, would you be okay with the above?

04.10.2014 17:38, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" пишет:
> On Friday, 3 October 2014 at 19:08:10 UTC, Ivan Timokhin wrote:
>> 29.09.2014 18:17, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>"
>> пишет:
>> ...
>> Now, an idea that I have is that bare scope should be just a syntactic
>> sugar for self-owned scope; i.e., `scope(int*) x;` would be just a
>> short way of writing `scope!x(int*) x;`. This would imply that every
>> scoped variable has its own unique type, which is probably not that
>> terrible, considering that they can't be assigned freely to each other
>> either way. This is also somewhat more natural, because it means that
>> there is no such thing as just "scoped" variable, it is always
>> connected to a particular lifetime.
>
> I've already suggested this as an implementation detail.
>

Sorry then, must have missed it.


>> ...
> This of course has the unfortunate side effect of incredible template
> bloat: For any distinct passed argument (which in practice means for
> almost every call), we'd get a new template instance. IMO this is not
> acceptable.
>

They could be merged in an executable, but a symbol table would be cluttered. That does sound like a major issue.

>>
>> As for passing unscoped strings to findSubstring, I see two alternatives:
>> 1) Declare that all unscoped references are implicitly convertible to
>> scope!GC or something like that (and this conversion is used in such
>> cases). This one is probably better.
>
> OTOH it would preclude automatic demoting of GC allocations to stack
> allocations. And it would marry us to the GC is "default" allocation
> strategy, which we might want to move away from (probably in favor of
> generic allocators changeable at any point in time).
>

It doesn't necessarily have to be GC, just any object that we can safely treat as an owner.

>> ...
>
> I think the key is in separating the scope attribute and the owner. The
> former needs to be part of the type, the latter doesn't. In this vein,
> it's probably a good idea to restrict the `scope!owner` syntax to
> function signatures, where it may only refer to other parameters and
> `this`. The use cases for it elsewhere are very marginal (if they exist
> at all).
>
> This naturally makes the return type of `findSubstring()` just
> `scope(string)`; declaring a variable of it simply works:
>
>      typeof(findSubstring("", "")) s = findSubstring("Hello, world",
> "world");
>
> is equivalent to:
>
>      scope(string) s = findSubstring("Hello, world", "world");
>
> This is a valid assignment, and owner propagation would even take care
> of preserving the owners (though only on declaration, but that's natural
> because the owners are only known at the call site):
>
>      string haystack, needle;
>      scope(string) s = findSubstring(haystack, needle);
>      // type of `s` is scope(string), owner is `haystack`
>
> In other words, the type part of `scope!a(T)` is just `scope(T)`, the
> owner is not part of the type and tracked separately.
>

That's ok with me (in fact, it looks very nice), but I think Manu's point was that dealing with anything that isn't part of the type is troublesome.

> ...

04.10.2014 21:01, Ivan Timokhin пишет:
> 04.10.2014 17:38, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" пишет:
>>
>> I think the key is in separating the scope attribute and the owner. The
>> former needs to be part of the type, the latter doesn't. In this vein,
>> it's probably a good idea to restrict the `scope!owner` syntax to
>> function signatures, where it may only refer to other parameters and
>> `this`. The use cases for it elsewhere are very marginal (if they exist
>> at all).
>>
>> This naturally makes the return type of `findSubstring()` just
>> `scope(string)`; declaring a variable of it simply works:
>>
>>      typeof(findSubstring("", "")) s = findSubstring("Hello, world",
>> "world");
>>
>> is equivalent to:
>>
>>      scope(string) s = findSubstring("Hello, world", "world");
>>
>> This is a valid assignment, and owner propagation would even take care
>> of preserving the owners (though only on declaration, but that's natural
>> because the owners are only known at the call site):
>>
>>      string haystack, needle;
>>      scope(string) s = findSubstring(haystack, needle);
>>      // type of `s` is scope(string), owner is `haystack`
>>
>> In other words, the type part of `scope!a(T)` is just `scope(T)`, the
>> owner is not part of the type and tracked separately.
>>
>
> That's ok with me (in fact, it looks very nice), but I think Manu's
> point was that dealing with anything that isn't part of the type is
> troublesome.
>

On the second thought, doesn't syntax look a bit awkward now? I mean, `scope!a(T)` certainly looks like `scope!a` as a whole is a type modifier. Since this syntax is pretty much limited to function return types with this proposal, maybe a better solution would be to have an owner as a separate annotation on a function?

Also, would it really make much sense to track the owner further than the assignment of a function's return value? That seems to complicate things a lot by adding a hidden attribute to a variable that is not only invisible at the declaration (even though the full type is spelled out), but, in fact, cannot be specified explicitly (because there's no syntax for that, now that scope with owners is limited to function signatures).

How about this:
---
    scope(string) haystack, needle;
    // next assignment is okay, because `s` is guaranteed not to outlive
    // `haystack`.
    scope(string) s = findSubstring(haystack, needle);
    // type of `s` is now scope(string), no additional information
    // attached

    // so the next assignment is disallowed:
    //needle = s; // error!
---

This could be unnecessarily limiting, but would it really cause much trouble?

On Saturday, 4 October 2014 at 18:13:03 UTC, Ivan Timokhin wrote:
> On the second thought, doesn't syntax look a bit awkward now? I mean, `scope!a(T)` certainly looks like `scope!a` as a whole is a type modifier. Since this syntax is pretty much limited to function return types with this proposal, maybe a better solution would be to have an owner as a separate annotation on a function?

I agree, but it should still stay closely associated to the return type.

>
> Also, would it really make much sense to track the owner further than the assignment of a function's return value? That seems to complicate things a lot by adding a hidden attribute to a variable that is not only invisible at the declaration (even though the full type is spelled out), but, in fact, cannot be specified explicitly (because there's no syntax for that, now that scope with owners is limited to function signatures).
>
> How about this:
> ---
>     scope(string) haystack, needle;
>     // next assignment is okay, because `s` is guaranteed not to outlive
>     // `haystack`.
>     scope(string) s = findSubstring(haystack, needle);
>     // type of `s` is now scope(string), no additional information
>     // attached
>
>     // so the next assignment is disallowed:
>     //needle = s; // error!
> ---
>
> This could be unnecessarily limiting, but would it really cause much trouble?

I think you're right, I thought about this after I replied to you. It would be the logical next step. On the other hand, I wouldn't want to lose const borrowing, because it turned out to be a requirement for safe moving. But I think it can still be tracked internally (owner tracking is necessary for implementation anyway).

On Saturday, 4 October 2014 at 19:22:45 UTC, Marc Schütz wrote:
> On Saturday, 4 October 2014 at 18:13:03 UTC, Ivan Timokhin wrote:
>> Also, would it really make much sense to track the owner further than the assignment of a function's return value? That seems to complicate things a lot by adding a hidden attribute to a variable that is not only invisible at the declaration (even though the full type is spelled out), but, in fact, cannot be specified explicitly (because there's no syntax for that, now that scope with owners is limited to function signatures).
>>
>> How about this:
>> ---
>>    scope(string) haystack, needle;
>>    // next assignment is okay, because `s` is guaranteed not to outlive
>>    // `haystack`.
>>    scope(string) s = findSubstring(haystack, needle);
>>    // type of `s` is now scope(string), no additional information
>>    // attached
>>
>>    // so the next assignment is disallowed:
>>    //needle = s; // error!
>> ---
>>
>> This could be unnecessarily limiting, but would it really cause much trouble?
>
> I think you're right, I thought about this after I replied to you. It would be the logical next step. On the other hand, I wouldn't want to lose const borrowing, because it turned out to be a requirement for safe moving. But I think it can still be tracked internally (owner tracking is necessary for implementation anyway).

Owner tracking is then completely limited to one expression. I think this will simplify the implementation a lot. Besides, it has precedences: uniqueness is also only tracked inside an expression, AFAIK.