On Wednesday, 17 December 2014 at 05:28:35 UTC, Walter Bright wrote:
> On 12/16/2014 5:37 AM, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>> The current proposal is either too limiting or not limiting enough.
>
> I'm afraid I don't understand at all what you wrote.

Well, look at it this way: D1 took 1990s style C++ application programming model and made it more convenient with a GC. Having a GC-memory model with the possibility of stack/heap allocations as optimizations can keep the language simple. D2 is more of a library author's language, but without GC you need to deal with different types of ownership.

If you toss out the GC, and want the same level of safety and generality you either:

1. need to track ownership add dynamic runtime checks
2. provide a full blown proof system (not realistic)
3. provide means for the programmer to "help" the semantic analysis

What you want is a combination of 1 and 3, but you don't need to keep them separate. Since refcounting is expensive, you don't want that everywhere. For stack allocated objects you have an implicit "refcounting" in the nesting of scopes, you only need to track scope-depth.

If the compiler knows that a returned object is stack allocated, then it can check that by comparing against the stack frame address since the stack grows downwards in an ordered fashion. And in most cases it should be able to elide the check.

If you also maintain information about order (stack depth) among the stack allocated objects a function receives through reference parameters (for every function call), then you can safely combine the stack allocated objects you receive through parameters in aggregates, like inserting an stack allocated element into a container.

Since D is increasingly becoming a library author's language, you probably also would benefit more from strengthening the template support by having a templated ref-type rather than "scope ref". You could define protocols (e.g. predefined UDAs) that tells the compiler what kind of key properties the library type provides, such as ownership. Then use that for a more generic approach to optimization and escape analysis and let the ownership-reference-types be library types. If semantic analysis gets rid of the dynamic aspects of a referencs (such as reference counting) then the compiler should be able to use the same function body, and in other cases just add a inferred wrapper to a generic function body. So the code bloat can be limited.

If all references to external resources (like the heap) in the stack allocated object is owned by the object, then you can treat those as a whole. But you need the compiler to understand what ownership is if it is to guarantee memory safety in the kind of situations where memory safety is known to be difficult to achieve.

The alternative is to state that @safe is not covering non-GC memory and that one should stick to vetted library code when writing @safe code that does not use the GC. I actually find that acceptable for D2. In D2 you would be better off spending the effort at finding ways to turn automatically turn GC allocations into stack allocations as optimizations. Then you can find a more general and powerful ownership-oriented approach for non-GC applications for a new major version of D.

It is not realistic that people will annotate with "scope ref", and compiler inference could make updates to libraries break application code if you get false negatives. Function signatures are contracts, they ought to be explicit for libraries. So having library functions act as "scope ref" without it being explicit in the signature is not necessarily a good idea.

On Wednesday, 17 December 2014 at 12:29:21 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 17 December 2014 at 11:13:02 UTC, Marc Schütz wrote:
>> On Wednesday, 17 December 2014 at 07:48:52 UTC, bearophile wrote:
>>> Walter Bright:
>>>
>>>> I'm afraid I don't understand at all what you wrote.
>>>
>>> Perhaps reading about linear type systems could help:
>>>
>>> http://en.wikipedia.org/w/index.php?title=Substructural_type_system&redirect=no#Linear_type_systems
>>
>> But note that we want a "relaxed" linear type system. In general, we are fine with multiple aliases, although there are
>
> Pointer aliasing is an optimization killer, which is why C99 added the «restrict» keyword. I think in most cases the use of stack allocated objects tend to be alias-free, so it is important that this knowledge is retained so that the compiler can make use of it.
>
> http://en.wikipedia.org/wiki/Pointer_aliasing

I know, this is one of those applications. The one I was thinking of is const-borrowing (useful to prevent iterator invalidation and a bunch of other things).

I think that enforcing alias-freeness by default (the way a strict linear type system requires) doesn't fit too well into the existing language. Some way of signalling this to the compiler will therefore be required. It might be as simple as C's `restrict` keyword (but it needs to be enforced!), or better yet, an `unrestrict` keyword. Parameters would be alias-free by default unless explicitly specified otherwise. Local aliases (inside one function) are however ok, as long as not more than one is passed to another function at the same time.

On Wednesday, 17 December 2014 at 12:29:21 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 17 December 2014 at 11:13:02 UTC, Marc Schütz wrote:
>> On Wednesday, 17 December 2014 at 07:48:52 UTC, bearophile wrote:
>>> Walter Bright:
>>>
>>>> I'm afraid I don't understand at all what you wrote.
>>>
>>> Perhaps reading about linear type systems could help:
>>>
>>> http://en.wikipedia.org/w/index.php?title=Substructural_type_system&redirect=no#Linear_type_systems
>>
>> But note that we want a "relaxed" linear type system. In general, we are fine with multiple aliases, although there are
>
> Pointer aliasing is an optimization killer, which is why C99 added the «restrict» keyword. I think in most cases the use of stack allocated objects tend to be alias-free, so it is important that this knowledge is retained so that the compiler can make use of it.
>
> http://en.wikipedia.org/wiki/Pointer_aliasing

If you have owner ship, you have free. If you have a pair alloc/free then you can promote on stack.

This is a much more powerful way to handle things, as this take advantage of inlining.

On Wednesday, 17 December 2014 at 13:48:24 UTC, deadalnix wrote:
> If you have owner ship, you have free. If you have a pair alloc/free then you can promote on stack.
>
> This is a much more powerful way to handle things, as this take advantage of inlining.

More powerful than what?

Rather than thinking in terms of alloc/free it might be helpful to think about it more generally as resources, use, reuse and side-effects. On the stack you get to reuse memory when the variable is not longer live, meaning you junk writes if the results cannot be read and reuse the memory "prematurely".

http://en.wikipedia.org/wiki/Live_variable_analysis

If you are saying that turning GC allocations automatically into stack allocations is a good direction for D2, then I agree.

On Wednesday, 17 December 2014 at 13:35:53 UTC, Marc Schütz wrote:
> I think that enforcing alias-freeness by default (the way a strict linear type system requires) doesn't fit too well into the existing language. Some way of signalling this to the compiler will therefore be required. It might be as simple as C's `restrict` keyword (but it needs to be enforced!), or better yet, an `unrestrict` keyword. Parameters would be alias-free by default unless explicitly specified otherwise. Local aliases (inside one function) are however ok, as long as not more than one is passed to another function at the same time.

Hmm, I kind of feel that D is having too many parameter keywords/types/qualifiers. Parameter specification is primarily useful where it provides information about constraints that the programmer need to know about when using a library.

"restricted scope ref" could be a useful default for non-templated functions, then use templated ownership reference typing or inference to go beyond it.

I think that if one improved the templating system so that it was capable of analysing similarities between instantiations and use heuristics to analyse when a new instance is profitable then you could make functions templated by default and propagate knowledge from the call site to the template-instantiation mechanism of the compiler. So if the reference is known to be aliasfree then the compiler can use heuristics to determine if it should instantiate a "restricted" version or use a slower general version of the templated function.

Similarily, if refcounters always are on offset 0 and destructors are virtual, then you only need a single container method template instance for refcouting references, even if you have many types using the same method. If refcounters are big enough, you can have a unique_ptr that just set it to a large value and use the same template instance for a unique_ptr and refcounted_ptr behind the scene when referencing objects that provide a refcount field.

There are also many other things that could be done to enhance the template system, like querying properties of the block in loops that iterate over a "range", so that you can instantiate different "optimized ranges" depending on whether the loop contains a "break" statement or not. Or like Jonathan Blow's language, where you can issue a "remove element and continue" like statement within a foreach-loop,  which might require a different "range iteration" implementation.

On Tuesday, 16 December 2014 at 01:00:47 UTC, Walter Bright wrote:
> On 12/15/2014 5:38 AM, deadalnix wrote:
>> On Monday, 15 December 2014 at 11:32:11 UTC, Dicebot wrote:
>>> For me "scopeness" is a property of "view", not object itself - this also
>>> makes ownership method of actual data irrelevant. Only difference between GC
>>> owned data and stack allocated one is that former can have scoped view
>>> optionally but for the latter compiler must force it as the only available.
>>
>> Ha finally something start to make sense here.
>
> Well, the DIP does defined scope in terms of a 'view' in just this manner.
>
> I am obviously terrible at explaining things.

Such notion of "view" requires at least some elements of transitivity to be practical in my opinion. Also with my definition in mind your example of tree that stores scope nodes makes absolutely no sense unless whole tree itself is scoped (and nodes are thus scoped transitively). Such view is always assumes worst case about ownership and shouldn't persist in any form (as that would require some serious ownership tracking).

I don't think you have explained your case that bad - we simply have a very different use cases in mind as "primary" use case.

On 12/19/2014 9:44 PM, Dicebot wrote:
> Such notion of "view" requires at least some elements of transitivity to be
> practical in my opinion.

I have no idea how "some elements of transitivity" can even work. It's either transitive or its not. Please don't think of scope in terms of ownership, ownership is an orthogonal issue.


> Also with my definition in mind your example of tree
> that stores scope nodes makes absolutely no sense unless whole tree itself is
> scoped (and nodes are thus scoped transitively). Such view is always assumes
> worst case about ownership and shouldn't persist in any form (as that would
> require some serious ownership tracking).

This is definitely conflating scope and ownership.

On 12/6/14 4:49 PM, Manu via Digitalmars-d wrote:
> I need, at least, forceinline to complete it, but that one*is*
> controversial - we've talked about this for years.

I'm still 883 messages behind so here's a drive-by comment - it's time to revisit this, I think the need has become a lot clearer. -- Andrei

On 12/6/14 4:49 PM, Manu via Digitalmars-d wrote:
> In the situation where templates are involved, it would be nice to be
> able to make that explicit statement that some type is ref or not at
> the point of template instantiation, and the resolution should work
> according to the well-defined rules that we are all familiar with.

Another drive-by comment: I understand the motivation for this and the difficulties involved. There needs to be a clear understanding that adding new type qualifiers is extremely intrusive and expensive. Because of that, I think we should best address binding generation via a set of tactical tools i.e. standard library artifacts that do all that mixin business in an encapsulated and reusable manner.

(As an aside forcing a template instantiation to decide ref vs. no ref should be easy but currently can't be done, i.e. this code should work but currently doesn't:

T fun(T)(ref T x) { return x + 1; }

void main(string[] group)
{
    int function(int) f1 = &fun!int;
    int function(ref int) f2 = &fun!int;
}

)


Andrei

On Saturday, 20 December 2014 at 21:39:44 UTC, Walter Bright wrote:
> On 12/19/2014 9:44 PM, Dicebot wrote:
>> Such notion of "view" requires at least some elements of transitivity to be
>> practical in my opinion.
>
> I have no idea how "some elements of transitivity" can even work. It's either transitive or its not. Please don't think of scope in terms of ownership, ownership is an orthogonal issue.

.. and here I was about you to do exactly the same :P What in example I show makes you think of ownership?

When I was speaking about "some elements of transitivity" I was thinking in a way of keeping scope storage class but transitively applying same restrictions to all data accessible through it AS IF it had scope storage class on its own - while still making illegal to use scope as a separate type qualifier.

>> Also with my definition in mind your example of tree
>> that stores scope nodes makes absolutely no sense unless whole tree itself is
>> scoped (and nodes are thus scoped transitively). Such view is always assumes
>> worst case about ownership and shouldn't persist in any form (as that would
>> require some serious ownership tracking).
>
> This is definitely conflating scope and ownership.

No, it is exactly the other way around. The very point of what I am saying is that you DOESN'T CARE about ownership as long as worst case scenario is assumed.   I have zero idea why you identify it is conflating as ownership when it is explicitly designed to be distinct.