Oren Tirosh:

> Bingo. Have some way to mark the function return type as a unique pointer. This does not imply full-fledged unique pointer type support in the language

Let's have full-fledged memory zones tracking in the D type system :-)

Bye,
bearophile

On 10/1/14, 6:52 AM, Sean Kelly wrote:
> On Wednesday, 1 October 2014 at 08:55:55 UTC, Andrei Alexandrescu wrote:
>> On 9/30/14, 9:10 AM, Sean Kelly wrote:
>>>
>>> Is this for exposition purposes or actually how you expect it to work?
>>
>> That's pretty much what it would take. The key here is that RCString
>> is almost a drop-in replacement for string, so the code using it is
>> almost identical. There will be places where code needs to be
>> replaced, e.g.
>>
>> auto s = "literal";
>>
>> would need to become
>>
>> S s = "literal";
>>
>> So creation of strings will change a bit, but overall there's not a
>> lot of churn.
>
> I'm confused.  Is this a general-purpose solution or just one that
> switches between string and RCString?

General purpose since your suggested change. -- Andrei

On 10/1/14, 7:03 AM, Sean Kelly wrote:
> So let the user supply a scratch buffer that will hold the result?  With
> the RC approach we're still allocating, they just aren't built-in
> slices, correct?

Correct. -- Andrei

On 10/1/14, 8:48 AM, Oren Tirosh wrote:
> On Tuesday, 30 September 2014 at 19:10:19 UTC, Marc Schütz wrote:
>> [...]
>>
>> I'm convinced this isn't necessary. Let's take `setExtension()` as an
>> example, standing in for any of a class of similar functions. This
>> function allocates memory, returns it, and abandons it; it gives up
>> ownership of the memory. The fact that the memory has been freshly
>> allocated means that it is (head) unique, and therefore the caller (=
>> library user) can take over the ownership. This, in turn, means that
>> the caller can decide how she wants to manage it.
>
> Bingo. Have some way to mark the function return type as a unique
> pointer.

I'm skeptical about this approach (though clearly we need to explore it for e.g. passing ownership of data across threads). For strings and other "casual" objects I think we should focus on GC/RC strategies. This is because people do things like:

auto s = setExtension(s1, s2);

and then attempt to use s as a regular variable (copy it etc). Making s unique would make usage quite surprising and cumbersome.


Andrei

On Wednesday, 1 October 2014 at 17:13:38 UTC, Andrei Alexandrescu wrote:
> On 10/1/14, 8:48 AM, Oren Tirosh wrote:
>> On Tuesday, 30 September 2014 at 19:10:19 UTC, Marc Schütz wrote:
>>> [...]
>>>
>>> I'm convinced this isn't necessary. Let's take `setExtension()` as an
>>> example, standing in for any of a class of similar functions. This
>>> function allocates memory, returns it, and abandons it; it gives up
>>> ownership of the memory. The fact that the memory has been freshly
>>> allocated means that it is (head) unique, and therefore the caller (=
>>> library user) can take over the ownership. This, in turn, means that
>>> the caller can decide how she wants to manage it.
>>
>> Bingo. Have some way to mark the function return type as a unique
>> pointer.
>
> I'm skeptical about this approach (though clearly we need to explore it for e.g. passing ownership of data across threads). For strings and other "casual" objects I think we should focus on GC/RC strategies. This is because people do things like:
>
> auto s = setExtension(s1, s2);
>
> and then attempt to use s as a regular variable (copy it etc). Making s unique would make usage quite surprising and cumbersome.

The idea is that the unique property is very short-lived: the caller immediately assigns it to a pointer of the appropriate policy: either RC or GC. This keeps the callee agnostic of the chosen policy and does not require templating multiple versions of the code. The allocator configured for the thread must match the generated code at the call site i.e. if the caller uses RC pointers the allocator must allocate space for the reference counter (at negative offset to keep compatibility).

On 10/1/14, 10:25 AM, Oren T wrote:
> The idea is that the unique property is very short-lived: the caller
> immediately assigns it to a pointer of the appropriate policy: either RC
> or GC. This keeps the callee agnostic of the chosen policy and does not
> require templating multiple versions of the code. The allocator
> configured for the thread must match the generated code at the call site
> i.e. if the caller uses RC pointers the allocator must allocate space
> for the reference counter (at negative offset to keep compatibility).

This all... looks arcane. I'm not sure how it can even made to work if user code just uses "auto". -- Andrei

On Wed, Oct 01, 2014 at 02:51:08AM -0700, Andrei Alexandrescu via Digitalmars-d wrote:
> On 9/30/14, 11:06 AM, Dmitry Olshansky wrote:
> >29-Sep-2014 14:49, Andrei Alexandrescu пишет:
> >>auto setExtension(MemoryManagementPolicy mmp = gc, R1, R2)(R1 path, R2
> >>ext)
> >>if (...)
> >>{
> >>     static if (mmp == gc) alias S = string;
> >>     else alias S = RCString;
> >>     S result;
> >>     ...
> >>     return result;
> >>}
> >
> >Incredible code bloat? Boilerplate in each function for the win? I'm at loss as to how it would make things better.
> 
> Sean's idea to make string an alias of the policy takes care of this concern. -- Andrei

But Sean's idea only takes strings into account. Strings aren't the only allocated resource Phobos needs to deal with. So extrapolating from that idea, each memory management struct (or whatever other aggregate we end up using), say call it MMP, will have to define MMP.string, MMP.jsonNode (since parseJSON() need to allocate not only strings but JSON nodes), MMP.redBlackTreeNode, MMP.listNode, MMP.userDefinedNode, ...

Nope, still don't see how this could work. Please clarify, kthx.


T

-- 
Sometimes the best solution to morale problems is just to fire all of the unhappy people. -- despair.com

On Wednesday, 1 October 2014 at 17:53:43 UTC, H. S. Teoh via Digitalmars-d wrote:
> On Wed, Oct 01, 2014 at 02:51:08AM -0700, Andrei Alexandrescu via Digitalmars-d wrote:
>> On 9/30/14, 11:06 AM, Dmitry Olshansky wrote:
>> >29-Sep-2014 14:49, Andrei Alexandrescu пишет:
>> >>auto setExtension(MemoryManagementPolicy mmp = gc, R1, R2)(R1 path, R2
>> >>ext)
>> >>if (...)
>> >>{
>> >>     static if (mmp == gc) alias S = string;
>> >>     else alias S = RCString;
>> >>     S result;
>> >>     ...
>> >>     return result;
>> >>}
>> >
>> >Incredible code bloat? Boilerplate in each function for the win?
>> >I'm at loss as to how it would make things better.
>> 
>> Sean's idea to make string an alias of the policy takes care of this
>> concern. -- Andrei
>
> But Sean's idea only takes strings into account. Strings aren't the only
> allocated resource Phobos needs to deal with. So extrapolating from that
> idea, each memory management struct (or whatever other aggregate we end
> up using), say call it MMP, will have to define MMP.string, MMP.jsonNode
> (since parseJSON() need to allocate not only strings but JSON nodes),
> MMP.redBlackTreeNode, MMP.listNode, MMP.userDefinedNode, ...
>
> Nope, still don't see how this could work. Please clarify, kthx.
>
>
> T


MMP.Ref!redBlackTreeNode ?

(where Ref is e.g. a ref-counted pointer type (like RefCounted but with class support) for RC MMP but plain GC reference for GC MMP, etc.)

I kinda like this idea, since it might possibly allow user-defined memory management policies (which wouldn't get special compiler treatment that e.g. RC may need, though).

On Wednesday, 1 October 2014 at 17:33:34 UTC, Andrei Alexandrescu wrote:
> On 10/1/14, 10:25 AM, Oren T wrote:
>> The idea is that the unique property is very short-lived: the caller
>> immediately assigns it to a pointer of the appropriate policy: either RC
>> or GC. This keeps the callee agnostic of the chosen policy and does not
>> require templating multiple versions of the code. The allocator
>> configured for the thread must match the generated code at the call site
>> i.e. if the caller uses RC pointers the allocator must allocate space
>> for the reference counter (at negative offset to keep compatibility).
>
> This all... looks arcane. I'm not sure how it can even made to work if user code just uses "auto". -- Andrei

At the moment, @nogc code can't call any function returning a pointer. Under this scheme @nogc is allowed to call either code that returns an explicitly RC ty

On Wednesday, 1 October 2014 at 17:33:34 UTC, Andrei Alexandrescu wrote:
> On 10/1/14, 10:25 AM, Oren T wrote:
>> The idea is that the unique property is very short-lived: the caller
>> immediately assigns it to a pointer of the appropriate policy: either RC
>> or GC. This keeps the callee agnostic of the chosen policy and does not
>> require templating multiple versions of the code. The allocator
>> configured for the thread must match the generated code at the call site
>> i.e. if the caller uses RC pointers the allocator must allocate space
>> for the reference counter (at negative offset to keep compatibility).
>
> This all... looks arcane. I'm not sure how it can even made to work if user code just uses "auto". -- Andrei

At the moment, @nogc code can't call any function returning a pointer. Under this scheme @nogc is allowed to call either code that returns an explicitly RC type (Exception, RCString) or code returning an "agnostic" unique pointer that may be used from either @gc or @nogc code.
I already see some holes and problems, but I wonder if something along these lines may be made to work.