On Thursday, 13 November 2014 at 10:10:34 UTC, Marc Schütz wrote:
> potentially have references to it. They either need to be stopped, or write barriers need to be utilized when references to immutable data are changed.

I sometimes wonder if the very pragmatic and brutal option of just having GC on the main thread and prevent all escapes of objects from the main thread without explicit ownership transfer would solve D's GC performance related problems.

It would work for real time applications and it would work for people who use D in a scripty fashion (but not for vibe.d).

On Wednesday, 12 November 2014 at 12:57:58 UTC, Marc Schütz wrote:
> On Wednesday, 12 November 2014 at 06:48:47 UTC, deadalnix wrote:
>> On Wednesday, 12 November 2014 at 03:13:20 UTC, Rikki Cattermole wrote:
>>> [...]
>>
>> yes and no. The ideas is similar, but it is not doable at library level if we want to get safety and the full benefit out of it, as it would require for the compiler to introduce some call to the runtime at strategic places and it does interact with @nogc.
>
> I'm not sure. A library implementation may be feasible. For invalidation, the objects can be returned to their init state. This is "safe", but maybe not ideal, as a compiler error might indeed be better. Even implicit conversion to shared & immutable will be possible with multiple alias this, though it's worth discussing whether an explicit conversion isn't preferable.
>
> As for @nogc, I see it as a clutch that's needed while no "real" solution is available, and as a tool for aiding transition once we have one.
>
> That said, some compiler support will definitely be necessary.

I now remember that I played around with a related idea when Andrei suggested his template solution (but without the islands part):

http://wiki.dlang.org/User:Schuetzm/RC,_Owned_and_allocators

The important part is that owning, RC and GC types all need to know the allocators they belong to. In my example I need that to allow Owned types to be converted to RC types. In your proposal, something similar will ultimately be needed for merging the islands into specific heaps. The nice thing is that this scheme also allows for switching allocators on the fly at runtime, without template bloat, at the cost of virtual calls for using the allocators (which is probably an acceptable compromise for an inherently expensive thing like allocation).

On Thursday, 13 November 2014 at 10:10:34 UTC, Marc Schütz wrote:
> On Wednesday, 12 November 2014 at 21:15:05 UTC, deadalnix wrote:
>> On Wednesday, 12 November 2014 at 12:49:41 UTC, Marc Schütz wrote:
>>> All this is unfortunately only true if there are no references between heaps, i.e. if the heaps are indeed "islands". Otherwise, there need to be at least write barriers.
>>>
>>
>> Yes, that is exactly why I'm listing the case where these can be created in @safe code and propose a solution to plug the hole (and which brings other benefits along the road).
>
> Hmm... I can't find that in what you wrote. To clarify: I'm talking about the fact that, for example, a thread-local heap can contain references into the immutable and shared heaps. Therefore, the immutable heap can _not_ be collected without disturbing any threads, because any TL heaps (and stacks!) can potentially have references to it. They either need to be stopped, or write barriers need to be utilized when references to immutable data are changed.

You need a set of root from the TL heap. The trick being to
consider everything that is allocated AFTER you get the roots as
automatically alive (you'll collect this in the next collection
cycle).

That way, new allocated chunk that have reference in the TL heap
will be kept alive, even if you don't know about the roots.

You'll find plenty of information about the details if look into
GC for ML family languages.

On Thursday, 13 November 2014 at 14:26:44 UTC, Marc Schütz wrote:
> The important part is that owning, RC and GC types all need to know the allocators they belong to. In my example I need that to allow Owned types to be converted to RC types. In your proposal, something similar will ultimately be needed for merging the islands into specific heaps. The nice thing is that this scheme also allows for switching allocators on the fly at runtime, without template bloat, at the cost of virtual calls for using the allocators (which is probably an acceptable compromise for an inherently expensive thing like allocation).

If you have a owned type, you don't need a builtin RC type. As
long as the wrapper that implement the RC own the RCed object,
everything is safe.

Also, this construct as the nice side effect that a single
reference count can own a whole island instead of tracking all
objects RC one by one.

On 11/11/2014 6:34 PM, deadalnix wrote:
> On an implementation level, a call to a pure function that return an owned could
> look like this :
>
> {
>    IslandID __saved = gc_switch_new_island();
>    scope(exit) gc_restore_island(__saved);
>
>    call_pure_function();
> }
>
> This allow us to rely much less on the GC and allow for a better GC implementation.

If that wrapper is automatically generated by the compiler, so the user doesn't have to mess with it, it could be workable.

On Friday, 14 November 2014 at 01:05:13 UTC, Walter Bright wrote:
> On 11/11/2014 6:34 PM, deadalnix wrote:
>> On an implementation level, a call to a pure function that return an owned could
>> look like this :
>>
>> {
>>   IslandID __saved = gc_switch_new_island();
>>   scope(exit) gc_restore_island(__saved);
>>
>>   call_pure_function();
>> }
>>
>> This allow us to rely much less on the GC and allow for a better GC implementation.
>
> If that wrapper is automatically generated by the compiler, so the user doesn't have to mess with it, it could be workable.

Yes, that is the intention. It means that my proposal does increase moderately the complexity of the language, but increase the complexity of the runtime (most specifically the GC) significantly.

On Thursday, 13 November 2014 at 22:12:22 UTC, deadalnix wrote:
> You need a set of root from the TL heap. The trick being to
> consider everything that is allocated AFTER you get the roots as
> automatically alive (you'll collect this in the next collection
> cycle).
>
> That way, new allocated chunk that have reference in the TL heap
> will be kept alive, even if you don't know about the roots.
>
> You'll find plenty of information about the details if look into
> GC for ML family languages.

Consider this:

    auto i = new immutable(int);
    immutable(int)* a, b;
    b = i;
    // GC kicks in here, scans `a` (== null)
    a = b;
    b = null;
    // GC goes on, scans `b` (== null)
    // => whoops, no reference to *i

Here, a and b are on the stack or in registers. They could also be on the TL heap.

13-Nov-2014 00:27, deadalnix пишет:
> On Wednesday, 12 November 2014 at 20:36:32 UTC, Dmitry Olshansky wrote:
>> Seems sane. owned(Exception) would be implicitly assumed i.e.:
>> catch(Exception e){ ... }
>>
>> would be seen by compiler as:
>> catch(owned(Exception) e){ ... }
>>
>> What happens if I throw l-value exception? Do I need to cast or
>> assumeOwned it?
>>
>> It's easy to see how it goes with r-values, such as new
>> Exception(...), since they are "unique expressions" whatever that
>> means ;)
>>
>
> Yes, the unsafe road must always be open, we are a system programming
> language :)
>
>> I take it that owned(T) is implicitly deduced by compiler in case of
>> pure functions? Also it seem templates should not take owned(T) into
>> consideration and let it decay... How does owned compose with other
>> qualifiers?
>>
>
> You mean what is I have an owned field into an object ? In the case you
> pass the owned where a TL, shared or immutable is expected, the island
> is merged so the question do not make sense.

Sorry, forgot to reply.

Here is the case I wanted to check:

try{
	...
}
catch(owned(Exception) e){
	foo(e);
}

void foo(T)(T arg){
	// what would this print? Exception or owned(Exception)
	// do we bloat a bit more on qualifiers?
	pragma(msg, T);
}


>
> An owned field in an object is interpreted as follow:
>   - immutable => immutable
>   - shared => owned (and can be touched only if the shared object is
> synchronized, which allow to hide a whole hierarchy behind a mutex. That
> is another selling point but I don't wanted to get into all the details
> as the post was already quite big).
>   - const => const owned (essentially unusable - except via burrowing if
> we ever want to go that road one day).
>

Thanks, looks sane.

>> Seems absolutely cool. But doesn't allocating exception touches heap
>> anyway? I take it that if I don't save exception explicitly anywhere
>> the owned island is destroyed at catch scope?
>>
>
> Yes it touches the heap. But as long as things are owned, they'll be
> freed automatically when going out of scope. That means, with that
> definition of things, what is forbidden in @nogc code is to consume the
> owned in such a fashion that its island is merged into TL, shared or
> immutable heap. If you don't do this, then your isolated will be freed
> when going out of scope and the GC won't need to kick in/no garbage will
> be produced.
>
> Doing so allow for relaxing the constraint in @nogc and allow for the
> same library code to be used with or without GC.


-- 
Dmitry Olshansky

On Friday, 14 November 2014 at 19:02:52 UTC, Dmitry Olshansky wrote:
> Here is the case I wanted to check:
>
> try{
> 	...
> }
> catch(owned(Exception) e){
> 	foo(e);
> }
>
> void foo(T)(T arg){
> 	// what would this print? Exception or owned(Exception)
> 	// do we bloat a bit more on qualifiers?
> 	pragma(msg, T);
> }

It needs to be `owned(Exception)`, otherwise, how could the compiler know how to treat it correctly? But declaring foo() in that way would be unhelpful, because it would move the exception on calling the function, which is usually not desired. What you want here, instead, is borrowing:

    void foo(T)(scope(T) arg) {
        pragma(msg, T);
    }

On Friday, 14 November 2014 at 11:46:51 UTC, Marc Schütz wrote:
> On Thursday, 13 November 2014 at 22:12:22 UTC, deadalnix wrote:
>> You need a set of root from the TL heap. The trick being to
>> consider everything that is allocated AFTER you get the roots as
>> automatically alive (you'll collect this in the next collection
>> cycle).
>>
>> That way, new allocated chunk that have reference in the TL heap
>> will be kept alive, even if you don't know about the roots.
>>
>> You'll find plenty of information about the details if look into
>> GC for ML family languages.
>
> Consider this:
>
>     auto i = new immutable(int);
>     immutable(int)* a, b;
>     b = i;
>     // GC kicks in here, scans `a` (== null)
>     a = b;
>     b = null;
>     // GC goes on, scans `b` (== null)
>     // => whoops, no reference to *i
>
> Here, a and b are on the stack or in registers. They could also be on the TL heap.

That is a well covered subject and told you what to google for as
well as the basic approach. Your example here simply told me you
haven't done your homework before posting.

Please go look into scientific documentation about GC for ML
languages.