On 2/24/15 1:55 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
> On Tuesday, 24 February 2015 at 19:40:35 UTC, Andrei Alexandrescu wrote:
>> So: does DIP25 allow safe slices? Looks that way
>
> I believe it does, but at the cost of forced reference counting. As I
> pointed out, the `ref` solution

What is the ref solution?

> is not applicable to slices, therefore
> it needs to return an RCArray. This in turn forces an inc/dec whenever
> it is sliced or copied, and - maybe worse - it requires all consumers to
> support RCArray (or whatever other idiosyncratic RC implementation users
> come up with).

It seems to me it's time to have a UniqueArray proof of concept.


Andrei

On Tuesday, 24 February 2015 at 19:40:35 UTC, Andrei Alexandrescu wrote:
> I modified Walter's sample code to this: http://dpaste.dzfl.pl/f3d854feede9. It uses malloc for both the array and the reference count, and also uses @trusted minimally. I inserted assert()s here and there to clarify the workings. Nothing big except for the careful use of @trusted.
>

Using malloc make the GC blind. It means you can't stored anything GCed in this. Which means you either need to enforce this at the interface level (I have no idea how) or remove all these @trusted in the code.

Generally using malloc is not the right way forward. I do think that, even for RCed resource, we want them backed by the GC. It will allow for cycle collection. The main problem with the GC is not allocation, it is collection, and collection won't fire if one do not leak.

> 2. Michel's point (https://issues.dlang.org/show_bug.cgi?id=14221) reveals the largest issue with RC/GC integration. We need to find a fix for it if we want to have the GC lift cycles.
>

There is a set of feature that create implicit sharing. GC is only one of them. They either need to be fixed (and here, you'll get my point about simple fix aggregating to something more complex) or make this thread safe.

> So: does DIP25 allow safe slices? Looks that way, though a proof would be nice. Does it allow other safe interesting structs that own data? Very likely.

As long as you don't plan to own an arbitrary sub graph.

> Does it allow really sophisticated ownership schemes? We need to explore that.

Limited, unless you wrap all indirection in some sort of struct down the road.

> Does it protect against bugs in implementations of safe ownership schemes that explicitly release memory? Not too well. I think the prevalent idiom will be to accompany such artifacts with unittests that make sure unsafe uses (such as fun() above) do not compile.
>

I though we wanted to do better than C++...

On 2/24/2015 2:45 PM, deadalnix wrote:
> Using malloc make the GC blind. It means you can't stored anything GCed in this.

GC.addRange() solves this problem.


> Generally using malloc is not the right way forward. I do think that, even for
> RCed resource, we want them backed by the GC. It will allow for cycle
> collection.

If cycles are not possible, malloc/free should be used. Cycles are not possible in objects without indirections, and we can assert they are not possible it the type can be introspected at compile time to see if cycles are not possible.

>> So: does DIP25 allow safe slices? Looks that way, though a proof would be
>> nice. Does it allow other safe interesting structs that own data? Very likely.
> As long as you don't plan to own an arbitrary sub graph.

To have an arbitrary sub graph be memory safe with return ref, the interface to it will have to be constructed to only allow access by values or return refs.


>> Does it protect against bugs in implementations of safe ownership schemes that
>> explicitly release memory? Not too well. I think the prevalent idiom will be
>> to accompany such artifacts with unittests that make sure unsafe uses (such as
>> fun() above) do not compile.
> I though we wanted to do better than C++...

No language offers a way to check system code for safety. Not Java, not Rust, not C#, not nobody, not no how. What is offered is safety for the client's usage of it. C++ doesn't have that.

On Wednesday, 25 February 2015 at 00:11:28 UTC, Walter Bright wrote:
>> Generally using malloc is not the right way forward. I do think that, even for
>> RCed resource, we want them backed by the GC. It will allow for cycle
>> collection.
>
> If cycles are not possible, malloc/free should be used. Cycles are not possible in objects without indirections, and we can assert they are not possible it the type can be introspected at compile time to see if cycles are not possible.
>

Why ? There is very little chance that malloc + GC.addRange becomes any faster that GC.malloc in the first place.

>>> So: does DIP25 allow safe slices? Looks that way, though a proof would be
>>> nice. Does it allow other safe interesting structs that own data? Very likely.
>> As long as you don't plan to own an arbitrary sub graph.
>
> To have an arbitrary sub graph be memory safe with return ref, the interface to it will have to be constructed to only allow access by values or return refs.
>

That means all library code must be duplicated. I could explain you why this is bad, but someone already made a very good point about it there: http://www.drdobbs.com/cpp/type-qualifiers-and-wild-cards/231902461

>>> Does it protect against bugs in implementations of safe ownership schemes that
>>> explicitly release memory? Not too well. I think the prevalent idiom will be
>>> to accompany such artifacts with unittests that make sure unsafe uses (such as
>>> fun() above) do not compile.
>> I though we wanted to do better than C++...
>
> No language offers a way to check system code for safety. Not Java, not Rust, not C#, not nobody, not no how. What is offered is safety for the client's usage of it. C++ doesn't have that.

A language is an API to the system you are running on. A very expressive and complex one, but an API nevertheless.

A good API will make correct and safe use easy and incorrect and unsafe use convoluted, so they are done on purpose and not by mistake.

That is not about proving system code correct (impossible) that is about making unprovable code harder to spit out than provable one (or getting an error).

On 2/24/2015 4:41 PM, deadalnix wrote:
> Why ? There is very little chance that malloc + GC.addRange becomes any faster
> that GC.malloc in the first place.

1. GC is designed to do GC, which is more than malloc/free is designed to do. Less work usually is faster.

2. There are many malloc/free implementations available, and people can and do swap them out to find one that works best for their application. Also, compiler vendors often expend a great deal of effort making malloc/free work well. Why not take advantage of that? It's like it's pretty hard to compete with the system memcpy() - people expend enormous effort on that.


>> To have an arbitrary sub graph be memory safe with return ref, the interface
>> to it will have to be constructed to only allow access by values or return refs.
> That means all library code must be duplicated.

Why?


> but someone already made a very good point about it there:
> http://www.drdobbs.com/cpp/type-qualifiers-and-wild-cards/231902461

Don't see how that applies.

Andrei Alexandrescu wrote:

> On 2/24/15 1:33 PM, Ivan Timokhin wrote:
>> Is there any plan to allow safe conversions to T[] (in restricted
>> circumstances, of course)?
> 
> We'd want to avoid it because that would necessitate the whole "scope" paraphernalia - e.g you can convert a reference counted slice to something like scope(T[]).
> 
> A possible option would be to allow the user to go "I want to use GC with this slice from now on" which gives back the T[] and lets the GC take care of it.
> 
> But generally I think we should let reference counted slices and built-in slices coexist. Generic range-based code is easy to write so factoring the nature of the slice away may be an option.
> 
> 
> Andrei

Oh. So, whenever you pass a reference-counted slice around, you need to do it with the full inc/dec protocol, which, as Walter has mentioned several times already, leads to code bloat and performance hits. So... no to efficient reference counting? Also, no slicing of static arrays in @safe code?

On Tuesday, 24 February 2015 at 22:11:36 UTC, Andrei Alexandrescu wrote:
> On 2/24/15 1:55 PM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm@gmx.net>" wrote:
>> On Tuesday, 24 February 2015 at 19:40:35 UTC, Andrei Alexandrescu wrote:
>>> So: does DIP25 allow safe slices? Looks that way
>>
>> I believe it does, but at the cost of forced reference counting. As I
>> pointed out, the `ref` solution
>
> What is the ref solution?
>

DIP25, in that it applies the semantics of `scope` to `ref`. The safety gained is thereby restricted to `ref`, but does not apply to slices, pointers and class references. That's the reason - as you already replied to Ivan - that RCArray cannot return a real slice.

>> is not applicable to slices, therefore
>> it needs to return an RCArray. This in turn forces an inc/dec whenever
>> it is sliced or copied, and - maybe worse - it requires all consumers to
>> support RCArray (or whatever other idiosyncratic RC implementation users
>> come up with).
>
> It seems to me it's time to have a UniqueArray proof of concept.

How is that related to what I wrote above?

On 2/25/15 1:58 AM, Ivan Timokhin wrote:
> Oh. So, whenever you pass a reference-counted slice around, you need to do
> it with the full inc/dec protocol, which, as Walter has mentioned several
> times already, leads to code bloat and performance hits. So... no to
> efficient reference counting? Also, no slicing of static arrays in @safe
> code?

Correct.

There's an issue of perception that I just figured, which explains a lot of the drama and trash talk about DIP25.

DIP25 is not a borrowing mechanism. Its charter is to make reference counted structs (and a variety of other structs) usable in @safe code. As RCSlice shows, with DIP25 a reference counted slice switches from unusable to usable in @safe code by adding one token to an otherwise unchanged implementation. That is Remarkable, and is Good Programming Language Design(tm).

Borrowing data in a scoped manner is the charter of DIP69. Given the excellent quality of DIP25, it's likely it is Here To Stay and DIP69, or any other proposal for borrowing, will work with it (and probably leverage it).


Andrei

On Wed, Feb 25, 2015 at 08:06:48AM -0800, Andrei Alexandrescu wrote:
> On 2/25/15 1:58 AM, Ivan Timokhin wrote:
> > Oh. So, whenever you pass a reference-counted slice around, you need to do it with the full inc/dec protocol, which, as Walter has mentioned several times already, leads to code bloat and performance hits. So... no to efficient reference counting? Also, no slicing of static arrays in @safe code?
> 
> Correct.
> 
> There's an issue of perception that I just figured, which explains a lot of the drama and trash talk about DIP25.
> 
> DIP25 is not a borrowing mechanism. Its charter is to make reference counted structs (and a variety of other structs) usable in @safe code. As RCSlice shows, with DIP25 a reference counted slice switches from unusable to usable in @safe code by adding one token to an otherwise unchanged implementation. That is Remarkable, and is Good Programming Language Design(tm).
> 
> Borrowing data in a scoped manner is the charter of DIP69. Given the excellent quality of DIP25, it's likely it is Here To Stay and DIP69, or any other proposal for borrowing, will work with it (and probably leverage it).
> 
> 
> Andrei
> 

OK, thank you. I wasn't asking about DIP25 in particular (I think I understand what it's about), but about your (and Walter's) plans on the topic in general. I apologise if it sounded offensive in any way.

On 2/25/15 11:04 AM, Ivan Timokhin wrote:
> OK, thank you. I wasn't asking about DIP25 in particular (I think I
> understand what it's about), but about your (and Walter's) plans on
> the topic in general.

The medium-term plan is two-pronged: (a) immediately apply DIP25 to http://wiki.dlang.org/?title=DIP74, which will propose @safe reference counted class objects; (b) examine scoped borrowing (DIP69 and friends) opportunities post DIP25.

Walter and I are working on DIP74 right now.


Andrei