Jump to page: 1 27  
Page
Thread overview
DIP 1008 Preliminary Review Round 1
May 19, 2017
Mike Parker
May 19, 2017
rikki cattermole
May 19, 2017
H. S. Teoh
May 19, 2017
Stanislav Blinov
May 19, 2017
H. S. Teoh
May 19, 2017
Stanislav Blinov
May 19, 2017
Jack Stouffer
May 19, 2017
H. S. Teoh
May 19, 2017
Stanislav Blinov
May 20, 2017
Jonathan M Davis
May 20, 2017
Walter Bright
May 20, 2017
Stefan Koch
May 20, 2017
H. S. Teoh
May 20, 2017
Jonathan M Davis
May 20, 2017
Jonathan M Davis
May 20, 2017
Stefan Koch
May 20, 2017
Jonathan M Davis
May 20, 2017
Stefan Koch
May 20, 2017
Jonathan M Davis
May 20, 2017
H. S. Teoh
May 21, 2017
Jack Stouffer
May 20, 2017
Stanislav Blinov
May 19, 2017
nkm1
May 19, 2017
Jack Stouffer
May 19, 2017
Adam D. Ruppe
May 19, 2017
Stanislav Blinov
May 20, 2017
Meta
May 20, 2017
Walter Bright
May 20, 2017
Stanislav Blinov
May 22, 2017
Nick Treleaven
May 20, 2017
Jonathan M Davis
May 20, 2017
Moritz Maxeiner
May 22, 2017
Nick Treleaven
May 24, 2017
Jack Stouffer
May 20, 2017
Stanislav Blinov
May 23, 2017
Martin Nowak
May 24, 2017
MysticZach
May 25, 2017
Walter Bright
May 26, 2017
Walter Bright
Jun 18, 2017
Martin Nowak
May 25, 2017
Atila Neves
May 26, 2017
Walter Bright
May 26, 2017
Walter Bright
May 26, 2017
Atila Neves
May 27, 2017
Walter Bright
May 27, 2017
Atila Neves
May 26, 2017
Atila Neves
May 26, 2017
Atila Neves
May 26, 2017
Atila Neves
May 27, 2017
Walter Bright
May 27, 2017
Atila Neves
May 27, 2017
Walter Bright
May 27, 2017
Walter Bright
May 27, 2017
via Digitalmars-d
May 27, 2017
via Digitalmars-d
Jun 11, 2017
Nick Treleaven
Jun 05, 2017
MysticZach
May 19, 2017
DIP 1008 is titled "Exceptions and @nogc".

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md

All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on June 2 (3:59 AM GMT June 3), or when I make a post declaring it complete.

At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors.

Extensive discussion of this DIP has already taken place in two threads, both linked from the document. You may find it beneficial to skim through those threads before posting any feedback here.

Thanks in advance to all who participate.

Destroy!
May 19, 2017
On 19/05/2017 4:45 PM, Mike Parker wrote:
> DIP 1008 is titled "Exceptions and @nogc".
>
> https://github.com/dlang/DIPs/blob/master/DIPs/DIP1008.md
>
> All review-related feedback on and discussion of the DIP should occur in
> this thread. The review period will end at 11:59 PM ET on June 2 (3:59
> AM GMT June 3), or when I make a post declaring it complete.
>
> At the end of Round 1, if further review is deemed necessary, the DIP
> will be scheduled for another round. Otherwise, it will be queued for
> the formal review and evaluation by the language authors.
>
> Extensive discussion of this DIP has already taken place in two threads,
> both linked from the document. You may find it beneficial to skim
> through those threads before posting any feedback here.
>
> Thanks in advance to all who participate.
>
> Destroy!

"Code that needs to leak the thrown exception object can clone the object."

Errors:
```D
import std.stdio;
void main() {
	auto e = new Exception("foo");
	e = e.dup;
	writeln(e.toString());	
}
```

Let's just say, I am unaware of a way to duplicate classes. Assuming I'm messing something, a code example would be very appreciated in the DIP.
May 19, 2017
On Fri, May 19, 2017 at 05:13:34PM +0100, rikki cattermole via Digitalmars-d wrote: [...]
> "Code that needs to leak the thrown exception object can clone the object."
> 
> Errors:
> ```D
> import std.stdio;
> void main() {
> 	auto e = new Exception("foo");
> 	e = e.dup;
> 	writeln(e.toString());
> }
> ```
> 
> Let's just say, I am unaware of a way to duplicate classes. Assuming I'm messing something, a code example would be very appreciated in the DIP.

AFAIK, there is no way to clone classes, unless the class writer implemented it explicitly. Only arrays support .dup, no other type does (unless user code explicitly implements it, or its equivalent).


T

-- 
This is not a sentence.
May 19, 2017
On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:
> On Fri, May 19, 2017 at 05:13:34PM +0100, rikki cattermole via Digitalmars-d wrote: [...]
>> "Code that needs to leak the thrown exception object can clone the object."
>> 
>> Errors:
>> ```D
>> import std.stdio;
>> void main() {
>> 	auto e = new Exception("foo");
>> 	e = e.dup;
>> 	writeln(e.toString());
>> }
>> ```
>> 
>> Let's just say, I am unaware of a way to duplicate classes. Assuming I'm messing something, a code example would be very appreciated in the DIP.
>
> AFAIK, there is no way to clone classes, unless the class writer implemented it explicitly. Only arrays support .dup, no other type does (unless user code explicitly implements it, or its equivalent).
>
>
> T

Well, not that it's a complete implementation or anything, but it definitely could be done with a library:

/// Make a shallow clone of a class instance
/// (members that are classes are not cloned)
C clone(C)(C src) if (is(C == class))
{
    // could probably just dynamic cast and copy
    // from most derived...
    assert(typeid(src) == typeid(C));
    // can use any memory allocator here,
    // assuming deallocation is handled correctly
    import core.memory : GC;
    import core.exception : onOutOfMemoryError;
    auto ptr = GC.malloc(__traits(classInstanceSize, C));
    ptr || onOutOfMemoryError;
    scope (failure) GC.free(ptr);

    C dst = cast(C)ptr;

    import std.traits : BaseClassesTuple;
    import std.meta : AliasSeq;
    import core.stdc.string : memcpy;
    foreach(T; AliasSeq!(C, BaseClassesTuple!C)) {
        T bsrc = src;
        T bdst = dst;
        foreach(i, ref _; bsrc.tupleof) {
            alias M = typeof(_);
            memcpy(&bdst.tupleof[i], &bsrc.tupleof[i], M.sizeof);
            static if (__traits(hasMember, M, "__postblit"))
                bdst.tupleof[i].__postblit;
        }
    }

    return dst;
}

void main() {
    import std.stdio;
    struct Struct {
        int[] data;
        this(this) {
            data = data.dup;
            writeln("Struct is copied");
        }
    }

    class Klass {
        int value;
        Struct s;
        this(int v) {
            value = v;
            s = Struct([1, 2, 3, 4, 5]);
        }
    }

    auto c1 = new Klass(42);
    auto c2 = c1.clone;
    assert(c2);
    assert(c2 !is c1);
    assert(c2.value == 42);
    assert(c2.s.data);
    assert(c2.s.data !is c1.s.data);
    assert(c2.s.data == c1.s.data);
}

May 19, 2017
On Fri, May 19, 2017 at 05:48:55PM +0000, Stanislav Blinov via Digitalmars-d wrote:
> On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:
[...]
> > AFAIK, there is no way to clone classes, unless the class writer implemented it explicitly. Only arrays support .dup, no other type does (unless user code explicitly implements it, or its equivalent).
> > 
> > 
> > T
> 
> Well, not that it's a complete implementation or anything, but it definitely could be done with a library:
> 
> /// Make a shallow clone of a class instance
> /// (members that are classes are not cloned)
[...]

That's the problem right there: you cannot guarantee that a shallow copy will have the correct semantics.  For example, modifying a class member through the original copy will also modify what is seen through the clone, which may not be the intention.

Also, you have to take care that if the class member is ref-counted, the clone function needs to increment the refcount, otherwise you could end up with a dangling pointer. Meaning, in the general case, you need to be careful about calling postblits and such.

Furthermore, a generic deep-copying clone function may not be possible in the general case, because sometimes you don't know if a reference member is intended to be a reference to external data, or an owned value that needs to be deep-copied. For example, if a class C encapsulates a container of references to data, then C.clone should not copy the data, but if a class D is a container of references to data that it owns, then the data should be copied. A generic function cannot decide which one it is unless the intent is made clear, e.g. via a standard UDA, or some other such means.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:
May 19, 2017
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:
> ...

I have already made my objections known in the other threads, but I'll list them here for posterity.

Firstly, and most importantly IMO, this does not solve the surface level problem, which is the lack of @nogc in much of Phobos. While the DIP is correct that in a lot of the absence of @nogc is due to exceptions, most notably the auto decoding functions in std.range, it not the only reason. There are many things in Phobos which stop @nogc, such as array literals, array appending, AAs, and delegates. While many functions can be made @nogc via this change, there will be many functions left in Phobos still allocating, and this is a major problem in my estimation. The reason I take this all or nothing view of the situation is that @nogc exists largely for a group of people who want to avoid the GC __completely__. If lack of any GC usage is actually a requirement for a project, having many very useful parts of Phobos off limits (a few that spring to mind are bigint, digest, and a lot of stdio) is still a problem when our competitors in this GC free arena are Rust and C++ which will both have a lot more functionality in this limited scope.

Secondly, I'm not a fan of special casing syntax, especially when I don't think the given benefits outweigh the above listed costs. Making operator new do something completely different in one specific context is a continuation of the trend in the recent past of making the language more and more complicated, which I find troubling as one of D's main selling points is it's practicality and its low learning curve.

And while I don't think the code breakage will be a huge problem, it still is something to consider when asking what are we getting for all of these costs.

Thirdly, I don't think the DIP asks the right question. The DIP asks

> How can we make exceptions, and thereby many Phobos functions, @nogc?

IMO the right question to ask is a level of analysis above that. As already mentioned, there are many things which will still be allocated on the GC. We need to fix these as well if we want to market D as a GC optional language. Instead, we should be asking

> How can allocations in Phobos and user code be transferred to an allocation strategy of the user's choosing/needs?

If the user values code simplicity and safety, then the GC is fine, and template functions which allocate can then be marked as @safe due to their use of GC. But if the user needs other speed or usage requirements, then the user needs to offer something else, and that will automatically be marked as @system. I think that integrating std.allocator across Phobos in a consistent manner will allow this, where functions can accept allocators, and their internal allocations can all be done on the given allocator. This would also allow exceptions and other objects can be manually freed after a function is called if needed, or just left to the GC to collect if the GC allocator was passed to the function.

In summary, I believe what Phobos needs is a wholistic approach to the problem of memory management, and not a bunch of small solutions over time. To be sure, this is a much harder task, but I think if it's done right, with D we can have the first language which has compete user control to those who need it while offering ease of use to everyone else.
May 19, 2017
On Fri, May 19, 2017 at 06:16:33PM +0000, Jack Stouffer via Digitalmars-d wrote: [...]
> Instead, we should be asking
> 
> > How can allocations in Phobos and user code be transferred to an allocation strategy of the user's choosing/needs?
> 
> If the user values code simplicity and safety, then the GC is fine, and template functions which allocate can then be marked as @safe due to their use of GC. But if the user needs other speed or usage requirements, then the user needs to offer something else, and that will automatically be marked as @system. I think that integrating std.allocator across Phobos in a consistent manner will allow this, where functions can accept allocators, and their internal allocations can all be done on the given allocator. This would also allow exceptions and other objects can be manually freed after a function is called if needed, or just left to the GC to collect if the GC allocator was passed to the function.

I agree with this, and having looked at std.experimental.allocator recently, I think Andrei may even have made certain design decisions with this in mind.  I think the ideal goal (I'm not sure how achievable it is) would be to make theAllocator part of *druntime*, upon which you can actually use array concatenations, AA's, etc., (almost) freely, and the user would be able to control exactly how allocations would work. At least in the current state of things, I don't see this as being particularly possible due to various issues, the main one being that code written with GC in mind generally does not track lifetimes, which is a big obstacle to successfully being able to just assign a different allocator to theAllocator and have things Just Work(tm).

Short of putting theAllocator in druntime outright, the next best thing would be for all Phobos code to use it by default, eschewing any constructs that might bypass it.  This seems more attainable, though it does imply a LOT of churn in Phobos, and may require significant time and effort to pull off.  Some Phobos code, of course, might be able to take in allocators as template parameters or some such mechanism of finer-grained control, but given the very large amount of code we're dealing with, defaulting to theAllocator seems like a (slightly) more reachable goal.


> In summary, I believe what Phobos needs is a wholistic approach to the problem of memory management, and not a bunch of small solutions over time.  To be sure, this is a much harder task, but I think if it's done right, with D we can have the first language which has compete user control to those who need it while offering ease of use to everyone else.

Yes, D needs to return to its original vision of getting the fundamental design decisions right, rather than trying to retroactively patch over existing flaws.  Otherwise, we risk becoming the next C++, which would be a great pity.


T

-- 
"Computer Science is no more about computers than astronomy is about telescopes." -- E.W. Dijkstra
May 19, 2017
On Friday, 19 May 2017 at 18:10:50 UTC, H. S. Teoh wrote:
> On Fri, May 19, 2017 at 05:48:55PM +0000, Stanislav Blinov via Digitalmars-d wrote:
>> On Friday, 19 May 2017 at 17:05:09 UTC, H. S. Teoh wrote:
> [...]
>> > AFAIK, there is no way to clone classes, unless the class writer implemented it explicitly. Only arrays support .dup, no other type does (unless user code explicitly implements it, or its equivalent).
>> > 
>> > 
>> > T
>> 
>> Well, not that it's a complete implementation or anything, but it definitely could be done with a library:
>> 
>> /// Make a shallow clone of a class instance
>> /// (members that are classes are not cloned)
> [...]
>
> That's the problem right there: you cannot guarantee that a shallow copy will have the correct semantics.  For example, modifying a class member through the original copy will also modify what is seen through the clone, which may not be the intention.

That's true.

> Also, you have to take care that if the class member is ref-counted, the clone function needs to increment the refcount, otherwise you could end up with a dangling pointer.

It does call postblits. But of course, a complete implementation should do more than that. It probably has to lock the monitor for the duration, correctly annotate the GC block if memory is being GC-allocated, etc...

> Meaning, in the general case, you need to be careful about calling postblits and such.
>
> Furthermore, a generic deep-copying clone function may not be possible in the general case...

Of course. For such cloning to be officially supported by the language, at least one additional primitive is needed: __postblit for classes. This will let user code correctly handle cloning as needed. Surely, a much better solution would be actual owning/non-owning pointers and references, but I doubt we'd ever get there.

May 19, 2017
On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:
> DIP 1008 is titled "Exceptions and @nogc".

As someone who iis interested in @nogc (more precisely: in avoiding GC pauses), I like this proposal... But it looks like people are concerned about 'new' becoming contextual keyword (that in some contexts it allocates with GC and in others it does something else). So maybe different syntax can be used? How about "throw scope Exception"? ("scope" already does various things :)

> Instead, we should be asking... How can allocations in Phobos and user code be transferred to an allocation strategy of the user's choosing/needs?

But memory management is not only about allocations, it's also about deallocations! Manually deallocating stuff (like freeing exception objects) is error prone and bothersome. Refcounted exceptions seems to me a pretty good solution (and I think the general plan for D is to use more reference counting? At least, that's my impression...)
May 19, 2017
On Friday, 19 May 2017 at 18:16:33 UTC, Jack Stouffer wrote:
> On Friday, 19 May 2017 at 15:45:28 UTC, Mike Parker wrote:
>> ...

> Secondly, I'm not a fan of special casing syntax, especially when I don't think the given benefits outweigh the above listed costs...

You're raising an extremely important point. Special-casing exceptions in such a way, while may help *some* code, is still a crutch, and a very ugly one at that. The bigger issue of @nogc is the implicit allocations. It would be much better to solve the fundamental problem, that is, give user code optional, but complete control over memory allocation.

All three main areas need to be addressed in concert: classes, arrays and delegates. Working on just one area, and even just a subset of it (exceptions) in the long run will further the problem, not help solve it.

As an example, operator new could accept additional syntax (outer [] mean "optional"):

new [(TAllocator)] Type [[count]] [(init)]

Where TAllocator is a class providing at least minimal allocator interface (allocate/deallocate). @nogc can be inferred from the allocator.
This lets user code decide on their class and array allocations. OTOH, this will incur storage penalty, since all dynamically-allocated objects (including arrays) will have to carry the allocator reference with them. It doesn't, however, solve the @nogc inference in general, since allocator is not part of the type. Attempting to concatenate two arrays at a distance from their allocation site would not be considered @nogc. That being said, arrays with complete inference support could be made library types, so perhaps that is not *that* big of an issue.

Delegates, as they are now, require a lot more attention. Firstly, there is the dubious design decision that any outer scope access is always by reference, which imposes the GC allocation of closures in the first place. @nogc cannot be solved for delegates without addressing control over captures. Then we have the allocation itself, in cases when it is desired. And that will completely destroy existing delegate syntax.

There is, however, another side to this problem, and that is interaction of exceptions with the runtime. Allowing users to employ any desired allocation scheme precludes reliable exception handling, since the runtime effectively loses control over exception lifetime. Catching garbage in a catch(...) block would never be a pleasant surprise. That being said, it's not that it is in control right now either:

void innocent() {
    throw new Exception("I am innocent");
}

void malicious() {
    try {
        innocent();
    } catch (Exception e) {
        // innocent, eh? well I am not...
        e.destroy;
        throw e;
    }
}

void oblivious() {
    try {
        malicious();
    } catch (Exception e) {
        writeln("Caught: ", e.msg);
    }
}

void main() {
    oblivious();
}

Hello, segmentation fault. If users want to be difficult, they'd find a way :)

So perhaps, instead of tackling exceptions on their own, we really should focus on allocation in general. Everything else should (hopefully) come automagically.
« First   ‹ Prev
1 2 3 4 5 6 7