I've been reading all the topics with those "radical" ideas about the GC and dtors and, honestly, i'd rather call them "insane". After all the reading and thinking, i came to conclusion, that what Andrey suggests is to call dtors only on stack-allocated structs. That also implies, that one can't put those in containers and gc-allocated objects.
Since all of them: containers, structs, classes -- are all first-class objects they must be all nicely combined in code, without any unintuitive stuff.
I mean, really, let us look at c++ strings. There are const char*, std::sting, QString, Poco::String, icu::UnicodeString, and every big project uses it's own strings implementation that can't be used together in a sane way. That is what great in D: you just threw the idea of library-implemented strings away and made it not only "standard", but special and that what makes it intuitive and simple (unless you interact with c++). Never seen any non-standart strings for D.
The point is, that every library and every coder add to the project incompatible and difficult to use together stuff, so every programmer already has to think of all the problems with other's people code, and you just can't add same shit to the language. All elements must nicely interact together and otherwise it's a total disaster.
Back to the dtors: i understand, that all the stuff you propose could make GC faster, simpler, and cooler, but it sounds insane to anyone, who uses the language, not develops it, that if you create struct object, dror will be called, but if you place the copy in a container, it wont. It's just unanderstandable from user's point of view.

Now, for the solution.

First, we can just fix this shit with arrays of structs and that's it. That still lives us with false pointers problem: not everything gets collected. That's no good. So, i propose to think of actually separating gc-memory management (via GC) and other resources management: via some new (or maybe old) mechanism.

Let me start with listing of existing solutions:

1) C.
That is the simplest way: fully-manual resource management.
It's obvious, we can't do that in D, because it's supposed to be simpler for coding, than C.

2) Go.
Actually, this one is not that different: it uses GC for memory only, and manual management for all the rest (with help of defer operator). We can't do it either, for the same reasons.

3) C++.
This one is semi-automatic (talking about user code, not some allocator libraries): you choose the scheme (refcounting, unique reference) and then it'll do the rest.

4) Rust.
I'm not a pro here, but as i understand, it uses C++ way, and adds gc-collected pointers, but not sure, so help me here.

5) Python.
GC-only, except one clever case: with statement calls close() method.


Please, if there are any pros in other platforms, add your knowledge to this list, i would very much love to learn (same, if a made any mistakes).


Now, for D: obviously D has GC-managed heap. First, we should, like in Go, leave only managing gc-memory to the GC -- this is just rephrasing Andreys proposal.
The simplest way o manage all other resources would be manual, Go-way:

A a = A();
scope(exit)
    a.~A();

But it's to annoying, to that all the time, so we really want dtors to save us lost of typing and debugging, but they can't be called all the time, because we can put stuff in GC-collected objects.

What i propose, is to include new concept in D: scoped objects.
Any object (no matter is it a class or struct instance) can be either scoped or not.
Dtors for scoped objects are called when out of scope, dtors for non-scoped objects are not called at all.

It is actually as simple as rewrite code

A a = A();

as

A a = A();
scope(exit)
    a.~A();

For all a's, which are scoped objects.

For me, it is both a simple concept and good rationalization for difficult dror-gets-called-or-not rules.

That leaves only to determine, what objects are scoped. Well, that is obviously stack-allocated structs, gc-allocated scope classes and gc-allocated structs in scope classes.

But that is just my idea. This post has so many words, because it's very important, that D devs make good decision on that deep problem, and the key to such decision is information and discussion.

UPD:
Also, about arrays and slices: if we could easily pass them around as cost ref-s, just like in C++, then we could make them value-types and they wouldn't require any ref counting. I would suggest, make all "in" function arguments const refs.

On Saturday, 3 May 2014 at 12:28:03 UTC, monnoroch wrote:
> I've been reading all the topics with those "radical" ideas about the GC and dtors and, honestly, i'd rather call them "insane". After all the reading and thinking, i came to conclusion, that what Andrey suggests is to call dtors only on stack-allocated structs. That also implies, that one can't put those in containers and gc-allocated objects.
> Since all of them: containers, structs, classes -- are all first-class objects they must be all nicely combined in code, without any unintuitive stuff.
> I mean, really, let us look at c++ strings. There are const char*, std::sting, QString, Poco::String, icu::UnicodeString, and every big project uses it's own strings implementation that can't be used together in a sane way. That is what great in D: you just threw the idea of library-implemented strings away and made it not only "standard", but special and that what makes it intuitive and simple (unless you interact with c++). Never seen any non-standart strings for D.
> The point is, that every library and every coder add to the project incompatible and difficult to use together stuff, so every programmer already has to think of all the problems with other's people code, and you just can't add same shit to the language. All elements must nicely interact together and otherwise it's a total disaster.
> Back to the dtors: i understand, that all the stuff you propose could make GC faster, simpler, and cooler, but it sounds insane to anyone, who uses the language, not develops it, that if you create struct object, dror will be called, but if you place the copy in a container, it wont. It's just unanderstandable from user's point of view.
+1 about dtors.

> Now, for the solution.
>
> First, we can just fix this shit with arrays of structs and that's it. That still lives us with false pointers problem: not everything gets collected. That's no good. So, i propose to think of actually separating gc-memory management (via GC) and other resources management: via some new (or maybe old) mechanism.
>
> Let me start with listing of existing solutions:
>
> 1) C.
> That is the simplest way: fully-manual resource management.
> It's obvious, we can't do that in D, because it's supposed to be simpler for coding, than C.
>
> 2) Go.
> Actually, this one is not that different: it uses GC for memory only, and manual management for all the rest (with help of defer operator). We can't do it either, for the same reasons.
>
> 3) C++.
> This one is semi-automatic (talking about user code, not some allocator libraries): you choose the scheme (refcounting, unique reference) and then it'll do the rest.
>
> 4) Rust.
> I'm not a pro here, but as i understand, it uses C++ way, and adds gc-collected pointers, but not sure, so help me here.
>
> 5) Python.
> GC-only, except one clever case: with statement calls close() method.
>
>
> Please, if there are any pros in other platforms, add your knowledge to this list, i would very much love to learn (same, if a made any mistakes).
>
>
> Now, for D: obviously D has GC-managed heap. First, we should, like in Go, leave only managing gc-memory to the GC -- this is just rephrasing Andreys proposal.
> The simplest way o manage all other resources would be manual, Go-way:
>
> A a = A();
> scope(exit)
>     a.~A();
>
> But it's to annoying, to that all the time, so we really want dtors to save us lost of typing and debugging, but they can't be called all the time, because we can put stuff in GC-collected objects.
>
> What i propose, is to include new concept in D: scoped objects.
> Any object (no matter is it a class or struct instance) can be either scoped or not.
> Dtors for scoped objects are called when out of scope, dtors for non-scoped objects are not called at all.
>
> It is actually as simple as rewrite code
>
> A a = A();
>
> as
>
> A a = A();
> scope(exit)
>     a.~A();
>
> For all a's, which are scoped objects.
>
> For me, it is both a simple concept and good rationalization for difficult dror-gets-called-or-not rules.
>
> That leaves only to determine, what objects are scoped. Well, that is obviously stack-allocated structs, gc-allocated scope classes and gc-allocated structs in scope classes.
>
> But that is just my idea. This post has so many words, because it's very important, that D devs make good decision on that deep problem, and the key to such decision is information and discussion.
>
> UPD:
> Also, about arrays and slices: if we could easily pass them around as cost ref-s, just like in C++, then we could make them value-types and they wouldn't require any ref counting. I would suggest, make all "in" function arguments const refs.

I like the idea of scoping dtor's. But I still want the ability to say:
Hey I have this global variable, if I assign a value to it and later null it, it'll call its destructor if its not referenced anywhere else.
Which in turn would make me think ref counting would be a good idea.

But either way, I think we are getting ahead of ourselves with all these 'major' proposed changes. I think its time to step back and say hey D2 shouldn't be changed much more lets freeze it.
Now lets plan for D3.
Breakage between D2 and D3 is acceptable but not in D2 to the extent some of these proposals is bringing to the table.

> Hey I have this global variable, if I assign a value to it and later null it, it'll call its destructor if its not referenced anywhere else.
> Which in turn would make me think ref counting would be a good idea.

It seems, that ARC is the only way. There were idea to make all non-scoped (in my terminology) objects ARC-d, if they have dtors. That makes sense to me.

Scoped-objects + ARC on non-scoped objects with dtors + GC on non-scoped objects w/o dtors would arguably solve the problem, especially, is arrays of scoped objects would be considered also scoped, or just add separate scoped arrays.

Am 03.05.2014 14:28, schrieb monnoroch:
> I've been reading all the topics with those "radical" ideas about the GC
> and dtors and, honestly, i'd rather call them "insane". After all the
> reading and thinking, i came to conclusion, that what Andrey suggests is
> to call dtors only on stack-allocated structs. That also implies, that
> one can't put those in containers and gc-allocated objects.
> Since all of them: containers, structs, classes -- are all first-class
> objects they must be all nicely combined in code, without any
> unintuitive stuff.
> I mean, really, let us look at c++ strings. There are const char*,
> std::sting, QString, Poco::String, icu::UnicodeString, and every big
> project uses it's own strings implementation that can't be used together
> in a sane way. That is what great in D: you just threw the idea of
> library-implemented strings away and made it not only "standard", but
> special and that what makes it intuitive and simple (unless you interact
> with c++). Never seen any non-standart strings for D.
> The point is, that every library and every coder add to the project
> incompatible and difficult to use together stuff, so every programmer
> already has to think of all the problems with other's people code, and
> you just can't add same shit to the language. All elements must nicely
> interact together and otherwise it's a total disaster.
> Back to the dtors: i understand, that all the stuff you propose could
> make GC faster, simpler, and cooler, but it sounds insane to anyone, who
> uses the language, not develops it, that if you create struct object,
> dror will be called, but if you place the copy in a container, it wont.
> It's just unanderstandable from user's point of view.
>
> Now, for the solution.
>
> First, we can just fix this shit with arrays of structs and that's it.
> That still lives us with false pointers problem: not everything gets
> collected. That's no good. So, i propose to think of actually separating
> gc-memory management (via GC) and other resources management: via some
> new (or maybe old) mechanism.
>
> Let me start with listing of existing solutions:
>
> 1) C.
> That is the simplest way: fully-manual resource management.
> It's obvious, we can't do that in D, because it's supposed to be simpler
> for coding, than C.
>
> 2) Go.
> Actually, this one is not that different: it uses GC for memory only,
> and manual management for all the rest (with help of defer operator). We
> can't do it either, for the same reasons.
>
> 3) C++.
> This one is semi-automatic (talking about user code, not some allocator
> libraries): you choose the scheme (refcounting, unique reference) and
> then it'll do the rest.
>
> 4) Rust.
> I'm not a pro here, but as i understand, it uses C++ way, and adds
> gc-collected pointers, but not sure, so help me here.
>
> 5) Python.
> GC-only, except one clever case: with statement calls close() method.
>
>
> Please, if there are any pros in other platforms, add your knowledge to
> this list, i would very much love to learn (same, if a made any mistakes).
>
>
> Now, for D: obviously D has GC-managed heap. First, we should, like in
> Go, leave only managing gc-memory to the GC -- this is just rephrasing
> Andreys proposal.
> The simplest way o manage all other resources would be manual, Go-way:
>
> A a = A();
> scope(exit)
>      a.~A();
>
> But it's to annoying, to that all the time, so we really want dtors to
> save us lost of typing and debugging, but they can't be called all the
> time, because we can put stuff in GC-collected objects.
>
> What i propose, is to include new concept in D: scoped objects.
> Any object (no matter is it a class or struct instance) can be either
> scoped or not.
> Dtors for scoped objects are called when out of scope, dtors for
> non-scoped objects are not called at all.
>
> It is actually as simple as rewrite code
>
> A a = A();
>
> as
>
> A a = A();
> scope(exit)
>      a.~A();
>
> For all a's, which are scoped objects.
>
> For me, it is both a simple concept and good rationalization for
> difficult dror-gets-called-or-not rules.
>
> That leaves only to determine, what objects are scoped. Well, that is
> obviously stack-allocated structs, gc-allocated scope classes and
> gc-allocated structs in scope classes.
>
> But that is just my idea. This post has so many words, because it's very
> important, that D devs make good decision on that deep problem, and the
> key to such decision is information and discussion.
>
> UPD:
> Also, about arrays and slices: if we could easily pass them around as
> cost ref-s, just like in C++, then we could make them value-types and
> they wouldn't require any ref counting. I would suggest, make all "in"
> function arguments const refs.


C# and Java also have scoped blocks (using, try-with-resources), similar to Python for resource management.

Ada has controlled types for it, where Finalize() plays the role of C++ destructor.

Additionally all languages with lambda support do offer resource management via implicit control structures. Mostly visible in Lisp and ML dialects. Great in the languages that allow the lambda as last parameter to appear outside of the call.

For example, assume a doTransaction (connection, lambda) function, then

doTransaction(connection) {
  db.insert (my data)
}


The approach taken is similar to implementing C++ templates for generic RAII use cases.

--
Paulo

On Saturday, 3 May 2014 at 13:21:04 UTC, Paulo Pinto wrote:
> C# and Java also have scoped blocks (using, try-with-resources), similar to Python for resource management.

Yeah,  but it doesn't work for graphs that maintain resources, such as a scene graph which hold onto texture memory. Unfortunately, you don't want GC collection to release it either.

I think in most cases resources can either be handled by owned pointers or a regional GC (in this case a GC that only trace SceneNodePointers).

Am 03.05.2014 16:02, schrieb "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>":
> On Saturday, 3 May 2014 at 13:21:04 UTC, Paulo Pinto wrote:
>> C# and Java also have scoped blocks (using, try-with-resources),
>> similar to Python for resource management.
>
> Yeah,  but it doesn't work for graphs that maintain resources, such as a
> scene graph which hold onto texture memory. Unfortunately, you don't
> want GC collection to release it either.
>
> I think in most cases resources can either be handled by owned pointers
> or a regional GC (in this case a GC that only trace SceneNodePointers).

Why not?

- Make all scene graph nodes IDisposable.

- Have a using(rootNode) {} on your render loop

--
Paulo

On Saturday, 3 May 2014 at 14:31:59 UTC, Paulo Pinto wrote:
> - Make all scene graph nodes IDisposable.
>
> - Have a using(rootNode) {} on your render loop

That would work for a static scene.

But you want to mark resources ready for release when nodes are removed from the graph dynamically. In most cases shared pointers (ref counting) would work, but if you allow "fractal" recursion then you will get cycles. (e.g. meshnode->scaleRND->rotateRND->stopIfTooSmall->meshnode )

Am 03.05.2014 16:43, schrieb "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>":
> On Saturday, 3 May 2014 at 14:31:59 UTC, Paulo Pinto wrote:
>> - Make all scene graph nodes IDisposable.
>>
>> - Have a using(rootNode) {} on your render loop
>
> That would work for a static scene.
>
> But you want to mark resources ready for release when nodes are removed
> from the graph dynamically. In most cases shared pointers (ref counting)
> would work, but if you allow "fractal" recursion then you will get
> cycles. (e.g. meshnode->scaleRND->rotateRND->stopIfTooSmall->meshnode )

Easy, you already know that you don't need the node, just call the dispose method, instead of marking for release.

Cannot release right away? Place them in a "to be removed" queue and do the cleaning in a specific controlled point.

--
Paulo

On Saturday, 3 May 2014 at 15:10:43 UTC, Paulo Pinto wrote:
> Easy, you already know that you don't need the node, just call the dispose method, instead of marking for release.

But you don't know unless you use RC or GC. Let's say you make a jungle. Lots of pointers to the tree root node. And you also don't want to wait with collection if you hold onto hardware resources (forcing perhaps lower resolution graphics if you are low on GPU resources).