On Wednesday, 7 October 2015 at 07:24:03 UTC, Paulo Pinto wrote:
> On Tuesday, 6 October 2015 at 20:43:42 UTC, bitwise wrote:
>> On Tuesday, 6 October 2015 at 18:43:42 UTC, Jonathan M Davis wrote:
>>> On Tuesday, 6 October 2015 at 18:10:42 UTC, bitwise wrote:
>>>> On Tuesday, 6 October 2015 at 17:20:39 UTC, Jonathan M Davis wrote:
>>>> I'm not sure what else I can say. The example I posted says it all, and it can't be done properly in D (or C#, but why lower the bar because of their mistakes? ;)
>>>
>>> It's a side effect of having the lifetime of an object managed by the GC. There's no way around that except to use something else like manual memory management or reference counting.
>>
>> You are literally repeating what I just said in different words.
>>
>>> in D, it's a good reason to use structs to manage resources like that, and since most objects really have no need of inheritance and have no business being classes, it's usually fine.
>>
>> This is an opinion.
>>
>> I want polymorphism AND deterministic destruction, and the least you could do is just admit that it's a downside to D not having it, instead of trying to tell me that everything I know is wrong..
>>
>>> But in the cases where you do have to use a class, it can get annoying.
>>
>> YES, its does, and it's not just an odd case here and there..
>>
>>> You simply do not rely on the GC or the destruction of the object to free system resources. You manually call a function on the object to free those resources when you're done with it.
>>
>> I'm sorry, but I almost can't believe you're saying this.
>>
>> So, you're saying you want me to just revert back to manual resource management and accept that huge resources like textures and such may just leak if someone doesn't use them right? or throws an exception? in a language like D that is supposed to be safe?
>>
>>> In the case of C#, they have a construct to help with it that (IIRC) is something like
>>>
>>> using(myObj)
>>> {
>>> } // myObj.dispose() is called when exiting this scope
>>
>> For the THIRD time, I'll post my example:
>>
>> class Texture { }
>> class Texture2D : Texture {
>>     this() { /* load texture... */ }
>>     ~this { /* free texture */ }     // OOPS, when, if ever, will this be called?
>> }
>>
>> Now, does this really seem like a realistic use case to you?
>>
>> using(Texture tex = new Texture2D) {
>>     // ...
>> }
>>
>
> That no, but this yes (at least in C#):
>
> using (LevelManager mgr = new LevelManager())
> {
>      //....
>      // Somewhere in the call stack
>      Texture text = mgr.getTexture();
> }
> --> All level resources gone that require manual management gone
> --> Ask the GC to collect the remaining memory right now
>
> If not level wide, than maybe scene/section wide.
>
> However I do get that not all architectures are amendable to be re-written in a GC friendly way.
>
> But the approach is similar to RAII in C++, reduce new to minimum and allocate via factory functions that work together with handle manager classes.
>
> --
> Paulo

Still no ;)

It's a Texture. It's meant to be seen on the screen for a while, not destroyed in the same scope which it was created.

In games though, we have a scene graph. When things happen, we often chip off a large part of it while the game is running, discard it, and load something new. We need to know that what we just discarded has been destroyed completely before we start loading new stuff when we're heavily constrained by memory. And even in cases where we aren't that constrained by memory, we need to know things have been destroyed, period, for non-memory resources. Also, when using graphics APIs like OpenGL, we need control over which thread an object is destroyed in, because you can't access OpenGL resources from just any thread. Now, you could set up some complicated queue where you send textures and so on to(latently) be destroyed, but this is just complicated. Picture a Hello OpenGL app in D and the hoops some noob would have to jump through. It's bad news.

Also, I should add, that a better example of the Texture thing would be a regular Texture and a RenderTexture. You can only draw to the RenderTexture, but you should be able to apply both to a primitive for drawing. You need polymorphism for this. A struct will not do.

    Bit

On Wednesday, 7 October 2015 at 09:49:27 UTC, Marc Schütz wrote:
> RefCounted isn't implemented for classes, but there's no reason why it shouldn't work.
>
>
> Really, I don't get why everyone wants to have builtin refcounting, when all that's required is a working way to make escape-proof references.

If you make a class that owns a resources that needs deterministic destruction, there is no guarantee that everyone will wrap that class in a RefCounted properly. Also, I've already stated many more problems with struct wrappers.

On Wednesday, 7 October 2015 at 12:56:32 UTC, bitwise wrote:
> On Wednesday, 7 October 2015 at 07:24:03 UTC, Paulo Pinto wrote:
>> On Tuesday, 6 October 2015 at 20:43:42 UTC, bitwise wrote:
>>>[...]
>>
>> That no, but this yes (at least in C#):
>>
>> using (LevelManager mgr = new LevelManager())
>> {
>>      //....
>>      // Somewhere in the call stack
>>      Texture text = mgr.getTexture();
>> }
>> --> All level resources gone that require manual management gone
>> --> Ask the GC to collect the remaining memory right now
>>
>> If not level wide, than maybe scene/section wide.
>>
>> However I do get that not all architectures are amendable to be re-written in a GC friendly way.
>>
>> But the approach is similar to RAII in C++, reduce new to minimum and allocate via factory functions that work together with handle manager classes.
>>
>> --
>> Paulo
>
> Still no ;)
>
> It's a Texture. It's meant to be seen on the screen for a while, not destroyed in the same scope which it was created.
>
> In games though, we have a scene graph. When things happen, we often chip off a large part of it while the game is running, discard it, and load something new. We need to know that what we just discarded has been destroyed completely before we start loading new stuff when we're heavily constrained by memory. And even in cases where we aren't that constrained by memory, we need to know things have been destroyed, period, for non-memory resources. Also, when using graphics APIs like OpenGL, we need control over which thread an object is destroyed in, because you can't access OpenGL resources from just any thread. Now, you could set up some complicated queue where you send textures and so on to(latently) be destroyed, but this is just complicated. Picture a Hello OpenGL app in D and the hoops some noob would have to jump through. It's bad news.
>
> Also, I should add, that a better example of the Texture thing would be a regular Texture and a RenderTexture. You can only draw to the RenderTexture, but you should be able to apply both to a primitive for drawing. You need polymorphism for this. A struct will not do.
>
>     Bit

I guess you misunderstood the // Somewhere in the call stack

It is meant as the logical region where that scene graph block you refer to is valid.

As for OpenGL being complex, fear not, Vulkan, Metal and DX 12 are here to help (ca 600 LOC for a triangle). :)

And both Java and .NET do offer support such type of queues as well.

Anyway I was just explaining what is possible when one embraces the tools GC languages offer.

In general, I advocate any form of automatic memory/resource management. With substructural type systems now being my favorite, but they still have an uphill battle for adoption.

Also as a note, Microsoft will be discussing their proposed C++ solution with the Rust team.

--
Paulo

On Wednesday, 7 October 2015 at 11:21:04 UTC, Namespace wrote:
>> Well, except that then it's less obvious that an object is ref-counted and less likely that the programmer using it will realize that the object expects to have a deterministic lifetime. So, it might actually make the situation worse and make it so that programmers are more likely to get wrong. I don't think that it's clear at all that the situation will be better with regards to programmers getting it right if it's in the language. Maybe it will be, but maybe it won't.
>>
>> - Jonathan M Davis
>
> Well then, there is another solution: enable inheritance for structs as well. Then we have polymorphie and deterministic lifetimes. Of course we cannot expect too much magic. But an example:

How does that solve anything? The problem is that some types need a deterministic lifetime, and no object whose lifetime is managed by the GC gets a deterministic lifetime. So, even if an object supports deterministic destruction, that stops working as soon as its put inside of something whose lifetime is managed by the GC. Whether the object with deterministic destruction has inheritance or not doesn't really matter, and whether it naturally has deterministic destruction or whether it has it because it's being used in smart pointer doesn't really matter. It's the fact that it was put in an object whose lifetime is managed by the GC that screws with things.

Even if D classes were identical to C++ classes, and we had no structs, the fact that we have a GC managing the lifetime of anything causes problems with any type that needs deterministic destruction.

What's needed is to have a smart pointer of some kind to manage the lifetime of objects on the heap that need deterministic destruction and then to have the programmer make sure that they do put any of such objects inside of an object whose lifetime is managed by the GC if they want the deterministic destruction to work. You can't get away from requiring the programmer to be smart about things here unless you simply have no GC (which then requires them to smart about other things), or any type with deterministic destruction simply isn't allowed to be on the GC heap, which is unnecessarily limiting, particularly since in many cases, it's perfectly okay to let objects that might normally be destroy deterministically to be destroyed at the GC's leisure.

std.typecons.RefCounted aside, it's quite possible as-is to implement smart pointers in D with structs, thus providing deterministic destruction for reference types. I don't know why RefCounted wasn't implemented to work with classes, and I don't know why Walter and Andrei think that something like DIP 74 is necessary to support ref-counting. I might have heard, but if so, I forgot. It probably comes down to a loophole somewhere in how structs are put together that makes it throw off the ref-count (maybe something to do with init values), but that's speculation on my part. Regardless, if we have a proper ref-counting mechanism for classes (built-in or not doesn't necessarily matter - it just needs to work), then the folks that need deterministic destruction with polymorphism get it. But they're always going to have be careful about having such objects on the GC heap due to the nature of garbage collection.

- Jonathan M Davis

On Wednesday, 7 October 2015 at 12:59:05 UTC, bitwise wrote:
> On Wednesday, 7 October 2015 at 09:49:27 UTC, Marc Schütz wrote:
>> RefCounted isn't implemented for classes, but there's no reason why it shouldn't work.
>>
>>
>> Really, I don't get why everyone wants to have builtin refcounting, when all that's required is a working way to make escape-proof references.
>
> If you make a class that owns a resources that needs deterministic destruction, there is no guarantee that everyone will wrap that class in a RefCounted properly. Also, I've already stated many more problems with struct wrappers.

Except it doesn't even matter if they always wrap it properly - or even if something like DIP 74 is implemented. The core problem is that no object (be it struct or class) which needs deterministic destruction can have its lifetime managed by the GC - be it be simply being directly allocated by the GC or by sitting inside of an object that was put on the GC heap without any kind of ref-counting. The programmer is always going to have to be careful about where they put an object that needs deterministic destruction. They either have to keep it out of objects whose lifetimes are managed by the GC or be okay with them not actually being destroyed deterministically in that particular case. The method of ref-counting doesn't change that. That's simply a restriction of dealing with the GC. The only way around that would be to make it illegal to put any kind of object with a destructor (rather than a finalizer) on the GC heap, which would be unnecessarily restrictive, particularly since there are plenty of cases where a programmer isn't going to care that an object that is normally destroyed deterministically ends up being destroyed at the GC's leisure instead. Rather, if we don't want to be stuck with techniques that Java and C# programmers use to solve these problems, we need a way to provide deterministic destruction for the programmer to use when they want to. Structs get that natively. Classes don't.

So, for classes to be able to work with deterministic destruction, we either need to have RefCounted (or something similar) work properly with classes, or we need something like DIP 74 which builds it into the language. It really doesn't matter which it is so long as it works. My guess is that RefCounted doesn't work with classes because of the fact that it uses alias this, and alias this off the table if you're trying to make it so that the wrapped object can't escape - though with that in mind, I question that RefCounted to should have alias this even it doesn't involve classes. opDispatch would be our only hope at that point, and I don't know if that would fully work or not (which may be why Walter and Andrei now are looking at putting ref-counting into the language rather than getting it to work with smart pointers like they were arguing for before).

Regardless of the ref-counting mechanism though, you can't guarantee that it's going to be used correctly. The best that we can do is guarantee that if it is used correctly, then the object will be destroyed deterministically.

- Jonathan M Davis

On Wednesday, 7 October 2015 at 15:13:40 UTC, Jonathan M Davis wrote:
> the GC heap, which is unnecessarily limiting, particularly since in many cases, it's perfectly okay to let objects that might normally be destroy deterministically to be destroyed at the GC's leisure.

This is a costly (in terms of collection) and unreliable approach to resource managment, so it ought to be removed and replaced with something more robust with less overhead.

> speculation on my part. Regardless, if we have a proper ref-counting mechanism for classes (built-in or not doesn't necessarily matter - it just needs to work), then the folks that need deterministic destruction with polymorphism get it.

Proper reference counting generally requires a lot of programmer attention if you want to get what C++ offers.

1. There is a need to add support for weak pointers so you can avoid circular references, this leads to double indirection.

2. There is a need to add support aliasing pointers (pointers to members) that increase the ownership refcount to prevent premature destruction if you retain a pointer to a member.

3. For multi threading you need to have atomics both on the counter and on the pointer  (coming in C++17). This is also needed for cache-objects IMO.

On Wednesday, 7 October 2015 at 13:15:11 UTC, Paulo Pinto wrote:
> In general, I advocate any form of automatic memory/resource management. With substructural type systems now being my favorite, but they still have an uphill battle for adoption.

Are you thinking about Rust, or some other language?

> Also as a note, Microsoft will be discussing their proposed C++ solution with the Rust team.

Are you thinking about more lintish tools that can give false positives, or something with guarantees that can be a language feature?

On Wednesday, 7 October 2015 at 15:13:40 UTC, Jonathan M Davis wrote:
> std.typecons.RefCounted aside, it's quite possible as-is to implement smart pointers in D with structs, thus providing deterministic destruction for reference types. I don't know why RefCounted wasn't implemented to work with classes, and I don't know why Walter and Andrei think that something like DIP 74 is necessary to support ref-counting.

There's a lot of hassles with a wrapper, the biggest blocker being how it interacts with inheritance.

interface Base {}
class Foo : Base {}

void test(RefCounted!Base b) {}

RefCounted!Foo foo = initialized;
test(foo);

What happens? Well, you could alias this to Base... but what about to Object? We'd need multiple alias this to get it all working right, and then it'd still be kinda a pain.

Also, what about covariant method overriding? Base class returns RefCounted!this and child class does too... will it be accepted?


Library refcounting is cool and great for a great many things, but it doesn't play well with classes at all, inheritance just makes it too complicated.


The magic of dip74 is that most the language remains the same so those cases are already covered.

On Wednesday, 7 October 2015 at 15:42:57 UTC, Ola Fosheim Grøstad wrote:
> On Wednesday, 7 October 2015 at 13:15:11 UTC, Paulo Pinto wrote:
>> In general, I advocate any form of automatic memory/resource management. With substructural type systems now being my favorite, but they still have an uphill battle for adoption.
>
> Are you thinking about Rust, or some other language?


All of the ones that explore this area. Rust, ATS, Idris, F*....
>
>> Also as a note, Microsoft will be discussing their proposed C++ solution with the Rust team.
>
> Are you thinking about more lintish tools that can give false positives, or something with guarantees that can be a language feature?

What Herb Sutter demoed at CppCon as compiler validation to CoreC++.

I can imagine that depending on how well the community takes those guidelines, they might become part of C++20.

On the other hand, on Herb's talk around 1% of the audience acknowledged the use of static analysers. Pretty much in sync what I see in enterprise developers.

On Wednesday, 7 October 2015 at 17:02:51 UTC, Paulo Pinto wrote:
> On Wednesday, 7 October 2015 at 15:42:57 UTC, Ola Fosheim Grøstad wrote:
>> On Wednesday, 7 October 2015 at 13:15:11 UTC, Paulo Pinto wrote:
>>> In general, I advocate any form of automatic memory/resource management. With substructural type systems now being my favorite, but they still have an uphill battle for adoption.
>>
>> Are you thinking about Rust, or some other language?
>
>
> All of the ones that explore this area. Rust, ATS, Idris, F*....
>>
>>> Also as a note, Microsoft will be discussing their proposed C++ solution with the Rust team.
>>
>> Are you thinking about more lintish tools that can give false positives, or something with guarantees that can be a language feature?
>
> What Herb Sutter demoed at CppCon as compiler validation to CoreC++.
>
> I can imagine that depending on how well the community takes those guidelines, they might become part of C++20.
>
> On the other hand, on Herb's talk around 1% of the audience acknowledged the use of static analysers. Pretty much in sync what I see in enterprise developers.

The CppCon demos were impressive, but I'm dying to see how Microsoft's analyser works out in real life. I've seen too many tools with too many false positives to be useful, and I'm sceptical that a library solution is all it takes to make C++ safe. As I asked Bjarne after his keynote, if it were that easy, why does Rust exist?

Atila