On Saturday, July 23, 2016 11:25:02 Steven Schveighoffer via Digitalmars-d- learn wrote:
> The real problem here is that there is a base method at all. We have been striving to remove it at some point, but it is very difficult due to all the legacy code which is written.
>
> Almost all the Object base methods need to be removed IMO. You can add them at a higher level if you need them, and then specify your requirements for derived classes.
>
> Including opHash, opCmp, toString, etc.

https://issues.dlang.org/show_bug.cgi?id=9769 https://issues.dlang.org/show_bug.cgi?id=9770 https://issues.dlang.org/show_bug.cgi?id=9771 https://issues.dlang.org/show_bug.cgi?id=9772

This PR would largely fix the opEquals problem (though it doesn't deal with actually removing opEquals from Object, just making it so that you don't need it):

https://github.com/dlang/druntime/pull/1439

However, it's just languished (probably because it's in druntime and not terribly interesting). But actually removing opEquals and replacing the others gets a lot more interesting (e.g. we have to finishing templatizing the built-in AAs as one of the steps), so as critical as that PR is, it's only one small step.

- Jonathan M Davis

On Saturday, 23 July 2016 at 17:27:24 UTC, Lodovico Giaretta wrote:
> On Saturday, 23 July 2016 at 17:04:42 UTC, Jonathan Marler wrote:
>> On Saturday, 23 July 2016 at 16:46:20 UTC, Jonathan Marler wrote:
>>> [...]
>>
>> Actually Im going to disagree with myself. This technique actually wouldn't work with virtual methods:)
>
> I don't think we have the big problems with @nogc that people points out.
>
> I mean, we cannot decide that specific methods or opXXX must always be @nogc. That's too restrictive.
>
> So, what we need to do is:
>
> - use templates: with them, we can have our algorithms be @safe when applied to @safe types, @nogc when applied to @nogc types, and so on; for example, instead of taking a specific delegate type, we shall always accept a generic type, and use traits to guarantee it is some delegate; in this way, we can accept @safe delegate and propagate @safety to our algorithm, or accept @system and have our algorithm usable in @system code; same with @nogc et al.
>
> - when we use virtual methods, we are giving up all compiler-checked attributes; in this situation we have two options:
>     - we trust what we are doing: e.g. we cannot mark a thing @nogc, but we know it is and the profiler confirms that no allocation happens, so we are happy; our aim is having code that doesn't freeze because of collections, and not marking code @nogc.

This is bad. It only creates a faulty foundation. The whole point of nogc is to enforce nogc behavior. If you don't use it your not enforcing anything and then things slip by only to create problems later. This mentality is completely wrong and leads to decay. This is exactly why we are discussing this right now, because someone decided that it was ok to ignore other use cases which eventually turn out to be quite important.

If, say, D was built without the GC, then adding the GC would be much easier and more logical than what has been done, which is built with GC and trying to remove it.

>     - we must have @nogc (or @whatever): then we know we cannot use certain classes, because they are definitely @nogc; so we cast the objects we get to the classes/interfaces that we know are @nogc (and are marked as such), and then our code is @nogc; as you see, you don't need Object to have @nogc methods; you only need the specific classes you use have it; if you want to work on generic objects, and as such cannot do specific casts, then you should definitely be using templates.
>
> The only real problems I found till now are:
> - some things in Phobos that shall be @nogc are not; they shall be refactored to have that attribute (but this is not always easy and requires time)
> - the interface IAllocator is mostly used with @nogc allocators, but cannot have the said attribute (as I explained above, it must not restrict the possibility of allocators); it shall have a sub-interface NoGCAllocator, so that code can accept a @nogc allocator parameter without using templates (of course templates are fine, and are what we use now, but if everyone uses templates, why have IAllocator in the first place? IMHO we must have or both or none).

Templates are not the end all be all. They don't allow for run-time polymorphism, which is an important aspect of software. What we should have is that when something is added or removed we are 100%(or close as we can get) that the feature is correct and causes no side effects. D's GC causes side effects because it was assumed that there would not be any.  Now the future is hear and it's up to us to try and unravel the mess the past created. It's always best to just do it right in the first place, it saves everyone from the headache.

DMD should probably be branched, and all GC stuff removed, then built back up to have the proper features that the GC version has. I think someone has essentially done this on their own, but never built it up to full capacity.

On Saturday, July 23, 2016 21:33:29 Rufus Smith via Digitalmars-d-learn wrote:
> I am trying to write some general code that works on arbitrary types. I need to compare, obviously, as that is relatively basic thing on objects.

That's part of of why attribute inferrence works on templates. That way, the template isn't constrained with regards to which attributes it uses. If it can be nothrow or @nogc or whatnot, then the compiler will infer it as such, and it can be used in code that requires those attributes, but if the template can't have those attributes with a particular set of template arguments, then they won't be inferred, and the template will still work, whereas if the attributes had been put on the template, it wouldn't.

Templates are straight-up the solution for dealing with an arbitrary mix
of attributes. Unfortunately, virtual functions are fundamentally
incompatible with templates, so even if the Object is fixed so that it
doesn't have opEquals, opCmp, toHash, and toString on it, the ones that are
declared on derived classes are still going to have to pick a set up
attributes and will restrict what further derived classed do. It's just that
the programmer will then have been able to choose those restrictions in
their base class rather than being forced by what druntime did with Object.
Classes and generic code really don't mix well and can't mix well when the
generic code needs to be part of the class instead of a free function.

- Jonathan M Davis

On Saturday, 23 July 2016 at 21:44:05 UTC, Rufus Smith wrote:
> Templates are not the end all be all. They don't allow for run-time polymorphism, which is an important aspect of software.

Ok, so you need runtime polymorphism. And you want it in @nogc code. That's not difficult. Just have the base class of your hierarchy declare its opXXX @nogc. This is possible, because non-@nogc functions can be overridden by @nogc ones (as logical). Now you can have your function accept parameters of your base class, and it will be @nogc. And without forcing every class of the D world to have opXXX @nogc.
I don't see why this wouldn't be enough...

On Saturday, 23 July 2016 at 21:44:05 UTC, Rufus Smith wrote:
> On Saturday, 23 July 2016 at 17:27:24 UTC, Lodovico Giaretta wrote:
>>     - we trust what we are doing: e.g. we cannot mark a thing @nogc, but we know it is and the profiler confirms that no allocation happens, so we are happy; our aim is having code that doesn't freeze because of collections, and not marking code @nogc.
>
> This is bad. It only creates a faulty foundation. The whole point of nogc is to enforce nogc behavior. If you don't use it your not enforcing anything and then things slip by only to create problems later. This mentality is completely wrong and leads to decay.

While you are right on this, we must be pragmatical: we currently do not enforce that pointers point to valid memory locations, that private members are not accessed outside the module with tricky arithmetics, we have tons of things that work by convention, because the compiler simply can't check them all and some are even intrinsically uncheckable. In this environment of "code by trust", @nogc is the least of the problems, also because, I insist, it can be checked with the profiler, or you can plug a custom GC to druntime that asserts every time you try to call its functions (work is being made to make the GC independent and pluggable).

> This is exactly why we are discussing this right now, because someone decided that it was ok to ignore other use cases which eventually turn out to be quite important.

Well, I think that deciding that opXXX must always be @nogc, ignoring other use cases that may need the GC, is blatantly wrong. By asking that Object.opXXX be @nogc, you are making the error the error you said others made.

On Saturday, 23 July 2016 at 22:48:07 UTC, Lodovico Giaretta wrote:
> On Saturday, 23 July 2016 at 21:44:05 UTC, Rufus Smith wrote:
>> On Saturday, 23 July 2016 at 17:27:24 UTC, Lodovico Giaretta wrote:
>>>     - we trust what we are doing: e.g. we cannot mark a thing @nogc, but we know it is and the profiler confirms that no allocation happens, so we are happy; our aim is having code that doesn't freeze because of collections, and not marking code @nogc.
>>
>> This is bad. It only creates a faulty foundation. The whole point of nogc is to enforce nogc behavior. If you don't use it your not enforcing anything and then things slip by only to create problems later. This mentality is completely wrong and leads to decay.
>
> While you are right on this, we must be pragmatical: we currently do not enforce that pointers point to valid memory locations, that private members are not accessed outside the module with tricky arithmetics, we have tons of things that work by convention, because the compiler simply can't check them all and some are even intrinsically uncheckable. In this environment of "code by trust", @nogc is the least of the problems, also because, I insist, it can be checked with the profiler, or you can plug a custom GC to druntime that asserts every time you try to call its functions (work is being made to make the GC independent and pluggable).
>
>> This is exactly why we are discussing this right now, because someone decided that it was ok to ignore other use cases which eventually turn out to be quite important.
>
> Well, I think that deciding that opXXX must always be @nogc, ignoring other use cases that may need the GC, is blatantly wrong. By asking that Object.opXXX be @nogc, you are making the error the error you said others made.

This just isn't right. What your saying is that because someone screwed up, we must live with the screw up and build everyone around the screw up. This mentality is why everyone is so screwed up in the first place, do you not see that?

And I think you really have a misconception about the GC vs nogc. One can rewrite GC code, such as an GC based opEquals, without limitations. They can allocate on the stack, use malloc and free when done, etc. opEquals generally doesn't have state. So you it is possible to around around. It's probably always possible to rewrite a GC opEquals to use nogc without too much difficulty.

BUT, it is impossible to use a GC opEquals in nogc code! This means there is no work around. What you claim is that we accept the impossiblity(which makes nogc useless) just to avoid having to rewrite some GC opEquals code. We can't rewrite the nogc side, it's set in stone. We are screwed from the start when we attempt to do nogc code because at some point we will have to do comparisons. It's the same problem with purity and any other transitive relationship.

All "workarounds" are just as limited because basically we added the relationship if A is nogc and A uses B, then B must be nogc. Yet, we start with B is GC. Hence we never ever have A use B, because A's can only use nogc.

To see it simpler, What if everything in D was GC based. All code was marked GC(even statements, types, etc). Do you agree that nogc would be absolutely useless? We couldn't build up anything because we couldn't include any code. This is the extreme case.

Conversely, if everything was built up using nogc, we could write GC based code just fine, could we not?

Therefore, claiming that we stay with GC based code just prevents using more and more nogc code. The more GC based code D gets, the less useful nogc gets and we are back were we started.

Since nogc is more critical in the foundational layers, as it affects everything built on it, all core features should be nogc. This way, the user isn't can decide when to break away from the GC code, which will only affect everything after that point.

This is a one way street, pretending it is two way only enriches the lawyers and eventually makes everyone unhappy. Making the D dependent on the GC was a mistake that many wasted man hours will go in to trying to unravel.  Trying to carry on this mistake just wastes more hours.

I understand that it is a mess, but it got that way from the mistake in the first place, not from trying to undo the mistake(which is illogical because there would be no nogc if there wasn't a gc in the first place).  I also understand that there is some desire to keep things "backwards compatible". This is also a mistake, not only does it prolong the pain and suffering but is irrational. 1. A fork can be made. Those people that have based their code on the GC can continue using an older version. Their code works at that point, does it not? So just stop going down that dead end path. They have what they need, it's not like they will lose anything(virtually nothing except in most cases). Moving in the correct direction is always better, regardless of who's panties get in a wad.

On Sunday, 24 July 2016 at 02:17:27 UTC, Rufus Smith wrote:
> On Saturday, 23 July 2016 at 22:48:07 UTC, Lodovico Giaretta wrote:
>> [...]
>
> This just isn't right. What your saying is that because someone screwed up, we must live with the screw up and build everyone around the screw up. This mentality is why everyone is so screwed up in the first place, do you not see that?
>
> [...]

I pretty much agree and had the same thoughts you've expressed here Rufus. Your arguments are logical and make sense. However, I can already tell you this kind of a change is going to elicit alot of negative feedback. I think you're gonna find yourself frustrated in a losing battle trying to get the community to see reason. I hope Im wrong, but know you got me on your side.

On Sunday, 24 July 2016 at 02:17:27 UTC, Rufus Smith wrote:
> This just isn't right. What your saying is that because someone screwed up, we must live with the screw up and build everyone around the screw up. This mentality is why everyone is so screwed up in the first place, do you not see that?
>
> And I think you really have a misconception about the GC vs nogc. One can rewrite GC code, such as an GC based opEquals, without limitations. They can allocate on the stack, use malloc and free when done, etc. opEquals generally doesn't have state. So you it is possible to around around. It's probably always possible to rewrite a GC opEquals to use nogc without too much difficulty.

Now you are telling me to "program by trust", because there's nothing ensuring that I remember to free everything I allocated with malloc/free, while a GC would guarantee no memory leaks. Again there's nothing stopping me from returning pointers to things allocated on the stack. And now there are lots...
Before you told me that programming by trust is a wrong attitude, and now you propose me to use it, risking memory leakage in a function that may be executed hundreds of times per second.

> BUT, it is impossible to use a GC opEquals in nogc code! This means there is no work around. What you claim is that we accept the impossiblity(which makes nogc useless) just to avoid having to rewrite some GC opEquals code. We can't rewrite the nogc side, it's set in stone. We are screwed from the start when we attempt to do nogc code because at some point we will have to do comparisons. It's the same problem with purity and any other transitive relationship.
>
> All "workarounds" are just as limited because basically we added the relationship if A is nogc and A uses B, then B must be nogc. Yet, we start with B is GC. Hence we never ever have A use B, because A's can only use nogc.
>
> To see it simpler, What if everything in D was GC based. All code was marked GC(even statements, types, etc). Do you agree that nogc would be absolutely useless? We couldn't build up anything because we couldn't include any code. This is the extreme case.
>
> Conversely, if everything was built up using nogc, we could write GC based code just fine, could we not?

No. If you put a big @nogc attribute on Object.opXXX, then nobody can write GC code in his classes. So if everything is @nogc, you cannont write GC code, because it woudn't interact with Phobos. Example: if you mark an algorithm that takes a delegate @nogc, then you cannot pass GC delegates to it. So you cannot use it in GC code.

> Therefore, claiming that we stay with GC based code just prevents using more and more nogc code. The more GC based code D gets, the less useful nogc gets and we are back were we started.
>
> Since nogc is more critical in the foundational layers, as it affects everything built on it, all core features should be nogc. This way, the user isn't can decide when to break away from the GC code, which will only affect everything after that point.

Yes. All building blocks must be as much @nogc as possible. But customization points (virtual functions, delegate arguments, ...) must not be @nogc, otherwise it is no possible to have classes that use the GC or callbacks that use the GC.

> This is a one way street, pretending it is two way only enriches the lawyers and eventually makes everyone unhappy. Making the D dependent on the GC was a mistake that many wasted man hours will go in to trying to unravel.  Trying to carry on this mistake just wastes more hours.
>
> I understand that it is a mess, but it got that way from the mistake in the first place, not from trying to undo the mistake(which is illogical because there would be no nogc if there wasn't a gc in the first place).  I also understand that there is some desire to keep things "backwards compatible". This is also a mistake, not only does it prolong the pain and suffering but is irrational. 1. A fork can be made. Those people that have based their code on the GC can continue using an older version. Their code works at that point, does it not? So just stop going down that dead end path. They have what they need, it's not like they will lose anything(virtually nothing except in most cases). Moving in the correct direction is always better, regardless of who's panties get in a wad.

I still don't understand why you want Object.opXXX @nogc. As I already said, you can still make your functions @nogc, just accepting parameters of @nogc types. It's obvious. If I wrote a wonderful library that uses the GC, you will not use it. If I have a class that uses the GC in opXXX (and I am free to have it, because maybe I need it, and maybe it's the most efficient way for my use case), you will not use it. The same applies here. You'll have your algorithms work only on classes that declare opXXX as @nogc.

Not all memory allocation patterns are good for malloc/free. Not all of them are good for stack allocations. Some of them are not even good for reference counting. Every class shall use the best solution for its job. And everybody must still be able to extend the base class.
If you want to use a method specific to a subclass, you downcast. If you want to use the @nogc opXXX when the base does not enforce it, you downcast. It's the same principle: more advanced functionalities require more derived types (and @nogc is more derived, because it is covariant to not-@nogc). Basic OOP.

On Sunday, 24 July 2016 at 09:03:04 UTC, Lodovico Giaretta wrote:
> On Sunday, 24 July 2016 at 02:17:27 UTC, Rufus Smith wrote:
>> [...]
>
> Now you are telling me to "program by trust", because there's nothing ensuring that I remember to free everything I allocated with malloc/free, while a GC would guarantee no memory leaks. Again there's nothing stopping me from returning pointers to things allocated on the stack. And now there are lots...
> Before you told me that programming by trust is a wrong attitude, and now you propose me to use it, risking memory leakage in a function that may be executed hundreds of times per second.
>
>> [...]
>
> No. If you put a big @nogc attribute on Object.opXXX, then nobody can write GC code in his classes. So if everything is @nogc, you cannont write GC code, because it woudn't interact with Phobos. Example: if you mark an algorithm that takes a delegate @nogc, then you cannot pass GC delegates to it. So you cannot use it in GC code.
>
>> [...]
>
> Yes. All building blocks must be as much @nogc as possible. But customization points (virtual functions, delegate arguments, ...) must not be @nogc, otherwise it is no possible to have classes that use the GC or callbacks that use the GC.
>
>> [...]
>
> I still don't understand why you want Object.opXXX @nogc. As I already said, you can still make your functions @nogc, just accepting parameters of @nogc types. It's obvious. If I wrote a wonderful library that uses the GC, you will not use it. If I have a class that uses the GC in opXXX (and I am free to have it, because maybe I need it, and maybe it's the most efficient way for my use case), you will not use it. The same applies here. You'll have your algorithms work only on classes that declare opXXX as @nogc.
>
> Not all memory allocation patterns are good for malloc/free. Not all of them are good for stack allocations. Some of them are not even good for reference counting. Every class shall use the best solution for its job. And everybody must still be able to extend the base class.
> If you want to use a method specific to a subclass, you downcast. If you want to use the @nogc opXXX when the base does not enforce it, you downcast. It's the same principle: more advanced functionalities require more derived types (and @nogc is more derived, because it is covariant to not-@nogc). Basic OOP.

I believe Rufus was only referring to the virtual methods defined in the object class.  That would be:

toHash (Note: this is already nothrow, that's intersting and quite restrictive)
opCmp
opEquals

I think all 3 of these are good candidates for @nogc.  However, AFAIK, making them @nogc would break any code that implements them because they would have to add the @nogc attribute to their implementations (unless I am mistaken?  Do subclass overrides need to explicitly have @nogc if the parent class does?).  If adding @nogc is not required in the implementation, then the only code that would break would be implementations that actually do allocate GC memory.

Some would think that restricting GC usage inside these virtual methods is a good thing because it has the benefit of discouraging memory allocation for these types of operations.  Really, you probably shouldn't be allocating memory to perform a comparison.  If you really need to, you can either allocate non GC memory, or use a different mechanism then the opCmp/opEquals methods.

On Sunday, 24 July 2016 at 14:54:11 UTC, Jonathan Marler wrote:
> I believe Rufus was only referring to the virtual methods defined in the object class.  That would be:
>
> toHash (Note: this is already nothrow, that's intersting and quite restrictive)
> opCmp
> opEquals
>
> I think all 3 of these are good candidates for @nogc.  However, AFAIK, making them @nogc would break any code that implements them because they would have to add the @nogc attribute to their implementations (unless I am mistaken?  Do subclass overrides need to explicitly have @nogc if the parent class does?).  If adding @nogc is not required in the implementation, then the only code that would break would be implementations that actually do allocate GC memory.
>
> Some would think that restricting GC usage inside these virtual methods is a good thing because it has the benefit of discouraging memory allocation for these types of operations.  Really, you probably shouldn't be allocating memory to perform a comparison.  If you really need to, you can either allocate non GC memory, or use a different mechanism then the opCmp/opEquals methods.

Yes, making them @nogc would require all existing overrides to be changed (overrides cannot throw away attributes, but must specify them explicitly as in the parent class, or stricter).
The real problem making these @nogc is that with the current state of things @nogc implies nothrow, thus preventing exceptions (of course we should work to change this thing, and I'm personally researching convenient ways of doing it; I'll soon write a post on General about this).

Remember that comparison of complex objects may require normalization, which may change the objects themselves and allocate memory. Also, comparisons may throw exceptions that need the GC (see above). So I'm personally against making those methods @nogc.

But I'm also against a singly-rooted hierarchy. Removing Object and having multiple class hierarchies would entirely solve the issue. But please note that you can already "do" that: if you never use Object, but always subclasses, the fact that Object isn't @nogc is no longer an issue. While Object (if it exists) must support any kind of descendant (and thus cannot pose arbitrary limitations like @nogc is), the roots of domain-specific hierarchies can exploit the fact that they are domain-specific to impose meaningful limitations, based on the expected usage and behaviour, and can thus be @nogc. And you can either limit your algorithm inputs to objects of a specific hierarchy or, if you want it generic, use a template (read as: you don't use Object explicitly, as if it doesn't exists).