On Wednesday, 10 April 2013 at 04:32:52 UTC, Manu wrote:
> final on the class means you can't derive it anymore (what's the point of a
> class?),

I think that you'd place final on a derived class, not a base class. So it can make perfect sense, although final on the methods of a final class makes little sense so it should probably be a compiler error.

> and the manual final blocks are totally prone to error.
> In my experience, 90% of functions are not (or rather, should not be)
> virtual. The common (and well performing) case should be default.

Believe it or not, but I actually have been in the reverse position more than once, so which way the default should go is debatable. In your case I can certainly see why you'd prefer the opposite as I've done RT programming before and will be doing it again. What could perhaps work is a module level specifier that indicates what the defaults should be as a matter of use case preference, but I can see that going horribly wrong.

The question I have, is why use a class if you do not need to use virtual functions? I think the problem goes a bit deeper than the defaults being opposite of what you want. I expect that you'd probably want struct inheritance or something like that but cannot get it from D?

> Any language with properties can't have virtual-by-default.
> Seriously, .length or any other trivial property that can no longer be
> inlined, or even just called.
> And human error aside, do you really want to have to type final on every
> function? Or indent every line an extra tab level?

Mark your properties as final?

[...]
> I don't want to ban the use of virtual functions. I want to ban the use of
> virtual functions that aren't marked virtual explicitly! ;)
>
> Likewise, I like the GC, I just want to be able to control it.
> Disable auto-collect, explicitly issue collect calls myself at controlled
> moments, and give the collect function a maximum timeout where it will
> yield, and then resume where it left off next time I call it.

I agree 100% and have that need too. I'd go further and also prefer the ability to optionally ban certain language features from use from within selective parts of my code base. As you say, I do not actually want to outright ban the GC or any other language feature (I really do use them!!!), it's only the desire to be able to have much better control over it for the situations that demand precision and certainty.

Having control over D and the GC like what we're taking about in here can turn D into a seriously awesome systems language unlike any other.

--rt

On Wednesday, 10 April 2013 at 04:32:52 UTC, Manu wrote:
> moments, and give the collect function a maximum timeout where it will
> yield, and then resume where it left off next time I call it.

Maximum collect period is perhaps the most significant missing feature of all. Having that alone would probably solve most of the complaints. The other thing is not having any control over when the thing decides to make a run. It runs while allocating large data sets without a reason to do so slowing things down by as much as 3x, although perhaps it has no way to know that there's no chance for anything to be collected without help from the programmer. What I have to do, is disable the GC, do the allocations, then re-enable, but without the maximum timeout, you can get a very large pause after re-enabling in the range of seconds which is completely unacceptable for some applications.

--rt

On 10 April 2013 15:08, Rob T <alanb@ucora.com> wrote:

> On Wednesday, 10 April 2013 at 04:32:52 UTC, Manu wrote:
>
>> final on the class means you can't derive it anymore (what's the point of
>> a
>> class?),
>>
>
> I think that you'd place final on a derived class, not a base class. So it can make perfect sense, although final on the methods of a final class makes little sense so it should probably be a compiler error.
>
>
>  and the manual final blocks are totally prone to error.
>> In my experience, 90% of functions are not (or rather, should not be)
>> virtual. The common (and well performing) case should be default.
>>
>
> Believe it or not, but I actually have been in the reverse position more than once, so which way the default should go is debatable. In your case I can certainly see why you'd prefer the opposite as I've done RT programming before and will be doing it again. What could perhaps work is a module level specifier that indicates what the defaults should be as a matter of use case preference, but I can see that going horribly wrong.
>
> The question I have, is why use a class if you do not need to use virtual functions? I think the problem goes a bit deeper than the defaults being opposite of what you want. I expect that you'd probably want struct inheritance or something like that but cannot get it from D?


I do use virtual functions, that's the point of classes. But most functions
are not virtual. More-so, most functions are trivial accessors, which
really shouldn't be virtual.
OOP by design recommends liberal use of accessors, ie, properties, that
usually just set or return a variable. Wen would you ever want @property
size_t size() { return size; } to be a virtual call?

A base class typically offers a sort of template of something, implementing
as much shared/common functionality as possible, but which you might
extend, or make more specific in some very controlled way.
Typically the base functionality and associated accessors deal with
variable data contained in the base-class.

The only situations I can imagine where most functions would be virtual are
either a) ridiculously small classes with only 2 functions (at least you'll
only type 'virtual' once or twice in this case), b) some sort of OOP-tastic
widget that 'can do anything!' or tries to be a generalisation of an
'anything', which is, frankly, horrible and immature software design, and
basically the entire reason OOP is developing a name as being a terrible
design pattern in the first place...
I wouldn't make the latter case an implicit recommendation through core
language design... but apparently that's just me ;)

No I don't want struct inheritance (although that would be nice! but I'm
okay with aggregating and 'alias this'), that would insist on using 'ref'
everywhere, and you can't create ref locals, so you can't really use
structs conveniently this way.
Classes are reference types, that's the point. I know what a class is, and
I'm happy with all aspects of the existing design, except this one thing.

Can you demonstrate a high level class, ie, not a primitive tool, but the
sort of thing a programmer would write in their daily work where all/most
functions would be virtual?
I can paste almost any class I've ever written, there is usually 2-4
virtuals, among 20-30 functions.


 Any language with properties can't have virtual-by-default.
>> Seriously, .length or any other trivial property that can no longer be
>> inlined, or even just called.
>> And human error aside, do you really want to have to type final on every
>> function? Or indent every line an extra tab level?
>>
>
> Mark your properties as final?
>

That's 90% of the class! You are familiar with OOP right? :) Almost everything is an accessor...

I usually have 2 virtual functions, update() and draw(), or perhaps there's
a purpose specific doWork() type of function to perform the derived
object's designated function, but basically everything else is trivial
accessors, or base class concepts that make absolutely no sense to override.
I also work with lots of C++ middleware, and the virtuals are usually
tightly controlled and deliberately minimised, and there's a good reason
for this too; the fewer virtuals that the user is expected to override, the
simpler it is to understand and work with your class!
Additionally, it's a nice self-documenting feature; at a glance you can see
the virtuals, ie, what you need to do to make use of a 3rd party OOP API.

[...]
>
>> I don't want to ban the use of virtual functions. I want to ban the use of
>>
>> virtual functions that aren't marked virtual explicitly! ;)
>>
>> Likewise, I like the GC, I just want to be able to control it.
>> Disable auto-collect, explicitly issue collect calls myself at controlled
>> moments, and give the collect function a maximum timeout where it will
>> yield, and then resume where it left off next time I call it.
>>
>
> I agree 100% and have that need too. I'd go further and also prefer the ability to optionally ban certain language features from use from within selective parts of my code base. As you say, I do not actually want to outright ban the GC or any other language feature (I really do use them!!!), it's only the desire to be able to have much better control over it for the situations that demand precision and certainty.
>

Precisely.

Having control over D and the GC like what we're taking about in here can
> turn D into a seriously awesome systems language unlike any other.


Correct, it's not quite a systems language while the GC does whatever it wants. But D needs the GC to be considered a 'modern', and generally productive language.

On 08.04.2013 05:12, Manu wrote:
> The GC really needs to be addressed in terms of performance; it can't stop
> the world for milliseconds at a time. I'd be happy to give it ~150us every
> 16ms, but NOT 2ms every 200ms.
> Alternatively, some urgency needs to be invested in tools to help
> programmers track accidental GC allocations.

I'm not sure if these have been proposed already in this long thread, but 2 very small patches could help a lot for realtime applications:

1. a thread local flag to disallow and detect GC allocations
2. a flag per thread to specify that the thread should not be paused by the GC during collections.

The latter would then put the responsibility on the programmer to ensure that no references are mutated in the non-pausing thread that don't exist anywhere else (to avoid the collection of objects used in the realtime thread). As anything in a realtime thread usually has to be pre-allocated anyway, that doesn't put a lot more constraints on it but to ensure having references to the pre-allocated data in other threads or global state.

On 10 April 2013 16:14, Rainer Schuetze <r.sagitario@gmx.de> wrote:

>
>
> On 08.04.2013 05:12, Manu wrote:
>
>> The GC really needs to be addressed in terms of performance; it can't stop
>> the world for milliseconds at a time. I'd be happy to give it ~150us every
>> 16ms, but NOT 2ms every 200ms.
>> Alternatively, some urgency needs to be invested in tools to help
>> programmers track accidental GC allocations.
>>
>
> I'm not sure if these have been proposed already in this long thread, but 2 very small patches could help a lot for realtime applications:
>
> 1. a thread local flag to disallow and detect GC allocations
> 2. a flag per thread to specify that the thread should not be paused by
> the GC during collections.
>
> The latter would then put the responsibility on the programmer to ensure that no references are mutated in the non-pausing thread that don't exist anywhere else (to avoid the collection of objects used in the realtime thread). As anything in a realtime thread usually has to be pre-allocated anyway, that doesn't put a lot more constraints on it but to ensure having references to the pre-allocated data in other threads or global state.
>

It's all rather useless without some powerful tools for tracking down leaks, and unintended allocations though. There will surely be bugs with this idea, and finding them will be a nightmare.

On 10.04.2013 08:26, Manu wrote:
> On 10 April 2013 16:14, Rainer Schuetze <r.sagitario@gmx.de> wrote:
>
>>
>>
>> On 08.04.2013 05:12, Manu wrote:
>>
>>> The GC really needs to be addressed in terms of performance; it can't stop
>>> the world for milliseconds at a time. I'd be happy to give it ~150us every
>>> 16ms, but NOT 2ms every 200ms.
>>> Alternatively, some urgency needs to be invested in tools to help
>>> programmers track accidental GC allocations.
>>>
>>
>> I'm not sure if these have been proposed already in this long thread, but
>> 2 very small patches could help a lot for realtime applications:
>>
>> 1. a thread local flag to disallow and detect GC allocations
>> 2. a flag per thread to specify that the thread should not be paused by
>> the GC during collections.
>>
>> The latter would then put the responsibility on the programmer to ensure
>> that no references are mutated in the non-pausing thread that don't exist
>> anywhere else (to avoid the collection of objects used in the realtime
>> thread). As anything in a realtime thread usually has to be pre-allocated
>> anyway, that doesn't put a lot more constraints on it but to ensure having
>> references to the pre-allocated data in other threads or global state.
>>
>
> It's all rather useless without some powerful tools for tracking down
> leaks, and unintended allocations though. There will surely be bugs with
> this idea, and finding them will be a nightmare.
>

It needs some kind of manual ownership handling that's more or less similar as what you do in C++ right now. IMO it allows to workaround some bad consequences due to the mere existence of the GC without giving up its benefits in other threads.

I don't expect a GC any time soon with pause times acceptable for real time tasks. I'd be happy if they were acceptable for standard interactive programs (hopefully the concurrent version can be ported to Windows aswell).

On Wednesday, 10 April 2013 at 06:03:08 UTC, Manu wrote:
> [...]
>
> I do use virtual functions, that's the point of classes. But most functions
> are not virtual. More-so, most functions are trivial accessors, which
> really shouldn't be virtual.
> OOP by design recommends liberal use of accessors, ie, properties, that
> usually just set or return a variable. Wen would you ever want @property
> size_t size() { return size; } to be a virtual call?

Yes, if you want to change its behavior in a derived class.

One nice feature of properties is that you can trigger actions when assigning/reading from properties.

This is very used in OO GUI and DB code in other languages.

>
> Can you demonstrate a high level class, ie, not a primitive tool, but the
> sort of thing a programmer would write in their daily work where all/most
> functions would be virtual?

I have lots of code from JVM and .NET languages with such examples.

OO code in the enterprise world is a beauty in itself, regardless of the language.

>>> Likewise, I like the GC, I just want to be able to control it.
>>> Disable auto-collect, explicitly issue collect calls myself at controlled
>>> moments, and give the collect function a maximum timeout where it will
>>> yield, and then resume where it left off next time I call it.
>>>
>>
>> I agree 100% and have that need too. I'd go further and also prefer the
>> ability to optionally ban certain language features from use from within
>> selective parts of my code base. As you say, I do not actually want to
>> outright ban the GC or any other language feature (I really do use
>> them!!!), it's only the desire to be able to have much better control over
>> it for the situations that demand precision and certainty.
>>
>
> Precisely.
>
> Having control over D and the GC like what we're taking about in here can
>> turn D into a seriously awesome systems language unlike any other.
>
>
> Correct, it's not quite a systems language while the GC does whatever it
> wants. But D needs the GC to be considered a 'modern', and generally
> productive language.

Maybe something like VisualVM would help as well.

--
Paulo

On Wednesday, 10 April 2013 at 06:14:24 UTC, Rainer Schuetze wrote:
> I'm not sure if these have been proposed already in this long thread, but 2 very small patches could help a lot for realtime applications:
>
> 1. a thread local flag to disallow and detect GC allocations
> 2. a flag per thread to specify that the thread should not be paused by the GC during collections.
>
> The latter would then put the responsibility on the programmer to ensure that no references are mutated in the non-pausing thread that don't exist anywhere else (to avoid the collection of objects used in the realtime thread). As anything in a realtime thread usually has to be pre-allocated anyway, that doesn't put a lot more constraints on it but to ensure having references to the pre-allocated data in other threads or global state.

One concept is abused quite nicely in Erlang VM/GC - there is one gc instance per process(fiber) and it gets own pre-allocated memory pool for all its needs. That memory is prohibited to escape to other processes. When process lifespan is short enough (and it is Erlang Way, spawning hell lot of small processes), true garbage collection is never called, whole memory block is just sent back to pool on process termination.

I have tested it few times and such approach allowed to meet some soft real-time requirements. Don't know if something like that can be abused in D with its fibers and scope storage class.

On Wednesday, 10 April 2013 at 06:03:08 UTC, Manu wrote:
> A base class typically offers a sort of template of something, implementing
> as much shared/common functionality as possible, but which you might
> extend, or make more specific in some very controlled way.
> Typically the base functionality and associated accessors deal with
> variable data contained in the base-class.

I believe that template mixins + structs are much more natural way to express this concept. Basically, if you need inheritance only for code reuse, you don't need inheritance and all polymorphic overhead. D provides some good tools to shift away from that traditional approach. Those can and should be improved, but I think the whole concept "classes are polymorphic virtual reference types, structs are plain aggregates" is very solid and area of struct-only development needs to be explored a bit more.

On 10 April 2013 17:01, Paulo Pinto <pjmlp@progtools.org> wrote:

> On Wednesday, 10 April 2013 at 06:03:08 UTC, Manu wrote:
>
>> [...]
>>
>>
>> I do use virtual functions, that's the point of classes. But most
>> functions
>> are not virtual. More-so, most functions are trivial accessors, which
>> really shouldn't be virtual.
>> OOP by design recommends liberal use of accessors, ie, properties, that
>> usually just set or return a variable. Wen would you ever want @property
>> size_t size() { return size; } to be a virtual call?
>>
>
> Yes, if you want to change its behavior in a derived class.
>

That really shouldn't be encouraged. Give me an example?


One nice feature of properties is that you can trigger actions when
> assigning/reading from properties.
>

That doesn't make the property virtual, that makes the virtual that the property calls virtual. You can't have a derived class redefining the function of a trivial accessor. If it has a side effect that is context specific, then it would call through to a separate virtual. And this would be a controlled and deliberate case, ie, 1 in 100, not the norm.


This is very used in OO GUI and DB code in other languages.


I know, it's an abomination, and the main reason OOP is going out of fashion.


Can you demonstrate a high level class, ie, not a primitive tool, but the
>> sort of thing a programmer would write in their daily work where all/most functions would be virtual?
>>
>
> I have lots of code from JVM and .NET languages with such examples.
>
> OO code in the enterprise world is a beauty in itself, regardless of the language.
>

That's not an exampe. I want to see a class where every function SHOULD be overloaded... That sounds like a nightmare, how can anyone other than the author ever expect to understand it? The fewer and more deliberately controlled the virtuals, the better, by almost every measure I can imagine.