May 31, 2013
On Friday, May 31, 2013 19:42:55 Manu wrote:
> On 31 May 2013 14:06, deadalnix <deadalnix@gmail.com> wrote:
> > On Friday, 31 May 2013 at 02:56:25 UTC, Andrei Alexandrescu wrote:
> >> On 5/30/13 9:26 PM, finalpatch wrote:
> >>> https://dl.dropboxusercontent.**com/u/974356/raytracer.d<https://dl.drop boxusercontent.com/u/974356/raytracer.d> https://dl.dropboxusercontent.**com/u/974356/raytracer.cpp<https://dl.d ropboxusercontent.com/u/974356/raytracer.cpp>>>
> >> Manu's gonna love this one: make all methods final.
> > 
> > I don't think going as far as making thing final by default make sense at this point. But we sure need a way to be able to finalize methods. We had an extensive discussion with Don and Manu at DConf, here are some idea that
> > 
> > came out :
> > - Final by default
> > 
> > This one is really a plus when it come to performance code. However,
> > virtual by default have proven itself very useful when performance isn't
> > that big of a deal (and it is the case for 90% of a program's code
> > usually)
> > and limiting the usage of some pattern like decorator (that also have been
> > proven to be useful). This is also huge breakage.
> 
> The correct choice :)
> 
> > virtual by default have proven itself very useful when performance isn't
> 
> that big of a deal
> 
> Has it? I'm not sure it's ever proven its self useful, can you point to any such proof, or evidence?

It's useful when you don't have override, because you avoid the problem of overriding non-virtual functions. So, going with virtual by default solves that nasty bug, but enforcing override _also_ solves it. So IMHO, requiring override (which we already do) solves the problem, making it pointless to make member functions virtual by default. At that point, it just makes more sense to make them non-virtual by default and avoid the performance issues caused by virtual by default. The _only_ thing that it costs you is that it forces you to put virtual on all functions which are going to be overridden, which is even fewer functions than we have to mark with override (because multiple classes can override whereas only the base class function would be marked with virtual).

> There's another detail that came up late at night at dconf; I was talking to Daniel Murphy about the extern(C++) class work he's working on to write the D-DMD front end, and apparently it would make the problem a lot easier if virtual was explicit there too. So perhaps it also offers improved interoperability with C++, which I think most of us have agreed is of heightened importance recently.

Yes. I intend to create a DIP for non-virtual by default (assuming that no one else does first), and I think that this issue gives even more weight to why we need to make the change.

> > This is also huge breakage.
> 
> I think this is another fallacy. The magnitude of the breakage is already
> known. It's comparable to the recent change where 'override' became an
> explicit requirement, which was hardly catastrophic.
> In that case, the breakage occurred in *every derived class*, which is a
> many:1 relationship to base classes.
> Explicit virtual would only affect _base_ classes. The magnitude of which
> is far smaller than what was recently considered acceptable, precisely:
> magnitudeOfPriorBreakage / numberOfDerivedClasses.
> 
> So considering explicit override was an argument won, I'd like to think that explicit virtual deserves the same treatment for even more compelling reasons, and at a significantly lesser cost in breakage.

Yes. The breakage is surprisingly small, and the change is arguably critical for both performance-critical code and for moving the compiler frontend to D.

- Jonathan M Davis
May 31, 2013
On 05/31/2013 05:12 PM, bearophile wrote:
> There are many kinds of D code, not everything is a high performance ray-tracer or 3D game. So I'm sure there are many many situations where using the C++ STL is more than enough. As most tools, you need to know where and when to use them. So it's not a Satan-spawn :-)

It's the Microsoft Office of programming solutions -- people get taught that this is the tool to use and don't get adequately taught about the alternatives or the rationale ...

I well remember my first experience with STL as a new postgraduate, _very_ inexperienced at programming (what's changed? :-P ) and having learned basically a small part of C.  I was writing some code where I needed at each time step to collect a list of indices of array entries whose value would change.

The only way I could think of how to do this in C was to use malloc to create an array of length N, and to store the actual number of indices stored at a given time step as a size_t which got reset to zero at the start of each step.  So, I could store as many indices as there were to list; the malloc was done at the start of the whole process, outside the loop of time steps.

Then I talked with a (much more experienced) colleague about the design of my code and this thing I was doing with a set of data that started from empty every time and built up the list of indices, and he said, "Oh, try using the vector type in C++."

So, I went, and discovered a bit about how it worked, and rewrote all my code using vectors in place of arrays and other nice C++ stuff, and I managed to come up with code that was much, much easier to read, and was ten zillion times slower.  (I can't remember now, but I suspect that my use of vector was carrying out lots of unnecessary allocs and frees within the core loop.)

I'd surely do much better these days (e.g. I'd make sure to define the vector outside the loop and to reserve the exact amount of memory required up front), but it was a nice lesson in how clever, generic-looking tools can be a big problem if you don't understand how to use them.
May 31, 2013
On 5/31/2013 4:42 AM, Manu wrote:
>
> People already have to type 'override' in every derived class, and
> they're happy to do that. Requiring to type 'virtual' in the base is
> hardly an inconvenience by contrast. Actually, it's quite orthogonal.
> D tends to prefer being explicit. Why bend the rules in this case,
> especially considering the counterpart (override) is expected to be
> explicit? Surely both being explicit is what people would expect?
>

Maybe the solution is to make everyone equally unhappy:

all (non constructor) class methods must be either final, override, or virtual, if you leave one of these off, you get an error :)


May 31, 2013
On Friday, 31 May 2013 at 19:17:05 UTC, Sean Cavanaugh wrote:
> On 5/31/2013 4:42 AM, Manu wrote:
>>
>> People already have to type 'override' in every derived class, and
>> they're happy to do that. Requiring to type 'virtual' in the base is
>> hardly an inconvenience by contrast. Actually, it's quite orthogonal.
>> D tends to prefer being explicit. Why bend the rules in this case,
>> especially considering the counterpart (override) is expected to be
>> explicit? Surely both being explicit is what people would expect?
>>
>
> Maybe the solution is to make everyone equally unhappy:
>
> all (non constructor) class methods must be either final, override, or virtual, if you leave one of these off, you get an error :)

Might not be so bad

class Foo {
final:
    foo();
    boo();
override:
    fun();
    kun();
virtual:
    baz();
    bar();
}
May 31, 2013
On 1 June 2013 01:12, bearophile <bearophileHUGS@lycos.com> wrote:

> Manu:
>
>
>  Frankly, this is a textbook example of why STL is the spawn of satan. For
>> some reason people are TAUGHT that it's reasonable to write code like this.
>>
>
> There are many kinds of D code, not everything is a high performance ray-tracer or 3D game. So I'm sure there are many many situations where using the C++ STL is more than enough. As most tools, you need to know where and when to use them. So it's not a Satan-spawn :-)
>

So why are we having this conversation at all then if faster isn't better in this instance?


May 31, 2013
On 1 June 2013 02:32, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Friday, May 31, 2013 19:42:55 Manu wrote:
> > On 31 May 2013 14:06, deadalnix <deadalnix@gmail.com> wrote:
> > > On Friday, 31 May 2013 at 02:56:25 UTC, Andrei Alexandrescu wrote:
> > >> On 5/30/13 9:26 PM, finalpatch wrote:
> > >>> https://dl.dropboxusercontent.**com/u/974356/raytracer.d<
> https://dl.drop
> > >>> boxusercontent.com/u/974356/raytracer.d> https://dl.dropboxusercontent.**com/u/974356/raytracer.cpp<
> https://dl.d
> > >>> ropboxusercontent.com/u/974356/raytracer.cpp>>>
> > >> Manu's gonna love this one: make all methods final.
> > >
> > > I don't think going as far as making thing final by default make sense
> at
> > > this point. But we sure need a way to be able to finalize methods. We
> had
> > > an extensive discussion with Don and Manu at DConf, here are some idea that
> > >
> > > came out :
> > > - Final by default
> > >
> > > This one is really a plus when it come to performance code. However, virtual by default have proven itself very useful when performance
> isn't
> > > that big of a deal (and it is the case for 90% of a program's code
> > > usually)
> > > and limiting the usage of some pattern like decorator (that also have
> been
> > > proven to be useful). This is also huge breakage.
> >
> > The correct choice :)
> >
> > > virtual by default have proven itself very useful when performance
> isn't
> >
> > that big of a deal
> >
> > Has it? I'm not sure it's ever proven its self useful, can you point to
> any
> > such proof, or evidence?
>
> It's useful when you don't have override, because you avoid the problem of
> overriding non-virtual functions. So, going with virtual by default solves
> that nasty bug, but enforcing override _also_ solves it. So IMHO, requiring
> override (which we already do) solves the problem, making it pointless to
> make
> member functions virtual by default. At that point, it just makes more
> sense
> to make them non-virtual by default and avoid the performance issues
> caused by
> virtual by default. The _only_ thing that it costs you is that it forces
> you
> to put virtual on all functions which are going to be overridden, which is
> even fewer functions than we have to mark with override (because multiple
> classes can override whereas only the base class function would be marked
> with
> virtual).
>
> > There's another detail that came up late at night at dconf; I was talking to Daniel Murphy about the extern(C++) class work he's working on to
> write
> > the D-DMD front end, and apparently it would make the problem a lot
> easier
> > if virtual was explicit there too. So perhaps it also offers improved interoperability with C++, which I think most of us have agreed is of heightened importance recently.
>
> Yes. I intend to create a DIP for non-virtual by default (assuming that no
> one
> else does first), and I think that this issue gives even more weight to
> why we
> need to make the change.
>
> > > This is also huge breakage.
> >
> > I think this is another fallacy. The magnitude of the breakage is already
> > known. It's comparable to the recent change where 'override' became an
> > explicit requirement, which was hardly catastrophic.
> > In that case, the breakage occurred in *every derived class*, which is a
> > many:1 relationship to base classes.
> > Explicit virtual would only affect _base_ classes. The magnitude of which
> > is far smaller than what was recently considered acceptable, precisely:
> > magnitudeOfPriorBreakage / numberOfDerivedClasses.
> >
> > So considering explicit override was an argument won, I'd like to think that explicit virtual deserves the same treatment for even more
> compelling
> > reasons, and at a significantly lesser cost in breakage.
>
> Yes. The breakage is surprisingly small, and the change is arguably
> critical
> for both performance-critical code and for moving the compiler frontend to
> D.
>

**applause**
Someone other than me said it, out loud!
This is a magnificent day! :)


May 31, 2013
Manu:

> On 1 June 2013 01:12, bearophile <bearophileHUGS@lycos.com> wrote:
>
>> Manu:
>>
>>
>>  Frankly, this is a textbook example of why STL is the spawn of satan. For
>>> some reason people are TAUGHT that it's reasonable to write code like
>>> this.
>>>
>>
>> There are many kinds of D code, not everything is a high performance
>> ray-tracer or 3D game. So I'm sure there are many many situations where
>> using the C++ STL is more than enough. As most tools, you need to know
>> where and when to use them. So it's not a Satan-spawn :-)
>>
>
> So why are we having this conversation at all then if faster isn't better in this instance?

Faster is better in this instance.
What's wrong is your thinking that the STL as the spawn of Satan in general.

Bye,
bearophile
May 31, 2013
On 1 June 2013 09:15, bearophile <bearophileHUGS@lycos.com> wrote:

> Manu:
>
>  On 1 June 2013 01:12, bearophile <bearophileHUGS@lycos.com> wrote:
>>
>>  Manu:
>>>
>>>
>>>  Frankly, this is a textbook example of why STL is the spawn of satan.
>>> For
>>>
>>>> some reason people are TAUGHT that it's reasonable to write code like this.
>>>>
>>>>
>>> There are many kinds of D code, not everything is a high performance ray-tracer or 3D game. So I'm sure there are many many situations where using the C++ STL is more than enough. As most tools, you need to know where and when to use them. So it's not a Satan-spawn :-)
>>>
>>>
>> So why are we having this conversation at all then if faster isn't better in this instance?
>>
>
> Faster is better in this instance.
> What's wrong is your thinking that the STL as the spawn of Satan in
> general.
>

Ah, but that's because it is ;)
Rule of thumb: never use STL in tight loops. problem solved (well,
mostly)...


June 01, 2013
On Saturday, June 01, 2013 09:04:50 Manu wrote:
> **applause**
> Someone other than me said it, out loud!
> This is a magnificent day! :)

Well, the discussions at dconf convinced me. Certainly, at this point, I think that the only semi-viable excuse for not making functions non-virtual by default is the code breakage that it would cause, and given how surprisingly minimal that is, I think that it's definitely worth it - especially when the kind of folks whose code Walter is most worried about breaking are the guys most interested in the change.

- Jonathan M Davis
June 01, 2013
On 05/31/2013 10:27 PM, Jonathan M Davis wrote:
> On Saturday, June 01, 2013 09:04:50 Manu wrote:
>> **applause**
>> Someone other than me said it, out loud!
>> This is a magnificent day! :)
> 
> Well, the discussions at dconf convinced me. Certainly, at this point, I think that the only semi-viable excuse for not making functions non-virtual by default is the code breakage that it would cause, and given how surprisingly minimal that is, I think that it's definitely worth it - especially when the kind of folks whose code Walter is most worried about breaking are the guys most interested in the change.
> 
> - Jonathan M Davis
> 

Making everything final by default would IMO kind of break
automated mock classes generation for unit testing,
automatic proxy class generation for DB entities, and
other OOP niceities.

And it wasn't just marking methods final which got the
D version of the raytracer in this thread faster than
the C++ version in the end. (it was a combination of
four or five things, which involved a bit of unrolling,
avoiding array literals, and so on).

--jm