On 06/06/2013 07:10 PM, Rob T wrote:
> My understanding is that final on it's own would be an error if the same named function was virtual in the base class, otherwise you would have to specify "final override". If that understanding is not correct, then I agree that would be a source of hidden errors.

The problem as I understand it is as follows.

First, consider this scenario: there's a base class.  You've created a subclass in it with a method, foo(), which is not in the base class.  Now, suppose that the base class adds its own foo(), which is virtual.

Now, your subclass will fail to compile [*] unless you tweak its own foo(), and
apart from renaming foo() in your subclass, you have only two options: (i) to
override the base class foo() or (ii) to "hide" or replace the base class foo().
 So, the language should require you to be unambiguous and indicate explicitly
which.  Obviously "override" or "final override" will do for the first case.

But what if you allow "final" to be the keyword for the second, "hide the base method" case?  (This is my understanding of the proposal I replied to.)

If you do, then consider the alternative scenario.  You have a base class with a virtual method bar().  You _mean_ to override it with a final method, but accidentally type just "final" and not "final override".  The compiler will accept this without warning, because it takes "final" to mean your explicit indication to "hide the base class method", and it may take you some time to realize that the base class method is incorrectly being hidden instead of overridden.

So, that's why I suggest an alternative keyword to indicate the programmer's desire to hide rather than override the base class method, and C#'s choice of "new" seems as good as any.

That's assuming that we want to allow base class methods to be hidden, but the
case made by C# is fairly compelling:
http://forum.dlang.org/thread/yzsqwejxqlnzryhrkfuq@forum.dlang.org?page=24#post-op.wx8biyx7eav7ka:40stevens-macbook-pro.local


[* Technically it need not fail to compile but the compiler would have to assume a default behaviour and warn you in the absence of an explicit indicator of your desires.  In C# the default is to hide rather than override the base class method, and it warns you if you don't explicitly indicate "override" or "new".]

On Friday, 7 June 2013 at 07:31:52 UTC, Walter Bright wrote:
> deadalnix mentioned enforcing 'export' on classes exposed to shared libraries, but then aren't we back to expecting user annotation rather than doing things automatically?

Yes, let me explain more.

The export problem isn't new, and you'll find 2 major approach : the UNIX approach of export everything, and the windows approach of requiring explicit export.

The first approach tend to show its limit. It limit what the optimizer can do, and create binary more and more costly to link (especially since the number of symbol tends to explode with metaprogramming) (linking a debug version of LLVM as shared object for instance is quite costly).

Both GCC and clang propose language extensions to manage export explicitly on UNIXes.

Additionally, it is beneficial to be very picky about what is exported. Once something is exported, you are tied to an API (as with public) but also to an ABI. Adding a function to a class for instance, can cause a relayout of the vtable. None of the existing interface is broken, but the shared object is broken anyway, because of symbols it don't use.

Considering this, we need either a way to export explicitly, or the other way around. It is also worth noting that consistency accross plateforms is a plus, and this is an area where we can do better that C (and we already do).

Regarding virtual in particular, it is known that calling a virtual method have a cost. It cause an indirect branch, but also prevent the compiler from optimizing, as the call is opaque. The same way, calling to a shared object is opaque to the compiler, and so, you can't get the full benefice of finalizing the exported function.

Requiring classes to be exported provide the following benefit :
 - LTO can finalize methods that aren't overridden. It include function that you want virtual by design, but you don't use that capability is your specific situation. In this regard, this is superior to the explicit virtual solution as it do not require to annotate virtual and can finalize method that would have to be annotated virtual.
 - Shared object can be stripped of all non exported symbols, reducing their size.
 - Reduce the surface area of breakage for shared objects.

It however require careful crafting of exported objects. I think this is mitigated by the fact that shared object interface require careful crafting anyway, as what is exported cannot be unexported (and cannot even change its ABI in way that aren't easily visible in the source).

The solution can be completed later by a tool as Andrei proposed (great idea), but by itself provide a lot of opportunity to finalize automagically. As you argued, this is the best way to go.

Executive summary :
 - We can provide a toll to finalize the whole program.
 - We can automatically finalize everything that isn't exported.
 - We don't break any code.
 - We get consistency between windows and UNIXes.
 - Manu will rant.

I see only benefits :D

On 6/7/13 3:42 AM, Walter Bright wrote:
> On 6/6/2013 10:14 PM, Andrei Alexandrescu wrote:
>> I don't buy all that "humans aren't rational" stuff.
>
> I don't either, but I don't think that was my point.
>
> My point is that a language where -O delivers the performance is
> preferable to a language that you have to do careful annotations and run
> extra tools on to get it.
>
> We discussed a while back an experimental Java compiler where
> annotations were used to signify uniqueness, etc. The compiler was a
> technical success, but the researcher was not very successful in getting
> users to use the annotations.
>
> This is why I am pushing for D doing attribute inference as much as
> possible, rather than expecting people to annotate carefully. I believe
> we are much more likely to be successful with the former approach.
>
> I'd very much prefer an auto-finalize mechanical solution.
>
> BTW, dustmite and rdmd are great tools. I'm happy to have been proven
> wrong about them.

The more I think of this the more I regress toward my initial opinion: just let it be. I understand the arguments involved and it is my opinion that the problem is overstated (it's just a default, not something that prevents people from doing what they want to do), the benefits are exaggerated (we're not going to see whopping performance improvements), and the costs are underplayed (let's not forget that many breaking changes look great _before_ the breakage).

We need to think holistically. If improving performance of generated code is a goal of ours, then there are so many better, nonbreaking vehicles for that: the backend interface, the inliner, the GC, the standard library, the SIMD builtins and associated library code, and many many more low-hanging fruit of lesser disruption and better returns.


Andrei

On Friday, 7 June 2013 at 07:22:18 UTC, Walter Bright wrote:
>
> A pretty good metric of some feature being used is the frequency it comes up in discussion here and the action it sees in bugzilla.
>
> There are two obvious reasons why a feature would not have much buzz:
>
> 1. it works perfectly
> 2. it isn't being used
>
> Value Range Propagation is a pretty good example of (1). Sadly, I suspect -cov is (2).
>
> I'd be happy to be wrong about that.

If D had a compiler option switch to collect statistics on feature usage, maybe you could get something a lot better than a guess for both 1 and 2.

Such a thing may be useful for other reasons too.

--rt

On 6/7/2013 2:52 AM, Timon Gehr wrote:
> You are certainly wrong about the value range propagation part. The transformers
> for the bitwise operators are not the best possible.
>
> ubyte x = ((y&252)^2)+1;
>
> The above term can be easily proven to fit into ubyte by just using an analysis
> of the ranges of its subterms, yet DMD rejects it.

Since it's easy, I expect a pull request from you, or at least a bugzilla entry with a description of the algorithm to use!

On Thursday, 6 June 2013 at 18:31:08 UTC, Walter Bright wrote:
> Ok, Manu, you win, I'm pretty much convinced.

So what does this mean? We got 'virtual' with the next release? :)

On 6/7/2013 9:51 AM, Rob T wrote:
> If D had a compiler option switch to collect statistics on feature usage, maybe
> you could get something a lot better than a guess for both 1 and 2.

And then you'd have to convince people to use it and send you the results!

On 6/7/2013 8:23 AM, deadalnix wrote:
> Yes, let me explain more.

Thanks for the detailed explanation.

On Friday, 7 June 2013 at 23:52:59 UTC, Walter Bright wrote:
> On 6/7/2013 8:23 AM, deadalnix wrote:
>> Yes, let me explain more.
>
> Thanks for the detailed explanation.

If you are looking further material on the topic, you might also want to try a quick web search for '-fvisibility=hidden', which is the GCC flag for hiding symbols by default.

David

On 8 June 2013 01:23, deadalnix <deadalnix@gmail.com> wrote:

> Requiring classes to be exported provide the following benefit :
>  - LTO can finalize methods that aren't overridden. It include function
> that you want virtual by design, but you don't use that capability is your
> specific situation. In this regard, this is superior to the explicit
> virtual solution as it do not require to annotate virtual and can finalize
> method that would have to be annotated virtual.
>  - Shared object can be stripped of all non exported symbols, reducing
> their size.
>  - Reduce the surface area of breakage for shared objects.
>
> It however require careful crafting of exported objects. I think this is mitigated by the fact that shared object interface require careful crafting anyway, as what is exported cannot be unexported (and cannot even change its ABI in way that aren't easily visible in the source).
>
> The solution can be completed later by a tool as Andrei proposed (great idea), but by itself provide a lot of opportunity to finalize automagically. As you argued, this is the best way to go.
>
> Executive summary :
>  - We can provide a toll to finalize the whole program.
>  - We can automatically finalize everything that isn't exported.
>  - We don't break any code.
>  - We get consistency between windows and UNIXes.
>  - Manu will rant.
>
> I see only benefits :D
>

It's like you've missed most of my points though.

So under this proposal, which *entails* said 'sufficiently advanced
optimiser/linker/etc', which doesn't exist. And even if it did, what
architectures are supported?
Many architectures will most certainly never receive support, and these are
usually the architectures that need it most.

Put that aside for a second, it's still flawed in a basic sense.

Anything 'exported' still falls under precisely the same set of problems as
we have now. By exporting something, NOTHING can be de-virtualised, even
when used internally. And exported things DO get used internally. So by
exporting, you lose all benefit within your own code anyway.
I can tell you that in my case, we export a lot(/most) things. Renderer
api, sound api, etc are often all in their own libraries. So none of them
are ever eligible for optimisation.

Additionally, one of my central points is completely un-satisfied, that is, 3rd party libraries. To make use of them at all implies they are exported, so there is never any possibility for optimisation there.

What am I left with? Basically nothing.

And finally, even if none of these problems existed, you still don't want
virtual methods across export boundaries.
Just because you use a class implemented in a DLL, that doesn't prevent you
from inlining the trivial accessors within your own code. Using a library
is an ABI commitment anyway, and it's completely normal to inline foreign
libraries trivial methods within your own code.

So, basically none of my usages are satisfied by your proposal. Sorry.