First, presentation:
https://www.youtube.com/watch?v=9OIEZAj243g

Some of this is available in LLVM today, and everything presented here will be in 4.0 . The long story short: ThinLTO can do most of what LTO does but with a price that is much closer to the one of a regular build than the one of a classic LTO build.

LTO optimization can devirtualize all function that do not need to be virtual, and even use profile infos to speculatively devirtualize - aka JVM grade devirtualization.

I would love to see this leveraged to finally put to rest the final vs virtual debate. If we use this tech properly, everything that do not need to be virtual can be finalized - except across shared object, which shouldn't be too much of an issue in practice.

On Sun, 04 Dec 2016 01:36:50 +0000, deadalnix wrote:
> LTO optimization can devirtualize all function that do not need to be virtual, and even use profile infos to speculatively devirtualize - aka JVM grade devirtualization.
> 
> I would love to see this leveraged to finally put to rest the final vs virtual debate. If we use this tech properly, everything that do not need to be virtual can be finalized - except across shared object, which shouldn't be too much of an issue in practice.

Unless someone can think of a compelling reason to shorten virtual function tables, anyway -- final member functions don't appear in vtables, but the linker can't remove them from the list.

On Sunday, 4 December 2016 at 01:36:50 UTC, deadalnix wrote:
> First, presentation:
> https://www.youtube.com/watch?v=9OIEZAj243g
>
> Some of this is available in LLVM today, and everything presented here will be in 4.0 . The long story short: ThinLTO can do most of what LTO does but with a price that is much closer to the one of a regular build than the one of a classic LTO build.
>
> LTO optimization can devirtualize all function that do not need to be virtual, and even use profile infos to speculatively devirtualize - aka JVM grade devirtualization.
>
> I would love to see this leveraged to finally put to rest the final vs virtual debate. If we use this tech properly, everything that do not need to be virtual can be finalized - except across shared object, which shouldn't be too much of an issue in practice.

IMHO it was always obvious that the proper way is "devirtualization" within compiler optimization process. The distinction between virtual and non-virtual methods is a break-in into optimization domain, which should be "transparent" to the developer. From the developer's POV, the correct semantics is when each method is virtual.

On Sunday, 4 December 2016 at 01:36:50 UTC, deadalnix wrote:
> First, presentation:
> https://www.youtube.com/watch?v=9OIEZAj243g
>
> Some of this is available in LLVM today, and everything presented here will be in 4.0 . The long story short: ThinLTO can do most of what LTO does but with a price that is much closer to the one of a regular build than the one of a classic LTO build.
>
> LTO optimization can devirtualize all function that do not need to be virtual, and even use profile infos to speculatively devirtualize - aka JVM grade devirtualization.
>
> I would love to see this leveraged to finally put to rest the final vs virtual debate. If we use this tech properly, everything that do not need to be virtual can be finalized - except across shared object, which shouldn't be too much of an issue in practice.

ldc-1.1.0-beta5 --help | grep -A2 lto
  -flto             - Set LTO mode, requires linker support
    =full           - Merges all input into a single module
    =thin           - Parallel importing and codegen (faster than 'full')
  -flto-binary=<file>              - Set the linker LTO plugin library file (e.g. LLVMgold.so (Unixes) or libLTO.dylib (Darwin))
  -fprofile-instr-generate=<filename>       - Generate instrumented code to collect execution counts (e.g. for PGO)
  -fprofile-instr-use=<filename>           - Use instrumentation data for profile-guided optimization

On Sun, 04 Dec 2016 10:36:30 +0000, angel wrote:
> IMHO it was always obvious that the proper way is "devirtualization" within compiler optimization process.

Incremental compilation. You can't in general do this in the compiler.

* I write a library that uses std.socket:InternetHost.
* You compile it into libawesome.a. The compiler must use virtual
dispatch (you might override members elsewhere).
* You write an app that uses libawesome.
* You compile it all together. Only now does the compiler know it can use
static dispatch -- but it's too late; libawesome.a was already compiled.

That's why this is a *link* time optimization.

You can get around this by only doing whole program compilation, but that can be slow. You *could* have a compiler flag that enables whole-program compilation and associated optimizations, but if you can do it faster at link time, that's better. What you test with should be as close to what you ship with as possible, and the longer the optimization step is, the more likely it is that I'll turn it off while developing.

> From the developer's POV, the
> correct semantics is when each method is virtual.

Assuming the developer made the function final for speed and not to indicate that this function should not be overridden.

On Sunday, 4 December 2016 at 01:36:50 UTC, deadalnix wrote:
> First, presentation:
> https://www.youtube.com/watch?v=9OIEZAj243g
>
> Some of this is available in LLVM today, and everything presented here will be in 4.0 . The long story short: ThinLTO can do most of what LTO does but with a price that is much closer to the one of a regular build than the one of a classic LTO build.
>
> LTO optimization can devirtualize all function that do not need to be virtual, and even use profile infos to speculatively devirtualize - aka JVM grade devirtualization.
>
> I would love to see this leveraged to finally put to rest the final vs virtual debate. If we use this tech properly, everything that do not need to be virtual can be finalized - except across shared object, which shouldn't be too much of an issue in practice.

My understanding was that the main argument for final by default was that it is easy to make the wrong decision for a method to be virtual and then going from virtual to final would break the compatibility.

On Sunday, December 04, 2016 19:49:15 NVolcz via Digitalmars-d wrote:
> On Sunday, 4 December 2016 at 01:36:50 UTC, deadalnix wrote:
> > First, presentation: https://www.youtube.com/watch?v=9OIEZAj243g
> >
> > Some of this is available in LLVM today, and everything presented here will be in 4.0 . The long story short: ThinLTO can do most of what LTO does but with a price that is much closer to the one of a regular build than the one of a classic LTO build.
> >
> > LTO optimization can devirtualize all function that do not need to be virtual, and even use profile infos to speculatively devirtualize - aka JVM grade devirtualization.
> >
> > I would love to see this leveraged to finally put to rest the final vs virtual debate. If we use this tech properly, everything that do not need to be virtual can be finalized - except across shared object, which shouldn't be too much of an issue in practice.
>
> My understanding was that the main argument for final by default was that it is easy to make the wrong decision for a method to be virtual and then going from virtual to final would break the compatibility.

That and that some of the performance-centric guys like Manu work in environments where non-virtual functions should be the norm, and you only want a small percentage of functons to be virtual.

Really, whether a function is virtual or not should be a design decision. For inheritance and polymorphism to work well, it really needs to be part of the design of a how a class works. The idea that it's reasonable to just inherit from a class and override random functions and expect everything to work well is a faulty one. Good design requires that the programmer actually make a design decision about whether overridability makes sense for that function and that class and make that function virtual or not, and having it be virtual when the programmer didn't explicitly planned for it to be overridden is just a recipe for disaster. So, the fact that virtual is then the default rather than requiring a decision by the programmer is just inviting problems. Those are problems that are dealt with in other languages (e.g. Java and C#), and they're survivable (especially if you don't work in an environment where the virtualness of a function is a performance problem), but since it's a problem for random functions to be overridden without the designer of the base class planning for it, and it _is_ a performance problem for some folks, it arguably really doesn't make sense for virtual to be the default. But we're stuck with it at this point.

Improvements to devirtualization may help with the performance problems associated with virtual functions, but they don't fix the design problems that go with virtual by default.

Fortunately, D makes it far less of an issue than it would be otherwise, because we have fully-featured structs which are non-polymorphic. So, this really only becomes a problem when you're using classes and don't need polymorphism for the whole class (and I think that part of Andrei's argument against final by default was the fact that he didn't think that it made any sense to even use classes if you weren't looking to have the functions be virtual).

- Jonathan M Davis

On Sunday, 4 December 2016 at 19:49:15 UTC, NVolcz wrote:
> My understanding was that the main argument for final by default was that it is easy to make the wrong decision for a method to be virtual and then going from virtual to final would break the compatibility.

I have a lot of experience with various language who are virtual by default, and this ended up being a big problem none of the time in years.

I understand the theoretical problem, but it is not that big of an issue in practice.