On Wed, 10 Apr 2013 11:48:18 +0100, Dicebot <m.strashun@gmail.com> wrote:

> My understanding as based on dlang.org:
>
> On Wednesday, 10 April 2013 at 10:29:05 UTC, Regan Heath wrote:
>> Ok, lets Rewind for a sec.  I need some educating on the actual issue and I want to go through the problem one case at a time..
>>
>>
>> #1 If I write a class..
>>
>> class A
>> {
>>   public int isVirt()  { return 1; }
>>   public int notVirt() { return 2; }
>> }
>>
>> and compile it with another class..
>>
>> class B : A
>> {
>>   override public int isVirt() { return 5; }
>> }
>>
>> and this main..
>>
>> void main()
>> {
>>   A a = new B();
>>   a.isVirt();    // compiler makes a virtual call
>>   a.notVirt();   // but not here
>> }
>>
>> Right?
>
> No.

Hmm..

> A is not final.

True.  But, I don't see how this matters.

> A has no internal linkage. It can be inherited from in other compilation unit.

False.  In this first example we are compiling A and B together (into an exe - I left that off) so the compiler has all sources and all uses of all methods of A (and B).

> notVirt is virtual.

It may actually be (I don't know) but it certainly does not have to be (compiler has all sources/uses) and my impression was that it /should/ not be.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On Wed, 10 Apr 2013 11:39:20 +0100, Manu <turkeyman@gmail.com> wrote:
> All your examples all hinge on the one case where the optimisation may
> possibly be valid, a is _allocated in the same scope_.

Correct, that was the whole point.  I am trying to establish where the "problem" arises, starting from the most basic examples :)

> Consider:

The example below is, I presume, supposed to be a function call in user code where A is defined in a library, correct?

If so, I agree with the issues stated.

If not, if A, and B, and any other classes derived from A are all compiled at the same time(*) and we're producing an exe then the compiler has all the information it needs to decide which methods MUST be virtual, and all others should be non-virtual.

This is the point I was trying to establish with my simpler examples :)

(*) Yes, perhaps you compile a.d then b.d separately - this may result in the same problem case as a library, or it may not - I don't have the understanding/data to say one way or the other.

> void func(A* a)
> {
>  a.isVirt(); // is it? I guess...
>  a.notVirt(); // what about this?
> }
>
> We don't even know what a B is, or whether it even exists.

Assuming A and B come from a library and are not present in the source we are compiling with this function, yes.

> All we know is that A's methods are virtual by default, and they could be
> overridden somewhere else. It's not possible to assume otherwise.

If we're compiling A, and some number of sub-classes of A, and producing an exe, then from the definition of A, B, and all sub-classes we know which methods have been overridden, those are virtual, all others are non-virtual.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On Wednesday, 10 April 2013 at 10:53:26 UTC, Regan Heath wrote:
> Hmm..
>
>> A is not final.
>
> True.  But, I don't see how this matters.
>
>> A has no internal linkage. It can be inherited from in other compilation unit.
>
> False.  In this first example we are compiling A and B together (into an exe - I left that off) so the compiler has all sources and all uses of all methods of A (and B).
>
>> notVirt is virtual.
>
> It may actually be (I don't know) but it certainly does not have to be (compiler has all sources/uses) and my impression was that it /should/ not be.
>
> R

If it is compiled all at once and compiled into executable binary than yes, you examples are valid and compiler _MAY_ omit virtual. But
a) DMD doesn't do it as far as I am aware.
b) It is a quite uncommon and restrictive build setup.

On 10 April 2013 20:53, Regan Heath <regan@netmail.co.nz> wrote:

>
> False.  In this first example we are compiling A and B together (into an
> exe - I left that off) so the compiler has all sources and all uses of all
> methods of A (and B).
>

And if the program calls LoadLibrary() somewhere?


 notVirt is virtual.
>>
>
> It may actually be (I don't know) but it certainly does not have to be
> (compiler has all sources/uses) and my impression was that it /should/ not
> be.


It doesn't have all sources. It could load a library. That can never be guaranteed.

On Wednesday, 10 April 2013 at 11:00:17 UTC, Manu wrote:
> On 10 April 2013 20:53, Regan Heath <regan@netmail.co.nz> wrote:
>
>>
>> False.  In this first example we are compiling A and B together (into an
>> exe - I left that off) so the compiler has all sources and all uses of all
>> methods of A (and B).
>>
>
> And if the program calls LoadLibrary() somewhere?
>
>
>  notVirt is virtual.
>>>
>>
>> It may actually be (I don't know) but it certainly does not have to be
>> (compiler has all sources/uses) and my impression was that it /should/ not
>> be.
>
>
> It doesn't have all sources. It could load a library. That can never be
> guaranteed.

That is the main reason why most JITs do code rewriting every time the world changes.

For example, on Hotspot, depending how many subclasses you have loaded many virtual calls are actually direct calls, until this is no longer possible and  code regeneration is required.

This of course works way better on server side, or with layered JITs like Java 7 has.

--
Paulo

On Wed, 10 Apr 2013 11:59:32 +0100, Dicebot <m.strashun@gmail.com> wrote:

> On Wednesday, 10 April 2013 at 10:53:26 UTC, Regan Heath wrote:
>> Hmm..
>>
>>> A is not final.
>>
>> True.  But, I don't see how this matters.
>>
>>> A has no internal linkage. It can be inherited from in other compilation unit.
>>
>> False.  In this first example we are compiling A and B together (into an exe - I left that off) so the compiler has all sources and all uses of all methods of A (and B).
>>
>>> notVirt is virtual.
>>
>> It may actually be (I don't know) but it certainly does not have to be (compiler has all sources/uses) and my impression was that it /should/ not be.
>>
>> R
>
> If it is compiled all at once and compiled into executable binary than yes, you examples are valid and compiler _MAY_ omit virtual.

Exactly the point I was trying to make.  I wanted to establish the point at which the design problems (what D defines/intends to do) arise, vs when the implementation problems arise (DMD not doing what D intends).

> But
> a) DMD doesn't do it as far as I am aware.

Maybe, maybe not.  I have no idea.  My understanding of the design decision is that DMD will eventually do it.

> b) It is a quite uncommon and restrictive build setup.

Maybe at present.

Lets assume DMD can remove virtual when presented with all sources compiled in one-shot.

Lets assume it cannot if each source is compiled separately.  Is that an insurmountable problem?  A design problem?  Or, is it simply an implementation issue.  Could an obj file format be designed to allow DMD to perform the same optimisation in this case, as in the one-shot case.  My impression is that this should be solvable.

So, that just leaves the library problem.  Is this also insurmountable?  A design problem?  Or, is it again an implementation issue.  Can D not mark exported library methods as virtual/non-virtual?  When user code derives from said exported class could D not perform the same optimisation for that class?  I don't know enough about compilation to answer that.  But, I can see how if the library itself manifests an object of type A - which may actually be an internal derived sub-class of A, there are clearly issues.  But, could DMD not have two separate definitions for A, and use one for objects manifested from the library, and another locally for user derived classes?  I don't know, these are all just ideas I have on the subject.  :p

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On Wednesday, 10 April 2013 at 09:15:26 UTC, Manu wrote:
> On 10 April 2013 19:07, Dicebot <m.strashun@gmail.com> wrote:
>
>> On Wednesday, 10 April 2013 at 08:57:55 UTC, Manu wrote:
>>
>>> That sounds horribly non-deterministic. What if you have 256mb of ram, and
>>> no pagefile, and you fill it up till you have 1mb headroom spare?
>>>
>>
>> It is Erlang, it is not meant to be run on 256Mb RAM ;) It kind of solves
>> the issue of response latency for GC-enabled software on powerful
>> enterprise servers. Because with stop-the-world GC you can't do it, does
>> not matter how powerful your hardware is.
>>
>> Does not help game dev and small-scale embedded though, that for sure ;)
>>
>
> Well there's always the standing question though, why is JVM and C# so much
> faster than D?
> They produce a squillion times more garbage than D, yet they're much much
> faster. I have come to accept the C# GC in less-intensive realtime
> software, it's not so bad.

First of all they require the use of safe references. Pointer manipulation is reserved to unsafe regions, which allows for more aggressive GC algorithms.

Secondly you have years of GC research invested into those runtimes.

Finally they don't offer a single GC, but tunable versions.

Additionally the garbage might be less than what you think, because you may use "new" but the JIT will actually do a stack allocation if it sees the object will be dead at the end of scope.

Some Java related information,
http://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html
http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html

--
Paulo

On Wednesday, 10 April 2013 at 11:09:30 UTC, Regan Heath wrote:
> ...
> R

Yes, this is insurmountable design problem, because it forces you to have an application that is not allowed to use dll's, not allowed to split part of its functionality in static libraries, requires full recompilation on smallest change. Sounds rather useless, to be honest. I don't see how changing obj format may help.

On 10 April 2013 21:09, Regan Heath <regan@netmail.co.nz> wrote:

> On Wed, 10 Apr 2013 11:59:32 +0100, Dicebot <m.strashun@gmail.com> wrote:
>
>  On Wednesday, 10 April 2013 at 10:53:26 UTC, Regan Heath wrote:
>>
>>> Hmm..
>>>
>>>  A is not final.
>>>>
>>>
>>> True.  But, I don't see how this matters.
>>>
>>>  A has no internal linkage. It can be inherited from in other
>>>> compilation unit.
>>>>
>>>
>>> False.  In this first example we are compiling A and B together (into an
>>> exe - I left that off) so the compiler has all sources and all uses of all
>>> methods of A (and B).
>>>
>>>  notVirt is virtual.
>>>>
>>>
>>> It may actually be (I don't know) but it certainly does not have to be
>>> (compiler has all sources/uses) and my impression was that it /should/ not
>>> be.
>>>
>>> R
>>>
>>
>> If it is compiled all at once and compiled into executable binary than yes, you examples are valid and compiler _MAY_ omit virtual.
>>
>
> Exactly the point I was trying to make.  I wanted to establish the point at which the design problems (what D defines/intends to do) arise, vs when the implementation problems arise (DMD not doing what D intends).
>
>
>  But
>> a) DMD doesn't do it as far as I am aware.
>>
>
> Maybe, maybe not.  I have no idea.  My understanding of the design decision is that DMD will eventually do it.


I feel like I'm being ignored. It's NOT POSSIBLE.

 b) It is a quite uncommon and restrictive build setup.
>>
>
> Maybe at present.
>
> Lets assume DMD can remove virtual when presented with all sources compiled in one-shot.
>
> Lets assume it cannot if each source is compiled separately.  Is that an insurmountable problem?  A design problem?  Or, is it simply an implementation issue.  Could an obj file format be designed to allow DMD to perform the same optimisation in this case, as in the one-shot case.  My impression is that this should be solvable.
>
> So, that just leaves the library problem.  Is this also insurmountable?  A
> design problem?  Or, is it again an implementation issue.  Can D not mark
> exported library methods as virtual/non-virtual?  When user code derives
> from said exported class could D not perform the same optimisation for that
> class?  I don't know enough about compilation to answer that.  But, I can
> see how if the library itself manifests an object of type A - which may
> actually be an internal derived sub-class of A, there are clearly issues.
>  But, could DMD not have two separate definitions for A, and use one for
> objects manifested from the library, and another locally for user derived
> classes?  I don't know, these are all just ideas I have on the subject.  :p


That sounds overly complex and error prone. I don't believe the source of
an object is actually trackable in the way required to do that.
I can't conceive any solution of this sort is viable. And even if it were
theoretically possible, when can we expect to see it implemented?
It's not feasible. The _problem_ is that functions are virtual by default.
It's a trivial problem to solve, however it's a major breaking change, so
it will never happen.
Hence my passing comment that spawned this whole thread, I see it as the
single biggest critical mistake in D, and I'm certain it will never be
changed. I've made my peace, however disappointing it is to me.

On 10 April 2013 21:18, Paulo Pinto <pjmlp@progtools.org> wrote:

> On Wednesday, 10 April 2013 at 09:15:26 UTC, Manu wrote:
>
>> On 10 April 2013 19:07, Dicebot <m.strashun@gmail.com> wrote:
>>
>>  On Wednesday, 10 April 2013 at 08:57:55 UTC, Manu wrote:
>>>
>>>  That sounds horribly non-deterministic. What if you have 256mb of ram,
>>>> and
>>>> no pagefile, and you fill it up till you have 1mb headroom spare?
>>>>
>>>>
>>> It is Erlang, it is not meant to be run on 256Mb RAM ;) It kind of solves the issue of response latency for GC-enabled software on powerful enterprise servers. Because with stop-the-world GC you can't do it, does not matter how powerful your hardware is.
>>>
>>> Does not help game dev and small-scale embedded though, that for sure ;)
>>>
>>>
>> Well there's always the standing question though, why is JVM and C# so
>> much
>> faster than D?
>> They produce a squillion times more garbage than D, yet they're much much
>> faster. I have come to accept the C# GC in less-intensive realtime
>> software, it's not so bad.
>>
>
> First of all they require the use of safe references. Pointer manipulation is reserved to unsafe regions, which allows for more aggressive GC algorithms.
>
> Secondly you have years of GC research invested into those runtimes.
>
> Finally they don't offer a single GC, but tunable versions.
>
> Additionally the garbage might be less than what you think, because you may use "new" but the JIT will actually do a stack allocation if it sees the object will be dead at the end of scope.
>

Good point. It'd be really great if D implemented optimisations of this sort too one of these days. There's a lot of such opportunities waiting for some attention. I'd be very interested to see what sort of practical difference they make.


Some Java related information,
> http://www.oracle.com/**technetwork/java/javase/tech/** g1-intro-jsp-135488.html<http://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html> http://docs.oracle.com/javase/**7/docs/technotes/guides/vm/** performance-enhancements-7.**html<http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html>
>