On Saturday, 22 December 2012 at 12:15:38 UTC, Russel Winder wrote:
> Interesting (or not)
> side observation: LDC generally creates faster executables than DMD,
> except in one or two cases that I have.

If you ever have time to do some quick profiling, could you please try to figure out why the LDC-generated executable is slower – or if the code you are working on is open source, put some instructions together on how to run the benchmark(s)? The LLVM backend really shouldn't produce slower code than DMD in just about any situation, so most likely we are doing something stupid in LDC.

I'm really interested in this, because apart from target platform support (druntime/ARM is still not there yet, though) and some very few DMD-specific bugs such as the one you seem to hit, there is little reason to use LDC other than for the its code generation.

Oh, and if you get around to doing this before the next LDC release, could you please try it on a Git master build, and if you are on a 32 bit system, ideally use LLVM 3.2+? We had to disable optimizations on earlier LLVM versions for x86 builds of druntime due to a LLVM codegen bug only fixed in 3.2, and the GC-to-stack-promotion pass now activated in master should catch a few cases where we do stupid things like allocating array manifest constants on every entry into function, even if they were only used for meta-programming.

There is still a single known issue which can drastically affect GC performance, though: https://github.com/ldc-developers/ldc/issues/233. There is an easy fix, but it completely breaks shared libraries (but given that those don't work reliably anyway, I think I'll just go with it for the time being…)

David

On Sat, 2012-12-22 at 17:41 +0100, David Nadlinger wrote: […]
> If you ever have time to do some quick profiling, could you please try to figure out why the LDC-generated executable is slower – or if the code you are working on is open source, put some instructions together on how to run the benchmark(s)? The LLVM backend really shouldn't produce slower code than DMD in just about any situation, so most likely we are doing something stupid in LDC.

Can you point me at tools and documentation for profiling that generates the data that would be useful to you. I am very much an LLVM n00b.  I just compile and use LDC and compile and install LLVMPy, so all of LLVM is under the covers for me.

The codes are all published on GitHub https://github.com/russel/Pi_Quadrature this is an embarrasingly parallel summation problem, approximating  π by quadrature. This is the example David used for one of the std.parallelism examples. The codes use SCons for compile and execution. So to compile and run pi_d_spawn.d:

	scons run_pi_d_spawn

Though you will want to use my "D tools fork" of SCons which is a Mercurial repository at BitBucket https://bitbucket.org/russel/scons_d_tooling No need to install SCons can be executed from the clone by:

	python /path/to/scons-clone/bootstrap.py run_pi_d_spawn

> I'm really interested in this, because apart from target platform support (druntime/ARM is still not there yet, though) and some very few DMD-specific bugs such as the one you seem to hit, there is little reason to use LDC other than for the its code generation.

The pattern of when DMD beats LDC on my machines is weird.  However, this is all currently just anecdotal evidence, I have not done a proper experiment that can give any statistical significance to the claims made.

LDC generally beats DMD by about 2%–4%, but in some cases by extraordinary amounts, but I think this shows failures in DMD rather than success of LDC since the LDC time is about what it should be.

The case where I thought DMD was beating LDC may be a misobservation (i.e. statistics needed) and that it is just that DMD is performing unusually as well as LDC. This is the pi_d_spawn.d code.

pi_d_threadsGlobalState_array_declarative causes both DMD and LDC to barf, with the same problem I think.

pi_d_threadsGlobalState_array_iterative.d LDC works and DMD barfs.

pi_d_threadsGlobalState_threadGroup.d DMD behaves weirdly, LDC gets things right.

This first one of this trio is the 2.059 → 2.060 regression reported if I remember correctly. Ithought I had reported this last one as well since I am fairly sure 2.059 did not behave as 2.060 does.

> Oh, and if you get around to doing this before the next LDC release, could you please try it on a Git master build, and if you are on a 32 bit system, ideally use LLVM 3.2+? We had to disable optimizations on earlier LLVM versions for x86 builds of druntime due to a LLVM codegen bug only fixed in 3.2, and the GC-to-stack-promotion pass now activated in master should catch a few cases where we do stupid things like allocating array manifest constants on every entry into function, even if they were only used for meta-programming.

I only ever use Git master, at least until Debian package a useful version of LDC, then I'll have to make a decision.

I am usually 64-bit Linux, I can also do 64-bit OS X. Windows exists only in a plane of the multiverse that I never frequent. :-)

> There is still a single known issue which can drastically affect GC performance, though: https://github.com/ldc-developers/ldc/issues/233. There is an easy fix, but it completely breaks shared libraries (but given that those don't work reliably anyway, I think I'll just go with it for the time being…)

:-)



-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Friday, 21 December 2012 at 22:12:47 UTC, Andrei Alexandrescu wrote:
> We plan to start building a new release on Sunday evening. To do so (pursuant to the embryonic process we're putting in place), at that time we'll create a new branch called "staging" for each of dmd, druntime, and phobos.
>
> All contributors - over the weekend please ping reviewers on what you believe are pull requests with a high importance*urgency product. Once we branch into staging, pull requests will only be merged into master.
>
>
> Thanks,
>
> Andrei

Is this the release for 2.061 which is being placed in staging?

Pull requests which fix regressions and bugs should go into staging and be made against staging.

On 12/22/12 1:39 PM, Jesse Phillips wrote:
> On Friday, 21 December 2012 at 22:12:47 UTC, Andrei Alexandrescu wrote:
>> We plan to start building a new release on Sunday evening. To do so
>> (pursuant to the embryonic process we're putting in place), at that
>> time we'll create a new branch called "staging" for each of dmd,
>> druntime, and phobos.
>>
>> All contributors - over the weekend please ping reviewers on what you
>> believe are pull requests with a high importance*urgency product. Once
>> we branch into staging, pull requests will only be merged into master.
>>
>>
>> Thanks,
>>
>> Andrei
>
> Is this the release for 2.061 which is being placed in staging?
>
> Pull requests which fix regressions and bugs should go into staging and
> be made against staging.

That is my understanding too.

Andrei

On 22-12-2012 06:11, Jonathan M Davis wrote:
> On Friday, December 21, 2012 17:12:47 Andrei Alexandrescu wrote:
>> We plan to start building a new release on Sunday evening. To do so
>> (pursuant to the embryonic process we're putting in place), at that time
>> we'll create a new branch called "staging" for each of dmd, druntime,
>> and phobos.
>>
>> All contributors - over the weekend please ping reviewers on what you
>> believe are pull requests with a high importance*urgency product. Once
>> we branch into staging, pull requests will only be merged into master.
>
> https://github.com/D-Programming-Language/dmd/pull/1287
>
> really should be resolved prior to 2.061, or we're going to be introducing a
> compiler flag (-di) which we're probably then going to turn around and
> deprecate (and making deprecations warn by default instead of giving you an
> error will be  _huge_ step forward in our ability to manage deprecations
> without breaking people's code).
>
> - Jonathan M Davis
>

+1 to this.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

On 12/22/2012 10:39 AM, Jesse Phillips wrote:
> On Friday, 21 December 2012 at 22:12:47 UTC, Andrei Alexandrescu wrote:
>> We plan to start building a new release on Sunday evening. To do so (pursuant to the embryonic process we're putting in place), at that time we'll create a new branch called "staging" for each of dmd, druntime, and phobos.
>>
>> All contributors - over the weekend please ping reviewers on what you believe are pull requests with a high importance*urgency product. Once we branch into staging, pull requests will only be merged into master.
>>
>>
>> Thanks,
>>
>> Andrei
> 
> Is this the release for 2.061 which is being placed in staging?
> 
> Pull requests which fix regressions and bugs should go into staging and be made against staging.

I strongly recommend requiring that all bugs be first fixed in the development branch and then being pushed backwards through the version history.  Quite a few projects follow this pattern based on the requirement that no fix can ever be accidentally left out of a future release.  You never want someone to pick up (using made up version numbers) 3.4.2 to get a fix and later upgrade to 4.1.1 and find out it's not yet fixed in that release.

On Saturday, 22 December 2012 at 21:48:51 UTC, Brad Roberts wrote:

> I strongly recommend requiring that all bugs be first fixed in the development branch and then being pushed backwards
> through the version history.  Quite a few projects follow this pattern based on the requirement that no fix can ever be
> accidentally left out of a future release.  You never want someone to pick up (using made up version numbers) 3.4.2 to
> get a fix and later upgrade to 4.1.1 and find out it's not yet fixed in that release.

Well, to have the easy merging the change must be made against the oldest applicable code. The benefit of merging into staging first is that staging can be merged into master, while master can not be merged into staging.

What is nice about making a pull request against staging is that the reviewer knows that the fix can be applied that far (not that comments wouldn't do the same).

On 12/22/2012 3:44 PM, Jesse Phillips wrote:
> On Saturday, 22 December 2012 at 21:48:51 UTC, Brad Roberts wrote:
> 
>> I strongly recommend requiring that all bugs be first fixed in the development branch and then being pushed backwards through the version history.  Quite a few projects follow this pattern based on the requirement that no fix can ever be accidentally left out of a future release.  You never want someone to pick up (using made up version numbers) 3.4.2 to get a fix and later upgrade to 4.1.1 and find out it's not yet fixed in that release.
> 
> Well, to have the easy merging the change must be made against the oldest applicable code. The benefit of merging into staging first is that staging can be merged into master, while master can not be merged into staging.
> 
> What is nice about making a pull request against staging is that the reviewer knows that the fix can be applied that far (not that comments wouldn't do the same).

I don't believe those assertions to be true.  Merging in either direction is possible and the difficulty lies in the nature of the drift between the two.  Neither direction is necessarily any easier than the other.

On Saturday, December 22, 2012 17:36:11 Brad Roberts wrote:
> On 12/22/2012 3:44 PM, Jesse Phillips wrote:
> > What is nice about making a pull request against staging is that the reviewer knows that the fix can be applied that far (not that comments wouldn't do the same).
> 
> I don't believe those assertions to be true.  Merging in either direction is possible and the difficulty lies in the nature of the drift between the two.  Neither direction is necessarily any easier than the other.

If you merge from the branch to master, then there's a higher risk of forgetting to merge fixes. If you merge from master to the branch, then there's a higher risk of putting changes in the branch that you don't want in the branch. However, as long as the changes on master aren't too large, you can simply cherry-pick the changes from master to the branch (or vice versa) without too much trouble. Overall though, I would think that the risk of screwing up is higher if commits go to the branch initially rather than master.

- Jonathan M Davis

On 12/22/2012 5:43 PM, Jonathan M Davis wrote:
> On Saturday, December 22, 2012 17:36:11 Brad Roberts wrote:
>> On 12/22/2012 3:44 PM, Jesse Phillips wrote:
>>> What is nice about making a pull request against staging is that the
>>> reviewer knows that the fix can be applied that far (not that comments
>>> wouldn't do the same).
>>
>> I don't believe those assertions to be true.  Merging in either direction is
>> possible and the difficulty lies in the nature of the drift between the
>> two.  Neither direction is necessarily any easier than the other.
>
> If you merge from the branch to master, then there's a higher risk of
> forgetting to merge fixes. If you merge from master to the branch, then there's
> a higher risk of putting changes in the branch that you don't want in the
> branch. However, as long as the changes on master aren't too large, you can
> simply cherry-pick the changes from master to the branch (or vice versa)
> without too much trouble. Overall though, I would think that the risk of
> screwing up is higher if commits go to the branch initially rather than
> master.

It makes more sense to me to put the commits into master, and then cherry pick for the branch.