On Tue, Feb 18, 2020 at 07:12:34PM +0300, drug via Digitalmars-d wrote:
> On 2/18/20 6:30 PM, jxel wrote:
> > 
> > The problem is that efforts are divided as a result.
[...]

That's the fallacy described in "The Mythical Man-Month".


> IMO you are wrong if you think that dropping dmd will increase man power in ldc/gdc land. Who wants to contribute to ldc/gdc already doing it. It is open source - people contribute to projects they like, you can not tell them what to do. Efforts are not divided - if dmd would be dropped people who doesn't contribute to ldc/gdc won't start doing that.

Yeah, this is not a commercial endeavour where people are paid to work on stuff they didn't choose to work on.  The people currently working on dmd do so because they *want* to work on dmd.  If we drop dmd, they may end up going elsewhere to find something else they're interested instead. There's no reason to expect that they will suddenly acquire a fresh interest in ldc/gdc -- if they already had such an interest they'd already be working on ldc/gdc.


> On 2/18/20 6:30 PM, jxel wrote:
[...]
> > The slowest part of D is the frontend and CTFE not the backend in LDC.

?!  That's clearly not true.  I'm compiling the same codebase with dmd, and it's measurably faster than ldc2.  Since it's identical source code, and the two compilers share the same frontend, the only possible difference in compile times must be due to the different backends.

Of course, compile speed isn't the only criterion for choosing between compilers, and that's why I still prefer using LDC in spite of DMD being faster at compiling.  The deciding factor is the fact that the LDC backend, in spite of being slower, emits better code.  So I'm gaining more runtime performance by paying with slightly longer compile times. The increase in compile times is (currently) outweighed by the superior performance of the resulting executable.


> > That's not even mentioning the constant out of memory problems
> > I experience and have to try to optimize the compiler for with my
> > code.

The frontend's memory issues are mainly because Walter switched to a bump-the-pointer allocator in interest of compile speed.  Even with -lowmem there are still fundamental problems that are unsolvable when running a low-memory system.  I've essentially given up using dmd on low-memory systems.  Fast compile times are worthless when the compiler never finishes running before it runs out of memory. It's an embarrassment.  (And this is why the whole "fast code fast" slogan makes me cringe. It epitomizes one of the worst aspects of D in my experience, in spite of everything else I love about D.)


T

-- 
"640K ought to be enough" -- Bill G. (allegedly), 1984. "The Internet is not a primary goal for PC usage" -- Bill G., 1995. "Linux has no impact on Microsoft's strategy" -- Bill G., 1999.

On Tuesday, 18 February 2020 at 18:00:03 UTC, H. S. Teoh wrote:
> 
>
> Plus, it's annoying to have to switch between different compiler options syntax just so I can use both dmd and ldc2 for my builds.
>

Why not use ldmd2 then?

On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via Digitalmars-d wrote:
> On Tuesday, 18 February 2020 at 18:00:03 UTC, H. S. Teoh wrote:
> > 
> > Plus, it's annoying to have to switch between different compiler options syntax just so I can use both dmd and ldc2 for my builds.
> 
> Why not use ldmd2 then?

Because some LDC flags are not accessible through the ldmd2 interface.


T

-- 
Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG

On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
> Unfortunately, the GDC and LDC optimizers consistently produce code that outperforms code generated by the DMD backend by 20-30%, sometimes as high as 40% for CPU-intensive code...

Well these numbers are huge, I never thought the difference would be higher than 10% between these compilers, thanks for pointing this out.

I use old DMD for my hobby projects in D, but now I'll test both GDC/LDC.

Matheus.

On Tuesday, 18 February 2020 at 18:40:34 UTC, H. S. Teoh wrote:
> On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via
>> Why not use ldmd2 then?
>
> Because some LDC flags are not accessible through the ldmd2 interface.

Just try it - unknown flags are forwarded to ldc2.exe.

On Tuesday, 18 February 2020 at 18:14:56 UTC, H. S. Teoh wrote:
> On Tue, Feb 18, 2020 at 07:12:34PM +0300, drug via Digitalmars-d wrote:
>> On 2/18/20 6:30 PM, jxel wrote:
>> > 
>> > The problem is that efforts are divided as a result.
> [...]
>
> That's the fallacy described in "The Mythical Man-Month".


Not really, how many people do you think work on and use LLVM? How many other compilers use it and as a result bugs and optimizations all happen " for free". Its not just the few people involved with D, and so much so the thousands involved with LLVM (including corporate funding).

>> IMO you are wrong if you think that dropping dmd will increase man power in ldc/gdc land.

Who said that? That's your misinterpretation.

On Tuesday, 18 February 2020 at 18:50:44 UTC, matheus wrote:
> On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
>> Unfortunately, the GDC and LDC optimizers consistently produce code that outperforms code generated by the DMD backend by 20-30%, sometimes as high as 40% for CPU-intensive code...
>
> Well these numbers are huge, I never thought the difference would be higher than 10% between these compilers, thanks for pointing this out.

Once you know that the DMD backend is a formidable one-man project, it should be clear as day that that cannot compete with huge eco-systems like LLVM and GCC with millions of man-hours, in terms of architecture support, optimizations and flexibility. It's nice for fast unoptimized codegen if you are only targeting x86, and probably features better debuginfos than LDC at the moment (well, ecept for Mac apparently), but to me, the DMD *backend* is clearly a dead end in the long run.

DMD itself runs faster by 58% when compiled with LDC for a random compilation test case, see https://github.com/dlang/installer/pull/425#issuecomment-580868218. For number crunching, you can definitely expect much higher speed-ups if the auto-vectorizer kicks in. Whole-program optimization via LTO, even across C(++) and D, and PGO can improve your runtime further.

On Tuesday, 18 February 2020 at 15:30:57 UTC, jxel wrote:
> The problem is that efforts are divided as a result.

As others have stated, that's not true. The vast majority of contributions to the dmd repo affect the front-end and thus implicitly benefit all compilers.

On Tuesday, 18 February 2020 at 20:10:57 UTC, kinke wrote:
> On Tuesday, 18 February 2020 at 18:50:44 UTC, matheus wrote:
>> On Sunday, 16 February 2020 at 00:16:24 UTC, H. S. Teoh wrote:
>>> Unfortunately, the GDC and LDC optimizers consistently produce code that outperforms code generated by the DMD backend by 20-30%, sometimes as high as 40% for CPU-intensive code...
>>
>> Well these numbers are huge, I never thought the difference would be higher than 10% between these compilers, thanks for pointing this out.
>
> Once you know that the DMD backend is a formidable one-man project, it should be clear as day that that cannot compete with huge eco-systems like LLVM and GCC with millions of man-hours, in terms of architecture support, optimizations and flexibility. It's nice for fast unoptimized codegen if you are only targeting x86, and probably features better debuginfos than LDC at the moment (well, ecept for Mac apparently), but to me, the DMD *backend* is clearly a dead end in the long run.

Just to be clear, I am NOT diminishing any work, I just wasn't aware of these differences, because 20 to 30 or maybe 40% on runtime is something that you can't just ignore.

I think that maybe is possible to balance things, like developing with DMD and generate final code with GDC/LDC to get faster runtime, well maybe some problems will happen mixing compilers and final result, but this is something that I will considerate for now on.

Matheus.

On Tue, Feb 18, 2020 at 08:10:57PM +0000, kinke via Digitalmars-d wrote: [...]
> Once you know that the DMD backend is a formidable one-man project, it should be clear as day that that cannot compete with huge eco-systems like LLVM and GCC with millions of man-hours, in terms of architecture support, optimizations and flexibility. It's nice for fast unoptimized codegen if you are only targeting x86, and probably features better debuginfos than LDC at the moment (well, ecept for Mac apparently), but to me, the DMD *backend* is clearly a dead end in the long run.

Yeah, not to mention Walter himself has been mostly working on the front end these days, as he himself said he hasn't been keeping the backend up-to-date.  A single person only has so many hours a day, no matter what a brilliant genius Walter is, there's still a limit as to what can be accomplished by a single person.  And if DMD's very limited target arch support is already languishing from lack of sufficient time to keep up-to-date, support for other targets like Android is pretty much never going to happen, whereas using LDC gives you that support *today*.

Add to that LDC's recent ability to cross-compile from Linux to Windows without needing to rebuild the entire toolchain, vs. dmd's inability to cross-compile at all (AFAIK), and the best choice from the user's POV is beyond obvious.


> DMD itself runs faster by 58% when compiled with LDC for a random compilation test case, see https://github.com/dlang/installer/pull/425#issuecomment-580868218.

IMNSHO, we should use LDC to compile official DMD releases, as someone has already mentioned. ;-)


> For number crunching, you can definitely expect much higher speed-ups if the auto-vectorizer kicks in. Whole-program optimization via LTO, even across C(++) and D, and PGO can improve your runtime further.
[...]

Yeah, LDC's optimizer is light-years ahead of DMD's.  I just tested it again on one of my latest projects, with a custom main() that runs a CPU-intensive part of the code in a loop 100 times.  With `dmd -O -inline`, a typical run is about 25-26 seconds.  With `ldmd2 -O3`, a typical run is about 16-17 seconds.  We're looking at a ~30% increase in performance here, on exactly the same code and maximum optimization flags on both compilers.

The numbers speak for themselves, really.

As for compile times, `dmd -O -inline` typically takes about 6 seconds, whereas `ldmd2 -O3` typically takes about 8 seconds, which is about a 25% slowdown in compile times.  Is a 2-second compile-time slowdown worth a 30% performance increase in the resulting executable?  To me, it's an unquestionable yes.


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond

On Tue, Feb 18, 2020 at 07:44:49PM +0000, kinke via Digitalmars-d wrote:
> On Tuesday, 18 February 2020 at 18:40:34 UTC, H. S. Teoh wrote:
> > On Tue, Feb 18, 2020 at 06:21:21PM +0000, jmh530 via
> > > Why not use ldmd2 then?
> > 
> > Because some LDC flags are not accessible through the ldmd2 interface.
> 
> Just try it - unknown flags are forwarded to ldc2.exe.

It works!!  Thanks for the tip!


T

-- 
Food and laptops don't mix.