"Rainer Schuetze" <r.sagitario@gmx.de> wrote in message news:i9f2ce$30re$1@digitalmars.com...
>
> I've used ddmd as a medium sized project for testing. It has more than 400 files with most modules mutually importing each other. Some remarks:
>
> - compilation speed is a lot worse, even when compared to what you would expect from a similar C/C++ project. It takes almost 8 minutes to compile ddmd on my system with single file compilation, while the standard compilation takes about 10 seconds.
>

Whaa...? Standard compilation of ddmd takes me a full two minutes.

"Nick Sabalausky" <a@a.a> wrote in message news:i9ibi0$ddn$1@digitalmars.com...
> "Rainer Schuetze" <r.sagitario@gmx.de> wrote in message news:i9f2ce$30re$1@digitalmars.com...
>>
>> I've used ddmd as a medium sized project for testing. It has more than 400 files with most modules mutually importing each other. Some remarks:
>>
>> - compilation speed is a lot worse, even when compared to what you would expect from a similar C/C++ project. It takes almost 8 minutes to compile ddmd on my system with single file compilation, while the standard compilation takes about 10 seconds.
>>
>
> Whaa...? Standard compilation of ddmd takes me a full two minutes.

As a sidebar, it's the *only* D program I've compiled that's ever taken more than a few seconds for me.

On Tue, 19 Oct 2010 00:41:54 +0400, Nick Sabalausky <a@a.a> wrote:

> "Rainer Schuetze" <r.sagitario@gmx.de> wrote in message
> news:i9f2ce$30re$1@digitalmars.com...
>>
>> I've used ddmd as a medium sized project for testing. It has more than 400
>> files with most modules mutually importing each other. Some remarks:
>>
>> - compilation speed is a lot worse, even when compared to what you would
>> expect from a similar C/C++ project. It takes almost 8 minutes to compile
>> ddmd on my system with single file compilation, while the standard
>> compilation takes about 10 seconds.
>>
>
> Whaa...? Standard compilation of ddmd takes me a full two minutes.
>
>

Do you compile the code on 486 or something? :) Running the build script takes 7.967 seconds for me (timed with timeit, part of the Windows Server 2003 Resource Kit Tools).

"Denis Koroskin" <2korden@gmail.com> wrote in message news:op.vkshh4bdo7cclz@korden-pc...
> On Tue, 19 Oct 2010 00:41:54 +0400, Nick Sabalausky <a@a.a> wrote:
>
>> "Rainer Schuetze" <r.sagitario@gmx.de> wrote in message news:i9f2ce$30re$1@digitalmars.com...
>>>
>>> I've used ddmd as a medium sized project for testing. It has more than
>>> 400
>>> files with most modules mutually importing each other. Some remarks:
>>>
>>> - compilation speed is a lot worse, even when compared to what you would
>>> expect from a similar C/C++ project. It takes almost 8 minutes to
>>> compile
>>> ddmd on my system with single file compilation, while the standard
>>> compilation takes about 10 seconds.
>>>
>>
>> Whaa...? Standard compilation of ddmd takes me a full two minutes.
>>
>>
>
> Do you compile the code on 486 or something? :)

Heh :)  Compared to your and Rainer's systems it probably might as well be.

> Running the build script  takes 7.967 seconds for me (timed with timeit, part of the Windows Server  2003 Resource Kit Tools).

My guess is it may be the multi-threaded "process multiple modules in parallel" optimization. DDMD does have a *lot* of source files. And I'm on a single-core (yea, yea, I know, but it works for me and I have no money).

I'm curious, is it possible for you or Rainer to time it running dmd in a "force single-core" mode? Ideally using the main system/OS core (if there even is one)? I would expect a normal out-of-the-box dmd would be noticably faster for you two just because of that multi-thread/core optimization and the cores themselves just simply being faster, but 120 sec vs 8 sec still seems an excessive difference.

And then considering also, outside of dmd bugs, I've never once had any D program besides ddmd take more than a few seconds - despite the fact that much of what I do complie is fairly template/mixin/ctfe-heavy, and ddmd appears to use very little (if any) of that - it's certainly still a conceivable speed difference, but it does make me wonder if something else might be going on.

BTW, anyone know of an easy way to check how much time in a process is spent waiting on disk IO? It's entirely possible that my disk is just fragmented all to hell.

On Tue, 19 Oct 2010 06:26:23 +0400, Nick Sabalausky <a@a.a> wrote:

> "Denis Koroskin" <2korden@gmail.com> wrote in message
> news:op.vkshh4bdo7cclz@korden-pc...
>> On Tue, 19 Oct 2010 00:41:54 +0400, Nick Sabalausky <a@a.a> wrote:
>>
>>> "Rainer Schuetze" <r.sagitario@gmx.de> wrote in message
>>> news:i9f2ce$30re$1@digitalmars.com...
>>>>
>>>> I've used ddmd as a medium sized project for testing. It has more than
>>>> 400
>>>> files with most modules mutually importing each other. Some remarks:
>>>>
>>>> - compilation speed is a lot worse, even when compared to what you would
>>>> expect from a similar C/C++ project. It takes almost 8 minutes to
>>>> compile
>>>> ddmd on my system with single file compilation, while the standard
>>>> compilation takes about 10 seconds.
>>>>
>>>
>>> Whaa...? Standard compilation of ddmd takes me a full two minutes.
>>>
>>>
>>
>> Do you compile the code on 486 or something? :)
>
> Heh :)  Compared to your and Rainer's systems it probably might as well be.
>
>> Running the build script  takes 7.967 seconds for me (timed with timeit,
>> part of the Windows Server  2003 Resource Kit Tools).
>
> My guess is it may be the multi-threaded "process multiple modules in
> parallel" optimization. DDMD does have a *lot* of source files. And I'm on a
> single-core (yea, yea, I know, but it works for me and I have no money).
>
> I'm curious, is it possible for you or Rainer to time it running dmd in a
> "force single-core" mode? Ideally using the main system/OS core (if there
> even is one)? I would expect a normal out-of-the-box dmd would be noticably
> faster for you two just because of that multi-thread/core optimization and
> the cores themselves just simply being faster, but 120 sec vs 8 sec still
> seems an excessive difference.
>
> And then considering also, outside of dmd bugs, I've never once had any D
> program besides ddmd take more than a few seconds - despite the fact that
> much of what I do complie is fairly template/mixin/ctfe-heavy, and ddmd
> appears to use very little (if any) of that - it's certainly still a
> conceivable speed difference, but it does make me wonder if something else
> might be going on.
>
> BTW, anyone know of an easy way to check how much time in a process is spent
> waiting on disk IO? It's entirely possible that my disk is just fragmented
> all to hell.
>
>

dmd doesn't use multiple core for anything BUT asynchronous reading from disk. Everything else is done is a single thread. I don't think it has THAT big of impact, but I'll try to recompile with all the cores disabled.

"Denis Koroskin" <2korden@gmail.com> wrote in message news:op.vksxyn15o7cclz@korden-pc...
> On Tue, 19 Oct 2010 06:26:23 +0400, Nick Sabalausky <a@a.a> wrote:
>
>> "Denis Koroskin" <2korden@gmail.com> wrote in message
>>> Running the build script  takes 7.967 seconds for me (timed with timeit, part of the Windows Server  2003 Resource Kit Tools).
>>
>> My guess is it may be the multi-threaded "process multiple modules in
>> parallel" optimization. DDMD does have a *lot* of source files. And I'm
>> on a
>> single-core (yea, yea, I know, but it works for me and I have no money).
>>
>> I'm curious, is it possible for you or Rainer to time it running dmd in a
>> "force single-core" mode? Ideally using the main system/OS core (if there
>> even is one)? I would expect a normal out-of-the-box dmd would be
>> noticably
>> faster for you two just because of that multi-thread/core optimization
>> and
>> the cores themselves just simply being faster, but 120 sec vs 8 sec still
>> seems an excessive difference.
>>
>> And then considering also, outside of dmd bugs, I've never once had any D
>> program besides ddmd take more than a few seconds - despite the fact that
>> much of what I do complie is fairly template/mixin/ctfe-heavy, and ddmd
>> appears to use very little (if any) of that - it's certainly still a
>> conceivable speed difference, but it does make me wonder if something
>> else
>> might be going on.
>>
>> BTW, anyone know of an easy way to check how much time in a process is
>> spent
>> waiting on disk IO? It's entirely possible that my disk is just
>> fragmented
>> all to hell.
>>
>>
>
> dmd doesn't use multiple core for anything BUT asynchronous reading from disk. Everything else is done is a single thread.

Ahh, I see. So much for that theory then.

> I don't think it has  THAT big of impact, but I'll try to recompile with all the cores disabled.

Thanks, that'll be interesting.

I did go ahead and re-time the compile. Apparently I must have been remembering it wrong, because this time it only took 1 min 20 sec (and this was with a ton of stuff running - bunch of misc apps, FF2 with 20 tabs, an HDD SMART monitor, a torrent manager and a bunch of other servers (but no clients connected)). But that's still quite a lot of time to compile a D app, even for my machine.

I also tried grabbing the latest ddmd, rebooted, killed all non-essential processes, and tried that way. Got it down to just slightly under one minute.

I thought about maybe it being a limited-memory issue (remembering that dmd never frees anything until it's done - or is that just CTFE?), but I don't think that's it - the highest memory usage it ever got was about 200MB, and I have 1GB, and it still took a whole minute with almost everything besides XP shut down, so I'm not sure that was it. (I could have sworn I had 2GB at one point, but I think I probably cannibalized one of the sticks when I built my linux box - not that that's really relevant ;) )

FWIW, this is all with compiling the just the debug version of ddmd only. Ie, not including building the release version or the one-time initial setup of building dmd.lib.

On Tue, 19 Oct 2010 07:53:45 +0400, Nick Sabalausky <a@a.a> wrote:

> "Denis Koroskin" <2korden@gmail.com> wrote in message
> news:op.vksxyn15o7cclz@korden-pc...
>> On Tue, 19 Oct 2010 06:26:23 +0400, Nick Sabalausky <a@a.a> wrote:
>>
>>> "Denis Koroskin" <2korden@gmail.com> wrote in message
>>>> Running the build script  takes 7.967 seconds for me (timed with timeit,
>>>> part of the Windows Server  2003 Resource Kit Tools).
>>>
>>> My guess is it may be the multi-threaded "process multiple modules in
>>> parallel" optimization. DDMD does have a *lot* of source files. And I'm
>>> on a
>>> single-core (yea, yea, I know, but it works for me and I have no money).
>>>
>>> I'm curious, is it possible for you or Rainer to time it running dmd in a
>>> "force single-core" mode? Ideally using the main system/OS core (if there
>>> even is one)? I would expect a normal out-of-the-box dmd would be
>>> noticably
>>> faster for you two just because of that multi-thread/core optimization
>>> and
>>> the cores themselves just simply being faster, but 120 sec vs 8 sec still
>>> seems an excessive difference.
>>>
>>> And then considering also, outside of dmd bugs, I've never once had any D
>>> program besides ddmd take more than a few seconds - despite the fact that
>>> much of what I do complie is fairly template/mixin/ctfe-heavy, and ddmd
>>> appears to use very little (if any) of that - it's certainly still a
>>> conceivable speed difference, but it does make me wonder if something
>>> else
>>> might be going on.
>>>
>>> BTW, anyone know of an easy way to check how much time in a process is
>>> spent
>>> waiting on disk IO? It's entirely possible that my disk is just
>>> fragmented
>>> all to hell.
>>>
>>>
>>
>> dmd doesn't use multiple core for anything BUT asynchronous reading from
>> disk. Everything else is done is a single thread.
>
> Ahh, I see. So much for that theory then.
>
>> I don't think it has  THAT big of impact, but I'll try to recompile with
>> all the cores disabled.
>
> Thanks, that'll be interesting.
>
> I did go ahead and re-time the compile. Apparently I must have been
> remembering it wrong, because this time it only took 1 min 20 sec (and this
> was with a ton of stuff running - bunch of misc apps, FF2 with 20 tabs, an
> HDD SMART monitor, a torrent manager and a bunch of other servers (but no
> clients connected)). But that's still quite a lot of time to compile a D
> app, even for my machine.
>
> I also tried grabbing the latest ddmd, rebooted, killed all non-essential
> processes, and tried that way. Got it down to just slightly under one
> minute.
>
> I thought about maybe it being a limited-memory issue (remembering that dmd
> never frees anything until it's done - or is that just CTFE?), but I don't
> think that's it - the highest memory usage it ever got was about 200MB, and
> I have 1GB, and it still took a whole minute with almost everything besides
> XP shut down, so I'm not sure that was it. (I could have sworn I had 2GB at
> one point, but I think I probably cannibalized one of the sticks when I
> built my linux box - not that that's really relevant ;) )
>
> FWIW, this is all with compiling the just the debug version of ddmd only.
> Ie, not including building the release version or the one-time initial setup
> of building dmd.lib.
>
>

It's 8.6 seconds for a single cores, 8.3s for all 4 cores (Core2 Quad Q8300 @ 2.5Ghz, Windows).

"Denis Koroskin" <2korden@gmail.com> wrote in message news:op.vks9nlljo7cclz@korden-pc...
> On Tue, 19 Oct 2010 07:53:45 +0400, Nick Sabalausky <a@a.a> wrote:
>
>> "Denis Koroskin" <2korden@gmail.com> wrote in message news:op.vksxyn15o7cclz@korden-pc...
>>> I don't think it has  THAT big of impact, but I'll try to recompile with all the cores disabled.
>>
>> Thanks, that'll be interesting.
>>
>> I did go ahead and re-time the compile. Apparently I must have been
>> remembering it wrong, because this time it only took 1 min 20 sec (and
>> this
>> was with a ton of stuff running - bunch of misc apps, FF2 with 20 tabs,
>> an
>> HDD SMART monitor, a torrent manager and a bunch of other servers (but no
>> clients connected)). But that's still quite a lot of time to compile a D
>> app, even for my machine.
>>
>> I also tried grabbing the latest ddmd, rebooted, killed all non-essential processes, and tried that way. Got it down to just slightly under one minute.
>>
>> I thought about maybe it being a limited-memory issue (remembering that
>> dmd
>> never frees anything until it's done - or is that just CTFE?), but I
>> don't
>> think that's it - the highest memory usage it ever got was about 200MB,
>> and
>> I have 1GB, and it still took a whole minute with almost everything
>> besides
>> XP shut down, so I'm not sure that was it. (I could have sworn I had 2GB
>> at
>> one point, but I think I probably cannibalized one of the sticks when I
>> built my linux box - not that that's really relevant ;) )
>>
>> FWIW, this is all with compiling the just the debug version of ddmd only.
>> Ie, not including building the release version or the one-time initial
>> setup
>> of building dmd.lib.
>>
>>
>
> It's 8.6 seconds for a single cores, 8.3s for all 4 cores (Core2 Quad Q8300 @ 2.5Ghz, Windows).

Hmm... that does make my situation seem odd then. Mine's a 1.7 GHz Celeron, so a little more than half the clock speed of yours. Of course, I'm well aware that my older architecture and less cache make mine slower than (2.5/1.7) of your speed, but that still seems like a strangely large difference.

I wonder if maybe RAM speed could account for it.

"Nick Sabalausky" <a@a.a> wrote in message news:i9jr29$6af$1@digitalmars.com...
> "Denis Koroskin" <2korden@gmail.com> wrote in message news:op.vks9nlljo7cclz@korden-pc...
>> On Tue, 19 Oct 2010 07:53:45 +0400, Nick Sabalausky <a@a.a> wrote:
>>
>>> "Denis Koroskin" <2korden@gmail.com> wrote in message news:op.vksxyn15o7cclz@korden-pc...
>>>> I don't think it has  THAT big of impact, but I'll try to recompile
>>>> with
>>>> all the cores disabled.
>>>
>>> Thanks, that'll be interesting.
>>>
>>> I did go ahead and re-time the compile. Apparently I must have been
>>> remembering it wrong, because this time it only took 1 min 20 sec (and
>>> this
>>> was with a ton of stuff running - bunch of misc apps, FF2 with 20 tabs,
>>> an
>>> HDD SMART monitor, a torrent manager and a bunch of other servers (but
>>> no
>>> clients connected)). But that's still quite a lot of time to compile a D
>>> app, even for my machine.
>>>
>>> I also tried grabbing the latest ddmd, rebooted, killed all
>>> non-essential
>>> processes, and tried that way. Got it down to just slightly under one
>>> minute.
>>>
>>> I thought about maybe it being a limited-memory issue (remembering that
>>> dmd
>>> never frees anything until it's done - or is that just CTFE?), but I
>>> don't
>>> think that's it - the highest memory usage it ever got was about 200MB,
>>> and
>>> I have 1GB, and it still took a whole minute with almost everything
>>> besides
>>> XP shut down, so I'm not sure that was it. (I could have sworn I had 2GB
>>> at
>>> one point, but I think I probably cannibalized one of the sticks when I
>>> built my linux box - not that that's really relevant ;) )
>>>
>>> FWIW, this is all with compiling the just the debug version of ddmd
>>> only.
>>> Ie, not including building the release version or the one-time initial
>>> setup
>>> of building dmd.lib.
>>>
>>>
>>
>> It's 8.6 seconds for a single cores, 8.3s for all 4 cores (Core2 Quad Q8300 @ 2.5Ghz, Windows).
>
> Hmm... that does make my situation seem odd then. Mine's a 1.7 GHz Celeron, so a little more than half the clock speed of yours. Of course, I'm well aware that my older architecture and less cache make mine slower than (2.5/1.7) of your speed, but that still seems like a strangely large difference.
>
> I wonder if maybe RAM speed could account for it.
>

s/(2.5/1.7)/(1.7/2.5)

Too late in the day for math...