Hi,

I'm not sure if this has been brought up before, but anyway:

I've noticed (in a project currently containing 37 source files) that building everything in one DMD call is much faster than using SCons (which is supposed to speed up building by not rebuilding everything every time). I guess this is because DMD has to repeatedly lex the imported modules.

I think that a) SCons should be modified to build everything in one DMD call and b) dmd might add some kind of caching of the lexed data between compiler calls.

Is any of this possible? How do other build tools (such as aap) handle this?

-Sebastian

Whoops, this probably should've been in the main newsgroup...

Sebastian Beschke wrote:
> Hi,
> 
> I'm not sure if this has been brought up before, but anyway:
> 
> I've noticed (in a project currently containing 37 source files) that building everything in one DMD call is much faster than using SCons (which is supposed to speed up building by not rebuilding everything every time). I guess this is because DMD has to repeatedly lex the imported modules.
> 
> I think that a) SCons should be modified to build everything in one DMD call and b) dmd might add some kind of caching of the lexed data between compiler calls.
> 
> Is any of this possible? How do other build tools (such as aap) handle this?
> 
> -Sebastian

Rebuilding everything is /much faster/ than just compiling the changed
source?  This sounds really counterintuitive.
Why is it so, Walter?

zwang schrieb:
> Sebastian Beschke wrote:
> 
>> Hi,
>>
>> I'm not sure if this has been brought up before, but anyway:
>>
>> I've noticed (in a project currently containing 37 source files) that building everything in one DMD call is much faster than using SCons (which is supposed to speed up building by not rebuilding everything every time). I guess this is because DMD has to repeatedly lex the imported modules.
>>
>> I think that a) SCons should be modified to build everything in one DMD call and b) dmd might add some kind of caching of the lexed data between compiler calls.
>>
>> Is any of this possible? How do other build tools (such as aap) handle this?
>>
>> -Sebastian
> 
> 
> Rebuilding everything is /much faster/ than just compiling the changed
> source?  This sounds really counterintuitive.
> Why is it so, Walter?

I guess I can already tell you the reason. SCons calls the compiler for every single source file. This means that dmd has to lex the imports once for every file. If I, however, supply all files on the command line at once, dmd can keep the lexed files in memory.

Basically, SCons seems to assume that the cost to compile any single file is a lot higher than the cost of lexing the imports. The reverse seems to be the case.

-Sebastian

Sebastian Beschke wrote:
> Hi,
> 
> I'm not sure if this has been brought up before, but anyway:
> 
> I've noticed (in a project currently containing 37 source files) that building everything in one DMD call is much faster than using SCons (which is supposed to speed up building by not rebuilding everything every time). I guess this is because DMD has to repeatedly lex the imported modules.
> 
> I think that a) SCons should be modified to build everything in one DMD call and b) dmd might add some kind of caching of the lexed data between compiler calls.
> 
> Is any of this possible? How do other build tools (such as aap) handle this?

I've noticed this as well, and I've been meaning to look into modifying SCons precisely as you suggest, but I haven't been able to find the time yet. :\

 -- andy

Sebastian Beschke wrote:
> I guess I can already tell you the reason. SCons calls the compiler for every single source file. This means that dmd has to lex the imports once for every file. If I, however, supply all files on the command line at once, dmd can keep the lexed files in memory.
> 
> Basically, SCons seems to assume that the cost to compile any single file is a lot higher than the cost of lexing the imports. The reverse seems to be the case.
> 
> -Sebastian

I see the point now.  It seems to me then that you don't need SCons at all :)

A-A-P has an option to build everything at once (the feature is only implemented for D AFAIK, and added because I found it useful). Thus the following commands in an .aap file will build a program without building each sourcefile as an object first;

------------

:import d

SOURCES = foo1.d
	  foo2.d
	  #etc.

:program {onestep} foo : $SOURCES

Sebastian Beschke wrote:

> Is any of this possible? How do other build tools (such as aap) handle this?

A-A-P can handle this (see my shameless plug on the main NG).

A-A-P need testers badly to get any further.

Lars Ivar Igesund

On Sun, 30 Jan 2005 14:28:48 +0100, Sebastian Beschke wrote:

> Hi,
> 
> I'm not sure if this has been brought up before, but anyway:
> 
> I've noticed (in a project currently containing 37 source files) that building everything in one DMD call is much faster than using SCons (which is supposed to speed up building by not rebuilding everything every time). I guess this is because DMD has to repeatedly lex the imported modules.
> 
> I think that a) SCons should be modified to build everything in one DMD call and b) dmd might add some kind of caching of the lexed data between compiler calls.
> 
> Is any of this possible? How do other build tools (such as aap) handle this?
> 
> -Sebastian

I also have a tool that builds D apps. It is derived from dmake, and just needs a few finishing touches before I can give to the community. I'll be working on that today.

-- 
Derek
Melbourne, Australia
31/01/2005 9:20:37 AM

On Mon, 31 Jan 2005 02:52:28 +0800, zwang wrote:

> Sebastian Beschke wrote:
>> I guess I can already tell you the reason. SCons calls the compiler for every single source file. This means that dmd has to lex the imports once for every file. If I, however, supply all files on the command line at once, dmd can keep the lexed files in memory.
>> 
>> Basically, SCons seems to assume that the cost to compile any single file is a lot higher than the cost of lexing the imports. The reverse seems to be the case.
>> 
>> -Sebastian
> 
> I see the point now.  It seems to me then that you don't need SCons at all :)

I said that over a year(?) ago.
But other compilers (GDC) aren't as fast.

Ant