Im running some tests with D. Was wondering whats the dependency
story. Cant find any info online, searched for dlang dependency
management and dlang dependency. Found bunch o dub stuff but not
the nitty gritty.

Unit of compilation is one D file but I saw if I pass several D
files to the compiler only one .o file is generated. Whats the
story there.

Plus if I change a non template function in one module then
theres no way to tell make no rebuild of modules importing it.
The dmd -deps call seems to generate all recursively. In
gcc/makedepend if one function changes only relinking is needed
or nothing if dynamic loading.

Overall as far as I understand compiler is fast but dependency
mgmt is coarse. Also whats the deal with cyclic dependencies. I
saw discussion that its not possible but wrote a few test modules
and it works fine.

Am I grokking this and how can I help it thanx.

On Thursday, 18 September 2014 at 16:48:22 UTC, Scott Wilson wrote:
> Unit of compilation is one D file but I saw if I pass several D
> files to the compiler only one .o file is generated. Whats the
> story there.

DMD will generate one object file per module file, unless you use the -of option. With -of, the compiler will compile all module files to a single object file.

> Plus if I change a non template function in one module then
> theres no way to tell make no rebuild of modules importing it.
> The dmd -deps call seems to generate all recursively. In
> gcc/makedepend if one function changes only relinking is needed
> or nothing if dynamic loading.

Yes, currently any changes, including function bodies, cause a rebuild of importing modules. One reason for this is CTFE - changing a function body can affect the code in importing modules if they invoke the function during compilation. You could use .di files to separate declarations from implementations.

> Overall as far as I understand compiler is fast but dependency
> mgmt is coarse. Also whats the deal with cyclic dependencies. I
> saw discussion that its not possible but wrote a few test modules
> and it works fine.

Cyclic dependencies between modules which all have module or static constructors are forbidden, because it is unknown in which order the constructors are expected to run.

On Thu, 18 Sep 2014 16:48:21 +0000
Scott Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> Plus if I change a non template function in one module then theres no way to tell make no rebuild of modules importing it. The dmd -deps call seems to generate all recursively. In gcc/makedepend if one function changes only relinking is needed or nothing if dynamic loading.
there is no way to tell what exactly was changed. this can be some template, for example, and with changed template all modules that import that one with template must be recompiled to accomodate new version. or think about CTFE.

On Thursday, 18 September 2014 at 17:04:43 UTC, ketmar via
Digitalmars-d wrote:
> On Thu, 18 Sep 2014 16:48:21 +0000
> Scott Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
>> Plus if I change a non template function in one module then
>> theres no way to tell make no rebuild of modules importing it.
>> The dmd -deps call seems to generate all recursively. In
>> gcc/makedepend if one function changes only relinking is needed
>> or nothing if dynamic loading.
> there is no way to tell what exactly was changed. this can be some
> template, for example, and with changed template all modules that import
> that one with template must be recompiled to accomodate new version. or
> think about CTFE.

Thanks fellas. But whats the thing with .di files? AFAIU the
compiler generates them and dependent modules can depend on them
instead of .d files directly.

Do .di files contain only templates (no comments and plain
functions? How well do they work? thanx

On Fri, 19 Sep 2014 01:42:58 +0000
Scott Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> Do .di files contain only templates (no comments and plain functions? How well do they work? thanx
as for 'how .di files work' question: '.di' is just a plain D source, just with stripped function bodies. nothing very special about that.


so yes, .di files contains only templates and function declarations (without function bodies). this *can* work, but when it comes to CTFE...

look at the following:

=== z00.d ===
  module z00;
  string foo(string name) { return `int `~name~`() {return 42;}`; }

=== z01.d ===
  import z00;
  mixin(foo(`bar`));
  void main () {
    import std.stdio;
    writeln(bar());
  }

and .di file generated with `dmd -H -c -o- z00.d`:
=== z00.di ===
  // D import file generated from 'z00.d'
  module z00;
  string foo(string name);


do you see any gotchas? heh:
# dmd z01.d
z01.d(2): Error: foo cannot be interpreted at compile time, because it
has no available source code
z01.d(2): Error: argument to mixin must be a string, not (foo("bar"))
of type string

the compiler has no source for foo() anymore, so it can't do CTFE.

you can avoid this by turning foo() into template:
  string foo()(string name) { return `int `~name~`() {return 42;}`; }

but then you should turn all your functions that can be used in CTFE into templates, and there will be no much sense in .di file anyway.


to make a long story short: don't use .di files unless you *REALLY* *KNOW* what you're doing. and even then think twice.

On 9/18/2014 9:48 AM, Scott Wilson wrote:
> Im running some tests with D. Was wondering whats the dependency
> story. Cant find any info online, searched for dlang dependency
> management and dlang dependency. Found bunch o dub stuff but not
> the nitty gritty.
>
> Unit of compilation is one D file but I saw if I pass several D
> files to the compiler only one .o file is generated. Whats the
> story there.

The idea is to essentially "pre-link" the object files that would have been generated if the files were compiled individually into one object file. This is faster and more convenient.

It also means that semantic analysis is done only once for each file, and the imports, rather than doing it over and over as is done with separate compilation.


> Plus if I change a non template function in one module then
> theres no way to tell make no rebuild of modules importing it.
> The dmd -deps call seems to generate all recursively. In
> gcc/makedepend if one function changes only relinking is needed
> or nothing if dynamic loading.

Dependency management is the same as in C++, if you are willing to use .di files to represent 'headers' of corresponding .d files.


> Overall as far as I understand compiler is fast but dependency
> mgmt is coarse. Also whats the deal with cyclic dependencies. I
> saw discussion that its not possible but wrote a few test modules
> and it works fine.

If two modules import each other, then if one changes, both should get recompiled.

On 9/18/2014 9:56 AM, Vladimir Panteleev wrote:
> On Thursday, 18 September 2014 at 16:48:22 UTC, Scott Wilson wrote:
>> Unit of compilation is one D file but I saw if I pass several D
>> files to the compiler only one .o file is generated. Whats the
>> story there.
>
> DMD will generate one object file per module file, unless you use the -of
> option.

This is incorrect. It'll generate one object file per invocation of dmd.

On Friday, 19 September 2014 at 05:19:07 UTC, Walter Bright wrote:
> On 9/18/2014 9:56 AM, Vladimir Panteleev wrote:
>> On Thursday, 18 September 2014 at 16:48:22 UTC, Scott Wilson wrote:
>>> Unit of compilation is one D file but I saw if I pass several D
>>> files to the compiler only one .o file is generated. Whats the
>>> story there.
>>
>> DMD will generate one object file per module file, unless you use the -of
>> option.
>
> This is incorrect. It'll generate one object file per invocation of dmd.

Erm, that's not quite correct either. I meant, one object file per module file passed on the command line.

On 9/18/2014 10:27 PM, Vladimir Panteleev wrote:
> On Friday, 19 September 2014 at 05:19:07 UTC, Walter Bright wrote:
>> On 9/18/2014 9:56 AM, Vladimir Panteleev wrote:
>>> On Thursday, 18 September 2014 at 16:48:22 UTC, Scott Wilson wrote:
>>>> Unit of compilation is one D file but I saw if I pass several D
>>>> files to the compiler only one .o file is generated. Whats the
>>>> story there.
>>>
>>> DMD will generate one object file per module file, unless you use the -of
>>> option.
>>
>> This is incorrect. It'll generate one object file per invocation of dmd.
>
> Erm, that's not quite correct either. I meant, one object file per module file
> passed on the command line.

Yeah, you're right. My mistake.

In general D with current compiler technology is not very suitable for incremental rebuilds. In C there is a very simple separation between implementation and the imported interface. C++ makes it much harder by introducing templates which also must be present in header files - it took quite a while for C++ compilers to stop screwing incremental compilation in presence of templates and optimization. For D it is even harder because of CTFE and the fact that by default .di headers are not generated.

For now compiling everything in one go is much superior strategy assuming you have enough memory. In future incremental compilation topic may be revisited when something like compiler daemon becomes feasible, one that will be able to cache AST level entities and their dependencies, as opposed to filesystem level entities.