On Thursday, 24 August 2017 at 17:37:12 UTC, Jonathan Marler wrote:
> On Thursday, 24 August 2017 at 16:49:08 UTC, Seb wrote:
>> On Thursday, 24 August 2017 at 16:32:32 UTC, Jonathan Marler wrote:
>>> On Thursday, 24 August 2017 at 15:56:32 UTC, H. S. Teoh wrote:
>>>> On Thu, Aug 24, 2017 at 03:53:05PM +0000, Jonathan Marler via Digitalmars-d wrote:
>>>>> Wanted to get peoples thoughts on this.  The idea is to have a way to tell the compiler (probably with a command line option) that you'd like to "compile imported modules".
>>>> [...]
>>>>
>>>> Isn't this what rdmd already does?
>>>>
>>>>
>>>> T
>>>
>>> That is one thing that rdmd does (as I mentioned in the original post).
>>>
>>> I just looked through the rdmd code (https://github.com/dlang/tools/blob/master/rdmd.d) and it looks like it invokes the compiler using "dmd -v" to get the list of modules and then invokes the compiler again with the modules it found to perform the full compile.  So my original thought that the logic to find modules is duplicated was incorrect.  Instead we just pay a performance hit to get the correct list of imports since running "dmd -v" seems to take almost as long as the actual compile itself.  So this method comes close to doubling the time it takes to compile than if the feature was implemented in the compiler itself.
>>>
>>> In any case, the idea is to allow the compiler to resolve this on it's own without help from rdmd.  This would remove the need to invoke the compiler twice, once to find the imports and once to compile.  It would also allow some projects/applications that don't use rdmd to take advantage of this feature, this may or may not include dub (not sure on that one).
>>
>> rdmd is really bad in terms of performance. If you call a single D file with rdmd, it will always compile it twice. There was an attempt to fix this (https://github.com/dlang/tools/pull/194), but this has been reverted as it introduced a regression and no one had time to look at the regression.
>> Moving rdmd into DMD has been on the TODO list for quite a while and there is a consensus that the performance overhead if rdmd isn't nice. However, IIRC there was no clear consensus on how the integration should happen. I recall that the plan was to do try this with "dmd as a library", but I'm not sure whether that's really feasible ATM.
>
> Well this should solve the rdmd performance problem as well as make other user cases easier that don't necessarilly use rdmd.
>
> I had another thought that instead of making this an "opt-in" feature, it would probably make more sense to be an "opt-out" feature.  So by default the compiler would compile missing imported modules unless you indicate otherwise, maybe a command line switch like "-dont-compile-imports".  And I don't see how this would break anything.  Everything should work the same as it did before, it's just now you can omit imported module files from the command line and it should just work.

I've looked through the DMD code to see how this could be implemented and I've run into a problem.  The solution I came up with was to go through all the imported modules and then determine which ones need to be compiled that haven't been given on the command line. The problem is, I don't know how to determine whether a module was already compiled and given in an obj/lib file.  For example,

dmd something.obj anotherthing.lib prog.d

As far as I know, the compiler has no idea which modules are contained in "something.obj" and "anotherthing.lib".  It just compiles the source given on then command line, then passes all the object files and libraries to the linker, at which point the concept of modules is lost.

Am I correct in saying that the compiler has no idea which modules an obj/lib file contains?

On Thursday, 24 August 2017 at 15:56:32 UTC, H. S. Teoh wrote:
>
> Isn't this what rdmd already does?
>

This is my favorite rdmd feature.
But there is local import bug :(
https://issues.dlang.org/show_bug.cgi?id=15533

On Friday, 25 August 2017 at 01:50:00 UTC, Jonathan Marler wrote:
> On Thursday, 24 August 2017 at 17:37:12 UTC, Jonathan Marler wrote:
>> On Thursday, 24 August 2017 at 16:49:08 UTC, Seb wrote:
>>> On Thursday, 24 August 2017 at 16:32:32 UTC, Jonathan Marler wrote:
>>>> [...]
>>>
>>> rdmd is really bad in terms of performance. If you call a single D file with rdmd, it will always compile it twice. There was an attempt to fix this (https://github.com/dlang/tools/pull/194), but this has been reverted as it introduced a regression and no one had time to look at the regression.
>>> Moving rdmd into DMD has been on the TODO list for quite a while and there is a consensus that the performance overhead if rdmd isn't nice. However, IIRC there was no clear consensus on how the integration should happen. I recall that the plan was to do try this with "dmd as a library", but I'm not sure whether that's really feasible ATM.
>>
>> Well this should solve the rdmd performance problem as well as make other user cases easier that don't necessarilly use rdmd.
>>
>> I had another thought that instead of making this an "opt-in" feature, it would probably make more sense to be an "opt-out" feature.  So by default the compiler would compile missing imported modules unless you indicate otherwise, maybe a command line switch like "-dont-compile-imports".  And I don't see how this would break anything.  Everything should work the same as it did before, it's just now you can omit imported module files from the command line and it should just work.
>
> I've looked through the DMD code to see how this could be implemented and I've run into a problem.  The solution I came up with was to go through all the imported modules and then determine which ones need to be compiled that haven't been given on the command line. The problem is, I don't know how to determine whether a module was already compiled and given in an obj/lib file.  For example,
>
> dmd something.obj anotherthing.lib prog.d
>
> As far as I know, the compiler has no idea which modules are contained in "something.obj" and "anotherthing.lib".  It just compiles the source given on then command line, then passes all the object files and libraries to the linker, at which point the concept of modules is lost.
>
> Am I correct in saying that the compiler has no idea which modules an obj/lib file contains?

I created a prototype implementation here (https://github.com/dlang/dmd/pull/7099).

It uses the same logic that rdmd uses to determine if a module exists in a given object/library file.  Pretty cool that now I can compile any code without having to list the import module files!

Instead of:

dmd prog.d -Isomelib somelib\foo\module1.d somelib\foo\module2.d somelib\foo\module3.d somelib\foo\module4.d somelib\foo\module5.d somelib\foo\module6.d -Ianotherlib anotherlib\bar\module1.d anotherlib\bar\module2.d anotherlib\bar\module3.d anotherlib\bar\module4.d anotherlib\bar\module5.d

I can do:

dmd -ci prog.d -Isomelib -Ianotherlib

On Friday, 25 August 2017 at 13:03:11 UTC, Jonathan Marler wrote:
> I can do:
>
> dmd -ci prog.d -Isomelib -Ianotherlib

I love it, thanks for doing this!

On Friday, 25 August 2017 at 13:15:35 UTC, Daniel N wrote:
> On Friday, 25 August 2017 at 13:03:11 UTC, Jonathan Marler wrote:
>> I can do:
>>
>> dmd -ci prog.d -Isomelib -Ianotherlib
>
> I love it, thanks for doing this!

Thanks, I think this is a really nice feature.  I've uploaded my build so that people can try it out here (https://github.com/marler8997/dmd/releases/tag/preview-compileimports).

Just download and unzip dmd2.zip, then run the addtopath.bat script in any shell you want to use this compiler in.  Thanks in advance to anyone who gives it a try and shares their thoughts.

On Friday, 25 August 2017 at 19:20:15 UTC, Jonathan Marler wrote:
> On Friday, 25 August 2017 at 13:15:35 UTC, Daniel N wrote:
>> On Friday, 25 August 2017 at 13:03:11 UTC, Jonathan Marler wrote:
>>> I can do:
>>>
>>> dmd -ci prog.d -Isomelib -Ianotherlib
>>
>> I love it, thanks for doing this!
>
> Thanks, I think this is a really nice feature.  I've uploaded my build so that people can try it out here (https://github.com/marler8997/dmd/releases/tag/preview-compileimports).
>
> Just download and unzip dmd2.zip, then run the addtopath.bat script in any shell you want to use this compiler in.  Thanks in advance to anyone who gives it a try and shares their thoughts.

How does that mix with implicit imports (public imports located in an explicit import) ?

On Saturday, 26 August 2017 at 06:31:11 UTC, user1234 wrote:
> On Friday, 25 August 2017 at 19:20:15 UTC, Jonathan Marler wrote:
>> On Friday, 25 August 2017 at 13:15:35 UTC, Daniel N wrote:
>>> On Friday, 25 August 2017 at 13:03:11 UTC, Jonathan Marler wrote:
>>>> I can do:
>>>>
>>>> dmd -ci prog.d -Isomelib -Ianotherlib
>>>
>>> I love it, thanks for doing this!
>>
>> Thanks, I think this is a really nice feature.  I've uploaded my build so that people can try it out here (https://github.com/marler8997/dmd/releases/tag/preview-compileimports).
>>
>> Just download and unzip dmd2.zip, then run the addtopath.bat script in any shell you want to use this compiler in.  Thanks in advance to anyone who gives it a try and shares their thoughts.
>
> How does that mix with implicit imports (public imports located in an explicit import) ?

All imported modules need to be compiled whether they were imported "explicitly" or "implicitly".  So both kinds work the same when it comes to this feature.

P.S. There are some cases where you can "get away" with not compiling a module, if it only contains declarations and templates for example.  In this case it kind of behaves like a header file in C/C++.