On Thursday, 13 August 2015 at 08:19:15 UTC, Ola Fosheim Grøstad wrote:
> On Thursday, 13 August 2015 at 00:54:37 UTC, JDemler wrote:
>> I am not sure if I understand your idea correctly:
>> The compiler would in case of an export(name, content) write the content both to a internal database and the filesystem and then only use the internal one?
>> Or could the content of the internal database be copied to the file system at the end of the compilation process? Or is only the output bundle written to the file system?
>
> The filesystem is a compiler issue and not a language issue, so sure, the compiler could do whatever it wants, flush everything to disk or into a SQL database or…
>
> Keep in mind that someone might want to compile on a diskless computer from an in-memory zip-file or similar.
>
> I think the important file generation that are relevant to the language would be for the output bundle, so that you e.g. can have a config file in D and generate platform specific files (.ini, .xml etc).

To summarize:
We would have a central registry handling all the generated and non generated files.
If an import is triggered, this registry is then searched for the imported file. If this file does not already exist, the import blocks. Writing these generated files to the file system would be optional, triggered by a compiler switch and happen after the compilation process is complete.

This solves the concurrency problem. But what about non concurrent compiler runs? A block there would never be resolved. I do not know enough about compilers to judge this problem. Can a sane way of compiling the files be found easily? Can the compiler switch to compile something else when it is blocked on an import?

And I do not understand why you differentiate between generated source files and generated non source files (.xml, .ini). As both types can be imported (or read at compile time) I do not think we should treat them differently. Also we would need a way to tell the one from the other (file extension? different parameter in the export syntax?).
In my understanding storage area 2 and 3 should be merged.

On Thursday, 13 August 2015 at 09:52:02 UTC, JDemler wrote:
> This solves the concurrency problem. But what about non concurrent compiler runs? A block there would never be resolved. I do not know enough about compilers to judge this problem. Can a sane way of compiling the files be found easily? Can the compiler switch to compile something else when it is blocked on an import?

You need a highly concurrent compiler. If you import in one definition and export in the next definition, then the first one would block and be queued, and the compiler would move on to the next definition.

In the actor-model (co-routines/fibers) each definition will spawn a new actor.

> And I do not understand why you differentiate between generated source files and generated non source files (.xml, .ini). As both types can be imported (or read at compile time) I do not think we should treat them differently. Also we would need a way to tell the one from the other (file extension? different parameter in the export syntax?).
> In my understanding storage area 2 and 3 should be merged.

Area 2 is pre-populated with area 1, so they are the same.

You need to be able to tell the compiler what files should be included in the output bundle.

Actually, I think the file-model is too simple. I think you need a more advanced key-value database so that you can update different fields in the same ".ini file" from different source files.

On Thursday, 13 August 2015 at 06:53:14 UTC, Jacob Carlborg wrote:
> On 2015-08-13 01:27, Tofu Ninja wrote:
>
>> A simple alternative to this would be a flag to the compiler to expand
>> mixins and output the new files. This would also be great for tooling,
>> an IDE could use this and allow you to expand a mixin in place to see
>> what it looks like. And currently all the auto complete engines I have
>> seen for D don't handle mixins very well. Expanding them would make
>> autocompletion a simpler job.
>
> I agree, this is much better. I remember the old Eclipse plugin, Descent. It had a compile time view which showed how mixins where expanded and how some language features where lowered, i.e. "scope" to try-catch-finally.

This would be great but we would need some support for the formatting I think, mixin code tends to be a mess, it would be nice to have some basic auto indent or general FMT style support.

On Thursday, 13 August 2015 at 10:06:32 UTC, Ola Fosheim Grøstad wrote:
> You need to be able to tell the compiler what files should be included in the output bundle.

While I basically agree that this would be a nice to have feature I do not think it is necessary and it would not make up for the added complexity inside the export syntax.
If a specific source file should not be written to the file system one can still use mixins.

>
> Actually, I think the file-model is too simple. I think you need a more advanced key-value database so that you can update different fields in the same ".ini file" from different source files.

This would a huge amount of complexity to the feature and would not work in general. One can add keys to .ini and maybe .config files but not to a .sql or .css file.
Also a append or prepend feature would make it impossible for the compiler to tell when a generated file is finished.

Based on this discussion I updated the DIP.
Included are now the idea of mixin and template expanding (in the section alternatives) and Ola's solution to the basic problems which I originally missed out on (in the section implementation considerations).

All in all I have to admit that the original approach was not designed well enough. It was simple but not functional. Alas, this new approach brings with it a completly different implementation complexity.
Nevertheless, the idea of writing files at compile time is still worth discussing about.

On Wednesday, 12 August 2015 at 18:37:40 UTC, JDemler wrote:
> Triggered by the original forum thread I wrote a DIP to further explain the idea of the writing files at compile time feature and its implications.
>
> http://wiki.dlang.org/DIP81
>
> Please discuss!

I like the idea of being able to write files at compile-time.
Is there any progress on this.

Am 08.03.2016 um 11:28 schrieb Stefan Koch:
> On Wednesday, 12 August 2015 at 18:37:40 UTC, JDemler wrote:
>> Triggered by the original forum thread I wrote a DIP to further
>> explain the idea of the writing files at compile time feature and its
>> implications.
>>
>> http://wiki.dlang.org/DIP81
>>
>> Please discuss!
>
> I like the idea of being able to write files at compile-time.
> Is there any progress on this.

No progress on the DIP, I made an alternative implementation for vibe.d's Diet templates using multiple `shared static this()` and writing the files at run time:

https://github.com/rejectedsoftware/vibe.d/pull/1385

Performance results are very promising. I don't know which way is nicer (or uglier), but at least for this use case, the DIP isn't strictly necessary.