Just in case anyone gives a ####:

I have submitted a pull request that adds ProgramAllAtOnce as a builder in the dmd, ldc, and gdc tools of SCons. This does an 'all at once' compilation in a single compiler instantiation, unlike the standard module at a time compilation and then link the program. There are lots of arguments about whether "all at once" is at all useful, I have added it simply because it is the only way Unit-Threaded works.

If you want to use it prior to it being accepted and released, then feel free to use the tools in the repository https://github.com/russel/ SCons_D_Experiment

Also in there you will spot a new tool dub. I am now using this to grab
unit-threaded from the Dub repository in a SCons build. See for example
 https://github.com/russel/ApproxGC

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Monday, 17 April 2017 at 17:51:33 UTC, Russel Winder wrote:
> Just in case anyone gives a ####:
>
> I have submitted a pull request that adds ProgramAllAtOnce as a builder in the dmd, ldc, and gdc tools of SCons. This does an 'all at once' compilation in a single compiler instantiation, unlike the standard module at a time compilation and then link the program. There are lots of arguments about whether "all at once" is at all useful, I have added it simply because it is the only way Unit-Threaded works.

Nice. I'm using SCons in my everyday development. I'll give your feature a try.

Meson has recently added a similar feature. Meson calls it a "unity build".

> If you want to use it prior to it being accepted and released, then feel free to use the tools in the repository https://github.com/russel/ SCons_D_Experiment
>
> Also in there you will spot a new tool dub. I am now using this to grab
> unit-threaded from the Dub repository in a SCons build. See for example
>  https://github.com/russel/ApproxGC

Cool.

Thanks.

On Monday, 17 April 2017 at 17:51:33 UTC, Russel Winder wrote:
> Just in case anyone gives a ####:
>
> I have submitted a pull request that adds ProgramAllAtOnce

I have been trying to do the opposite with scons - incremental builds. In c++ the .h and .cpp files allowed the compilation from many edits to be limited to one translation unit.

One thing people are often confused about is whether everything needs to be in the header (or di file).

An influence is that most of the code they see written by skilled language users is libraries like the standard library. These typically benefit little from interfaces files because everything is templates and must be in the header to compile. Hence why boost is mostly headers in c++ land and the D ecosystem encourages all at once / all everytime compilation.

In a large code base this becomes cumbersome. Changing a unit test should not result in everything recompiling.

I think there is a difference between library style code and application code. My application code doesn't have templates and can make use of di files. It makes sense that the language developers spend most of their time on library type code, and have been focused on it.

I just need to work out how to get scons to recognise two targets from one builder properly : the object file and the di file.
I can add DFLAGs to make a Object builder make the di files too, and the d scanner finds di files where d files are not available, but the build tree dependency from d to di file is missing. Dies anyone know how to do this ?

On Monday, 17 April 2017 at 17:51:33 UTC, Russel Winder wrote:
> Just in case anyone gives a ####:
>
> I have submitted a pull request that adds ProgramAllAtOnce as a builder in the dmd, ldc, and gdc tools of SCons. This does an 'all at once' compilation in a single compiler instantiation, unlike the standard module at a time compilation and then link the program. There are lots of arguments about whether "all at once" is at all useful, I have added it simply because it is the only way Unit-Threaded works.

It's the only way __traits(getUnitTests) works. Unfortunately. I'm just going to fix that dmd bug myself.

Atila

On Monday, 17 April 2017 at 20:56:55 UTC, Alex wrote:
> On Monday, 17 April 2017 at 17:51:33 UTC, Russel Winder wrote:
>> Just in case anyone gives a ####:
>>
>> I have submitted a pull request that adds ProgramAllAtOnce
>
> I have been trying to do the opposite with scons - incremental builds. In c++ the .h and .cpp files allowed the compilation from many edits to be limited to one translation unit.

In D, this is nearly always slower (I measured). Compiling per package, however, is a lot faster.

Atila

On 4/17/17 6:58 PM, Atila Neves wrote:
> On Monday, 17 April 2017 at 17:51:33 UTC, Russel Winder wrote:
>> Just in case anyone gives a ####:
>>
>> I have submitted a pull request that adds ProgramAllAtOnce as a builder in the dmd, ldc, and gdc tools of SCons. This does an 'all at once' compilation in a single compiler instantiation, unlike the standard module at a time compilation and then link the program. There are lots of arguments about whether "all at once" is at all useful, I have added it simply because it is the only way Unit-Threaded works.
> 
> It's the only way __traits(getUnitTests) works. Unfortunately. I'm just going to fix that dmd bug myself.

Thanks!! -- Andrei

On Mon, 2017-04-17 at 20:56 +0000, Alex via Digitalmars-d wrote:
> On Monday, 17 April 2017 at 17:51:33 UTC, Russel Winder wrote:
> > Just in case anyone gives a ####:
> > 
> > I have submitted a pull request that adds ProgramAllAtOnce
> 
> I have been trying to do the opposite with scons - incremental builds. In c++ the .h and .cpp files allowed the compilation from many edits to be limited to one translation unit.

Per module compilation then linking is the default for the Program builder in SCons, that is already there and has been for over a decade.

> One thing people are often confused about is whether everything needs to be in the header (or di file).

Uuurrr… there is no separation of header and code in D, just modules. And packages. SCons currently has no "per package" builder. I guess I need to add this.

> An influence is that most of the code they see written by skilled language users is libraries like the standard library. These typically benefit little from interfaces files because everything is templates and must be in the header to compile. Hence why boost is mostly headers in c++ land and the D ecosystem encourages all at once / all everytime compilation.
> 
> In a large code base this becomes cumbersome. Changing a unit test should not result in everything recompiling.
> 
> I think there is a difference between library style code and application code. My application code doesn't have templates and can make use of di files. It makes sense that the language developers spend most of their time on library type code, and have been focused on it.
> 
> I just need to work out how to get scons to recognise two targets
> from one builder properly : the object file and the di file.
> I can add DFLAGs to make a Object builder make the di files too,
> and the d scanner finds di files where d files are not available,
> but the build tree dependency from d to di file is missing. Dies
> anyone know how to do this ?

Currently there is no explicit handling of di files in SCons. If you can come up with a small project that exhibits the problem you need solved, I can add it as a test case in the SCons system thereby creating a need for a fix.

(Tests in SCons for the tools are end-to-end tests, so need a project
exhibiting a problem.)

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Mon, 2017-04-17 at 22:59 +0000, Atila Neves via Digitalmars-d wrote:
> […]
> In D, this is nearly always slower (I measured). Compiling per
> package, however, is a lot faster.

I have only ever needed file-by-file or whole source compilation, I have never needed package-oriented compilation. It sounds like this needs adding to SCons and Meson. What is needed is a really small project showing the idea so that it can be used as a test case.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Mon, 2017-04-17 at 18:53 +0000, Nordlöw via Digitalmars-d wrote:
> 
[…]
> Nice. I'm using SCons in my everyday development. I'll give your feature a try.

My pull request is currently being reviewed and some updates are needed. These will appear immediately in the SCons_D_Experiment. I'll post once the pull request gets merged into SCons mainline. Then it just awaits a release, unless you use default/tip from the mainline repository.

> Meson has recently added a similar feature. Meson calls it a "unity build".

Is there documentation? I need this immediately as unit-threaded only works with whole source compilation.

> > If you want to use it prior to it being accepted and released, then feel free to use the tools in the repository https://github.com/russel/ SCons_D_Experiment
> > 
> > Also in there you will spot a new tool dub. I am now using this
> > to grab
> > unit-threaded from the Dub repository in a SCons build. See for
> > example
> >  https://github.com/russel/ApproxGC
> 
> Cool.
> 

:-)

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Tuesday, 18 April 2017 at 06:51:51 UTC, Russel Winder wrote:
>> I have been trying to do the opposite with scons - incremental builds. In c++ the .h and .cpp files allowed the compilation from many edits to be limited to one translation unit.
>
> Per module compilation then linking is the default for the Program builder in SCons, that is already there and has been for over a decade.
>
I know, I have been using your work in SCons to build in this mode since it was first available.

>> One thing people are often confused about is whether everything needs to be in the header (or di file).
>
> Uuurrr… there is no separation of header and code in D, just modules. And packages. SCons currently has no "per package" builder. I guess I need to add this.

I realise there are no headers, that is the problem. SCons can't tell whether a change to a module affects the interface, only affects the implementation or only affects a unit test. This means everything that imports a module is recompiled on every edit, even if only the unit test was modified, or a formatting change was made.

I am not suggesting the D way is bad compared to C++, I am suggesting we can have the best of both worlds by automatically generating 'headers' or .di files from modules. Then having dependent modules read only the .di file instead of the .d file.
This way SCons won't recompile dependencies unless necessary. This is the main purpose of using SCons (perfect incremental builds) which is largely defeated currently by D.


> Currently there is no explicit handling of di files in SCons.
I know, I have been working through the guts of SCons for (a little bit too long to mention)  trying to make it both output two products with proper dependencies (.o and .di) and to find the di files correctly. SCons seems to have an ability to find the original .d files in the project even if the compiler can't.

> If you can come up with a small project that exhibits the problem you need solved, I can add it as a test case in the SCons system thereby creating a need for a fix.
>
Thanks for that kind offer Russel, I will make a minimal example project.

> (Tests in SCons for the tools are end-to-end tests, so need a project
> exhibiting a problem.)

OK.