On Friday, 5 June 2015 at 19:17:14 UTC, Steven Schveighoffer wrote:
> On 6/5/15 11:03 AM, Atila Neves wrote:
>> On Friday, 5 June 2015 at 09:22:14 UTC, Jacob Carlborg wrote:
>>> On 2015-06-04 23:50, Atila Neves wrote:
>>>> http://dpaste.dzfl.pl/562f1ddc1aad
>>>
>>> Reggae doesn't support shell globbing of files? Something like:
>>>
>>> ExcludeFiles(["std/c/windows/**/*.d"]);
>>
>> Not right now, no. The good thing about how it's designed is you could
>> write your own. Of course, the whole point is to have usable high-level
>> builtins as well. It's a good idea, but like I said before, I hardly
>> ever have to exclude anything in my own projects. Feel free to open an
>> enhancement request.
>
> This sounds like a job for CAPTAIN RANGE!!!
>
> Seriously though, the thought of using ranges to allow specifying files fits perfectly. Imagine the power!
>
> "std/c/windows/".allSubFiles.filter!(a => a.name.extension == ".d");
>
> Need that allSubFiles range, it probably already exists :)
>
> -Steve

So, given the range input, I realised that trying to accomodate what every user would never need when selecting source files, instead... I settled on this:

Sources!(["dir1, "dir2"]) //or
Sources!(Dirs(["dir1", "dir2"),
         Files(["extrafile1.d", "extrafile2.d"]),
         Filter!(a => ...));

Simple. Define source directories, define extra files, filter whatever isn't wanted out. Why reinvent the wheel when D already has pretty good wheels?


The result of all this (and many bug discoveries trying to copy posix.mak) is this:

https://github.com/atilaneves/phobos/blob/reggae/reggaefile.d

It doesn't completely replicate the UT build with its shared library (I got lazy), but... right now it builds both static and shard phobos, creates the symlinks just like posix.mak, _and_ does both unit tests runs (debug and release), building the unit tests binaries in the process. Oh, and the html documentation too. I tested with the make, ninja and binary backends, and everything seems to work. I wouldn't be surprised if there are still bugs, especially with the binary backend.

Worth making a PR for? The PR would only add this file, it wouldn't change anything about the current build, but could be used for testing purposes in the meanwhile.

Oh, and given the manual filtering in the config, it obviously won't work on Mac OS. I have to get a hold of a Mac and make it work there too. Then... Windows.

Atila

On Monday, 29 June 2015 at 10:35:34 UTC, Atila Neves wrote:

>
> The result of all this (and many bug discoveries trying to copy posix.mak) is this:
>
> https://github.com/atilaneves/phobos/blob/reggae/reggaefile.d
>

Ah... so that's what posix.mak is doing.  And 1/3 the size.  Nice work!

On Monday, 29 June 2015 at 10:53:46 UTC, Mike wrote:
> On Monday, 29 June 2015 at 10:35:34 UTC, Atila Neves wrote:
>
>>
>> The result of all this (and many bug discoveries trying to copy posix.mak) is this:
>>
>> https://github.com/atilaneves/phobos/blob/reggae/reggaefile.d
>>
>
> Ah... so that's what posix.mak is doing.  And 1/3 the size.  Nice work!

No, there's more. I don't think much more (reading makefiles is tedious), I concentrated on getting the important ones right first.

Notably, that reggaefile.d doesn't define the module.test targets to only test one phobos module, but that can be added easily.

It also hard-codes a lot of what are configurable variables in posix.mak, like the 64-bit build. All of this is doable, I just haven't gotten around to it yet.

Then there's the issue of making sure the functionality really is the same. For some reason the reggaefile is producing more HTML files than posix.mak but to figure out why I'll have to go and debug.

I wouldn't do a PR until the outputs are exactly the same, I posted this trying to gauge what the interest / opinions were.

Atila

On Monday, 29 June 2015 at 10:35:34 UTC, Atila Neves wrote:
> Worth making a PR for? The PR would only add this file, it wouldn't change anything about the current build, but could be used for testing purposes in the meanwhile.

Yes, those who want reggae can use it, and the makefile can remain the default for now.  I can't wait to ditch the makefiles.

> Oh, and given the manual filtering in the config, it obviously won't work on Mac OS. I have to get a hold of a Mac and make it work there too. Then... Windows.

You could probably use the auto-tester to test reggae: simply add the reggae source to your PR, build it with a patched makefile, and then use reggae to build druntime/phobos.  I've used the auto-tester to test a couple PRs for various OSs that I don't have, especially OS X.

On Monday, 29 June 2015 at 12:06:39 UTC, Atila Neves wrote:
> I wouldn't do a PR until the outputs are exactly the same, I posted this trying to gauge what the interest / opinions were.

Moving away from Makefiles to something like this definitely appeals to me (editing the makefiles is _not_ fun, and sadly, in spite of how bad it is, posix.mak is far less of a pain than win32.mak). However, as far as actually moving away from the Makefiles goes, you'd have to convince Walter and Andrei, and I don't know how convincible they are or aren't. And I don't think that we really want to have multiple ways to build the standard D toolchain, since then that increases maintenance to keep them in line with one another. So, if we have raggae builds (or anything like it) for the D toolchain, then I think that we need to fully move over to them and ditch the makefiles altogether.

- Jonathan M Davis

On 29/06/15 12:35, Atila Neves wrote:

> https://github.com/atilaneves/phobos/blob/reggae/reggaefile.d

One major advantage with this is that the files are not hard coded, still don't understand why they are in the makefiles.  I think in druntime every file is listed at least three times. When adding a new file, it's very easy to forget adding in all places.

-- 
/Jacob Carlborg

On 29-Jun-2015 13:35, Atila Neves wrote:
[snip]

> The result of all this (and many bug discoveries trying to copy
> posix.mak) is this:
>
> https://github.com/atilaneves/phobos/blob/reggae/reggaefile.d
>
> It doesn't completely replicate the UT build with its shared library (I
> got lazy), but... right now it builds both static and shard phobos,
> creates the symlinks just like posix.mak, _and_ does both unit tests
> runs (debug and release), building the unit tests binaries in the
> process. Oh, and the html documentation too. I tested with the make,
> ninja and binary backends, and everything seems to work. I wouldn't be
> surprised if there are still bugs, especially with the binary backend.
>
> Worth making a PR for? The PR would only add this file, it wouldn't
> change anything about the current build, but could be used for testing
> purposes in the meanwhile.

Awesome! Since raggae has Makefile backend it seems to me that we may switch over to raggae and keep generated Makefiles as a transitional measure.

Ideally if reggae doesn't depend on Phobos/druntime (e.g. builds with -betterC switch) then Makefile has no use at all - builds are completely bootstrappable.


-- 
Dmitry Olshansky

On Monday, 29 June 2015 at 18:10:32 UTC, Dmitry Olshansky wrote:
> Ideally if reggae doesn't depend on Phobos/druntime (e.g. builds with -betterC switch) then Makefile has no use at all - builds are completely bootstrappable.

We already require valid install of previous release for bootstrapping because of DDMD. No reason to require -betterC - just getting reggae as a library into Phobos is enough.

On Monday, 29 June 2015 at 16:52:34 UTC, Jonathan M Davis wrote:
> On Monday, 29 June 2015 at 12:06:39 UTC, Atila Neves wrote:
>> I wouldn't do a PR until the outputs are exactly the same, I posted this trying to gauge what the interest / opinions were.
>
> Moving away from Makefiles to something like this definitely appeals to me (editing the makefiles is _not_ fun, and sadly, in spite of how bad it is, posix.mak is far less of a pain than win32.mak). However, as far as actually moving away from the Makefiles goes, you'd have to convince Walter and Andrei, and I don't know how convincible they are or aren't. And I don't think that we really want to have multiple ways to build the standard D toolchain, since then that increases maintenance to keep them in line with one another. So, if we have raggae builds (or anything like it) for the D toolchain, then I think that we need to fully move over to them and ditch the makefiles altogether.
>
> - Jonathan M Davis

Well, yes, the idea would be to ultimately move away from the current makefiles (all 3 of them). And yes, that would mean convincing the big two. But I had to show how it'd look like first instead of just saying "we should get rid of the makefiles". The idea behind adding the file if a PR is ever approved is so enough people use it and find any other lurking bugs. Once proven to work, the switch would be made.

Even if it never happens, this exercise has only ironed out bugs for reggae so it was still worthwhile.

Atila

On Monday, 29 June 2015 at 16:05:07 UTC, Joakim wrote:
> On Monday, 29 June 2015 at 10:35:34 UTC, Atila Neves wrote:
>> Worth making a PR for? The PR would only add this file, it wouldn't change anything about the current build, but could be used for testing purposes in the meanwhile.
>
> Yes, those who want reggae can use it, and the makefile can remain the default for now.  I can't wait to ditch the makefiles.
>
>> Oh, and given the manual filtering in the config, it obviously won't work on Mac OS. I have to get a hold of a Mac and make it work there too. Then... Windows.
>
> You could probably use the auto-tester to test reggae: simply add the reggae source to your PR, build it with a patched makefile, and then use reggae to build druntime/phobos.  I've used the auto-tester to test a couple PRs for various OSs that I don't have, especially OS X.

You mean make a posix.mak that actually calls reggae instead? Hmm... I guess that's doable. Otherwise I'd have to edit the script the auto-tester is using.

I'm not sure that'd guarantee the same output though.

Atila