On Friday, 26 October 2012 at 22:15:13 UTC, Rob T wrote:
> I'm trying to do a very simple build, but with scons I find myself spending much more time with it (and getting nowhere) than the actual coding, and that tells me that it's no better and may be even worse than Make.
>
> As an example of the sort of nonsense scons dishes out, I cannot in a reasonable way, specify a build folder above my root source folder, such a thing should be extremely easy to specify, but no it isn't.
>
> I also found that scons is lacking basic features that any build tool should have. For example, when I tried to generate a list of my source files, I discovered that scons it has no built in ability to recursively scan a folder to include all subfolders. WTF!?
>
> I know I can write custom code in python to get around these problems, but that will depeat the entire purpose of having a build tool in the first place.
>
> scons also comes with a ton of dependency baggage that I simply do not need, therefore I am giving up on it entirely.
>
> At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want.
>
> If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated.
>
> --rt

I am currently learning more D to design a new one, that should really end the need for other ones. If you can wait 1 or 2 months for an alpha release...

If anyone has any suggestion, I would be thankful.

>
> If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated.
>
> --rt

Rob,

I added D support to Premake (http://industriousone.com/premake) which is available here (https://bitbucket.org/goughy/premake-dev-d) which supports make & VStudio.

Note that some parts are not feature complete yet (out of source builds) but _may_ work.  Its a work in progress.  Premake is written in Lua and packaged as a single executable so there are zero runtime dependencies.

I'd appreciate feedback as to whether it works for your use cases.

Goughy

On Sat, Oct 27, 2012 at 04:11:02AM +0200, RenatoUtsch wrote:
> On Friday, 26 October 2012 at 22:15:13 UTC, Rob T wrote:
[...]
> >At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want.
> >
> >If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated.
> >
> >--rt
> 
> I am currently learning more D to design a new one, that should really end the need for other ones. If you can wait 1 or 2 months for an alpha release...
> 
> If anyone has any suggestion, I would be thankful.

If you're going to write a new build tool, you should at least take a look at some of the ideas and concepts in tup:

	http://gittup.org/tup/

Do take some time to read the PDF under "additional info"; it is very insightful and contains some possibly revolutionary algorithms.  Don't just reinvent the same old broken system that make was 30 years ago, just with different packaging and eye-candy.

(Caveat: I've never used tup before. I just read up on it today, and was quite impressed by some of the innovations. Even if you decide to take another approach, you should at least be aware of what's cutting-edge these days so that you aren't just rehashing the same old stuff.)


T

-- 
Nothing in the world is more distasteful to a man than to take the path that leads to himself. -- Herman Hesse

On Saturday, 27 October 2012 at 03:33:37 UTC, H. S. Teoh wrote:
> On Sat, Oct 27, 2012 at 04:11:02AM +0200, RenatoUtsch wrote:
>> On Friday, 26 October 2012 at 22:15:13 UTC, Rob T wrote:
> [...]
>> >At this point I'm considering looking at those old build tools
>> >written in D, perhaps I can patch one of them up to get it to do
>> >what I want.
>> >
>> >If anyone has a suggestion as to which of the (I think) 2 or 3
>> >build tools coded in D looked the best, that would be appreciated.
>> >
>> >--rt
>> 
>> I am currently learning more D to design a new one, that should
>> really end the need for other ones. If you can wait 1 or 2 months
>> for an alpha release...
>> 
>> If anyone has any suggestion, I would be thankful.
>
> If you're going to write a new build tool, you should at least take a
> look at some of the ideas and concepts in tup:
>
> 	http://gittup.org/tup/
>
> Do take some time to read the PDF under "additional info"; it is very
> insightful and contains some possibly revolutionary algorithms.
>  Don't
> just reinvent the same old broken system that make was 30 years ago,
> just with different packaging and eye-candy.
>
> (Caveat: I've never used tup before. I just read up on it today, and was
> quite impressed by some of the innovations. Even if you decide to take
> another approach, you should at least be aware of what's cutting-edge
> these days so that you aren't just rehashing the same old stuff.)
>
>
> T

Tup has some interesting concepts, maybe I can adapt them to my project.

But I was thinking in making something more authomatized, like CMake or SCons but with tons of innovations.

On Sat, 2012-10-27 at 00:15 +0200, Rob T wrote:
> I'm trying to do a very simple build, but with scons I find myself spending much more time with it (and getting nowhere) than the actual coding, and that tells me that it's no better and may be even worse than Make.

Or it says you know Make but not SCons.  All build frameworks have their computational models, idiosyncrasies , and points of pain. Make and SCons both have these.

The problem most people have when moving from Make to SCons is that they think Make computational models and idioms. It takes a while to get over these and appreciate that SCons is very different from Make even though it is fundamentally the same.

The short form is probably:  Make is about a set of relationship rules and actions to achieve a target from a source. SCons is about specifying a toolchain and then construction a graph of the dependencies.

> As an example of the sort of nonsense scons dishes out, I cannot in a reasonable way, specify a build folder above my root source folder, such a thing should be extremely easy to specify, but no it isn't.

Hummm... Whilst I am a fan of "out of source tree builds" I have always build within the project tree, so I have never noticed that trying to build in a directory that can only be reached from .. of the SConstruct appears to be impossible – without use of symbolic links. Will you put in the bug report or should I?

> I also found that scons is lacking basic features that any build tool should have. For example, when I tried to generate a list of my source files, I discovered that scons it has no built in ability to recursively scan a folder to include all subfolders. WTF!?

Does Make? CMake, Autotools, Waf?

> I know I can write custom code in python to get around these problems, but that will depeat the entire purpose of having a build tool in the first place.

Yes and no. Clearly there is a core of idiomatic things that every build framework should have. Then there is stuff that is unique.

> scons also comes with a ton of dependency baggage that I simply do not need, therefore I am giving up on it entirely.

SCons depends only on Python. What are these other dependencies that you speak of?

Giving up on SCons is fine, if it doesn't work for you.

> At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want.

Or fix SCons?

> If anyone has a suggestion as to which of the (I think) 2 or 3 build tools coded in D looked the best, that would be appreciated.

As far as I am aware there are no D coded build frameworks that can handle C, C++, Fortran, D, LaTeX, Vala, Haskell, OCaml, Java, Scala.

Well SCons isn't very good at Java and Scala, but that is a whole different argument. Probably a full half hour and not 5 mins. (*)


(*) Think SCons → Python → Monty Python.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 27 October 2012 at 18:11:30 UTC, Russel Winder wrote:
> Or it says you know Make but not SCons.  All build frameworks have their
> computational models, idiosyncrasies , and points of pain. Make and
> SCons both have these.

I definitely do not like Make. The scripts are made out of garbage, and maintaining garbage just produces more waste. Unfortunately for me, my attempts to make use of scons is not encouraging. It may be better than Make, but not enough for me to settle down with it.

> The problem most people have when moving from Make to SCons is that they
> think Make computational models and idioms. It takes a while to get over
> these and appreciate that SCons is very different from Make even though
> it is fundamentally the same.

The two problems I mentioned were encountered almost immediately. These are inability to scan subfolders recursively, and inability to build to a level above the source folder. I don't think that neither requirement has anything to do with thinking in terms of Make. It could be that solving these two deficiencies may be enough to keep me going with scons, I don't know.

>
> Hummm... Whilst I am a fan of "out of source tree builds" I have always
> build within the project tree, so I have never noticed that trying to
> build in a directory that can only be reached from .. of the SConstruct
> appears to be impossible – without use of symbolic links. Will you put
> in the bug report or should I?

I don't think it's a bug, because it's actually documented as a feature. It may however be a "bug" in terms of the assumptions about how applications should be built.

>
> Does Make? CMake, Autotools, Waf?

I have only used Make, and as bad as it is, at least I can scan subfolders with one built-in command.


> Yes and no. Clearly there is a core of idiomatic things that every build
> framework should have. Then there is stuff that is unique.

Scons is far too rigid with the assumptions it makes, and IMO some of the assumptions are plain wrong.

For example, building to a location out of the source tree has the obvious advantage that your source tree remains a source tree. I don't understand how anyone can consider this unusual or not necessary. If a source tree is to be a tree containing source code, then recursive scanning and building out of the tree is an essential requirement.

Scons however assumes that your source tree must be flat, and that your source tree must be polluted with build files.

> SCons depends only on Python. What are these other dependencies that you
> speak of?

You are correct, only Python, which on a Linux system is normally installed by default. I was refering to the need to manually build scons from from a source repository in order to get latest D support. I know I'm in the bleeding edge zone when it comes to D, so a certain amount of hacking is needed, but I'd like to minimize it as much as possible.

>> At this point I'm considering looking at those old build tools written in D, perhaps I can patch one of them up to get it to do what I want.
>
> Or fix SCons?

I thought of that, however in order to fix scons, I would have to learn a lot about scons, and also learn Python. The flaws that I see with scons are so basic, I probably would not fit in with the scons culture, so I see nothing but pain in trying to fix scons. I'm also learning D, and would rather spend more of my time learning D than something else. My only interest with scons is for using it, not fixing it, and I have no interest in learning Python.

> As far as I am aware there are no D coded build frameworks that can
> handle C, C++, Fortran, D, LaTeX, Vala, Haskell, OCaml, Java, Scala.

I'm currently only interested in building C++ and D, generalized tools that can manage multiple languages tend to be much more complex than I need.

>
> (*) Think SCons → Python → Monty Python.

That's how I view most of what is going on in programming land.

--rt

On Tue, Oct 30, 2012 at 12:19:46AM +0100, Rob T wrote: [...]
> Scons is far too rigid with the assumptions it makes, and IMO some of the assumptions are plain wrong.
> 
> For example, building to a location out of the source tree has the obvious advantage that your source tree remains a source tree. I don't understand how anyone can consider this unusual or not necessary. If a source tree is to be a tree containing source code, then recursive scanning and building out of the tree is an essential requirement.
> 
> Scons however assumes that your source tree must be flat, and that your source tree must be polluted with build files.

I don't know where you got this assumption from, but it's plain wrong. SCons supports out-of-source-tree builds. In fact, I have a project in which I generate multiple builds from the same source tree (and I can build *all* build variants in a single command, parallelized - something that will cause make to keel over and die).

The only bug is that SCons assumes that the source tree and build tree(s) must be under a common root, which may not be the case. Nevertheless, even this is not a fatal problem, as you can just put your SConstruct in the directory above the source tree, then you can build to other subdirectories easily.


> >SCons depends only on Python. What are these other dependencies that you speak of?
> 
> You are correct, only Python, which on a Linux system is normally installed by default. I was refering to the need to manually build scons from from a source repository in order to get latest D support. I know I'm in the bleeding edge zone when it comes to D, so a certain amount of hacking is needed, but I'd like to minimize it as much as possible.

This is just what one puts up with when working with bleeding edge technology. If there weren't kinks in the works, it'd be mainstream already.


[...]
> >As far as I am aware there are no D coded build frameworks that can handle C, C++, Fortran, D, LaTeX, Vala, Haskell, OCaml, Java, Scala.
> 
> I'm currently only interested in building C++ and D, generalized tools that can manage multiple languages tend to be much more complex than I need.

To each his own, but I honestly don't see what's so difficult about this:

	# src/SConscript
	Import('env')
	env.Program('mydprogram', ['main.d', 'abc.d', 'def.cc'])

	# lib1/SConscript
	Import('env')
	env.Library('mylib1', ['mod1.d', 'mod2.d'])

	# lib2/SConscript
	Import('env')
	env.Library('mylib2', ['mod2.d', 'mod3.d'])

	# SConstruct
	objdir = 'build'
	env = Environment()
	Export(env)
	env.SConscript('src/SConscript',  build_dir=objdir)
	env.SConscript('lib1/SConscript', build_dir=objdir)
	env.SConscript('lib2/SConscript', build_dir=objdir)

Main program in src/, two libraries in lib1, lib2, and everything builds in build/ instead of the respective source trees. No problem. I even threw in a C++ file for kicks.

Now granted, SCons does have its own flaws, but railing about how useless it is when one hasn't even bothered to learn what it can do sounds rather unfair to me.


T

-- 
Ruby is essentially Perl minus Wall.

On Tuesday, 30 October 2012 at 01:22:17 UTC, H. S. Teoh wrote:
> To each his own, but I honestly don't see what's so difficult about
> this:
>
> 	# src/SConscript
> 	Import('env')
> 	env.Program('mydprogram', ['main.d', 'abc.d', 'def.cc'])
>
> 	# lib1/SConscript
> 	Import('env')
> 	env.Library('mylib1', ['mod1.d', 'mod2.d'])
>
> 	# lib2/SConscript
> 	Import('env')
> 	env.Library('mylib2', ['mod2.d', 'mod3.d'])
>
> 	# SConstruct
> 	objdir = 'build'
> 	env = Environment()
> 	Export(env)
> 	env.SConscript('src/SConscript',  build_dir=objdir)
> 	env.SConscript('lib1/SConscript', build_dir=objdir)
> 	env.SConscript('lib2/SConscript', build_dir=objdir)
>
> Main program in src/, two libraries in lib1, lib2, and everything builds
> in build/ instead of the respective source trees. No problem. I even
> threw in a C++ file for kicks.

You are right, Make cannot do something like that in a reasonable way, and it looks great.

You are describing one main project, with sub-level projects inside, so the build dump is still going into the project tree. This may work for some people, but it is frustrating for someone who wishes to dump the build files outside of the project tree. I guess there's just no solution to do different at this time. Not what I want, but not a terminal problem either.

Also scons has no built-in ability to scan subfolders for source files. You can manually specify sub-level source files, but that's unreasonable to do for large projects. You can write custom Python code to scan subfolders, but that is a lot more complicated than it should be. Do you know of a decent solution to scan sub-folders?

Scons does look like a very powerful tool, and I really do appreciate your input and the other posters as well. I am determined not to continue with Make, so maybe I will have to keep trying to get scons to do what I want.

One more question: I am wondering how the scons D support locates dependencies from the imports specfied in the source files? So far I have not been able to get automatic dependency inclusions to work, so I end up manually specifying the import files. With C++ and Make, I could get gcc to scan source files for the #include files, and dump the list to a dependency file (.dep), and then I could include the .dep file(s) into the make process. Can scons with D support do something similar, or deal with it better?

--rt

On Tue, 2012-10-23 at 14:58 -0700, H. S. Teoh wrote:
[…]
> Well, dmd tends to work best when given the full list of D files, as opposed to the C/C++ custom of per-file compilation. (It's also faster that way---significantly so.) The -op flag is your friend when it comes to using dmd with multi-folder projects.
> 
> And I just tried: gdc works with multiple files too. I'm not sure how well it handles a full list of D files, though, if some of those files may not necessarily be real dependencies.

So perhaps the D tooling for SCons should move more towards the Java approach than the C/C++/Fortran approach, i.e. a compilation step is a single one depending only on source files and generating a known set of outputs (which is easier than Java since it can generate an almost untold number of output files, Scala is even worse).

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Tue, 2012-10-30 at 00:19 +0100, Rob T wrote:
[…]
> I definitely do not like Make. The scripts are made out of garbage, and maintaining garbage just produces more waste. Unfortunately for me, my attempts to make use of scons is not encouraging. It may be better than Make, but not enough for me to settle down with it.

I would suggest you haven't given SCons long enough to get into the SCons idioms, but you have to go with the route that is most comfortable for you.

> The two problems I mentioned were encountered almost immediately. These are inability to scan subfolders recursively, and inability to build to a level above the source folder. I don't think that neither requirement has anything to do with thinking in terms of Make. It could be that solving these two deficiencies may be enough to keep me going with scons, I don't know.

I do not see why you need to scan subfolders recursively. This is a traditional Make approach. SCons does things differently even for highly hierarchical systems. In particular use of SConscript files handles everything. So I do not think this is a SCons problem.

Of course if you have to do things recursively then os.walk is the function you need.

[…]

> I don't think it's a bug, because it's actually documented as a feature. It may however be a "bug" in terms of the assumptions about how applications should be built.

As noted in the exchanges to which I cc you in everything I sent, SCons does insist on having all directories in use under the directory with the SConstruct – unless you have with Default. On reflection I am now much less worried about this that I was at first.  Out of source tree I find essential in any build, SCons does this well, Make less so. Out of project tree builds I am now not really worried about.

[…]
> I have only used Make, and as bad as it is, at least I can scan subfolders with one built-in command.

But why do you want to do this?  Why doesn't os.walk achieve what you need with SCons?

[…]
> Scons is far too rigid with the assumptions it makes, and IMO some of the assumptions are plain wrong.

I disagree. I find Make totally rigid and unyielding. Not to mention rooted in 1977 assumptions of code.

> For example, building to a location out of the source tree has the obvious advantage that your source tree remains a source tree. I don't understand how anyone can consider this unusual or not necessary. If a source tree is to be a tree containing source code, then recursive scanning and building out of the tree is an essential requirement.

I always build out of source tree using SCons, to do otherwise is insanity, yes Autotools I am looking at you. However I have a source directory in my project directory and can then have many build directories in the project directory. Building a project for multiple platforms makes this essential. SCons supports this very well with the Variant system.

[…]
> You are correct, only Python, which on a Linux system is normally installed by default. I was refering to the need to manually build scons from from a source repository in order to get latest D support. I know I'm in the bleeding edge zone when it comes to D, so a certain amount of hacking is needed, but I'd like to minimize it as much as possible.

You don't need to install SCons to use it, you can use it from a clone directly using the bootstrap system.  I have an alias

        alias scons='python /home/users/russel/Repositories/Mercurial/Masters/SCons_D_Tooling/bootstrap.py'

[…]
> > Or fix SCons?
> 
> I thought of that, however in order to fix scons, I would have to learn a lot about scons, and also learn Python. The flaws that I see with scons are so basic, I probably would not fit in with the scons culture, so I see nothing but pain in trying to fix scons. I'm also learning D, and would rather spend more of my time learning D than something else. My only interest with scons is for using it, not fixing it, and I have no interest in learning Python.

Please stick with the "I don't want to learn Python" as your reason for not working with SCons. That is fine. I have no problem with that.

I do have a problem with you saying "the flaws with SCons are so basic". This is just FUD from a person who hasn't really learnt the SCons way of doing things.

So the resolution here is to stop mud-slinging at SCons and say "I am not going to use SCons because it involve working with Python and I don't want to do that." Then people can respect your position.

[…]
> > (*) Think SCons → Python → Monty Python.
> 
> That's how I view most of what is going on in programming land.

:-)

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder