July 13, 2011
"Jacob Carlborg" <doob@me.com> wrote in message news:ivkpdp$8dj$1@digitalmars.com...
>>
>> FWIW, my Drake system takes that as a high priority, too. For example, if
>> you have project "foo", then you can get the cross-platform binary
>> filename,
>> object filename, shared lib filename, static lib filename, etc like this:
>>
>> "foo".exe  // "foo.exe" or "foo"
>> "foo".obj  // "foo.obj" or "foo.o"
>> "foo".lib  // "foo.lib" or "foo.a"
>> "foo".slib  // "foo.dll" or "foo.so"
>> "foo".bat  // "foo.bat" or "foo"
>> "foo".sh   // "foo.bat" or "foo.sh"
>
> Don't forget "foo.dylib" on Mac OS X.
>

Thanks, I didn't know about that. Does OSX use .dylib instead of .so, or is it another thing that it has in addition to .so?

Are there any other OSX-specific (or BSD-specific, for that matter) extensions to be aware of? Until now, I thought it just used all the same extensions as Linux.

Oh also, to Posix-people: Could "slib" be easily confused as being to "lib" what "sbin" is to "bin"?


July 13, 2011
2011/7/13 Nick Sabalausky <a@a.a>:
> "Jacob Carlborg" <doob@me.com> wrote in message news:ivke5k$2m78$1@digitalmars.com...
>>
>> First I have to say that I know you are doing this because you want to use D as the language for the build scripts. The reason I did choose Ruby because I think D will be too verbose and when I'm looking at drakefile.d I do think it's too verbose.
>
> Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
>

Not trying to be argumentative, but what exactly do you see as the gains in having a D-buildtool built in D (or D-specific build-tool in any language, for that matter)? Seriously, I'm really asking, since I'm having a hard time seeing it? Personally, I can only see the drawbacks;

 * Building druntime and phobos might be difficult with a d-based buildtool
 * The build-tool itself will need bootstrapping. A user that wants to
test some D-project, will first have to aquire (build) and install
some D-compiler with custom tools. Then install druntime with another
custom build-system. Then Phobos. Then drake. And THEN, the
application/library he/she was interested in. Shortening this path is
IMHO REALLY important to see more D adoption. From my personal
experience, convincing developers and testers to fight through this
path is HARD.
 * Cross-language builds (project with bindings), and builds with
external targets might be more difficult than need be, if the "2nd"
language is not supported by drake.
 * Verbose build-script is IMHO a _really_ undesireable trait.
 * How soon will I as a developer be able to "just build" a D-binding
to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
Gentoo, be able to count on this working in his/her envrironment too?
 * Non-compilation actions will have to be reimplemented; document
generation, test execution, install-tasks following OS-specific
install procedures (XDG-guidelines etc.), ...

IMHO, it sounds like a case of the NIH-syndrome, but I might be missing something obvious?
July 13, 2011
I completely agree. I definitely believe that work on a new build tool is worthwhile especially if it has built in support for D. It's a shame that so many build tools at the moment are being developed using interpreted languages, it's obviously for the benefits of creating DSLs that extend the base language itself. They tend to suffer from slowness though (my subjective opinion ;-) ).

I don't think building the tool in D is a good idea either for reasons mentioned by Ulrik. It's worth considering using Lua <http://www.lua.org/> or Io <http://iolanguage.com/> for the build description language but writing the tool in C (or other).

Cheers,

Chris


On 07/13/11 23:32, Ulrik Mikaelsson wrote:
> 2011/7/13 Nick Sabalausky<a@a.a>:
>> "Jacob Carlborg"<doob@me.com>  wrote in message news:ivke5k$2m78$1@digitalmars.com...
>>> First I have to say that I know you are doing this because you want to use D as the language for the build scripts. The reason I did choose Ruby because I think D will be too verbose and when I'm looking at drakefile.d I do think it's too verbose.
>> Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
>>
> Not trying to be argumentative, but what exactly do you see as the gains in having a D-buildtool built in D (or D-specific build-tool in any language, for that matter)? Seriously, I'm really asking, since I'm having a hard time seeing it? Personally, I can only see the drawbacks;
>
>   * Building druntime and phobos might be difficult with a d-based buildtool
>   * The build-tool itself will need bootstrapping. A user that wants to
> test some D-project, will first have to aquire (build) and install
> some D-compiler with custom tools. Then install druntime with another
> custom build-system. Then Phobos. Then drake. And THEN, the
> application/library he/she was interested in. Shortening this path is
> IMHO REALLY important to see more D adoption. From my personal
> experience, convincing developers and testers to fight through this
> path is HARD.
>   * Cross-language builds (project with bindings), and builds with
> external targets might be more difficult than need be, if the "2nd"
> language is not supported by drake.
>   * Verbose build-script is IMHO a _really_ undesireable trait.
>   * How soon will I as a developer be able to "just build" a D-binding
> to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
> Gentoo, be able to count on this working in his/her envrironment too?
>   * Non-compilation actions will have to be reimplemented; document
> generation, test execution, install-tasks following OS-specific
> install procedures (XDG-guidelines etc.), ...
>
> IMHO, it sounds like a case of the NIH-syndrome, but I might be missing something obvious?


July 13, 2011
On 7/13/11 11:41 PM, Nick Sabalausky wrote:
>> "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible
>> wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and
>> translate them to the correct "native" flags.
>>
>
> That's great, I didn't know that. […]

Be aware though that neither of them supports the -lib switch. By the way, does anyone have a wrapper emulating it for LDC lying around? That way, the Phobos build system wouldn't have to be duplicated for it, and e.g. the test suite could be built easily.

David
July 14, 2011
On 7/13/11 5:32 PM, Ulrik Mikaelsson wrote:
> Not trying to be argumentative, but what exactly do you see as the
> gains in having a D-buildtool built in D (or D-specific build-tool in
> any language, for that matter)?

I think it's a matter of positioning D and eating one's dogfood. If D is inconceivable for the kind of tasks that Python or Ruby are adept at, then sure, we could and should use either. On the other hand, if we advocate D as a good tool for short scripts, using the competition would hurt its brand.

Personally I believe D is plenty adequate for short scripts, of which I've written a ton of. So the path of least resistance for a package manager or for a build tool is D, not any other language. I'd question much harder the decision of using another language (D is, however, not a competitor for the likes of bash or make).

> Seriously, I'm really asking, since
> I'm having a hard time seeing it? Personally, I can only see the
> drawbacks;
>
>   * Building druntime and phobos might be difficult with a d-based buildtool

Druntime and phobos are distributed, and only built by a small number of users. We should not worry about building them. Also, a tool based on other language would require that language's environment to be already available.

>   * The build-tool itself will need bootstrapping. A user that wants to
> test some D-project, will first have to aquire (build) and install
> some D-compiler with custom tools. Then install druntime with another
> custom build-system. Then Phobos. Then drake. And THEN, the
> application/library he/she was interested in. Shortening this path is
> IMHO REALLY important to see more D adoption. From my personal
> experience, convincing developers and testers to fight through this
> path is HARD.

The installation provides the adequate tools. Installing and building from scratch is unnecessary. A user wanting to test some project needs to (a) install dmd using OS-specific installation tools (which we already have) and (b) use the provided tools to build whatever project.

>   * Cross-language builds (project with bindings), and builds with
> external targets might be more difficult than need be, if the "2nd"
> language is not supported by drake.

This is a good advantage of generic build tools.

>   * Verbose build-script is IMHO a _really_ undesireable trait.

I don't understand this.

>   * How soon will I as a developer be able to "just build" a D-binding
> to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
> Gentoo, be able to count on this working in his/her envrironment too?

This pretty much depends on the capabilities of the tool, which would have to be fairly D-specific.

>   * Non-compilation actions will have to be reimplemented; document
> generation, test execution, install-tasks following OS-specific
> install procedures (XDG-guidelines etc.), ...

Here you're conflating the capabilities of the tool with the capabilities of the _langauge_ used to implement the tool.

> IMHO, it sounds like a case of the NIH-syndrome, but I might be
> missing something obvious?

I see no NIH here. D is an ample language scaling up to large programs and down to scripts. If the question is to build a tool from scratch, D is the obvious choice and any other choice is just odd. It's like some tools I've seen (none successful) that required the competition's product to be installed in order to work.


Andrei
July 14, 2011
Why build a tool anyway? Why not make a build library that we can use in our build scripts?

Then we just import the library in our `build.d` script, write the build script with the help of those library functions, and let the user run `rdmd build.d`. No need to install any tools, just run rdmd.

I'm already using this approach for DWinProj., the only issue is that the build script is a little large (~300 lines), but a lot of its code could be put into a library (e.g. it has file traversing, parallel builds, checking if all tools are present, checking if all libraries are present, converting header files, etc..).

This would reduce the script to a few dozen lines. And that's probably what I'll do to make my other projects easier to build.
July 14, 2011
On Wed, 2011-07-13 at 20:32 +0200, Jacob Carlborg wrote:
[ . . . ]
> I think that one of the problem with these language independent build tools is that they don't make it as easy as it could, because they usually don't know enough about a given language.

This is generally due to incomplete support in the tools, this is not a problem with general build frameworks per se.  In systems such as Waf and SCons, the framework provides facilities for creating tools and for those tools to build DAGs that then get "resolved" in the second phase of activity.  The language specific tool does things as a first phase of activity -- i.e. building the DAG.  The tools should realize all the language specific things that that languages users needs and/or the language expects.  Anything that is not handled is a problem that should be fixed by the tool maintainer.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


July 14, 2011
Nick,

On Wed, 2011-07-13 at 17:41 -0400, Nick Sabalausky wrote:
[ . . . ]
> Yea, D is likely to be a little more verbose than what could be done in Ruby (or Python). Personally, I think that's well worth it, though. I don't know how many others would agree or not.
[ . . . ]

You might want to take a quick look at SBT -- the standard Scala build framework.  It's advocates started it because Ant and Maven are XML hell and are generally problematic, and Gradle is build on Java and Groovy, and (as you might expect) Scala folk abhor all things dynamic (i.e. Groovy) and insist on static type checking.  I have some doubts about the huge downloads they dump into each project hierarchy, but there is no doubt that they have made excellent use of a statically type language to create a DSL for building Scala things.

If D can be used to go down this sort of road, and if it can support the build and install facilities of SCons and Waf, then it could be a winner.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@russel.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


July 14, 2011
On 2011-07-13 23:41, Nick Sabalausky wrote:
> "Jacob Carlborg"<doob@me.com>  wrote in message
> news:ivke5k$2m78$1@digitalmars.com...
>>
>> First I have to say that I know you are doing this because you want to use
>> D as the language for the build scripts. The reason I did choose Ruby
>> because I think D will be too verbose and when I'm looking at drakefile.d
>> I do think it's too verbose.
>
> Yea, D is likely to be a little more verbose than what could be done in Ruby
> (or Python). Personally, I think that's well worth it, though. I don't know
> how many others would agree or not.
>
>
>> But instead of starting a debate between Ruby and D I'm going to give you
>> a few suggestions and comments instead.
>>
>> The minimal drakefile doesn't look that minimal.
>
> I think I may be able to make the "immutable modes = [];" optional by
> checking for it with some compile-time reflecion.
>
> The other parts I'll address further below...
>
>
>> This is the idea I have for a minimal build script file:
>>
>> target("main.d"); // builds "main.d" as an executable
>>
>> Or:
>>
>> target("foobar"); // builds the directory "foobar" as a library
>>
>> DSSS did this very good.
>>
>
> I've never been very comfortable with build systems magically understanding
> and inferring all those details about a language and filetypes. It just
> feels too "magic", it's much more difficult to expand to new
> languages/tools, and every time I use a system like that I feel completely
> lost every time I need to include even just one custom buildstep.
>
> However, Drake's design is expandable. Add-on modules can be created that
> define new target types, so you could do:
>
>      module drake_dlang;
>      class DLang : Target (or File)
>      {
>          //...etc
>      }
>
>      target!DLang("main.d");
>
> And that would essentially wrap a File target, adding in any D-specific
> things such as "The target is 'main.exe'". And if you really wanted you
> could take it one more step and do:
>
>      void targetD(string source)
>      {
>          target!DLang(source);
>      }
>
>      targetD("main.d");
>

If that's possible, then great, I was hoping this could be default. I see no reason why it should be more complicated or requires more syntax to such a simple thing as building a standard executable with default configurations. If it's not this easy by default, then I think the build tool have failed, just my opinion.

> Good add-ons could even be folded into the standard Drake.
>
>
>> Now looking in the drakefile.d. You have targets and tasks all over the
>> place, almost every target has "Task" as a template parameter. Why is this
>> necessary? Have one function for targets and one for tasks. Example from
>> the drakefile.d:
>>
>>          target!Task("upload", "all",
>>                  (Target t)
>>                  {
>>                          system("ftp ...");
>>                  }
>>          );
>>
>> Seems this could look like this:
>>
>> task("upload", "all", {
>>      system("ftp...");
>> }); // if D just could get a better looking delegate syntax
>>
>
> Some Drake terminology first: Every "node" in the dependency tree is a
> "Target". There are three pre-defined types of targets: "Task", "File" and
> "Dir". Other new target types can also be defined. So the template param is
> due to that.
>
> However, I suppose it may be a good idea for every target type to come with
> a matching shortcut:
>
>      alias target!Task task;
>      alias target!File file;
>      alias target!Dir  dir;
>
> Then your "task("upload", ...);" example should work.

Ok, now I see. I think this is quite important the you first show the most simple forms and then show what they actually are.

> I agree that D's delegate syntax could be better. I think something like
> your example would be possible, but I ran into what seemed like some type
> inference limitations with all the overloading and templates, etc.

Ok.

>> "getDCompiler" seems unnecessary, both ldc and gdc have a dmd compatible
>> wrapper, ldmd and gdmd. These wrappers takes the same flags as dmd and
>> translate them to the correct "native" flags.
>>
>
> That's great, I didn't know that. But, you still have to say "ldmd" or
> "gdmd" instead of "dmd". And since the idea of "getDCompiler" is to allow
> the user to select which compiler to use (if the drakefile's author wants to
> support that), something roughly like "getDCompiler" may still be needed,
> albeit in a greatly simplified form.
>
> In any case, it was just an example of using the "modes" feature.

Ok, I see. I would prefer if the tool had built-in support for this. Just calling a function named "compiler" and it will use that particular compiler.

>> The "drakfile" function seems unnecessary. I would use a string import (or
>> what it's called), importing the whole build script into a method in a
>> class. All code in the "drakefile" function would instead be at top level.
>>
>
> I'm not entirely opposed to that idea, but here are the (perhaps minor?)
> reasons I did it this way:
>
> - Using "import" inside functions is brand-new and likely still buggy. And
> unless I'm mistaken, I think there's other problems that still exist with
> using the same code inside a function as outside (such as some
> order-of-declaration limitations, IIRC). I wanted to get moving on something
> that would work reliably right away without being too sensitive to bleeding
> edge-cases. I figured if all that gets sorted out later on, then maybe a
> Drake v2 could go that route.

I was referring to this:

void main ()
{
    writeln(import("main.d"));
}

This has been working as long as I've been using D. It works in D1 as well.

> - I wasn't sure if a D file that isn't strictly a proper D file would be too
> weird, too confusing, too magical, or would confuse the fuck out of advanced
> IDE's.

I have no idea how an IDE would behave with an incomplete file like that. I pretty sure this only matters if the IDE does some semantic processing, i.e. something more than just syntax highlighting. Textmate, which basically only does syntax highlighting, has no problem with an incomplete D file.

> If those are really just totally bullshit reasons, then I'm certainly open
> to the idea of doing it as you suggest.
>
>
>> Most of the functions called in the build script should be instance
>> methods, this will allow to use threading, possible invoke multiple
>> targets/tasks simultaneously and running several build scripts
>> simultaneously.
>>
>
> My understanding is that with thread-local being the default, they *don't*
> need to be instance methods to be thread-safe.
>
> Also, dependency-checking and task-invocation (once they're implemented)
> won't occur at all until *after* the drakefile() function completes. The
> target!...(...) function merely defines the targets, adding them to an
> internal list - nothing more. So I'm not sure that's not really a
> thread-appropriate matter.
>
> Not really sure what you mean about running several build scripts
> simultaneously. The buildscript gets compiled into an exe, so the
> buildscripts all run in separate processes anyway.

Hm, it may actually not be a problem. I'll have to give this some more thought, if there is any problem or not. It's probably me doesn't think straight.

>> I see installation related code in the build script. I think a package
>> manager should be responsible for that. A build tool should deal with
>> single files and a package manager should deal with packages (of files).
>>
>
> Those are just there as examples. You can do it however you want.
>
>
> -------------------------------
> Not sent from an iPhone.
>
>

Ok, I see.

-- 
/Jacob Carlborg
July 14, 2011
On 14/07/2011 03:53, Andrej Mitrovic wrote:
> Why build a tool anyway? Why not make a build library that we can use
> in our build scripts?
>
> Then we just import the library in our `build.d` script, write the
> build script with the help of those library functions, and let the
> user run `rdmd build.d`. No need to install any tools, just run rdmd.
>
> I'm already using this approach for DWinProj., the only issue is that
> the build script is a little large (~300 lines), but a lot of its code
> could be put into a library (e.g. it has file traversing, parallel
> builds, checking if all tools are present, checking if all libraries
> are present, converting header files, etc..).
>
> This would reduce the script to a few dozen lines. And that's probably
> what I'll do to make my other projects easier to build.

Oooh, I like this build lib idea!

A...