View mode: basic / threaded / horizontal-split · Log in · Help
July 14, 2011
Re: Prototype buildsystem "Drake"
On 2011-07-13 23:41, Nick Sabalausky wrote:
> "Jacob Carlborg"<doob@me.com>  wrote in message
>> Most of the functions called in the build script should be instance
>> methods, this will allow to use threading, possible invoke multiple
>> targets/tasks simultaneously and running several build scripts
>> simultaneously.
>>
>
> My understanding is that with thread-local being the default, they *don't*
> need to be instance methods to be thread-safe.
>
> Also, dependency-checking and task-invocation (once they're implemented)
> won't occur at all until *after* the drakefile() function completes. The
> target!...(...) function merely defines the targets, adding them to an
> internal list - nothing more. So I'm not sure that's not really a
> thread-appropriate matter.
>
> Not really sure what you mean about running several build scripts
> simultaneously. The buildscript gets compiled into an exe, so the
> buildscripts all run in separate processes anyway.

Now I remember how I was thinking, I was thinking in Ruby :)
This will most likely not work in D but let me explain anyway. Say that 
you have a build script with two targets, two executables:

target("foo.d", (Target t) {
    t.buildflags ~=  "-L-ldwt";
});

target("main.d" (Target t) {
    t.buildflags ~=  "-release"
});

In Ruby this would work the same:

target "foo.d" do |t|
    t.buildflags << "-L-ldwt"
end

target "main.d" do |t|
    t.buildflags << "-release"
end

In Ruby you could improve this syntax a little, like this:

target "foo.d" do
    buildflags << "-L-ldwt"
end

target "main.d" do
    buildflags << "-release"
end

In Ruby you can take advantage of the instance_eval method, allowing to 
evaluate a block/delegate in the context of an instance. In the above 
example "buildflags" would be a an instance method in the class that the 
block is evaluated in. Then you don't need to pass the a Target instance 
to the block.

In D, with this syntax:

target("foo.d", {
    buildflags ~=  "-L-ldwt";
});

target("main.d" {
    buildflags ~=  "-release"
});

"buildflags" would probably be a global function or an instance method. 
If this should work "buildflags" needs to keep some data structure with 
buildflags for each target. This seems quite complicated, making sure 
the correct build flags are used with the correct target.
July 14, 2011
Re: Prototype buildsystem "Drake"
"Jacob Carlborg" <doob@me.com> wrote in message 
news:ivme1u$31i8$1@digitalmars.com...
>
[...snip...]
> In D, with this syntax:
>
> target("foo.d", {
>     buildflags ~=  "-L-ldwt";
> });
>
> target("main.d" {
>     buildflags ~=  "-release"
> });
>
> "buildflags" would probably be a global function or an instance method. If 
> this should work "buildflags" needs to keep some data structure with 
> buildflags for each target. This seems quite complicated, making sure the 
> correct build flags are used with the correct target.

That's an interestng idea. I think it could be done fairly easily by having 
Drake invoke the delegates indirectly like this:

Target t; // global
void invokeBuildStep(Target currTarget, {type} dg)
{
   t = currTarget;
   dg();
   t = null;
}

Only thing though, is if there's any special members added to the Target 
subclass being used, the user would have to cast 't' to access them. Then 
again, that's an issue in the current design, too. I'll have to think about 
this...
July 14, 2011
Re: Prototype buildsystem "Drake"
On 2011-07-13 23:03, jdrewsen wrote:
> Den 13-07-2011 03:02, Nick Sabalausky skrev:
>> The recent discussions about package managers and buildsystems has
>> prompted
>> me to get off my ass (or rather, *on* my ass as per how programming is
>> usually performed...) and start on the D-based rake-inspired buildtool
>> I've
>> been meaning to make for awhile now. It's not actually usable yet, but
>> there's a sample drakefile demonstrating everything, and it does actually
>> compile and run (but just doesn't build the dependency tree or
>> actually run
>> any of the build steps). *Should* work on Posix, but I only tested on
>> Windows, so I may have fucked up the Posix version...
>>
>> Apologies to Jacob Carlborg for the name being so close to "dake". Didn't
>> really know what else to call it ("Duck", maybe?) Like dake, it's
>> inspired
>> by Ruby's Rake. But unlike dake, the buildscript is written in D
>> instead of
>> Ruby, which was my #1 reason for making a Rake alternative.
>>
>> Before I go implemeting everything, I'd like to get input on it. Is it
>> something that could be a major D tool?
>>
>> Overview of Drake and links to all the code are here:
>>
>> https://bitbucket.org/Abscissa256/drake/wiki/Home
>
> A good start I think. The first thing that I think should be fixed is
> the naming.
>
> Drake should be called dbuild
> Orb should be called dpack
>
> You read the name and you have an idea of what it does - nothing fancy.

"Drake" was probably chosen because it's basically "rake for D" and 
possibly because "dake" was already taken. "rake" was most likely chosen 
because it's "make" but the makefiles are written in Ruby instead.

I chose "Orbit" as the name of the package manager and named the tool 
"Orb". Why? No particular reason, but after that someone mentioned it 
has the same theme as (Digital)Mars and Phobos.

> People coming from ruby may recognize the drake name, but I guess many
> c++ peoply have no idea.

"Drake" is derived from Rake which is derived from Make.

> Orb is not derived from anything I guess.
>
> Regarding drake itself:
>
> I really like how general it is but at the same time miss some
> simplicity. "Make the common task easy and the uncommon possible"

So true, so true.

> Some steps that you could take in that direction is to predefine
> commonly used modes like "build", "clean" etc. And the define
> buildMode(), cleanMode() as builtins. That would get rid of the
> "immutable modes = ..." lines in simple projects.
>
> Maybe let the wrapper that loads the drakefile insert the "import
> drake.all" and also wrap the "void drakefile() { ... }" around the file.
> Maybe that is what your have already listed on the todo?
>
> As suggested elsewhere you could get rid of target!Task or target!File
> and just defined the target() and file() functions.
>
> Also a convenience could be to just include all .d files found for an
> exe target if no .d files are explicitly listed.
>
> I guess this would make it possible to have a drakefile as simple as:
>
> task("appA".exe);
>
>
> It seems to lack the configure part of the build process e.g. locate the
> libfoobar.a file for linking or locate a D compiler.
>
> Thanks,
> /Jonas

I agree with most of this.

-- 
/Jacob Carlborg
July 14, 2011
Re: Prototype buildsystem "Drake"
"Jacob Carlborg" <doob@me.com> wrote in message 
news:ivm9lq$2q1d$1@digitalmars.com...
> On 2011-07-13 23:41, Nick Sabalausky wrote:
>> "Jacob Carlborg"<doob@me.com>  wrote in message
>> news:ivke5k$2m78$1@digitalmars.com...
>>>
>>> The "drakfile" function seems unnecessary. I would use a string import 
>>> (or
>>> what it's called), importing the whole build script into a method in a
>>> class. All code in the "drakefile" function would instead be at top 
>>> level.
>>>
>>
>> I'm not entirely opposed to that idea, but here are the (perhaps minor?)
>> reasons I did it this way:
>>
>> - Using "import" inside functions is brand-new and likely still buggy. 
>> And
>> unless I'm mistaken, I think there's other problems that still exist with
>> using the same code inside a function as outside (such as some
>> order-of-declaration limitations, IIRC). I wanted to get moving on 
>> something
>> that would work reliably right away without being too sensitive to 
>> bleeding
>> edge-cases. I figured if all that gets sorted out later on, then maybe a
>> Drake v2 could go that route.
>
> I was referring to this:
>
> void main ()
> {
>     writeln(import("main.d"));
> }
>
> This has been working as long as I've been using D. It works in D1 as 
> well.
>

I assume you meant "mixin" instead of "writeln".

What I meant is that, suppose wr have this:

   // internal_to_drake.d:
   void main()
   {
       mixin(import("main.d"));
   }

   // main.d:
   import blah;
   // rest of buildscript...

Then that becomes:

   void main()
   {
       import blah;
       // rest of buildscript...
   }

And *that's* a brand-new feature. And I think there may be some other 
things, too, that might still be a little buggy if they're stuck inside a 
function like that.


>> - I wasn't sure if a D file that isn't strictly a proper D file would be 
>> too
>> weird, too confusing, too magical, or would confuse the fuck out of 
>> advanced
>> IDE's.
>
> I have no idea how an IDE would behave with an incomplete file like that. 
> I pretty sure this only matters if the IDE does some semantic processing, 
> i.e. something more than just syntax highlighting. Textmate, which 
> basically only does syntax highlighting, has no problem with an incomplete 
> D file.
>

Right, but a lot of people use fancy IDE's like Eclipse and Visual Studio.


>> If those are really just totally bullshit reasons, then I'm certainly 
>> open
>> to the idea of doing it as you suggest.
>>
>>
>>> Most of the functions called in the build script should be instance
>>> methods, this will allow to use threading, possible invoke multiple
>>> targets/tasks simultaneously and running several build scripts
>>> simultaneously.
>>>
>>
>> My understanding is that with thread-local being the default, they 
>> *don't*
>> need to be instance methods to be thread-safe.
>>
>> Also, dependency-checking and task-invocation (once they're implemented)
>> won't occur at all until *after* the drakefile() function completes. The
>> target!...(...) function merely defines the targets, adding them to an
>> internal list - nothing more. So I'm not sure that's not really a
>> thread-appropriate matter.
>>
>> Not really sure what you mean about running several build scripts
>> simultaneously. The buildscript gets compiled into an exe, so the
>> buildscripts all run in separate processes anyway.
>
> Hm, it may actually not be a problem. I'll have to give this some more 
> thought, if there is any problem or not. It's probably me doesn't think 
> straight.
>

Maybe you're thinking D1? I *think* the thread-local-by-default is D2-only 
(though I'm not certain).
July 14, 2011
Re: Prototype buildsystem "Drake"
"Jacob Carlborg" <doob@me.com> wrote in message 
news:ivnfk8$2v3p$1@digitalmars.com...
> On 2011-07-13 23:03, jdrewsen wrote:
>>
>> A good start I think. The first thing that I think should be fixed is
>> the naming.
>>
>> Drake should be called dbuild
>> Orb should be called dpack
>>
>> You read the name and you have an idea of what it does - nothing fancy.
>
> "Drake" was probably chosen because it's basically "rake for D" and 
> possibly because "dake" was already taken. "rake" was most likely chosen 
> because it's "make" but the makefiles are written in Ruby instead.
>
> I chose "Orbit" as the name of the package manager and named the tool 
> "Orb". Why? No particular reason, but after that someone mentioned it has 
> the same theme as (Digital)Mars and Phobos.
>

Between "Orb" and "Orange" I was thinking you just liked using names based 
on round things :)
July 14, 2011
Re: Prototype buildsystem "Drake"
On 2011-07-13 23:52, Nick Sabalausky wrote:
> Hmm, yea, configure didn't even occur to me. I think it could acually be
> done as just another File target that everything else depends on. Only thing
> is then you'd have to save/load the configuration file manually, so maybe
> there should be some convenience mechnism for that (Maybe utilizing Jacob's
> Orange serialization library once that rewrite is done? Or maybe that would
> be overkill? Is there some INI-file util, or am I just imagining that?)

What about doing something like DSSS/rebuild does. There are config 
files for dmd, ldc and gdc on various platforms. You can choose to 
invoke different compilers and it will use the config file for that 
particular compiler on the current platform.

I think a complete serialization library might be a little overkill (but 
I don't know). The rewrite of Orange will take me longer time then I 
expected to finish, it's (de)serializing pointers that give me a 
headache, almost everything else is finished. For now Orbit has higher 
priority.

Tango has a INI-file module. Otherwise JSON might be a good idea.

For linking with libraries I think integration with a package manager 
would be the best idea. This is how I'm planning with Orbit and the 
build tool (Dake) I'm working on. If Drake becomes the standard build 
tool for D and Orbit the package manager I'm more than happy to help 
integrate Drake with Orbit.

-- 
/Jacob Carlborg
July 14, 2011
Re: Prototype buildsystem "Drake"
On Thursday 14 July 2011 15:18:46 Nick Sabalausky wrote:
> "Jacob Carlborg" <doob@me.com> wrote in message
> news:ivnfk8$2v3p$1@digitalmars.com...
> 
> > On 2011-07-13 23:03, jdrewsen wrote:
> >> A good start I think. The first thing that I think should be fixed is
> >> the naming.
> >> 
> >> Drake should be called dbuild
> >> Orb should be called dpack
> >> 
> >> You read the name and you have an idea of what it does - nothing
> >> fancy.
> > 
> > "Drake" was probably chosen because it's basically "rake for D" and
> > possibly because "dake" was already taken. "rake" was most likely chosen
> > because it's "make" but the makefiles are written in Ruby instead.
> > 
> > I chose "Orbit" as the name of the package manager and named the tool
> > "Orb". Why? No particular reason, but after that someone mentioned it
> > has
> > the same theme as (Digital)Mars and Phobos.
> 
> Between "Orb" and "Orange" I was thinking you just liked using names based
> on round things :)

That and names starting with "Or".

- Jonathan M Davis
July 14, 2011
Re: Prototype buildsystem "Drake"
"Andrej Mitrovic" <andrej.mitrovich@gmail.com> wrote in message 
news:mailman.1624.1310612000.14074.digitalmars-d@puremagic.com...
> Why build a tool anyway? Why not make a build library that we can use
> in our build scripts?
>
> Then we just import the library in our `build.d` script, write the
> build script with the help of those library functions, and let the
> user run `rdmd build.d`. No need to install any tools, just run rdmd.
>
> I'm already using this approach for DWinProj., the only issue is that
> the build script is a little large (~300 lines), but a lot of its code
> could be put into a library (e.g. it has file traversing, parallel
> builds, checking if all tools are present, checking if all libraries
> are present, converting header files, etc..).
>
> This would reduce the script to a few dozen lines. And that's probably
> what I'll do to make my other projects easier to build.

Yea, I've given that some thought. Heck, I wrote the D-based buildscript for 
the recent versions of DDMD, so I know D's good for scripts like that :) 
It's definitely not a bad idea.

Ultimately, I think going with a rake/scons-like build-system approach just 
allows for a little bit more boilerplate to be removed, which is good for 
something like a buildscript. For example, if done as a library, once you've 
defined all your targets, you'd have to call some function to actually do 
the build. As a build system, defining the targets is all you need to do, 
and the build gets invoked automatically. It's kinda like how dmain() calls 
the user's main() instead of the work inside dmain() being a library 
functions that the main() author needs to call. Plus, as a build-system, the 
"void main(string[] args) {" part can eventually be removed (once sticking 
normally-module-level things inside a function is mature enough, if it isn't 
already).

That said, making Drake's functionality at least available in library form 
does sound like a good idea. And the buildscript-wrapping system (for people 
who want it) can, of course, be built on top of that.
July 14, 2011
Re: Prototype buildsystem "Drake"
On 2011-07-14 00:29, Nick Sabalausky wrote:
> "Jacob Carlborg"<doob@me.com>  wrote in message
> news:ivkpdp$8dj$1@digitalmars.com...
>>>
>>> FWIW, my Drake system takes that as a high priority, too. For example, if
>>> you have project "foo", then you can get the cross-platform binary
>>> filename,
>>> object filename, shared lib filename, static lib filename, etc like this:
>>>
>>> "foo".exe  // "foo.exe" or "foo"
>>> "foo".obj  // "foo.obj" or "foo.o"
>>> "foo".lib  // "foo.lib" or "foo.a"
>>> "foo".slib  // "foo.dll" or "foo.so"
>>> "foo".bat  // "foo.bat" or "foo"
>>> "foo".sh   // "foo.bat" or "foo.sh"
>>
>> Don't forget "foo.dylib" on Mac OS X.
>>
>
> Thanks, I didn't know about that. Does OSX use .dylib instead of .so, or is
> it another thing that it has in addition to .so?

Yes it uses .dylib instead of .so. It also has frameworks and other 
bundles which probably is a little overkill for the tool to support. A 
framework is just a folder name <name>.framework that contains a 
specific file and directory layout. It contains a dylib, header files, 
resources like images and sounds and possibly other dynamic libraries. 
Basically a package with everything the library needs to run.

BTW, DMD can build dynamic libraries on Mac OS X using the "-dylib" switch.

> Are there any other OSX-specific (or BSD-specific, for that matter)
> extensions to be aware of? Until now, I thought it just used all the same
> extensions as Linux.

It depends on what the build tool should be able to do and know about. 
For example, GUI applications on Mac OS X is expected to be in an 
application bundle (<name>.app), similar to a framework but containing 
an executable instead of a dynamic library.

> Oh also, to Posix-people: Could "slib" be easily confused as being to "lib"
> what "sbin" is to "bin"?

-- 
/Jacob Carlborg
July 14, 2011
Re: Prototype buildsystem "Drake"
On 2011-07-14 00:32, Ulrik Mikaelsson wrote
> Not trying to be argumentative, but what exactly do you see as the
> gains in having a D-buildtool built in D (or D-specific build-tool in
> any language, for that matter)? Seriously, I'm really asking, since
> I'm having a hard time seeing it? Personally, I can only see the
> drawbacks;
>
>   * Building druntime and phobos might be difficult with a d-based buildtool
>   * The build-tool itself will need bootstrapping. A user that wants to
> test some D-project, will first have to aquire (build) and install
> some D-compiler with custom tools. Then install druntime with another
> custom build-system. Then Phobos. Then drake. And THEN, the
> application/library he/she was interested in. Shortening this path is
> IMHO REALLY important to see more D adoption. From my personal
> experience, convincing developers and testers to fight through this
> path is HARD.

The build tool should provide pre-compiled binaries on all supported 
platforms. Then the build tool should provide a build script for it 
self, so the build tool can build it self.

>   * Cross-language builds (project with bindings), and builds with
> external targets might be more difficult than need be, if the "2nd"
> language is not supported by drake.

You can offer to run system commands.

>   * Verbose build-script is IMHO a _really_ undesireable trait.

I agree with this.

>   * How soon will I as a developer be able to "just build" a D-binding
> to a C++-app (with needed C-glue-code) in Drake? Will a user of, say
> Gentoo, be able to count on this working in his/her envrironment too?
>   * Non-compilation actions will have to be reimplemented; document
> generation, test execution, install-tasks following OS-specific
> install procedures (XDG-guidelines etc.), ...

Yes, this is preferable to have.

> IMHO, it sounds like a case of the NIH-syndrome, but I might be
> missing something obvious?

In my experience other build tools aren't good enough in at least:

1. Making it as easy as it could be to build simple executables and 
libraries

2. Syntax is not as good/clean as it could be

1. Is because they don't know about D and D compilers.
2. I don't know about this one. This is an example of a Rubygems file:

spec = Gem::Specification.new do |s|
  s.name = 'example'
  s.version = '1.0'
  s.summary = 'Example gem specification'
end

Even this tool, which uses Ruby, has unnecessary noise in the syntax. 
This could instead be written as:

gem do
  name 'example'
  version '1.0'
  summary 'Example gem specification'
end

In this case the Ruby syntax doesn't allow to use the equal sign, like 
this: "name = 'example'".

-- 
/Jacob Carlborg
1 2 3 4 5 6 7 8 9
Top | Discussion index | About this forum | D home