On 11/29/12 3:39 PM, Jacob Carlborg wrote:
> On 2012-11-29 15:28, Andrei Alexandrescu wrote:
>
>> Since you think (as opposed to believe), then there are reasons. What
>> are those reasons, and what steps can we take to obviate them from the D
>> side?
>
> Some features Ruby has that makes it less verbose to use:
>
> * No semicolons
> * Calling a method without parentheses
> * Code can be executed basically every where. Global scope, module scope
> , class scope, method scope and function scope
> * Blocks
> * Easy to integrate into an application
> * Relaxed syntax for associative array literals
> * Dynamic typing can probably help as well

That's a good list. Some don't really affect small scripts, but some are important to address (e.g. running code everywhere). The way I've seen this done in frameworks is by using a declarative approach in conjunction with the Hollywood principle.

> Actually, I'm a bit concerned about how I would do the actual
> integration if the build script was written in D. In Ruby it's easy,
> just do something like (written in D using libruby) :
>
> # Build script in Ruby
>
> target :foo do
> end

You should picture that this doesn't make any sense to someone not knowing Ruby (beyond a possible misspelling of "voodoo").

> // Handling the build script in D
>
> string content = read(buildScript);
> BuildScriptContext context = new BuildScriptContext;
> context.instance_eval(content);
>
> // access what's needed from "context"
> string target = context.target;
>
> writeln(target); // prints "foo"

I don't understand how this comparison works (but maybe that's not important).

>> Then why not work on it? A tool using D is much more likely to be
>> accepted by the community than one using Ruby, and the latter will
>> probably never be part of the official distro.
>
> That's not fair. I'm doing the best I can. I'm working on several D
> relate projects (including a build tool) and I don't have much time to
> work at D at all. I would really like to be able to work on these
> projects full time. But I don't know how to make money on that.

Money has nothing to do with it. It's your time. Why work on something that may be successful instead of something that will definitely not be? You have been pushing for Ruby ever since build and distro tools came about. Drop Ruby. I know you like it. But it's not going to work here. Just drop it.

>> I plan to change your
>> Ruby installer creation scripts into shell scripts as soon as I'll have
>> a minute.
>
> How is that any better?

It is better because it's only requiring the shell and understanding of shell scripting instead of yet another language. Everyone building stuff on Unix would know the shell.

Just so it's clear what we're talking about:

https://github.com/D-Programming-Language/installer/blob/master/osx/make.rb

This is a good example of gratuitous use of Ruby in a framework that ALREADY uses a makefile AND a shell script. I actually have the rewrite of this to a shell script somewhere, I found it so needless that I sat down and figured whether there's any advantage to not using the shell there.

> Yeah I do know that you prefer shell scripts
> over Ruby.

It's not me, it's you. You prefer Ruby over the more sensible shell scripting for this case.

> But you're arguing that I should use D instead of Ruby and
> then you're going to use shell script.

Drop Ruby.


Andrei

On 11/29/12 3:48 PM, Jacob Carlborg wrote:
> On 2012-11-29 16:27, Gor Gyolchanyan wrote:
>> Sure, but that will be part of the code, so there will still be no build
>> system, because the compiler will be able read the build info from the
>> source.
>
> For the 10th time, how will it handle import paths?

Actually I figured it out - rdmd can simply read its own file argument and look at the shebang line. Then there's no more issue of space coalescing, line length limitations etc.

Andrei

> Actually I figured it out - rdmd can simply read its own file argument and look at the shebang line. Then there's no more issue of space coalescing, line length limitations etc.

Smart! :-)

On Thursday, 29 November 2012 at 20:47:49 UTC, Jacob Carlborg wrote:
> On 2012-11-29 19:53, Rob T wrote:
> Ruby would not be a runtime dependency. It's embedded in the tool just like any other library.

OK, that's better,

> If I recall correctly, you need to use that exact syntax. If you move the first brace to a new line it won't work. That's the problem one get when using custom "languages" for these kind of things.

OK, but the idea is to get rid of DSSS like systems, partly for the same sort of reasons.

> What's the difference compared to any other build tool. Where you have to learn some kind of special syntax. This is a special syntax as well, just happens to be a real language as well.

For the moment, let's talk about ddoc, or unit testing in D. That's the difference, it's not an external tool set, it's instead a a part of the standard language feature set. BTW, IMO ddoc was implemented poorly, using comments, which I fully agree with you would be a vary bad way to go about implementing the feature. In that case, I would rather use Ruby.

>> the language for real, like CTFE or something.
>
> I don't think that will work. How will the compiler handling imports paths? I need to have all imports path before starting to compile. Otherwise it needs to rescan the source files several times.

Perhaps a better choice of an analogy would be the way unit testing is done rather than CTFE, however I will have to be specialized for the particular task.

I would say that the build specific code would have to go into the main.d file, but since libraries do not have a "main" file, the build process code would have to go into another file, perhaps named "build.d", that contains the entire build section which the compiler uses to kick off the build process. The build.d files can be chained so that multiple related projects can be constructed.

In any event, I'd ask how do the current build systems do it? They read and parse through the source files to learn about dependencies OR a programmer specifies all of it manually, which is a complete nightmare to maintain on large projects. Now, if things were designed correctly, I don't see why a semi-automated build, using internally specified build options for manual tweaking where needed, cannot be done with an embedded build system.

Another analogy is to think about languages that have embedded reflection - unfortunately D currently lacks a generalized solution for reflection, but you can see hints of the potential it offers. Simplifying how builds are performed through "reflection" will increase productivity, and will potentially offer new ways of solving old problems well past what was considered possible before.

Clearly, I don't have all the answers, this is an idea being thought out on the fly, so how it will work is entirely up for discussion. I'm just trying to think outside the box, looking for something much more sensible than the same old messy build system.

--rt

On 2012-11-29 22:45, Andrei Alexandrescu wrote:

>> # Build script in Ruby
>>
>> target :foo do
>> end
>
> You should picture that this doesn't make any sense to someone not
> knowing Ruby (beyond a possible misspelling of "voodoo").

That would look like this in D :

target("foo", {

});

> I don't understand how this comparison works (but maybe that's not
> important).

I'm just showing how to handle the Ruby code in D.

> Money has nothing to do with it.

It has everything to do with it.

1. I want to live
2. To live I need to eat
3. Food cost money
4. To make money I need to work
5. I don't use D in my work
6. I basically need to work eight hours a day
7. And I need to sleep eight hours a night
8. As said, I need to eat and do other things as well that take time
9. Not much time left for D

If I could use D in my work or even better by job was to work on these D projects I could get things done a lot sooner.

It has everything to do with money.

> It's your time. Why work on something
> that may be successful instead of something that will definitely not be?

These comments will more than definitely not be successful.

> You have been pushing for Ruby ever since build and distro tools came
> about. Drop Ruby. I know you like it. But it's not going to work here.
> Just drop it.

I said a build tool need to support build scripts written in a full blown language. I mentioned Ruby, Python and D. I don't know where Ruby was brought into the discussion.

> It is better because it's only requiring the shell

That doesn't matter. What's matter is what comes pre-installed on a given operating system. Since this will only run on Mac OS X and Mac OS X comes pre-installed with Ruby there's no problem.

 and understanding of
> shell scripting instead of yet another language.

Shell script is yet another language and a quite horrible one. I mean, I can't have spaces around the equal sign when assigning a variable.

>Everyone building stuff on Unix would know the shell.

Fair enough.

> Just so it's clear what we're talking about:
>
> https://github.com/D-Programming-Language/installer/blob/master/osx/make.rb

I know.

> This is a good example of gratuitous use of Ruby in a framework that
> ALREADY uses a makefile AND a shell script. I actually have the rewrite
> of this to a shell script somewhere, I found it so needless that I sat
> down and figured whether there's any advantage to not using the shell
> there.

No, there's no advantage of not using the shell. The Ruby script mostly call shell command and other commands anyway. I just don't like shell script.

>> Yeah I do know that you prefer shell scripts
>> over Ruby.
>
> It's not me, it's you. You prefer Ruby over the more sensible shell
> scripting for this case.
>
>> But you're arguing that I should use D instead of Ruby and
>> then you're going to use shell script.
>
> Drop Ruby.

Now you're just mean.

-- 
/Jacob Carlborg

On 2012-11-29 23:06, Rob T wrote:

> For the moment, let's talk about ddoc, or unit testing in D. That's the
> difference, it's not an external tool set, it's instead a a part of the
> standard language feature set. BTW, IMO ddoc was implemented poorly,
> using comments, which I fully agree with you would be a vary bad way to
> go about implementing the feature. In that case, I would rather use Ruby.

The built-in support for unit testing is too simplistic. I think one needs an external tool anyway that makes use of the built-in support unit tests.

I just want to be able to do something like:

$ test a.d b.d

And it will run all unit tests in the modules "a" and "b". In D I need to manually creating a test module which imports all modules I want to test. This will give the most basic functionality. This is a few things of that's missing:

* Run a single test
* Names or context for the tests
* Nice report of which tests failed
* Continue running other tests if a given test failed

> I would say that the build specific code would have to go into the
> main.d file, but since libraries do not have a "main" file, the build
> process code would have to go into another file, perhaps named
> "build.d", that contains the entire build section which the compiler
> uses to kick off the build process. The build.d files can be chained so
> that multiple related projects can be constructed.

That's the exact same thing as I'm proposing, except it's the compiler handling it.

> In any event, I'd ask how do the current build systems do it? They read
> and parse through the source files to learn about dependencies

No, they:

1. Run "$ dmd -o- -c main.d -deps=deps.txt" which will write out all dependencies of "main.d" to "deps.txt"

2. Read the "deps.txt" file and pass all files to dmd

> OR a programmer specifies all of it manually, which is a complete nightmare
> to maintain on large projects.

I completely agree.

> Now, if things were designed correctly, I
> don't see why a semi-automated build, using internally specified build
> options for manual tweaking where needed, cannot be done with an
> embedded build system.

If it was designed correctly the compiler would be built as a library. The build tool could the use this library to get all the dependencies of a given source file.

> Another analogy is to think about languages that have embedded
> reflection - unfortunately D currently lacks a generalized solution for
> reflection, but you can see hints of the potential it offers.
> Simplifying how builds are performed through "reflection" will increase
> productivity, and will potentially offer new ways of solving old
> problems well past what was considered possible before.

Do you have any example?

-- 
/Jacob Carlborg

On 2012-11-29 22:46, Andrei Alexandrescu wrote:

> Actually I figured it out - rdmd can simply read its own file argument
> and look at the shebang line. Then there's no more issue of space
> coalescing, line length limitations etc.

I don't know if you intended to answer to my post but having a separate build system is no problem. What Gor suggested was having the build system in the compiler, that will cause problems.

-- 
/Jacob Carlborg

On Thursday, 29 November 2012 at 20:47:49 UTC, Jacob Carlborg wrote:
> What's the difference compared to any other build tool. Where you have to learn some kind of special syntax. This is a special syntax as well, just happens to be a real language as well.

In D could be quite elegant. Here a simplified sample how I think it could look like:

-----------------------
// Builder Library
module dlang.builder;

struct Target
{
    string output;
    string[] source;
    string[] libs;
}

alias Target Executable;

alias Target StaticLib;

struct Environment
{
    bool tests = false;
    bool verbose = false;
    string[] importDirs;
}

mixin template Builder()
{
    int main(string[] args)
    {
        // enumerate and build the targets...
    }
}

-----------------------
// build.d in project folder
#!/usr/bin/env rdmd

import dlang.builder;

// version(X) to differentiate
Environment env = {
    tests: true,
    verbose: true,
    importDirs: ["../deimos"]
}

Executable myapp = {
    output: "myapp.exe",
    source: ["app/a.d", "app/b.d"],
    libs: ["libutil"]
}

StaticLib libutil = {
    source: ["util/*.d"]
}

mixin Builder;

------------------------
invocation would be basically rdmd build.d
>./build.d --variant=release --toolset=dmd_2_060

On 2012-11-30 09:19, Tavi Cacina wrote:

> // version(X) to differentiate
> Environment env = {
>      tests: true,
>      verbose: true,
>      importDirs: ["../deimos"]
> }

That's quite a clever syntax, but you forgot the trailing semicolon and isn't that syntax intended to be deprecated? It's also quite limiting, it would be nice to be able to execute arbitrary code for a given target.

-- 
/Jacob Carlborg

On Friday, 30 November 2012 at 08:05:25 UTC, Jacob Carlborg wrote:
> That's the exact same thing as I'm proposing, except it's the compiler handling it.

Which would be better because it is integrated and not an external tool.

>> In any event, I'd ask how do the current build systems do it? They read
>> and parse through the source files to learn about dependencies
>
> No, they:
>
> 1. Run "$ dmd -o- -c main.d -deps=deps.txt" which will write out all dependencies of "main.d" to "deps.txt"

Personally, I don't see how that would work using the current form of the output. I tried it with Make to figure out dependencies and the problem I immediately ran into was that the output did not contain full path information for the projects modules, and without that information, there was no way to combine builds from related project under a separate folder. What I find, is that with D, people seem to be building in simple ways, everything under one folder. This works perhaps for many people, but not for everyone. Currently I want send all build output to a separate folder outside my project folder onto a separate drive, but I can't do something even that simple. Sure I can hack it with perhaps a symbolic, but that's a hack which sucks.

> 2. Read the "deps.txt" file and pass all files to dmd
>> OR a programmer specifies all of it manually, which is a complete nightmare
>> to maintain on large projects.
>
> I completely agree.
>
>> Now, if things were designed correctly, I
>> don't see why a semi-automated build, using internally specified build
>> options for manual tweaking where needed, cannot be done with an
>> embedded build system.
>
> If it was designed correctly the compiler would be built as a library. The build tool could the use this library to get all the dependencies of a given source file.

No doubt the compiler should be a library. Why isn't it? If it was a library, then perhaps it could use itself in some very interesting ways. The compiler should also accept plugins for extensibility. I have not looked at the code yet, but I suspect what we have under the hood will make me want to cry.

>> Another analogy is to think about languages that have embedded
>> reflection - unfortunately D currently lacks a generalized solution for
>> reflection, but you can see hints of the potential it offers.
>> Simplifying how builds are performed through "reflection" will increase
>> productivity, and will potentially offer new ways of solving old
>> problems well past what was considered possible before.
>
> Do you have any example?

If there's information inside the source, then the compiler could use that information during a build. A very simple example of this, would be the imports. So instead of manually dumping a deps file, and working some build script magic, the compiler could have that information available internally, thereby saving the programmer from hacking away at an external build script to get it. My guess that's the least of the advantages, there's probably a lot more that could be done.

To me, building is just an ugly hack and patch process caused by a broken system that is unable to build itself. It's a total mess. The best place to fix the problem is right at the source.

--rt