On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:

> At least I don't expect Dub to support every single language out there natively.

that's why other build tools allows to manually specify dependencies and commands as a last resort. but dub can't.

On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:
> On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:
>
>> At least I don't expect Dub to support every single language out there
>> natively.
>
> that's why other build tools allows to manually specify dependencies and
> commands as a last resort. but dub can't.

Wrong on both points.

On Mon, 02 Feb 2015 15:01:02 +0000, Dicebot wrote:

> On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:
>> On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:
>>
>>> At least I don't expect Dub to support every single language out there natively.
>>
>> that's why other build tools allows to manually specify dependencies and commands as a last resort. but dub can't.
> 
> Wrong on both points.

please, can i see how dub can support building C library with gcc? nope, not "just execute this very long command line and hope for the best" -- i need dependency tracking, selective rebuilds and so on.

On Monday, 2 February 2015 at 15:26:36 UTC, ketmar wrote:
> On Mon, 02 Feb 2015 15:01:02 +0000, Dicebot wrote:
>
>> On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:
>>> On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:
>>>
>>>> At least I don't expect Dub to support every single language out there
>>>> natively.
>>>
>>> that's why other build tools allows to manually specify dependencies
>>> and commands as a last resort. but dub can't.
>> 
>> Wrong on both points.
>
> please, can i see how dub can support building C library with gcc? nope,
> not "just execute this very long command line and hope for the best" -- i
> need dependency tracking, selective rebuilds and so on.

It is not the same as "manually specify dependencies and commands".

As for dependency tracking - afaik adding dub.json to C library that does have any sourceFiles but simply calls existing build tool via" preBuildCommands" should work. Not pretty but adds all the metadata you have with native D package.

On Monday, 2 February 2015 at 14:35:03 UTC, John Colvin wrote:
> a little script that calls `dub describe`, extracts the import paths and then calls dmd -o- appropriately would be easy to do. There's no fundamental blocker here.

As long as the source code is open, there is no fundamental blocker at all - only a scalar of required effort, and the resulting gain per effort expended. But in my case, nothing is blocked - it is not a question of "I want to use dub but can't", but "I can't use dub and don't really need to".

On Mon, 02 Feb 2015 15:29:54 +0000, Dicebot wrote:

> On Monday, 2 February 2015 at 15:26:36 UTC, ketmar wrote:
>> On Mon, 02 Feb 2015 15:01:02 +0000, Dicebot wrote:
>>
>>> On Monday, 2 February 2015 at 14:59:09 UTC, ketmar wrote:
>>>> On Mon, 02 Feb 2015 14:18:47 +0000, Tobias Pankrath wrote:
>>>>
>>>>> At least I don't expect Dub to support every single language out there natively.
>>>>
>>>> that's why other build tools allows to manually specify dependencies and commands as a last resort. but dub can't.
>>> 
>>> Wrong on both points.
>>
>> please, can i see how dub can support building C library with gcc?
>> nope,
>> not "just execute this very long command line and hope for the best" --
>> i need dependency tracking, selective rebuilds and so on.
> 
> It is not the same as "manually specify dependencies and commands".

i list dependencies in my jamfile (if i can't write regexp to extact them from sources, which i usually can), i specify compiler for unknown file extension, i run my jam. voila, everything else is done by my jam. for make it's almost similar.

On Monday, 2 February 2015 at 11:00:20 UTC, Manu wrote:
> I have an issue with your proposed module management solution that I
> wonder if you can address.
> You suggest placing source in the root of the repo. Just... no. I will
> never do that. The root directory is for readme's, licenses, maybe a
> build script. It's not the place for the source, that lives in 'src'.
> What's the solution to this problem?
> Is it that your src directory should have symlinks to the src
> directories of the submodules? Is that why you raise the symlink
> support on windows issue? I haven't thought of that before, and have
> no idea if it works...
>
> Does git support relative symlinks?

If you're going to start using submodules, how about putting your source innto it's own submodule?

fruit ->
   README
   src -> (git submodule "fruit_src"?)
       apple.d
       banana.d

Just a thought.  If git supported cloning a partial repository you could do this without a sub module as well.  What do you think?  Seems a little odd to me but maybe it could work.

I have to say that the directory naming issue bit me; I had to move all of my code to "source" in order to use dub, and then I end up with "cerealed/source/cerealed/*.d" which is kind of silly.

rdmd is awesome, I use dub when the D program I'm writing has dub package dependencies. For vibe.d, for instance, I really don't want to have to figure out how to build it myself, and rdmd won't cut it there, at least not without major contortions.

But both rdmd and dub fall short with anything remotely complicated. I've used a lot of CMake features that I simply don't have with either of them.

Atila

On Monday, 2 February 2015 at 08:09:39 UTC, Vladimir Panteleev wrote:
> On Monday, 2 February 2015 at 05:23:52 UTC, Daniel Murphy wrote:
>> "Vladimir Panteleev"  wrote in message news:viqwfixznbdbdwvhavuk@forum.dlang.org...
>>
>>> I don't use Dub
>>
>> You really should!  I put it off for months and months but I'm quite happy with it now.
>
> Even if I had faith that dub was a perfectly polished piece of software, it doesn't solve any problems I have with building D programs, and in fact would make said task more complicated. Here's why.
>
> 1. rdmd
>
> rdmd is a simple and effective way to build D programs, and I'm sad to see its use declining.
>
> rdmd leverages two things: D's module system, and the compiler's import search path. Dub's design seems to ignore both of the above.
>
> 1a. rdmd and D's module system:
>
> When you run `dmd -o- program.d`, the compiler will automatically read all modules imported by your program, and their imports, and so on. It does so by searching the filesystem across its search path for matches which correspond with D's module system, and only reads those files that are needed.
>
> rdmd leverages this by collecting (and caching) the list of modules used in the program, and passing that list to the compiler. The compiler will then compile no more and no less than the exact set of modules transitively imported by the program.
>
> In contrast, Dub's default modus operandi is to blindly send to the compiler all *.d files found in the "src" folder, whether they're actually used or not. Not only can this be slower if not all modules are always used, but it also won't work if the source code contains multiple entry points, forcing you to write complicated configuration files (i.e. do the computer's work for it).
>
> 1b. rdmd and D's search path
>
> rdmd does not have any additional parameters to set up for where it needs to look for source files, because it relies on the compiler's search mechanism. Thus, if you can build your program with rdmd, "dmd -o- program" will succeed, and usually vice versa.
>
> In contrast, Dub builds its own search path using its JSON configuration files, and has no equivalent of "dmd -o-".
>
> There is no simple way to syntax-check just one file in a project when using Dub. I rely on this a lot in my workflow - I configured a syntax-check key in my editor, which I use almost as often as I save. A syntax check (dmd -o-) is much faster than a full build, as it skips parsing other parts of the project, code generation, and linking.
>
> 2. git
>
> I've found that git's submodule feature is an excellent way to manage dependencies across D libraries, and fits really well with D's package system. For clarity to readers not too familiar with Git, I'll explain by example:
>
> 2a. Directory structure
>
> If you have a library "fruit" with the modules "fruit.apple" and "fruit.banana", create a git repository (in a directory called "fruit") with the files apple.d and banana.d.
>
> Now, in your project (e.g. "orchard"), add a symlink called "fruit" pointing to the repository:
>
> ~/
> |- fruit/
> |  |- .git/
> |  |- apple.d
> |  '- banana.d
> '- orchard/
>    |- .git/
>    |- fruit -> ~/fruit/
>    '- tree.d
>
> tree.d can now import the "fruit.apple" module without any additional compiler switches. `rdmd tree` will just work. This is because the modules of the "fruit" library are placed at the repository root, and the repository's directory name matches with the library's package name, so when the compiler will look for the "fruit.apple" module, it will find it at "fruit/apple.d".
>
> 2b. Distribution
>
> Upload the "fruit" repository to GitHub. In your "orchard" project, delete the symlink, and run:
>
> git submodule add https://github.com/You/fruit fruit
>
> This will clone the repository from GitHub, create a .gitmodules file, and add an entry to the git index. If you commit all these, and put "orchard" on GitHub too, someone can run the command:
>
> git clone --recursive https://github.com/You/orchard
>
> ... to get the orchard project, and its dependencies (the fruit library).
>
> 2c. Versioning
>
> Git stores the exact commit of each submodule in the parent repository.
>
> So, if you make a commit in orchard/fruit/, and run `git status` in orchard/, git will show you that the "fruit" submodule has been modified.
>
> Most importantly, this means that any breaking change in the "fruit" library will never affect existing projects which use it, since they will continue to use the registered commit which was known to work.
>
> This gives you dependency versioning with commit granularity - you can hardly ask for something better - all without messing with configuration files.
>
> ---
>
> The one thing I wish git had is symlink support for Windows. Windows has symlinks and directory junctions, but msysgit still doesn't support them.
>
> For a practical instance of the above example, see Digger:
>
> https://github.com/CyberShadow/Digger
>
> Note the simple build instructions:
>
> https://github.com/CyberShadow/Digger#building
>
> 2d. Git vs. Dub
>
> Unfortunately, the above-described approach is not compatible with Dub:
>
> - Dub packages are generally expected to have their source code in a "src" subdirectory, although you can set the source directory to "." in the configuration file.
>
> - When cloning repositories, dub does not preserve the repository's directory name (so e.g. fruit will be cloned to ~/.dub/fruit-1.0.0/).
>
> Somebody has created a Dub package for my library (excluding certain packages, due to point 1a above), and the other day someone complained that it doesn't work with Dscanner, because of the above issue - the module path "ae.dir.module" does not correspond to the filesystem path "ae-1.0.1/dir/module.d".
>
> So, in order to start using Dub, I'd need to:
>
> - restructure the directory layout of my library (breaking change)
> - update all projects which use this library to use Dub instead
> - give up quick syntax checking
> - give up commit-granularity versioning
> - begin maintaining JSON configuration files
> - begin versioning libraries by hand
> - install Dub on all my computers, servers, and virtual machines
>
> No thanks.
>
> I could invest time in improving Dub to fix or ameliorate some of the above points, but I don't see a compelling reason to. In fact, I think we should integrate rdmd into dmd - dmd clearly already knows which source files participate in compilation, as all rdmd does is basically take dmd's output and feed it back to it. This will greatly speed up compilation, too.
>
> Change my view.

This thread has just made me decide to write a D build system, in D and configured in D that builds on dub and its packages intead of reinventing the wheel. I might ask the community for inputs on requirements. Maybe this will be my DConf 2015 submission.

Also, Dicebot, I agree with nearly everything you said here. It tends to happen that I usually either completely agree or completely disagree with you ;)

Atila

On Monday, 2 February 2015 at 14:41:00 UTC, Dicebot wrote:
> Quick summary of my opinion can look like this:
>
> 1) dub is a horrible build tool
> 2) git submodules is a horrible way for dependency management
>
> First thing is something we can improve on in future and, actually, dub supports using rdmd as a build tool already.
>
> Second thing is an inherent issue that won't change without some fundamental breaking changes in how hit submodules are implemented.
>
> Because of that I am trying to push dub despite the fact I don't like lot of things about it personally.
>
> ----------------------------------------
>
> Explaining why dub is a horrible build tool is hardly necessary - there are plenty of cases in this thread already.
>
> So why git submodules don't really work?
>
> 1) Lack of version management
>
> submodule remembers specific commit hash and that is all. As of recent git version one can also remember remote tracking branch (but it will still keep specific hash in history!)
>
> This may not seem an issue if you have only few dependencies and control them all. But it scales badly once bulding the project needs many incompatible versions of the same dependency and dependency count increases in general.
>
> 2) Lack of local cache
>
> This is really annoying when working with many project with similar dependencies - those will get cloned over and over again, taking quite some time. Can possibly be fixed with clever scripts that replace remotes with local repo paths but that is not out-of-the box solution
>
> 3) Lack of package repository
>
> Biggest thing about dub is not dub tool itself. It is code.dlang.org and what it can become in future. Providing simple centralized way to discover new libraries and quickly try those in uniform matter is the way you build decentralized ecosystem (ironically).
>
> With git submodules you are pretty much stick with doing GitHub search over and over again
>
> 4) Inherently coupled with specific DVCS
>
> git and GitHub are most common (and officially endorsed) ways of version control in D world but those are not the only one. dub registry allows to use GitHub, GitBucket and source tarball packages in the same project - in simple uniform matter. It is a 100% in-house tool that does no depend on decisions of external project maintainers that provides a common API for everything else to interoperate

On Monday, 2 February 2015 at 16:26:45 UTC, Atila Neves wrote:
> This thread has just made me decide to write a D build system, in D and configured in D that builds on dub and its packages intead of reinventing the wheel. I might ask the community for inputs on requirements. Maybe this will be my DConf 2015 submission.

I have recently been experimenting with meta-dlang repository (to aggregate exisiting ones) and using D scripts instead of makefiles for uniform cross-platform build experience.

That quickly has made me wanting to get something like `std.make` into Phobos - small utility module that would allow declaratively defining target tree + shell scripts similar to makefiles, as well as set of small nice shell wrappers (like one that effectively enables `set -e` for called commands). Having such module in standard library would make it much easier to replace makefiles with a quick D scripts and potentially build a robust build system on top.