On 06/01/2015 09:22 AM, Rikki Cattermole wrote:
>
> Something I've thought about but haven't suggested up to now is making
> dub-repo to support personal projects.
> Basically one repo per person preconfigured if e.g. Github/Bitbucket is
> specified.
>
> github.com/rikkimax/dubcode
>
> Just as an idea for what I'm meaning url wise.
>
> So then when specifying a dependency in dub, its just a matter of saying
> <person>:<subproject>
>

Already works.

$ git clone https://github.com/rikkimax/dubcode.git path-to-dubcode
$ dub add-local path-to-dubcode

Done.

On 06/01/2015 02:51 PM, Nick Sabalausky wrote:
> On 06/01/2015 09:22 AM, Rikki Cattermole wrote:
>>
>> Something I've thought about but haven't suggested up to now is making
>> dub-repo to support personal projects.
>> Basically one repo per person preconfigured if e.g. Github/Bitbucket is
>> specified.
>>
>> github.com/rikkimax/dubcode
>>
>> Just as an idea for what I'm meaning url wise.
>>
>> So then when specifying a dependency in dub, its just a matter of saying
>> <person>:<subproject>
>>
>
> Already works.
>
> $ git clone https://github.com/rikkimax/dubcode.git path-to-dubcode
> $ dub add-local path-to-dubcode
>
> Done.
>

Ehh, to clarify a few things:

$ cd my-project
$ cat dub.json
{
    "name": "my-project",
    "dependencies": {
        "not-in-dub-repo": "~>1.1.0"
    }
}
$ git clone https://github.com/user/not-in-dub-repo.git path/to/not-in-dub-repo
$ cat path/to/not-in-dub-repo/dub.json
{
    "name": "not-in-dub-repo"
}
$ dub add-local path/to/not-in-dub-repo

On 06/01/2015 01:36 PM, Nick Sabalausky wrote:
>
> There is one part of that old proposal that I have come to like though,
> and wouldn't mind seeing: There was an idea to have DMD automatically
> invoke an external tool
>

Oops, forgot to finish writing that thought...

As part of that old proposal, there was an idea to have DMD, when importing a module it couldn't find, automatically invoke an external tool which would tell DMD where to find the module. That way, a tool like RDMD would NOT need to invoke DMD twice (once to find the dependencies, and again to actually compile the dependencies).

I've sort of come around more to that particular idea (not the entire proposal though), although actually using it would likely increase compile times on windows (notoriously slow at launching processes, just look at dustmite on win), and frankly it'd probably be better just to incorporate rdmd's dependency searching as an optional feature of dmd.

On 6/1/15 12:16 PM, Nick Sabalausky wrote:
> On 06/01/2015 01:36 PM, Nick Sabalausky wrote:
>>
>> There is one part of that old proposal that I have come to like though,
>> and wouldn't mind seeing: There was an idea to have DMD automatically
>> invoke an external tool
>>
>
> Oops, forgot to finish writing that thought...
>
> As part of that old proposal, there was an idea to have DMD, when
> importing a module it couldn't find, automatically invoke an external
> tool which would tell DMD where to find the module. That way, a tool
> like RDMD would NOT need to invoke DMD twice (once to find the
> dependencies, and again to actually compile the dependencies).

rdmd could collect dependencies while performing the actual compilation. Then, if the dependencies are stable (compare with the saved ones), all done in one run. Otherwise, discard the output and rebuild.

This can be implemented today. Would you want to work on it?

Andrei

On 06/01/2015 02:49 PM, Steven Schveighoffer wrote:
> On 6/1/15 1:36 PM, Nick Sabalausky wrote:
>>
>> That sort of did what dub does, but badly. Among other things, it
>> directly encouraged developers to hardcode their exact dependencies. It
>> would have been a mess. Dub's package management side already handles
>> that DIP's goals WAY better. The *only* issue now is trying to opt-out
>> of dub's buildsystem side without causing problems (and I absolutely
>> believe that to be easily solvable, see my other posts).
>
> I'm not defending DIP11, just that the fact that it could be done
> without requiring an extra "dependencies" file. One thing I absolutely
> LOVE about git, is that when I do git status, if it sees a
> not-just-cloned repository, it gives me all the commands I would need to
> run to do all the various things. I rarely have to go to the git manual.
> I'd love to see something like the following:
>
>  > dmd x.d
> Error: Your import of some.lib.module didn't work. Running dub
> --with-some-cool-arg some.lib.module would fetch the correct module from
> code.dlang.org
>  > dub --with-some-cool-arg some.lib.module
> Dub : got it, fetched.
>  > dmd x.d
>  > echo 'yay, it worked! Too bad this is a fantasy :('
>

Yea, true, that would be nice.

However, once the dub PRs I'm working on now are finished, then you should be able to do something like this:

> dmd x.d
Error: Your import of some.lib.module didn't work.
> dub fetch libfoobar
> dmd `dub describe libfoobar --dmd-args` x.d

And it should even be possible to add support for this stuff, even without a dub.json for the current project:

> dmd x.d
Error: Your import of some.lib.module didn't work.
> dub find-module some.lib.module
(searches the repo)
libfoobar
> dmd `dub use libfoobar [optional-version] --dmd-args` x.d
(Auto-fetches libfoobar if it hasn't already been fetched, and then outputs the necessary CLI args for DMD to use.)


>>> The problem I have with dub is it makes you as a consumer of libraries
>>> live within its realm. You have to make a dub file to use it, and that's
>>> not a small burden.
>>>
>>
>> The dub.json file is an absolutely trivial "burden". It's a really,
>> REALLY simple little file. Or at least, it should be...
>
> Sorry, I suppose it's a small burden if you use dub frequently. I don't.
> Look what happens when I initialize a dub project:
>
> {
>      "name": "testdub",
>      "description": "A minimal D application.",
>      "copyright": "Copyright © 2015, steves",
>      "authors": ["steves"],
>      "dependencies": {
>      }
> }
>
> Cool, I can figure out how to massage all the existing strings, except,
> hm... dependencies. What goes in there?
>
> Now, it's time to get out the dub manual. annoying. Where is the comment
> that says "put your dependencies in here, they should look like this:
> ..."???
>
> And dub uses versions. How does that work? Again no comment.
>

There's really not much to remember. All you really need to remember is this:

"dependencies": {
    "libname": "==1.3.0"
}

That's really it.

There are fancier things you *can* do, but that right there is really all you need, nothing more.

Once dub supports a format that can actually freaking handle comments (*grumble grumble* stupid json *grumble grumble*), then dub init's generated file could be way more self-descriptive:

"dependencies": {
    // Uncomment these to use them:

    // Use an exact version:
    //     "name-of-lib": "==1.3.4",

    // Restrict to a certain minor version,
    // equivalent to ">=1.3.4 <1.4.0":
    //     "name-of-lib": "~>1.3.4",
}

> And where does the e.g. "name" get stuck into the executable? Oh, it
> doesn't. Why is it generated? I don't freaking know (except that it was
> the directory I was in when running dub init).
>

Name is just the name of the project. To identify it. Just like a Github project name. Or, pretty much anything called a "name", really. Every project has a name, dub or not. Not really hard to remember. There's no magic going on there.

And again, dub needs a format here that actually freaking supports comments.

> I'm sure dub is great for people who want to learn dub. For those who
> don't, it should work in a minimal way, and ESPECIALLY if we are to
> include it with the distribution.
>

It could be improved in some ways, but I think dub.json is way simpler then non-users really expect it to be. It just needs to communicate the minimal basics a little better, so people don't have to go trying to find <http://code.dlang.org/package-format> and then get intimidated by all the (mostly non-essential) stuff there. And of course, it needs the PRs I'm doing to make it way more friendly to non-dub buildsystems.

On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:
>
> rdmd could collect dependencies while performing the actual compilation.
> Then, if the dependencies are stable (compare with the saved ones), all
> done in one run. Otherwise, discard the output and rebuild.
>

Hmm, actually, that's a good idea.

> This can be implemented today. Would you want to work on it?
>

I've gotten rusty on RDMD's internals, but I'll take a look, see what I can manage.

On 6/1/15 3:53 PM, Nick Sabalausky wrote:
> On 06/01/2015 02:49 PM, Steven Schveighoffer wrote:
>> On 6/1/15 1:36 PM, Nick Sabalausky wrote:
>>>
>>> That sort of did what dub does, but badly. Among other things, it
>>> directly encouraged developers to hardcode their exact dependencies. It
>>> would have been a mess. Dub's package management side already handles
>>> that DIP's goals WAY better. The *only* issue now is trying to opt-out
>>> of dub's buildsystem side without causing problems (and I absolutely
>>> believe that to be easily solvable, see my other posts).
>>
>> I'm not defending DIP11, just that the fact that it could be done
>> without requiring an extra "dependencies" file. One thing I absolutely
>> LOVE about git, is that when I do git status, if it sees a
>> not-just-cloned repository, it gives me all the commands I would need to
>> run to do all the various things. I rarely have to go to the git manual.
>> I'd love to see something like the following:
>>
>>  > dmd x.d
>> Error: Your import of some.lib.module didn't work. Running dub
>> --with-some-cool-arg some.lib.module would fetch the correct module from
>> code.dlang.org
>>  > dub --with-some-cool-arg some.lib.module
>> Dub : got it, fetched.
>>  > dmd x.d
>>  > echo 'yay, it worked! Too bad this is a fantasy :('
>>
>
> Yea, true, that would be nice.
>
> However, once the dub PRs I'm working on now are finished, then you
> should be able to do something like this:
>
>  > dmd x.d
> Error: Your import of some.lib.module didn't work.
>  > dub fetch libfoobar
>  > dmd `dub describe libfoobar --dmd-args` x.d
>
> And it should even be possible to add support for this stuff, even
> without a dub.json for the current project:
>
>  > dmd x.d
> Error: Your import of some.lib.module didn't work.
>  > dub find-module some.lib.module
> (searches the repo)
> libfoobar
>  > dmd `dub use libfoobar [optional-version] --dmd-args` x.d
> (Auto-fetches libfoobar if it hasn't already been fetched, and then
> outputs the necessary CLI args for DMD to use.)

I would like to see more automation like this, even if it means there are some generated meta-files. But I'd like it to be dmd and dub working together. For instance, dmd should check to see if dub can has already fetched the dependency and use it (maybe with a switch enabled). Having to do the whole "`dub use blah blah`" on the command line is hacky.

A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step.

>> And where does the e.g. "name" get stuck into the executable? Oh, it
>> doesn't. Why is it generated? I don't freaking know (except that it was
>> the directory I was in when running dub init).
>>
>
> Name is just the name of the project. To identify it. Just like a Github
> project name. Or, pretty much anything called a "name", really. Every
> project has a name, dub or not. Not really hard to remember. There's no
> magic going on there.

I think I mistakenly hastily chose "name" as the first thing that stuck out as not being used, but it's actually probably the only one that *is* used (to name the resulting binary), which actually makes sense. Pick any of the other ones :)

> And again, dub needs a format here that actually freaking supports
> comments.

Yes, definitely. Does it have to be straight json?

-Steve

> rdmd could collect dependencies while performing the actual compilation. Then, if the dependencies are stable (compare with the saved ones), all done in one run. Otherwise, discard the output and rebuild.

I wrote a silly tool that gathers dependencies as it compiles. Essentially, it calls dmd -v and parses the result as rdmd does. The net effect is something akin to gcc's -MF -MD combo.

It's part of the build system I keep pestering everyone about. I had to write it because Ninja depends on it for proper dependency tracking. Make needs it too, but the Makefile code I have to generate is best not looked at. Make's fault, not mine.

Atila

On 6/1/15 12:59 PM, Nick Sabalausky wrote:
> I'll take a look, see what I
> can manage.

Thanks! -- Andrei

P.S. Note the crafty editing of the quote :o).

On Monday, 1 June 2015 at 19:16:31 UTC, Nick Sabalausky wrote:
> On 06/01/2015 01:36 PM, Nick Sabalausky wrote:
>>
>> There is one part of that old proposal that I have come to like though,
>> and wouldn't mind seeing: There was an idea to have DMD automatically
>> invoke an external tool
>>
>
> Oops, forgot to finish writing that thought...
>
> As part of that old proposal, there was an idea to have DMD, when importing a module it couldn't find, automatically invoke an external tool which would tell DMD where to find the module. That way, a tool like RDMD would NOT need to invoke DMD twice (once to find the dependencies, and again to actually compile the dependencies).
>
> I've sort of come around more to that particular idea (not the entire proposal though), although actually using it would likely increase compile times on windows (notoriously slow at launching processes, just look at dustmite on win), and frankly it'd probably be better just to incorporate rdmd's dependency searching as an optional feature of dmd.

Of course there is always shared libraries as an alternative to executables.