On 6/14/11 10:27 AM, Graham Fawcett wrote:
> On Tue, 14 Jun 2011 08:53:16 -0500, Andrei Alexandrescu wrote:
>
>> http://www.wikiservice.at/d/wiki.cgi?LanguageDevel/DIPs/DIP11
>>
>> Destroy.
>>
>>
>> Andrei
>
> What's the expected payload format? A text/plain D source-file? A zip or
> tar archive?

Text for now.

> If an archive, what's the required directory layout? Should
> "dsource.foo.baz" be required to be in "/dsource/foo/baz.d" within the
> archive?

I agree we need to address these in the future, and also binary distributions (e.g. .di files + .a/.lib files).

> And if not an archive, how to reasonably handle multi-file packages?

Consider a library "acme" consisting of three files: widgets.d, gadgets.d, fidgets.d in "http://acme.com/d/". It also depends on the external library monads on "http://nad.mo/d".

// User code:
pragma(lib, acme, "http://acme.com/d/");
import acme.widgets;
... use ...

// widgets.d
// Assume it depends on other stuff in the same lib
// and on monads.d
pragma(lib, monads, "http://nad.mo/d/");
import acme.gadgets, acme.fidgets, monads.io;

This is all that's needed for the compiler to download and compile everything needed.


Andrei

On 6/14/11 10:31 AM, Daniel Gibson wrote:
> Am 14.06.2011 17:31, schrieb Adam D. Ruppe:
>> Daniel Gibson:
>>> Is this possible with plain webspace, i.e. when you don't control
>>> the server?
>>
>> If you have your own domain name, you can certainly set this up.
>>
>> But if your host is fully managed, probably not. A fix is to set
>> up a redirection server people can share. This is kinda like a
>> central repository, but you wouldn't have to upload your files
>> directly. You might just put up a url (and other metadata?) to point
>> people to the final location.
>>
>> However, I'd prefer to have simple files. Downloading from git
>> is imo a mistake - those files are probably in development...
>> meaning they are mutable. If the files are mutable, it includes a
>> lot of pain for versioning and caching.
>
> There could be stable branches in the git repositories for exactly this
> purpose.
> Furthermore it's be nice as an additional feature - some people may
> always want to use the latest bleeding edge version from git/svn/whatever.

I noticed that many online git/svn repos have bridges that serve raw files via http directly. No need for using the git/svn tool on the client.

Andrei

Am 14.06.2011 17:33, schrieb Andrei Alexandrescu:
> On 6/14/11 10:31 AM, Daniel Gibson wrote:
>> Am 14.06.2011 17:31, schrieb Adam D. Ruppe:
>>> Daniel Gibson:
>>>> Is this possible with plain webspace, i.e. when you don't control the server?
>>>
>>> If you have your own domain name, you can certainly set this up.
>>>
>>> But if your host is fully managed, probably not. A fix is to set
>>> up a redirection server people can share. This is kinda like a
>>> central repository, but you wouldn't have to upload your files
>>> directly. You might just put up a url (and other metadata?) to point
>>> people to the final location.
>>>
>>> However, I'd prefer to have simple files. Downloading from git is imo a mistake - those files are probably in development... meaning they are mutable. If the files are mutable, it includes a lot of pain for versioning and caching.
>>
>> There could be stable branches in the git repositories for exactly this
>> purpose.
>> Furthermore it's be nice as an additional feature - some people may
>> always want to use the latest bleeding edge version from
>> git/svn/whatever.
> 
> I noticed that many online git/svn repos have bridges that serve raw files via http directly. No need for using the git/svn tool on the client.
> 
> Andrei

Maybe "many" do this, but i.e. custom SVN repos often don't to it.
I like using SVN with trac - trac does have an interface for displaying
the code, you can even download the raw files, but it's rather slow (due
to trac overhead) and usually shouldn't be used to download the source.

Cheers,
- Daniel

On 6/14/11 10:26 AM, Daniel Gibson wrote:
> One additional file. I don't think having one file would be a burden to
> the programmer, not much more than adding pragmas in his code.
>
> But if there's central metadata repository even this additional file
> isn't needed - neither are pragmas - (the build-tool will ask that repo
> where to find the lib/module), unless the lib is kind of obscure or
> brand-new and thus not known by the metadata repo. And in that case:
> it's just a single file.
>
> (Of course it would be possible to periodically or via "build-tool
> update" - like apt-get update - fetch the metadata, so the server
> doesn't have to be asked each time.)

I agree that a build tool is an alternative. The problem is that it's front-heavy - we need to design the config file format, design the tool to deal with various dependency issues etc. Many of these issues are solved by design or don't even exist with the pragma, for example there's no need for a config file format or for config files in the first place (although they can be easily defined as small .d files consisting of pragmas).

One thing I like about the pragma is that it's mostly mechanism and very little policy, while at the same time fostering very simple, straightforward policies. Source files can carry their own dependencies (but they don't need to), transitivity just works, (re|in)direction and central repos are possible without being required.

One other interesting aspect is that the string literal can be CTFE-constructed, i.e. may include a path to a library depending on the OS, version, etc. An external tool would need to give up on that (and use multiple configuration files) or invent its own string manipulation primitives.


Andrei

On Tue, 14 Jun 2011 10:51:06 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 6/14/11 9:45 AM, Michel Fortin wrote:
>> On 2011-06-14 09:53:16 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> said:
>>
>>> http://www.wikiservice.at/d/wiki.cgi?LanguageDevel/DIPs/DIP11
>>
>> I disagree about the model where you specify URLs in source code.
>
> Why?

Because the location of software is not controllable by the builder.  For example, if the owner of the code decides to move his software, the user compiling code using that software is at his mercy.

Contrast that with import paths, where the user compiling is in control of the paths on his hard drive.

So essentially, code can "break" outside of your control, and the only recourse you have is to modify the source code (not a pleasant prospect for people who are not developers).

>
>> I'd suggest an alternative solution: If DMD finds a dmd.conf file at the
>> root of one of the -I directories, it uses the download parameters in
>> that file to download to populate the directory with modules as needed.
>> This would also apply to the current directory. The content of that
>> dmd.conf file could look like this:
>>
>> [module-download]
>> package.* = http://someserver.net/d-repo/latest/
>> otherpack.* = http://otherserver.net/d-repo/1.2/
>>
>> That way it's easy to understand what comes from where without having to
>> scan all your source code for pragmas.
>
> I agree this would be good to have, but not exclusively. The problem is that anyone who wants to share some code must require potential users to modify their config files, instead of offering them a key-ready solution.

Just unpack them?  I mean, there's no reason you can't install their code on your hard drive.  If you want to publish your code for others to use, you have three options:

1. provide a download.
2. provide a config line for people to add to their dmd.conf
2. put your code in a central repository (like dsource.org) that is already added to everyone's configuration by default.

What might be useful is a tool to allow simplified management of the sources, i.e.:

dmdpkg addsource http://...

I also think it would be nice to provide wildcards for centralized servers.  So that as packages are added to the server, you don't need to modify your rules.  For example, if someone adds a package to dsource, there should be a rule that allows people to publish the code without people having to add lines to their dmd.conf.  This might require some sort of metadata file system.

I really think this project needs more design work.

-Steve

> One other interesting aspect is that the string literal can be CTFE-constructed,

Oh, or it could be in version {} blocks. I like that.

I think we should actually whip up a working model. It needn't be
a compiler feature at this point - we can use
pragma(msg, "BUILD: " ~ param) for now and have a helper program
scan dmd's output.

On Tue, 14 Jun 2011 10:31:59 -0500, Andrei Alexandrescu wrote:

> On 6/14/11 10:27 AM, Graham Fawcett wrote:
>> On Tue, 14 Jun 2011 08:53:16 -0500, Andrei Alexandrescu wrote:
>>
>>> http://www.wikiservice.at/d/wiki.cgi?LanguageDevel/DIPs/DIP11
>>>
>>> Destroy.
>>>
>>>
>>> Andrei
>>
>> What's the expected payload format? A text/plain D source-file? A zip or tar archive?
> 
> Text for now.
> 
>> If an archive, what's the required directory layout? Should "dsource.foo.baz" be required to be in "/dsource/foo/baz.d" within the archive?
> 
> I agree we need to address these in the future, and also binary distributions (e.g. .di files + .a/.lib files).
> 
>> And if not an archive, how to reasonably handle multi-file packages?
> 
> Consider a library "acme" consisting of three files: widgets.d, gadgets.d, fidgets.d in "http://acme.com/d/". It also depends on the external library monads on "http://nad.mo/d".
> 
> // User code:
> pragma(lib, acme, "http://acme.com/d/"); import acme.widgets;
> ... use ...
> 
> // widgets.d
> // Assume it depends on other stuff in the same lib // and on monads.d
> pragma(lib, monads, "http://nad.mo/d/"); import acme.gadgets,
> acme.fidgets, monads.io;
> 
> This is all that's needed for the compiler to download and compile everything needed.

So, to clarify:

> pragma(lib, acme, "http://acme.com/d/");

...establishes a "prefix" relationship: modules names prefixed with "acme." may be found at URLs prefixed with "http://acme.com/d/". So we would expect to find:

  acme.widgets         at  http://acme.com/d/widgets.d
  acme.widgets.core.x  at  http://acme.com/d/widgets/core/x.d

Correct?

Graham

Am 14.06.2011 17:41, schrieb Andrei Alexandrescu:
> On 6/14/11 10:26 AM, Daniel Gibson wrote:
>> One additional file. I don't think having one file would be a burden to the programmer, not much more than adding pragmas in his code.
>>
>> But if there's central metadata repository even this additional file isn't needed - neither are pragmas - (the build-tool will ask that repo where to find the lib/module), unless the lib is kind of obscure or brand-new and thus not known by the metadata repo. And in that case: it's just a single file.
>>
>> (Of course it would be possible to periodically or via "build-tool update" - like apt-get update - fetch the metadata, so the server doesn't have to be asked each time.)
> 
> I agree that a build tool is an alternative. The problem is that it's front-heavy - we need to design the config file format, design the tool to deal with various dependency issues etc.

And that would be harder than getting support for this in the compiler? (Especially since there are already tools that could be enhanced)

> Many of these issues are
> solved by design or don't even exist with the pragma, for example
> there's no need for a config file format or for config files in the
> first place (although they can be easily defined as small .d files
> consisting of pragmas).
> 
> One thing I like about the pragma is that it's mostly mechanism and very little policy, while at the same time fostering very simple, straightforward policies. Source files can carry their own dependencies (but they don't need to), transitivity just works, (re|in)direction and central repos are possible without being required.
> 
> One other interesting aspect is that the string literal can be CTFE-constructed, i.e. may include a path to a library depending on the OS, version, etc. An external tool would need to give up on that (and use multiple configuration files) or invent its own string manipulation primitives.
> 
> 
> Andrei

On Tue, 14 Jun 2011 15:59:15 +0000, Adam D. Ruppe wrote:

>> One other interesting aspect is that the string literal can be CTFE-constructed,
> 
> Oh, or it could be in version {} blocks. I like that.
> 
> I think we should actually whip up a working model. It needn't be a compiler feature at this point - we can use pragma(msg, "BUILD: " ~ param) for now and have a helper program scan dmd's output.

+1, sounds fun. :)

Rather than pragma(msg), you could also use pragma(liburl), and run dmd with "-ignore -v". You can parse the pragma from there. (I think you'd need to write `pragma(liburl, "name-in-quotes", "url-in-quotes")`, a slight diversion from Andrei's syntax, but otherwise it would work.)

Graham

On 6/14/11 11:26 AM, Graham Fawcett wrote:
> On Tue, 14 Jun 2011 10:31:59 -0500, Andrei Alexandrescu wrote:
>
>> On 6/14/11 10:27 AM, Graham Fawcett wrote:
>>> On Tue, 14 Jun 2011 08:53:16 -0500, Andrei Alexandrescu wrote:
>>>
>>>> http://www.wikiservice.at/d/wiki.cgi?LanguageDevel/DIPs/DIP11
>>>>
>>>> Destroy.
>>>>
>>>>
>>>> Andrei
>>>
>>> What's the expected payload format? A text/plain D source-file? A zip
>>> or tar archive?
>>
>> Text for now.
>>
>>> If an archive, what's the required directory layout? Should
>>> "dsource.foo.baz" be required to be in "/dsource/foo/baz.d" within the
>>> archive?
>>
>> I agree we need to address these in the future, and also binary
>> distributions (e.g. .di files + .a/.lib files).
>>
>>> And if not an archive, how to reasonably handle multi-file packages?
>>
>> Consider a library "acme" consisting of three files: widgets.d,
>> gadgets.d, fidgets.d in "http://acme.com/d/". It also depends on the
>> external library monads on "http://nad.mo/d".
>>
>> // User code:
>> pragma(lib, acme, "http://acme.com/d/"); import acme.widgets;
>> ... use ...
>>
>> // widgets.d
>> // Assume it depends on other stuff in the same lib // and on monads.d
>> pragma(lib, monads, "http://nad.mo/d/"); import acme.gadgets,
>> acme.fidgets, monads.io;
>>
>> This is all that's needed for the compiler to download and compile
>> everything needed.
>
> So, to clarify:
>
>> pragma(lib, acme, "http://acme.com/d/");
>
> ...establishes a "prefix" relationship: modules names prefixed with
> "acme." may be found at URLs prefixed with "http://acme.com/d/". So we
> would expect to find:
>
>    acme.widgets         at  http://acme.com/d/widgets.d
>    acme.widgets.core.x  at  http://acme.com/d/widgets/core/x.d
>
> Correct?
>
> Graham

Yes, that's the intent. I added an explanation and an example to subsection "Package case".

Andrei