On Thursday, 14 April 2016 at 18:42:49 UTC, Jordi Sayol wrote:
> El 14/04/16 a les 17:54, Matthias Klumpp via Digitalmars-d-announce ha escrit:
>> On Tuesday, 12 April 2016 at 13:28:29 UTC, Jordi Sayol wrote:
>> [...]
>> I think with "property" you mean "virtual package". See https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual
>> Basically, the dmd package needs a "Provides: d-compiler" line, then it should be able to satisfy the dependencies of the dub package.
>
> Thanks. What happen is multiple packages, all of them not installed, sets "Provides: d-compiler"? Which one is installed?

I think in that case the (alphabetically) first real package is installed. This is an uncommon case though, usually when virtual packages are used, a default dependency is provided (so you have "default | virtual").

On Thursday, 14 April 2016 at 17:46:55 UTC, Johannes Pfau wrote:
> [...]
>
> (1) Interface files
>
> We have .di interface files as a replacement for C/C++ headers (although the .di extension is only a convention, you can also use the .d extension). These files do not contain function bodies, but they still need the complete source code for templates. The compilers can generate interface files from normal source code.
>
> https://dlang.org/dmd-linux.html#interface-files

Looks like a pretty good choice for handling the interface-with-library issue.

> OSS projects do not use interface files though: It prevents inlining of functions and there's no real benefit for OSS projects. Interface files are (theoretically) useful for closed source libraries if you don't want to ship the source code.

I think those would also be useful for FLOSS projects, if you ship a compiled binary and don't want to recompile the code.
This is the case for example for very huge projects which take long to compile (think in WebKit or Eigen3 dimensions), or for Linux distributions which want to separate as much code as possible and prevent code duplication and statically linked stuff to make security uploads easier and faster.

> (2) Linking against installed libraries
>
> The compilers can of course link against pre-compiled D libraries. IIRC dub does not have integrated support for this. The real problem is the D compilers are not ABI compatible.

Urgh...

> If you built a library with GDC you can't use it with LDC or DMD. This is one reason why dub compiles everything from source. (Even different frontend versions can sometimes break ABI compatibility).

Is there any way this can be fixed? https://dlang.org/spec/abi.html gave me the impression D has a defined ABI the compilers adhere too (which would be a great advantage over C++).
Fixing this would be pretty useful, not only for the distro usecase, I think.

Thanks for the explanations, this is useful to know and helps me to work around some of the issues short-term (long-term we would really need a solution for these issues, since this will become a huge issue if/when more D code makes it into distros).

Am Thu, 14 Apr 2016 23:16:49 +0000
schrieb Matthias Klumpp <matthias@tenstral.net>:

> On Thursday, 14 April 2016 at 17:46:55 UTC, Johannes Pfau wrote:
> > OSS projects do not use interface files though: It prevents inlining of functions and there's no real benefit for OSS projects. Interface files are (theoretically) useful for closed source libraries if you don't want to ship the source code.
> 
> I think those would also be useful for FLOSS projects, if you ship a compiled binary and don't want to recompile the code. This is the case for example for very huge projects which take long to compile (think in WebKit or Eigen3 dimensions), or for Linux distributions which want to separate as much code as possible and prevent code duplication and statically linked stuff to make security uploads easier and faster.
> 

I totally agree it makes sense to ship precompiled libraries. But even with precompiled libraries you can still ship the full source code instead of header files. An example:

a/foo.d:
------------------------------------------------
module foo;
int fooFunc() {return 42;}
------------------------------------------------

=> dmd -H a/foo.d => b/foo.di
------------------------------------------------
// D import file generated from 'foo.d'
module foo;
int fooFunc();
------------------------------------------------

dmd -lib a/foo.d -oflibfoo.a

main.d
------------------------------------------------
import foo;
void main(){fooFunc()}
------------------------------------------------

// We need foo.di or foo.d in the import path
dmd main.d -L-L. -L-lfoo
main.d(1): Error: module foo is in file 'foo.d' which cannot be read

// This uses the foo.di header
dmd main.d -L-L. -L-lfoo -Ib

// This uses foo.d as a header, but does _not_ compile foo.d dmd main.d -L-L. -L-lfoo -Ia


The important point here is that you can use the normal source code as headers. The source code of a library is not recompiled, it's only used to parse function definitions and similar stuff. The compilation speed should be very similar for .d and .di files as well. The benefit of shipping the complete source code is that fooFunc can be inlined into main.d with the foo.d source, but not with foo.di.

Shipping .di files only makes sense if you have to hide the source code.

> > If you built a library with GDC you can't use it with LDC or DMD. This is one reason why dub compiles everything from source. (Even different frontend versions can sometimes break ABI compatibility).
> 
> Is there any way this can be fixed?
> https://dlang.org/spec/abi.html gave me the impression D has a
> defined ABI the compilers adhere too (which would be a great
> advantage over C++).
> Fixing this would be pretty useful, not only for the distro
> usecase, I think.

The ABI is partially standardized:
* Name mangling is the same for all compilers.
* D has a special calling convention on X86 windows, but all other
  architectures and all other OS use the C calling convention. The
  compilers should be compatible, but this is something which needs
  some testing.

* Exception handling: DMD recently implemented DWARF exception
  handling, so the compilers could be compatible but the personality
  functions are not. I'm not sure if the implementations are
  compatible, but not even the function names match
  (__dmd_personality_v0 / __gdc_personality_v0)
* The biggest problem is druntime: we need to make sure that the
  druntime libraries used by different compilers have a compatible ABI.

> Thanks for the explanations, this is useful to know and helps me to work around some of the issues short-term (long-term we would really need a solution for these issues, since this will become a huge issue if/when more D code makes it into distros).

I agree having a common ABI should be prioritized. It's really
annoying for distributions (E.g. archlinux installs druntime/phobos into
different directories /usr/include/dlang/{gdc,dmd,ldc} and never
installs compiled libraries). Shared D libraries are useless in
practice because of this issue.

This needs coordinated work from all compiler teams though. For GDC I'll finally have to finish shared library support first...

El 15/04/16 a les 01:09, Matthias Klumpp via Digitalmars-d-announce ha escrit:
> On Thursday, 14 April 2016 at 18:42:49 UTC, Jordi Sayol wrote:
>> El 14/04/16 a les 17:54, Matthias Klumpp via Digitalmars-d-announce ha escrit:
>>> On Tuesday, 12 April 2016 at 13:28:29 UTC, Jordi Sayol wrote:
>>> [...]
>>> I think with "property" you mean "virtual package". See https://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual
>>> Basically, the dmd package needs a "Provides: d-compiler" line, then it should be able to satisfy the dependencies of the dub package.
>>
>> Thanks. What happen is multiple packages, all of them not installed, sets "Provides: d-compiler"? Which one is installed?
> 
> I think in that case the (alphabetically) first real package is installed. This is an uncommon case though, usually when virtual packages are used, a default dependency is provided (so you have "default | virtual").
> 

I'll include "Provides: d-compiler" on dlang dmd deb package and d-apt dmd-bin deb too. Many thanks!

On 04/15/2016 01:34 PM, Jordi Sayol via Digitalmars-d-announce wrote:
> I'll include "Provides: d-compiler" on dlang dmd deb package and d-apt dmd-bin deb too. Many thanks!

Awesomne, Jordi. I recently got a notice that the dmd compiler has been updated by Ubuntu's package manager. Clicked, got it, all went smoothly. Should I take it we owe all of that to you? -- Andrei

El 15/04/16 a les 19:52, Andrei Alexandrescu via Digitalmars-d-announce ha escrit:
> Awesomne, Jordi. I recently got a notice that the dmd compiler has been updated by Ubuntu's package manager. Clicked, got it, all went smoothly. Should I take it we owe all of that to you? -- Andrei

I think so :-)
Many thanks Andrei, and all d-apt users.

On Thursday, 14 April 2016 at 16:29:31 UTC, Matthias Klumpp wrote:
> FTR, I filed https://bugs.launchpad.net/ubuntu/+source/ldc/+bug/1570006 - if we are lucky, this still has a chance to go in.

That's great, thank you very much.

> As for further dub stuff, it is important that https://github.com/D-Programming-Language/dub/issues/811 is addressed, so we can build software using dub without downloading stuff from the internet.
> Btw, since D doesn't know traditional headers like C/C++, how would I link against a library without having the full sourcecode present somewhere?

Is there anything wrong -- in principle -- with (for now) treating open-source D libraries much like the various Boost C++ libraries, which AFAICT are almost entirely header-based?

I ask because so far as I can tell that would work around the immediate problems of compiler incompatibilities, it would not add any extra work compiler-wise that's not already there with dub's current way of doing things, and for template-heavy D code (which is quite a lot of code), it makes no difference, since that would have to be available in .d or .di files anyway.

On Saturday, 16 April 2016 at 09:48:01 UTC, Joseph Rushton Wakeling wrote:
> [..]
>> As for further dub stuff, it is important that https://github.com/D-Programming-Language/dub/issues/811 is addressed, so we can build software using dub without downloading stuff from the internet.
>> Btw, since D doesn't know traditional headers like C/C++, how would I link against a library without having the full sourcecode present somewhere?
>
> Is there anything wrong -- in principle -- with (for now) treating open-source D libraries much like the various Boost C++ libraries, which AFAICT are almost entirely header-based?

It's a bit ugly, because we are essentially embedding code copies into every binary, instead of having a proper shared library stuff can link to (and we also need to recompile everything again).
Aside from that, there is no real issue - Debian policy allows doing that, and in fact, this is the thing I am currently attempting to do to work around the ABI issues.

> I ask because so far as I can tell that would work around the immediate problems of compiler incompatibilities, it would not add any extra work compiler-wise that's not already there with dub's current way of doing things, and for template-heavy D code (which is quite a lot of code), it makes no difference, since that would have to be available in .d or .di files anyway.

Jup - what dub would still need to do though is to find the globally installed software, so we don't need to patch each dub.(json|sdl) file to find the code which was installed via distro packages.
At time, I install the D code into `/usr/include/d/<package>`[1] where dub could theoretically automatically find it.

[1]: Probably not the best location since this isn't actually a header... (alternative could be `/usr/share/dlang/`)

On Friday, 15 April 2016 at 09:15:05 UTC, Johannes Pfau wrote:
> Am Thu, 14 Apr 2016 23:16:49 +0000
> schrieb Matthias Klumpp <matthias@tenstral.net>:
>
>> On Thursday, 14 April 2016 at 17:46:55 UTC, Johannes Pfau wrote:
>> > OSS projects do not use interface files though: It prevents inlining of functions and there's no real benefit for OSS projects. Interface files are (theoretically) useful for closed source libraries if you don't want to ship the source code.
>> 
>> I think those would also be useful for FLOSS projects, if you ship a compiled binary and don't want to recompile the code. This is the case for example for very huge projects which take long to compile (think in WebKit or Eigen3 dimensions), or for Linux distributions which want to separate as much code as possible and prevent code duplication and statically linked stuff to make security uploads easier and faster.
>
> I totally agree it makes sense to ship precompiled libraries. But even with precompiled libraries you can still ship the full source code instead of header files. An example:
>
> [...]
>
> The important point here is that you can use the normal source code as headers. The source code of a library is not recompiled, it's only used to parse function definitions and similar stuff. The compilation speed should be very similar for .d and .di files as well. The benefit of shipping the complete source code is that fooFunc can be inlined into main.d with the foo.d source, but not with foo.di.
>
> Shipping .di files only makes sense if you have to hide the source code.

Or when the source-code is very large, I guess. But the way D handles this makes much sense to me!

>> [...]
>
> I agree having a common ABI should be prioritized. It's really
> annoying for distributions (E.g. archlinux installs druntime/phobos into
> different directories /usr/include/dlang/{gdc,dmd,ldc} and never
> installs compiled libraries). Shared D libraries are useless in
> practice because of this issue.

Yeah, and this makes D pretty hard to sell to distributors. While I would argue that for applications it is no longer essential to be packages and shipped by distributions (or rather it won't be in future), for a "new" programming language this is a crucial thing.
As soon as it is easy to try out for developers, more people will "just try it" and maybe stick with it. If testing it is hard due to incompatible standard libraries, runtimes, or simply missing free D compilers, people won't try it.
In fact, that it also is pretty hard for people to compile applications written in D on distributions other than Debian (and even there it was not without issues...) was *the* biggest concern for me.

> This needs coordinated work from all compiler teams though. For GDC I'll finally have to finish shared library support first...

That would be awesome to have! Thank you for working on this!

On Fri, 2016-04-15 at 20:03 +0200, Jordi Sayol via Digitalmars-d- announce wrote:
> […]
> Many thanks Andrei, and all d-apt users.

Thanks for creating and running D-Apt – all it's users a A-D-Apt-able. ;-)


I wish there was an equivalent RPM store for Fedora Rawhide. I'd be happy to help out creating something, but not if I am the only user.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder