Hello,

I am working on a large cross-platform project, which will be written primarily in D, interfacing to C as necessary. To get finer control over memory safety, binary size, and use of the GC, we would like to disclude libphobos as a dependency in lieu of our own code. The project is compiled using LDC.

I am not sure if this is possible though, as it seems there are certain things in libphobos that are tightly coupled into the D runtime. There are several things in the core namespace that would be helpful for us (SIMD, C bindings, etc), but I am not sure if that is not also part of libphobos along with the other namespaces.

How do I remove libphobos as a runtime dependency with ld and MSVC’s link.exe? Is it possible to decouple core from other parts of the runtime, and if so, how?

Regards,
A. Nicholi

On 4/25/18 11:04 PM, A. Nicholi wrote:
> Hello,
> 
> I am working on a large cross-platform project, which will be written primarily in D, interfacing to C as necessary. To get finer control over memory safety, binary size, and use of the GC, we would like to disclude libphobos as a dependency in lieu of our own code. The project is compiled using LDC.
> 
> I am not sure if this is possible though, as it seems there are certain things in libphobos that are tightly coupled into the D runtime. There are several things in the core namespace that would be helpful for us (SIMD, C bindings, etc), but I am not sure if that is not also part of libphobos along with the other namespaces.
> 
> How do I remove libphobos as a runtime dependency with ld and MSVC’s link.exe? Is it possible to decouple core from other parts of the runtime, and if so, how?
> 
> Regards,
> A. Nicholi

It sounds like you want to simply link against druntime and not phobos.

In many cases, phobos and druntime are bundled together as a static library. If you just want druntime (core and rt) then you probably have to build druntime from source.

I'm generally working on dmd, so I don't know how it works for ldc. But I'm sure it's quite similar (building druntime with dmd is just one make command).

-Steve

On Thursday, 26 April 2018 at 03:04:55 UTC, A. Nicholi wrote:

> I am working on a large cross-platform project, which will be written primarily in D, interfacing to C as necessary. To get finer control over memory safety, binary size, and use of the GC, we would like to disclude libphobos as a dependency in lieu of our own code. The project is compiled using LDC.
>
> I am not sure if this is possible though, as it seems there are certain things in libphobos that are tightly coupled into the D runtime. There are several things in the core namespace that would be helpful for us (SIMD, C bindings, etc), but I am not sure if that is not also part of libphobos along with the other namespaces.
>
> How do I remove libphobos as a runtime dependency with ld and MSVC’s link.exe? Is it possible to decouple core from other parts of the runtime, and if so, how?

I suggest reading the following 2 items before digging deeper:
https://dlang.org/blog/2017/08/23/d-as-a-better-c/
https://dlang.org/changelog/2.079.0.html#minimal_runtime

Next you should realize that Phobos and DRuntime are actually 2 different things that are, unfortunately, often compiled together into one "libphobos2" binary when the compiler is released.

Phobos is the standard library:
https://github.com/dlang/phobos

DRuntime contains C language bindings, OS bindings, and some D language feature implementations, including the GC.
https://github.com/dlang/druntime

It's quite unfortunate there so much hard coupling between all of these components, but that's the way it is.  Phobos depends on DRuntime.  DRuntime depends on C language bindings and OS bindings.

Compiling the simplest of programs with DMD (https://run.dlang.io/is/KMjKsJ) results in the following C compiler invocation:
------
cc onlineapp.o -o /tmp/dmd_run3nK4IS -g -m64 -Xlinker -v -L/dlang/dmd/linux/bin64/../lib64 -Xlinker --export-dynamic -Xlinker -Bstatic -lphobos2 -Xlinker -Bdynamic -lpthread -lm -lrt -ldl

Which in turn calls the following linker invocation:
------
GNU gold (GNU Binutils for Ubuntu 2.29.1) 1.14
collect2 version 7.2.0
/usr/bin/ld -plugin /usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper -plugin-opt=-fresolution=/tmp/ccg3GKxT.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro -o /tmp/dmd_run3nK4IS /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o -L/dlang/dmd/linux/bin64/../lib64 -L/usr/lib/gcc/x86_64-linux-gnu/7 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/dlang/dmd/linux/lib64 -L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. onlineapp.o -v --export-dynamic -Bstatic -lphobos2 -Bdynamic -lpthread -lm -lrt -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o

The compiler uses the C compiler (unfortunately again) to do its linking; and it becomes evident that D is in some ways a layer on top of C.

You can compile some D programs without linking libphobos2, but will require separate compilation and linking because the compiler itself actually hard-codes the call to the linker (actually the C compiler as demonstrated above).  Example 3 at https://dlang.org/changelog/2.079.0.html#minimal_runtime demonstrates this.

If you use that method, you won't be able to use certain features of D that have runtime implementations.  The obvious ones are classes, dynamic arrays, and exceptions.

I could go on, but I'd have to make some assumptions about what you're really after.  Feel free to ask more specific questions and I'll be happy to share what I know (or at least what I think I know; sometimes I'm wrong).

Mike

On Thursday, 26 April 2018 at 03:04:55 UTC, A. Nicholi wrote:
> Hello,
>
> I am working on a large cross-platform project, which will be written primarily in D, interfacing to C as necessary. To get finer control over memory safety, binary size, and use of the GC, we would like to disclude libphobos as a dependency in lieu of our own code. The project is compiled using LDC.
>
> I am not sure if this is possible though, as it seems there are certain things in libphobos that are tightly coupled into the D runtime. There are several things in the core namespace that would be helpful for us (SIMD, C bindings, etc), but I am not sure if that is not also part of libphobos along with the other namespaces.
>
> How do I remove libphobos as a runtime dependency with ld and MSVC’s link.exe? Is it possible to decouple core from other parts of the runtime, and if so, how?
>
> Regards,
> A. Nicholi

LDC allows specifying default libs, something like `-defaultlib=phobos2-ldc,druntime-ldc`; you can remove phobos from the list.

I think you can use any symbol that is compile time only from phobos (like std.traits and std.meta), and you can take stuff from phobos and adapt it to your own needs. I use the compiler explorer (https://d.godbolt.org/) to check the generate code for symbols added.

LDC is flexible enough, from my experience, to allow certain degree of customization. You can override the default C compiler and add your own paths, or you can create a custom C compiler driver that overrides defaults. Check the ldc command line spec for inspiration https://wiki.dlang.org/Using_LDC

Something to remember though, if you intend to commit to serious commercial use, you should be prepared to dig in and fix stuff that doesn't work, or add new functionality, or sponsor the foundation for support. D and LDC are opensource projects, and they have mostly good quality implementation, but they are not commercial grade top of the line products.

On Thursday, 26 April 2018 at 03:53:54 UTC, Mike Franklin wrote:
>
> I suggest reading the following 2 items before digging deeper:
> https://dlang.org/blog/2017/08/23/d-as-a-better-c/
> https://dlang.org/changelog/2.079.0.html#minimal_runtime

I didn’t know D had begun offering serious decoupling like that. With something like this, we may very well be able to avoid writing C entirely, at least in-house! Thank you for bringing that up.

On Thursday, 26 April 2018 at 03:53:54 UTC, Mike Franklin wrote:
> The compiler uses the C compiler (unfortunately again) to do its linking; and it becomes evident that D is in some ways a layer on top of C.
>
> You can compile some D programs without linking libphobos2, but will require separate compilation and linking because the compiler itself actually hard-codes the call to the linker (actually the C compiler as demonstrated above).  Example 3 at https://dlang.org/changelog/2.079.0.html#minimal_runtime demonstrates this.
>
> If you use that method, you won't be able to use certain features of D that have runtime implementations.  The obvious ones are classes, dynamic arrays, and exceptions.
>
> I could go on, but I'd have to make some assumptions about what you're really after.  Feel free to ask more specific questions and I'll be happy to share what I know (or at least what I think I know; sometimes I'm wrong).
>
> Mike

So in a way, the D runtime is similar to libstdc++, providing implementations of runtime language features. But it is also like C++ in that those language features can be avoided, correct? At least with the use of minimal D, I mean. This means that as a language, there is enough granularity to theoretically provide as few or as many features as one desires for their use case, making the penning of new C and C++ code redundant? Do I get this right?

On Thursday, 26 April 2018 at 06:32:14 UTC, Radu wrote:
>
> LDC allows specifying default libs, something like `-defaultlib=phobos2-ldc,druntime-ldc`; you can remove phobos from the list.

Alright, that sounds simple enough. Thanks!

On Thursday, 26 April 2018 at 06:32:14 UTC, Radu wrote:
> I think you can use any symbol that is compile time only from phobos (like std.traits and std.meta), and you can take stuff from phobos and adapt it to your own needs. I use the compiler explorer (https://d.godbolt.org/) to check the generate code for symbols added.

Didn’t know Godbolt supported D. Thanks for sharing :)

On Thursday, 26 April 2018 at 06:32:14 UTC, Radu wrote:
> Something to remember though, if you intend to commit to serious commercial use, you should be prepared to dig in and fix stuff that doesn't work, or add new functionality, or sponsor the foundation for support. D and LDC are opensource projects, and they have mostly good quality implementation, but they are not commercial grade top of the line products.

Right. We’re prepared for working around that, especially since we’ll be targeting platforms and architectures that LDC boasts only fleeting support for.

On Thursday, 26 April 2018 at 09:24:19 UTC, A. Nicholi wrote:

> So in a way, the D runtime is similar to libstdc++, providing implementations of runtime language features.

I would argue that Phobos is more analogous to libstc++, but there are some language features in C++ that are implemented in libstc++.  For example, if you try to compile the following code.

---main.cpp
int main()
{
    throw 1;
    return 0;
}

$g++ -c main.cpp
$ld main.o
ld: warning: cannot find entry symbol _start; defaulting to 00000000004000e8
main.o: In function `main':
main.cpp:(.text+0xa): undefined reference to `__cxa_allocate_exception'
main.cpp:(.text+0x1c): undefined reference to `typeinfo for int'
main.cpp:(.text+0x24): undefined reference to `__cxa_throw'

So, yeah those "__cxa_..." functions are very similar to what you'll find in Druntim.  D calls them runtime hooks, and you can find an unmaintained list of them here:  https://wiki.dlang.org/Runtime_Hooks

Here's similar experiment in D.  IMO D is actually better than C++ in this regard because it emits a compile-time error, instead of a linker error, when it can't find something in the runtime that it needs.

---object.d
module object;
// this useless object.d file is currently required, though I'm trying to
// find a way to get rid of such nonsense.
// See https://github.com/dlang/dmd/pull/7825

---main.d
void main()
{
    throw new Exception("whatever");
}

$dmd -conf= -defaultlib= main.d
main.d(3): Error: Cannot use throw statements because object.Throwable was not declared

> But it is also like C++ in that those language features can be avoided, correct?

That was the goal with the "minimal runtime" features released in v2.079.  It's still a work in progress though.  See https://github.com/dlang/dmd/pull/8204

But, yes, as long as you avoid certain features that require runtime implementations in your code, you should still be able to write software in D.  And, if I am allowed to have may way, it should get even better.

> At least with the use of minimal D, I mean. This means that as a language, there is enough granularity to theoretically provide as few or as many features as one desires for their use case, making the penning of new C and C++ code redundant? Do I get this right?

Not sure if I totally understand that, but it sounds right on.  The goal with the minimal runtime features is to allow one use D on any platform in a pay-as-you-go fashion.  That is, you only need to implement what you intend to use.  The primary use case I have in mind is using D for resource-constrained microcontrollers, though it would also be useful for libraries written in D, but intended to be linked in by applications written in other languages.  I'm there are also use cases that I haven't even thought of.

Mike

On Thursday, 26 April 2018 at 03:04:55 UTC, A. Nicholi wrote:
> I am not sure if this is possible though

I think you've got the technical answer already (just don't link in phobos2) but I'll add my 2c that Phobosless programming isn't just possible but realistically doable.  It's a shame to go without Phobos because it's got so much higher-level functionality, but if you're okay with writing things from the ground up in C-like code, it works.  Most third-party D libraries depend on Phobos, though, so it's mostly DIY and using bindings to other languages.

Have a look at the standard library here:
https://dlang.org/phobos/index.html

Basically, all the std.* is Phobos, and most of the core.* stuff needs linking with the D runtime library. However, there's a bunch of bindings to standard C stuff in core.stdc.* that only need the C runtime and standard library, and these come in really handy when you're not using Phobos.  (There are more platform-specific C bindings in core.sys.* that aren't listed on that page.)  Also, there are some other things in core.bitop and core.atomic (probably more) that don't need linking to druntime.

On Friday, 27 April 2018 at 00:21:51 UTC, sarn wrote:
> I think you've got the technical answer already (just don't link in phobos2) but I'll add my 2c that Phobosless programming isn't just possible but realistically doable.  It's a shame to go without Phobos because it's got so much higher-level functionality, but if you're okay with writing things from the ground up in C-like code, it works.  Most third-party D libraries depend on Phobos, though, so it's mostly DIY and using bindings to other languages.

Right, that’s the plan for the program itself. Even without D we would be talking to services as directly as possible, and third party libraries would come in as C code and be tweaked to work in-house. Performance is paramount with this, which is why I was so pleased to find out about the ‘-betterC’ switch in development, even though we may have gotten by with a minimal D runtime. The ecosystem at work here is really something else :)

On Friday, 27 April 2018 at 00:21:51 UTC, sarn wrote:
> Have a look at the standard library here:
> https://dlang.org/phobos/index.html
>
> Basically, all the std.* is Phobos, and most of the core.* stuff needs linking with the D runtime library. However, there's a bunch of bindings to standard C stuff in core.stdc.* that only need the C runtime and standard library, and these come in really handy when you're not using Phobos.  (There are more platform-specific C bindings in core.sys.* that aren't listed on that page.)  Also, there are some other things in core.bitop and core.atomic (probably more) that don't need linking to druntime.

Right. So there isn’t anything in core.* that would be in libphobos, only D runtime? And std.* depends on both, that is correct? Just want to be sure there aren’t edge cases or exceptions, it would be a handy rule of thumb.

On Saturday, 28 April 2018 at 10:36:03 UTC, A. Nicholi wrote:
> Right. So there isn’t anything in core.* that would be in libphobos, only D runtime? And std.* depends on both, that is correct? Just want to be sure there aren’t edge cases or exceptions, it would be a handy rule of thumb.

That's right, and I think it's intended to stay that way.  (I.e., you could file a bug report if you somehow found reverse dependencies from druntime to phobos2.)

Things from std.* are in the https://github.com/dlang/phobos repo, which builds libphobos2, and things from core.* are in https://github.com/dlang/druntime, which builds libdruntime.