I'm examining the possibility to move from Python+C/C++ to D or Python+D. I read (https://wiki.dlang.org/Programming_in_D_for_Python_Programmers) and
(https://jackstouffer.com/blog/nd_slice.html), where is mentioned PyD, Mir-algorithm, all seems very promising. But I did not test it yet.

The main motivation why I like D over Julia (https://julialang.org/) is that D has smaller footprint (Julia basically needs kind of managed environment (like java virtual machine), it does not really compile to standalone native executable or library) and compilation of D seems to be paradoxically faster than initialization of Julia (despite Julia is JIT, in practice many operations (like recompiling, installing, importing libraries) are quite slow) - waiting 1 minute for that considerably hamper fast development cycle.

Nevertheless I have several concerns now:

1) rdmd - it seems like exactly what I want (to use D as a script language, with super fast compilation). However, I cannot figure out how to make it work with libraries (e.g. If I use mir-algorithm, derelict-gl3, derelict-sdl). It does not seem to work together with dub and ignores dependencies from dub.json.

2) D footprint - the size of binary produced by D is kind of large. This simple project https://github.com/ProkopHapala/SimpleSimulationEngine/tree/master/Dlang/TEMP/firstTriangle-derelict produce 2.9MB with dmd and 3.5MB with ldc. This is because libraries are lineked statically, I guess. There should be a way how to link them dynamically (https://forum.dlang.org/thread/eimyvbojdmanbbshwvmc@forum.dlang.org), at least with ldc, but I cannot figure out how. (sorry for cross posting, but I got no answer to the previous post, I guess because it is >1year old). Still I would like dynamic linking with dmd, since it has faster compilation time than ldc.

I'm concerned about binary size because I want to generate many different small dynamic libraries (.so) on-the-fly. With 3MB per library I would easily waste Gigabytes of disk space that way. Basically I want to achieve functionality a bit like python weave (https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/weave.html).

3) Dynamic loading of D-libraries with ctypes - from what I see in PyD examples (https://github.com/ariovistus/pyd/tree/master/examples) it all uses distutils/setup.py. That is perhaps nice for installation by end user, but not so convenient for rapid prototyping and development. There is this special construction:

extern(C) void PydMain() {
    def!(hello)();
    module_init();
}

I'm not sure what def! exactly does and how limiting it is? I would prefer just to generate standard .so library which exposes standard extern(C) functions. And than to load it dynamically with ctypes like any other c-library. Is that possible? (I did not yet tried to investicate it in detail). The reason is that while I often use python for prototyping, I'd like to produce .so libraries which I can than load from anywhere else (not just from python, but e.g. from C/C++) without changes. And it seems there is some problem loading D-library functions from C (https://dlang.org/spec/betterc.html).

On Tue, Oct 22, 2019 at 8:00 AM Prokop Hapala via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>
> I'm examining the possibility to move from Python+C/C++ to D or
> Python+D. I read
> (https://wiki.dlang.org/Programming_in_D_for_Python_Programmers)
> and
> (https://jackstouffer.com/blog/nd_slice.html), where is mentioned
> PyD, Mir-algorithm, all seems very promising. But I did not test
> it yet.
 >...

You should try to use https://github.com/BindBC/bindbc-opengl and
https://github.com/BindBC/bindbc-sdl. There seems to be an issue with
derelict packages (mainly with the gl3 one)
And as far as I know derelict should be replaced by bindbc anyway in future.

And if you plan to have *.so libs you should add "targetType" : "dynamicLibrary", to you dub.json

On Tuesday, 22 October 2019 at 07:23:46 UTC, Daniel Kozak wrote:
> On Tue, Oct 22, 2019 at 8:00 AM Prokop Hapala via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>>
>> I'm examining the possibility to move from Python+C/C++ to D or
>> Python+D. I read
>> (https://wiki.dlang.org/Programming_in_D_for_Python_Programmers)
>> and
>> (https://jackstouffer.com/blog/nd_slice.html), where is mentioned
>> PyD, Mir-algorithm, all seems very promising. But I did not test
>> it yet.
>  >...
>
> You should try to use https://github.com/BindBC/bindbc-opengl and
> https://github.com/BindBC/bindbc-sdl. There seems to be an issue with
> derelict packages (mainly with the gl3 one)
> And as far as I know derelict should be replaced by bindbc anyway in future.
>
> And if you plan to have *.so libs you should add "targetType" : "dynamicLibrary", to you dub.json

OK, thanks. That is useful to know. But just to not turn the topic elsewhere I should make clear that:

1) I'm not speaking about OpenGL and SDL specifically (that was just small example which I tried first)

2) I'm more concerned about how to D compiler links dependencies when it compiles simple .d program (with lot of dependencies).

I think if I can make it link everything dynamically, It would considerably reduce both size of binary target (whether it is executable or .so) and compilation speed (since it would not re-compile dependencies).

What I want is to recompile and run quite large programs/projects composed composed of many little sub-programs/sub-libraries from Python+D in fast cycles (<< 1 second), because that would make debugging workflow much more pleasant and efficient (in comparison to Python+C/C++ or Julia).

On Tuesday, 22 October 2019 at 07:40:01 UTC, Prokop Hapala wrote:
> On Tuesday, 22 October 2019 at 07:23:46 UTC, Daniel Kozak wrote:
>> On Tue, Oct 22, 2019 at 8:00 AM Prokop Hapala via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>>>
>>> I'm examining the possibility to move from Python+C/C++ to D or
>>> Python+D. I read
>>> (https://wiki.dlang.org/Programming_in_D_for_Python_Programmers)
>>> and
>>> (https://jackstouffer.com/blog/nd_slice.html), where is mentioned
>>> PyD, Mir-algorithm, all seems very promising. But I did not test
>>> it yet.
>>  >...
>>
>> You should try to use https://github.com/BindBC/bindbc-opengl and
>> https://github.com/BindBC/bindbc-sdl. There seems to be an issue with
>> derelict packages (mainly with the gl3 one)
>> And as far as I know derelict should be replaced by bindbc anyway in future.
>>
>> And if you plan to have *.so libs you should add "targetType" : "dynamicLibrary", to you dub.json
>
> OK, thanks. That is useful to know. But just to not turn the topic elsewhere I should make clear that:
>
> 1) I'm not speaking about OpenGL and SDL specifically (that was just small example which I tried first)
>
> 2) I'm more concerned about how to D compiler links dependencies when it compiles simple .d program (with lot of dependencies).
>
> I think if I can make it link everything dynamically, It would considerably reduce both size of binary target (whether it is executable or .so) and compilation speed (since it would not re-compile dependencies).
>
> What I want is to recompile and run quite large programs/projects composed composed of many little sub-programs/sub-libraries from Python+D in fast cycles (<< 1 second), because that would make debugging workflow much more pleasant and efficient (in comparison to Python+C/C++ or Julia).

If you are building individual files, use ldc2 with --link-defaultlib-shared flag:

arun@home-pc:/tmp$ cat a.d
void main() { import std; writeln("Hai"); }
arun@home-pc:/tmp$ ldc2 a.d
arun@home-pc:/tmp$ ldd a
        linux-vdso.so.1 (0x00007fff6395b000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1ec91ea000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1ec91c7000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1ec9078000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1ec905e000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1ec8e6d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1ec92c4000)
arun@home-pc:/tmp$ ldc2 a.d --link-defaultlib-shared
arun@home-pc:/tmp$ ldd a
        linux-vdso.so.1 (0x00007ffcbda7f000)
        libphobos2-ldc-shared.so.88 => /home/arun/.bin/ldc2-1.18.0-linux-x86_64/bin/../lib/libphobos2-ldc-shared.so.88 (0x00007f1b57be8000)
        libdruntime-ldc-shared.so.88 => /home/arun/.bin/ldc2-1.18.0-linux-x86_64/bin/../lib/libdruntime-ldc-shared.so.88 (0x00007f1b57abc000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b57a84000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b57893000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b57744000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b57721000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b57714000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b5770e000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1b58067000)
arun@home-pc:/tmp$

On Tuesday, 22 October 2019 at 07:40:01 UTC, Prokop Hapala wrote:

>
> 1) I'm not speaking about OpenGL and SDL specifically (that was just small example which I tried first)

FYI, the BindBC bindings can be configured as dynamic (in which all the of C library functions are declared as function pointers and the C shared library must be loaded manually at runtime) or static (in which the C library functions become normal function declarations with no implementation and there is a link-time dependency on the C static or shared library). The exception is OpenGL, for which there is no static binding.

The dynamic bindings add the overhead of the function pointer declarations when linked. It's insignificant for most of them. The biggest bindbc-opengl, depending on how many extensions you configure (it's extremely high for DerelictGL3 most of the time, as it tends to pull in more extensions).

>
> 2) I'm more concerned about how to D compiler links dependencies when it compiles simple .d program (with lot of dependencies).

When using dub, it depends on the package configuration of each dependency. All of the Derelict and BindBC packages, for example, are configured to build as static libraries. It didn't (and doesn't) make sense to me to compile a binding as a shared library. Dub packages can also be configured as shared libraries, or simply as "library" (I haven't investigated what that means). Some might provide custom subconfigurations to choose static or shared.

The biggest overhead with DMD output tends to come from the standard library. On Windows, the static library the only option. IIRC on Linux it's necessary to pass -defaultlib=libphobos2.so to DMD to link the shared lib. I don't know how LDC handles it.

On Tuesday, 22 October 2019 at 07:51:16 UTC, Arun Chandrasekaran wrote:
> On Tuesday, 22 October 2019 at 07:40:01 UTC, Prokop Hapala wrote:
>> [...]
>
> If you are building individual files, use ldc2 with --link-defaultlib-shared flag:
>
> arun@home-pc:/tmp$ cat a.d
> void main() { import std; writeln("Hai"); }
> arun@home-pc:/tmp$ ldc2 a.d
> arun@home-pc:/tmp$ ldd a
>         linux-vdso.so.1 (0x00007fff6395b000)
>         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1ec91ea000)
>         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1ec91c7000)
>         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1ec9078000)
>         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1ec905e000)
>         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1ec8e6d000)
>         /lib64/ld-linux-x86-64.so.2 (0x00007f1ec92c4000)
> arun@home-pc:/tmp$ ldc2 a.d --link-defaultlib-shared
> arun@home-pc:/tmp$ ldd a
>         linux-vdso.so.1 (0x00007ffcbda7f000)
>         libphobos2-ldc-shared.so.88 => /home/arun/.bin/ldc2-1.18.0-linux-x86_64/bin/../lib/libphobos2-ldc-shared.so.88 (0x00007f1b57be8000)
>         libdruntime-ldc-shared.so.88 => /home/arun/.bin/ldc2-1.18.0-linux-x86_64/bin/../lib/libdruntime-ldc-shared.so.88 (0x00007f1b57abc000)
>         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b57a84000)
>         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b57893000)
>         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b57744000)
>         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b57721000)
>         librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b57714000)
>         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b5770e000)
>         /lib64/ld-linux-x86-64.so.2 (0x00007f1b58067000)
> arun@home-pc:/tmp$

If you want similar behavior with dub, here is a sample dub.json

arun@home-pc:/tmp/test$ cat dub.json
{
    "authors": [
        "Arun"
    ],
    "copyright": "Copyright © 2019, Arun",
    "description": "A minimal D application.",
    "license": "proprietary",
    "dflags-ldc": ["-link-defaultlib-shared"],
    "name": "test"
}
arun@home-pc:/tmp/test$

Dub settings can be at times intimidating, but still give it a read https://dub.pm/package-format-json#build-settings

On Tuesday, 22 October 2019 at 08:04:32 UTC, Arun Chandrasekaran wrote:
> On Tuesday, 22 October 2019 at 07:51:16 UTC, Arun Chandrasekaran wrote:
>> On Tuesday, 22 October 2019 at 07:40:01 UTC, Prokop Hapala wrote:
>>> [...]
>>
>> If you are building individual files, use ldc2 with --link-defaultlib-shared flag:
>>
>> arun@home-pc:/tmp$ cat a.d
>> void main() { import std; writeln("Hai"); }
>> arun@home-pc:/tmp$ ldc2 a.d
>> arun@home-pc:/tmp$ ldd a
>>         linux-vdso.so.1 (0x00007fff6395b000)
>>         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1ec91ea000)
>>         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1ec91c7000)
>>         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1ec9078000)
>>         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1ec905e000)
>>         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1ec8e6d000)
>>         /lib64/ld-linux-x86-64.so.2 (0x00007f1ec92c4000)
>> arun@home-pc:/tmp$ ldc2 a.d --link-defaultlib-shared
>> arun@home-pc:/tmp$ ldd a
>>         linux-vdso.so.1 (0x00007ffcbda7f000)
>>         libphobos2-ldc-shared.so.88 => /home/arun/.bin/ldc2-1.18.0-linux-x86_64/bin/../lib/libphobos2-ldc-shared.so.88 (0x00007f1b57be8000)
>>         libdruntime-ldc-shared.so.88 => /home/arun/.bin/ldc2-1.18.0-linux-x86_64/bin/../lib/libdruntime-ldc-shared.so.88 (0x00007f1b57abc000)
>>         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1b57a84000)
>>         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1b57893000)
>>         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1b57744000)
>>         libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1b57721000)
>>         librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1b57714000)
>>         libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1b5770e000)
>>         /lib64/ld-linux-x86-64.so.2 (0x00007f1b58067000)
>> arun@home-pc:/tmp$
>
> If you want similar behavior with dub, here is a sample dub.json
>
> arun@home-pc:/tmp/test$ cat dub.json
> {
>     "authors": [
>         "Arun"
>     ],
>     "copyright": "Copyright © 2019, Arun",
>     "description": "A minimal D application.",
>     "license": "proprietary",
>     "dflags-ldc": ["-link-defaultlib-shared"],
>     "name": "test"
> }
> arun@home-pc:/tmp/test$
>
> Dub settings can be at times intimidating, but still give it a read https://dub.pm/package-format-json#build-settings

Thank you Arun, that is very helpful!

Do you have also some example which use some other than default library ?

I see there is quite comprehensive description how to use dmd with dynamic libraries
https://dlang.org/articles/dll-linux.html

But there is nothing how to integrate it with dub and deb.json (I really like DUB, it's one of the best improvements over C/C++ enviroment I noticed)

Also where is RDMD in the equation? I really like the idea run binary programs like:

#!/usr/bin/env rdmd
import std.stdio;
void main(){
    writeln("Hello, world!");
}

But I cannot find any documentation how to use rdmd with any libraries/dependencies and dub.json ? Not even statically linked, not to say dynamic.

On Tue, Oct 22, 2019 at 10:25 AM Prokop Hapala via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>
> ...
>
> Also where is RDMD in the equation? I really like the idea run binary programs like:
>
> #!/usr/bin/env rdmd
> import std.stdio;
> void main(){
>      writeln("Hello, world!");
> }
>
> But I cannot find any documentation how to use rdmd with any libraries/dependencies and dub.json ? Not even statically linked, not to say dynamic.
>

You can't use rdmd with dub, but for simple scripts you can use dub directly:

https://dub.pm/advanced_usage.html

On Tuesday, 22 October 2019 at 08:30:36 UTC, Daniel Kozak wrote:
> On Tue, Oct 22, 2019 at 10:25 AM Prokop Hapala via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>>
>> ...
>>
>> Also where is RDMD in the equation? I really like the idea run binary programs like:
>>
>> #!/usr/bin/env rdmd
>> import std.stdio;
>> void main(){
>>      writeln("Hello, world!");
>> }
>>
>> But I cannot find any documentation how to use rdmd with any libraries/dependencies and dub.json ? Not even statically linked, not to say dynamic.
>>
>
> You can't use rdmd with dub, but for simple scripts you can use dub directly:
>
> https://dub.pm/advanced_usage.html

Here an example, save it as example.d:

```d
#!/usr/bin/env dub

/+ dub.json:
{
    "name":"example-s3",
    "dependencies":{
        "aws-sdk:core": "1.0.0",
        "aws-sdk:s3": "1.0.0"
    }
}
+/

import aws.sdk.core, aws.sdk.s3;

void main()
{
    auto client = new AwsClient();
    auto s3 = new S3Service(client);

    ...
}
```

On linux you could start this file using command ```./example.d```.
Or with command ```dub example.d```.

Kind regards
André

On Tuesday, 22 October 2019 at 05:58:50 UTC, Prokop Hapala wrote:
> I'm examining the possibility to move from Python+C/C++ to D or Python+D. I read (https://wiki.dlang.org/Programming_in_D_for_Python_Programmers) and
> (https://jackstouffer.com/blog/nd_slice.html), where is mentioned PyD, Mir-algorithm, all seems very promising. But I did not test it yet.
>
> [...]

See autowrap.  PyNih might eventually be a bit nicer than pyd but it's not yet there.

We did some very early work on Julia integration and probably will finish when time.

You can call R libraries from D too.

If you do use pyd then ppyd might make it a bit more pleasant.

Some rough edges around pyd but it's okay if you don't mind figuring things out.