Hi D gurus.

Did read an interesting post form GitLab [1] how they improved performance by 30x by just going to go_v1.9... because they again went from "fork" to "posix_spawn"...

I've searched the GitHub DLANG org for "posix_spawn" and didn't find a hit... so asking myself and you.... is DLANG still on "fork" and could there be some performance improvement potential?

[1] https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/

On Sunday, 14 October 2018 at 11:36:28 UTC, notna wrote:
> Hi D gurus.
>
> Did read an interesting post form GitLab [1] how they improved performance by 30x by just going to go_v1.9... because they again went from "fork" to "posix_spawn"...
>
> [1] https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/

Hi notna,

Thanks for the interesting article.

> I've searched the GitHub DLANG org for "posix_spawn" and didn't find a hit...

Indeed currently (as of DMD 2.082.1) druntime does not include bindings of posix_spawn [1]. The nice thing about D is that if you need to use a C function that's not yet declared in druntime it's very easy to add the declaration to your app and use it.

I looked at glibc's implementation and translated the declarations. You can find them along with the example from the man page here: https://run.dlang.io/gist/ZombineDev/536eb8b456cc365b7ee05133ec452627?compiler=dmd
(if you want to use posix_spawn in your app/library copy the code from line 133 till the end of the file).

> so asking myself and you.... is DLANG still on "fork" and could there be some performance improvement potential?

One cannot say whether D is "on fork" or not because unlike Go, everyone is free to use whatever system function they need, without explicit cooperation from upstream. Though probably many apps or libraries use the high-level functions declared in https://dlang.org/phobos/std_process instead of using the low-level system functions directly. In particular, spawnProcess [2] is currently implemented via fork/exec. I found an open issue about optimizing std.process: https://issues.dlang.org/show_bug.cgi?id=14770.

I'll look into making a pull request to druntime about posix_spawn, though no promises for phobos, as there the code is a bit more involved and would need more careful analysis.

[1]: http://man7.org/linux/man-pages/man3/posix_spawn.3.html
[2]: https://dlang.org/phobos/std_process#.spawnProcess

On Sunday, 14 October 2018 at 17:33:37 UTC, Petar Kirov [ZombineDev] wrote:
>
> I'll look into making a pull request to druntime about posix_spawn [..]

Here's the full deal:
https://github.com/dlang/druntime/pull/2329/files

with support for:
Linux Glibc
Linux Bionic libc
Linux Musl libc
Linux uClibc
Darwin (macOS and iOS)
FreeBSD
NetBSD
OpenBSD
DragonFlyBSD
Solaris/Illumos (thanks @n8sh!)

On 10/14/18 7:36 AM, notna wrote:
> Hi D gurus.
> 
> Did read an interesting post form GitLab [1] how they improved performance by 30x by just going to go_v1.9... because they again went from "fork" to "posix_spawn"...
> 
> I've searched the GitHub DLANG org for "posix_spawn" and didn't find a hit... so asking myself and you.... is DLANG still on "fork" and could there be some performance improvement potential?
> 
> [1] https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/ 
> 
> 

Related:

https://issues.dlang.org/show_bug.cgi?id=14770

-Steve