On 11/17/21 7:14 AM, tchaloupka wrote:
- have you tried to write a shared D lib used from some other language from multiple threads? I know that you must register/unregister threads in GC, but it just hasn't work for me reliably in any way and you would have to track the lifetime of the thread in the calling language - not pleasant experience at all, no tutorials of how to do that properly that actually works - it's some years old experience now so maybe something has changed
- as GC is
stop the world kind, only way to make it to not intervene with you and still use it in other places is make a thread (with a @nogc function) that is not registered in the GC and make some own mechanism to exchange data between GC and @nogc threads (as std.concurrency won't help you here)
- GC won't magically stop the leaks. Nowadays one want's to have a long running service that just works. But try that with a 'flagship' vibe-d framework and you probably get this experience
I have to say, the second I see "How to X with DLL" in this forum, I mark as read and continue. D has a horrible story on DLLs, and it's been like that for as long as I can remember. If there is some infrastructure project that needs attention, it's DLLs. You are right that
importC are useless if using D from C is nigh impossible or so difficult even the experts can't tell you the answer.
HOWEVER -- there is no excuse for the runtime hanging when a possible error is detected from a system call. Your linked discussion was not resolved how it should have been. Either the runtime should deal with that result properly, or it should crash the application completely. Not properly handling system call errors deep in the runtime is not acceptable.
If no bug has been filed on this, please do.
* I don't like much when GC.stats reports something like: 239MB free from 251MB used memory that is a lot of unused space in a microservice world (and we had a cases when OS just OOM killed the service as it just grows indefinitely regardles there is a majority of free space in GC - as GC.stats says)
I believe the GC can be tuned to reduce this, as long as you haven't at one point needed that much memory at once.
However, it is worth noting that GC (in any language) generally does require more memory than manual allocation or reference counting. D does not have the ability to use a moving GC, because of the type system, which makes compacting GC impossible unfortunately.
* one of the cases that caused a lot of unnecessary small allocations was something like this
row["count"].as!long when reading the columns from a database. Seems like a totally normal way right? But there is much more to it. As it (
dpq2 library) uses
libpq internally that addresses columns by their index, it uses C method with
char* column name to get it and so is using
toStringz that allocates, for every single use of that column for every single row being read. You can imagine where it goes handling some complex queries on thousands of rows. And that is not something that a programmer like 'original me' wants to care about, he just wants to use the available libraries and get the work done, that is what GC should help with right?
dpq2 library should fix that (probably malloc/free the temporary C string). Have you filed a bug on that?
* there are leaks caused by druntime bugs itself (for example https://issues.dlang.org/show_bug.cgi?id=20680)
After those and some other experiences with GC I just became a bit GC phobic (I'm ok with GC for CLI tools, scripts, short running programs, no problem with that there) and try to avoid it as much as I can. But when you want to
get shit done you can't write all on your own, but use the libraries that get you there with no much hassle between.
If I can separate the use cases here, using D as a main application, on a normal modern server/desktop, I have found using the GC to be a non-issue.
There are problems clearly with:
- using D with GC as a plugin
- using GC in a memory/resource constrained environment
Does that sound fair?
Overall my 2 cents on D state:
- druntime relies too much on the GC
* no Fiber usable in @betterC or @nogc
* no Thread usable in @betterC or @nogc
I think it's more of an either-or. Using D on constrained environments likely needs a separate runtime that is catered to those environments.
I might be wrong, and a reasonable base that can deal with both is possible, but it hasn't materialized in the 14+ years that I've been using D.
await - it's hard to find a modern language without it, D is one of them and there doesn't seem to be any interest in it by the leadership (it would deserve a workgroup to work on it)
This I have to disagree on, I'm not used to async/await, but I absolutely love fiber-based i/o where I don't have to deal with those (i.e. vibe-core or mecca).
There are also library solutions to it, which might be good enough, I don't know.
A good demonstration of how async/await can help D other than "because I'm used to it" would be great to see.
* but I'm afraid even if it would potentially be added to the language it would still use the GC as GC is great..
This doesn't make sense, as I believe async/await in languages enables the compiler to rewrite your functions into a state machine. How it stores the state might not need GC usage, because the compiler is in control of the state creation and the state usage.
I wouldn't expect it to need more GC abilities than
- pure tooling compared to others - I'm using VSCode in linux (sorry I'm lazy to learn vim in a way I'd be effective with it), it somewhat works, but code completion breaks too often for me (I'm used to it over the years, but I can imagine it doesn't look good to newcomers)
If you refer to UFCS and/or templates, I'm not sure how solvable a problem this is.
- dub and code.dlang.org doesn't seems to be too official, and being cared about
It is official, it does not receive enough care.
- it's hard to tell anyone that GC in D is fine when you look at techempower benchmark and searching the vibe-d (or anything in D) somewhere up where it should be and isn't (event other GC languages are much higher there)
More solid web frameworks are needed for sure. D can shine in this area, it just needs some champions for it.
- I think there are 2 sorts of group in D community - one more low level that won't like to use GC much, and GC 'likers', for whom GC is just 'good enough'
There are fundamental tradeoffs that are ingrained in these choices. If you want @safe code, you need GC, some reference counting scheme, or rust-style borrow checking.
There are people who don't care about any of these, and don't care about @safe code. If you want @safe though, you have to pick one, and the GC is a perfectly good option for that.
Having the other options is a nice possibility, but I would be very much against removing the GC, or refactoring everything so I have to think about non-GC options.
* I'm afraid that there can't be consensus of what D should look as those groups has different priorities and points of view
D does pretty well letting you choose a lot of things. Even the GC has some choice in it, but for sure the ecosystem for it isn't there. Most other languages don't let you make these choices.
- most libraries on code.dlang.org are high level, and mostly when you want to use
betterC or avoid GC, you are on your own. That is a problem when you just want to use some component and be done (if there is no C alternative or it would mean to write a more idiomatic wrapper for it).
Such is life. If you want an ecosystem built around a certain paradigm or low level design choice, you have to either create it or get others to join you in that paradigm.
D does give you the opportunity to do that, it certainly does take a lot of critical mass to do that.