Thread overview
stub out your gc without hacking on druntime
Jun 10, 2013
Adam D. Ruppe
Jun 11, 2013
H. S. Teoh
Jun 11, 2013
Adam D. Ruppe
Jun 30, 2016
Adam D. Ruppe
Jun 11, 2013
Simen Kjaeraas
June 10, 2013
I decided to boil going without gc down to one simple move: put this code into a file called nogc.d and then add it to your dmd command line:

===

import core.stdc.stdio;
import core.stdc.stdlib;

extern(C):

void* gc_malloc() {
        fprintf(stderr, "GC allocations are disabled\nProgram terminated\n");
        asm { hlt; }
        assert(0);
}

// druntime calls these, but we can just stub them out
void gc_init() { }
void gc_addRange() { }
void gc_term() { }

// druntime makes some classes too. we'll malloc them. This technically
// leaks but since it is startup code I'm pretty sure it doesn't matter.

// this also makes new class available to user code, but remember to free your classes and call the destructor:
/*
void free(Object object) {
        auto dtor = cast(void function(Object o)) object.classinfo.destructor;
        if(dtor)
                dtor(object);
        free(cast(void*) object);
}
*/
extern(C) Object _d_newclass(const ClassInfo ci) {
        void* memory = malloc(ci.init.length);
        if(memory is null) {
                fprintf(stderr, "Out of memory to allocate class\n");
                asm { hlt; }
                assert(0);
        }
        (cast(byte*) memory)[0 .. ci.init.length] = ci.init[];
        return cast(Object) memory;
}

===



You shouldn't have to modify your code nor druntime/phobos (though you'll probably find them being killed by hidden allocations!), unlike the minimal D stuff I've been talking about the last few weeks which replaces them entirely.

The reason it works is the gc functions come from a library file. .lib functions are overridden by functions with the same name in an object file.

So this redefines crucial gc functions, and then the linker uses them instead of the ones druntime provides.Thereby stubbing out the garbage collector in this individual exe. I tried it on both Windows and Linux and it seemed to work as I expected.

The resulting executable is slightly smaller too, since the linker can dump more functions that are never called by the stubs:

$ dmd test2.d
You have mail in /var/spool/mail/me
$ ls -lh test2
-rwxr-xr-x 1 me users 683K 2013-06-10 15:06 test2
$ dmd test2.d nogc.d
$ ls -lh test2
-rwxr-xr-x 1 me users 626K 2013-06-10 15:06 test2

(test2.d is just a random program that does writeln("ctor") and writeln("dtor") on a few classes to see when/if they are still running, nothing special there)


On IRC someone suggested an even simpler solution to me too: set a breakpoint at gc_malloc in your debugger. Then you can see where it is called and continue/stop at any time.



I found a hidden allocation in druntime using this instantly and already filed to bugzilla. If you are on an AMD processor you'll probably see it too if you try to run a program
http://d.puremagic.com/issues/show_bug.cgi?id=10323

so you won't get far with nogc.d! But if you fix that up and try again I was able to get my test to run.
June 11, 2013
On Mon, Jun 10, 2013 at 09:16:04PM +0200, Adam D. Ruppe wrote:
> I decided to boil going without gc down to one simple move: put this code into a file called nogc.d and then add it to your dmd command line:
> 
> ===
> 
> import core.stdc.stdio;
> import core.stdc.stdlib;
> 
> extern(C):
> 
> void* gc_malloc() {
>         fprintf(stderr, "GC allocations are disabled\nProgram
> terminated\n");
>         asm { hlt; }
>         assert(0);
> }
> 
> // druntime calls these, but we can just stub them out
> void gc_init() { }
> void gc_addRange() { }
> void gc_term() { }

Should these return errors too? Or is this just a quick-n-dirty way to get a no-GC environment without needing hack druntime?


[...]
> extern(C) Object _d_newclass(const ClassInfo ci) {
>         void* memory = malloc(ci.init.length);
>         if(memory is null) {
>                 fprintf(stderr, "Out of memory to allocate
> class\n");
>                 asm { hlt; }
>                 assert(0);
>         }
>         (cast(byte*) memory)[0 .. ci.init.length] = ci.init[];
>         return cast(Object) memory;
> }

Hmm. What if the user code instantiates classes? Will they leak memory then? Though I suppose the point is really just to find *hidden* allocations, and 'new' is a pretty blatant use of the GC.


[...]
> You shouldn't have to modify your code nor druntime/phobos (though you'll probably find them being killed by hidden allocations!), unlike the minimal D stuff I've been talking about the last few weeks which replaces them entirely.
> 
> The reason it works is the gc functions come from a library file. .lib functions are overridden by functions with the same name in an object file.
> 
> So this redefines crucial gc functions, and then the linker uses them instead of the ones druntime provides.Thereby stubbing out the garbage collector in this individual exe. I tried it on both Windows and Linux and it seemed to work as I expected.
[...]

Very nice! Clever way of quickly testing for GC-dependence in a piece of code.


T

-- 
Questions are the beginning of intelligence, but the fear of God is the beginning of wisdom.
June 11, 2013
On Tue, 11 Jun 2013 05:06:53 +0200, H. S. Teoh <hsteoh@quickfur.ath.cx> wrote:

>> void gc_init() { }
>> void gc_addRange() { }
>> void gc_term() { }
>
> Should these return errors too? Or is this just a quick-n-dirty way to
> get a no-GC environment without needing hack druntime?

They should not return errors. Basically, these turn gc initialization
(which druntime does at startup) and gc termination (which druntime does
at close time) into no-ops. Stubbing out gc_addRange makes sure the GC
does not try to track anything.


> Hmm. What if the user code instantiates classes? Will they leak memory
> then? Though I suppose the point is really just to find *hidden*
> allocations, and 'new' is a pretty blatant use of the GC.

They will leak, yes:

On Mon, 10 Jun 2013 21:16:04 +0200, Adam D. Ruppe <destructionator@gmail.com> wrote:

> // this also makes new class available to user code, but remember to free your classes and call the destructor:

-- 
Simen
June 11, 2013
On Tuesday, 11 June 2013 at 03:08:36 UTC, H. S. Teoh wrote:
> Or is this just a quick-n-dirty way to
> get a no-GC environment without needing hack druntime?

Yea, that's the idea here. Like Simen said, if these were errors, your program wouldn't even start, so noop is better. Though gc_init should call thread_init, which I didn't do here. Without that call, stack traces on exceptions will segfault on linux.

So extern (C) void thread_init(); extern(C) void gc_init() { thread_init(); } will be better than doing nothing.

> Hmm. What if the user code instantiates classes? Will they leak memory then?

Unless you free() it! But like with the stub functions, druntime uses new in the startup code, so if this was removed entirely, you wouldn't get far. Exceptions are also useful and you gotta new them.

> Though I suppose the point is really just to find *hidden*
> allocations, and 'new' is a pretty blatant use of the GC.

Exactly, it isn't that hard to match new with free since it is clear that you used it and can follow the object's lifetime manually. A hidden allocation though, even though you might be able to free it, can slip past you.
June 30, 2016
On Tuesday, 11 June 2013 at 12:13:05 UTC, Adam D. Ruppe wrote:
> On Tuesday, 11 June 2013 at 03:08:36 UTC, H. S. Teoh wrote:
>> [...]
>
> Yea, that's the idea here. Like Simen said, if these were errors, your program wouldn't even start, so noop is better. Though gc_init should call thread_init, which I didn't do here. Without that call, stack traces on exceptions will segfault on linux.
>
> [...]



Can one use this to hook in to any phobo's lib functions? What about simply "sniffing" the functions? Can we call the original functions somehow?
June 30, 2016
On Thursday, 30 June 2016 at 04:07:03 UTC, Hiemlick Hiemlicker wrote:
> Can one use this to hook in to any phobo's lib functions?

Yeah, you could. Easier though is to just copy the phobos lib file, modify it, then compile it into your program explicitly:

cp dmd2/src/phobos/std/file.d .
# edit your local ./file.d to play with it
dmd yourprogram.d your_other_modules.d file.d


notice that I added the copied Phobos file to the compile command line. Then the compiler will prefer the one from your file to the one in Phobos itself. Nothing else needs to change, you still import std.file;

> Can we call the original functions somehow?

No, with this technique the linker will remove the original functions from the program.