View mode: basic / threaded / horizontal-split · Log in · Help
July 31, 2012
Allocating Executable Memory
New to the D language here. I'm working on a tracing JIT compiler 
and will need to allocate chunks of memory that are marked as 
executable.

Is there a standard way of doing this in D, or do I need to 
directly call into mprotect? If I'm going to be calling mprotect, 
what's the cleanest way to do that, do I need to declare my own 
prototype for the function and its flags, or should I write some 
C code that does the call?

Thanks for your help.
July 31, 2012
Re: Allocating Executable Memory
On 31-07-2012 19:23, Maxime Chevalier wrote:
> New to the D language here. I'm working on a tracing JIT compiler and
> will need to allocate chunks of memory that are marked as executable.
>
> Is there a standard way of doing this in D, or do I need to directly
> call into mprotect? If I'm going to be calling mprotect, what's the
> cleanest way to do that, do I need to declare my own prototype for the
> function and its flags, or should I write some C code that does the call?
>
> Thanks for your help.

There's no standard way to do it. You'll have to either use mprotect or 
just pass the relevant flags when you call mmap.

The functions and constants related to this are all in core.sys.posix.mman.

See also:

* https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71 
(and further down)
* https://github.com/lycus/mci/blob/master/src/mci/vm/code.d

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
July 31, 2012
Re: Allocating Executable Memory
> The functions and constants related to this are all in 
> core.sys.posix.mman.
>
> See also:
>
> * 
> https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71 
> (and further down)
> * https://github.com/lycus/mci/blob/master/src/mci/vm/code.d

Thanks. Forgive my ignorance though, but does 
core.sys.posix.sys.mman ship standard with D compilers?
July 31, 2012
Re: Allocating Executable Memory
On Tue, Jul 31, 2012 at 9:36 PM, Maxime Chevalier <
maximechevalierb@gmail.com> wrote:

> The functions and constants related to this are all in core.sys.posix.mman.
>>
>> See also:
>>
>> * https://github.com/lycus/mci/**blob/master/src/mci/core/**memory.d#L71<https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71>(and further down)
>> * https://github.com/lycus/mci/**blob/master/src/mci/vm/code.d<https://github.com/lycus/mci/blob/master/src/mci/vm/code.d>
>>
>
> Thanks. Forgive my ignorance though, but does core.sys.posix.sys.mman ship
> standard with D compilers?
>

It's part of druntime. Druntime is D's runtime library, without which D
code above version 2.0 can't work.

-- 
Bye,
Gor Gyolchanyan.
July 31, 2012
Re: Allocating Executable Memory
On 31-07-2012 19:36, Maxime Chevalier wrote:
>> The functions and constants related to this are all in
>> core.sys.posix.mman.
>>
>> See also:
>>
>> * https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71
>> (and further down)
>> * https://github.com/lycus/mci/blob/master/src/mci/vm/code.d
>
> Thanks. Forgive my ignorance though, but does core.sys.posix.sys.mman
> ship standard with D compilers?

Yep. Anything core.* and std.* is part of druntime and phobos which both 
ship with any D 2.0 compiler.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
July 31, 2012
Re: Allocating Executable Memory
> Yep. Anything core.* and std.* is part of druntime and phobos 
> which both ship with any D 2.0 compiler.

Thanks very much for the quick responses.

My tracing JIT will likely need to have the compiled code write 
into data structures of the host VM. This will mean the compiled 
code holds pointers to host data.

How well does the D GC deal with immovable objects. Would it be a 
problem if I allocated many chunks of immovable data?

Would it be preferable to minimize immovable data by having 
compiled code refer to an (immovable) table with pointers to 
(movable) host data?

Also, if you don't mind me asking, which D compiler do you guys 
prefer? Which one is most widely supported, most up to date?
July 31, 2012
Re: Allocating Executable Memory
On 31-07-2012 20:06, Maxime Chevalier wrote:
>> Yep. Anything core.* and std.* is part of druntime and phobos which
>> both ship with any D 2.0 compiler.
>
> Thanks very much for the quick responses.
>
> My tracing JIT will likely need to have the compiled code write into
> data structures of the host VM. This will mean the compiled code holds
> pointers to host data.
>
> How well does the D GC deal with immovable objects. Would it be a
> problem if I allocated many chunks of immovable data?

Can you clarify what you mean by immovable data? I don't think I follow.

In any case, if the compiled code's executable memory regions aren't 
added as root ranges to D's GC (which is entirely reasonable to not do), 
then you can keep the data alive by simply keeping pointers to the D 
GC-managed data in the host environment. D doesn't use a moving collector.

>
> Would it be preferable to minimize immovable data by having compiled
> code refer to an (immovable) table with pointers to (movable) host data?
>
> Also, if you don't mind me asking, which D compiler do you guys prefer?
> Which one is most widely supported, most up to date?

DMD: Latest updates, fixes, and enhancements. Very fast compilation. Not 
that great optimization.
GDC: Best optimization and fantastic debug info generation. Somewhat 
slow at compilation.
LDC: Fast compilation and good optimization. Sits somewhere between DMD 
and GDC.

Note that DMD can only target x86 while LDC and GDC (due to using LLVM 
and GCC respectively) can target many other 32-bit and 64-bit architectures.

Also, LDC and GDC tend to lag one release behind DMD sometimes, since 
they rely on the DMD front end code.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
July 31, 2012
Re: Allocating Executable Memory
On Tuesday, 31 July 2012 at 17:23:08 UTC, Maxime Chevalier wrote:
> New to the D language here. I'm working on a tracing JIT 
> compiler and will need to allocate chunks of memory that are 
> marked as executable.
>
> Is there a standard way of doing this in D, or do I need to 
> directly call into mprotect? If I'm going to be calling 
> mprotect, what's the cleanest way to do that, do I need to 
> declare my own prototype for the function and its flags, or 
> should I write some C code that does the call?

 The x86 chip it's a simple flag that the OS can set. For use 
with like UPX, the whole section is marked read, write & execute 
I believe (since it has to expand it first before it can execute 
the code; That is of course for loaded memory, not 
allocating...). I would say check their sources, may prove 
interesting.
July 31, 2012
Re: Allocating Executable Memory
On 07/31/2012 08:06 PM, Maxime Chevalier wrote:
>
> How well does the D GC deal with immovable objects. Would it be a
> problem if I allocated many chunks of immovable data?
>

The current GC in druntime never moves data.
July 31, 2012
Re: Allocating Executable Memory
Maxime Chevalier:

> How well does the D GC deal with immovable objects. Would it be 
> a problem if I allocated many chunks of immovable data?

The GC supports flags to define memory as movable or immovable,
but I think at the moment it performs no memory movements.


> Also, if you don't mind me asking, which D compiler do you guys 
> prefer? Which one is most widely supported, most up to date?

DMD has a back-end that produces less efficient binaries compared
to GDC and LDC. But DMD is more up to date (but not a lot), and
it's the most supported, it's the reference implementation. DMD
works well on Windows too, GDC works on Windows too. I think LDC
doesn't work well on Windows.

Bye,
bearophile
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home