New to the D language here. I'm working on a tracing JIT compiler and will need to allocate chunks of memory that are marked as executable.

Is there a standard way of doing this in D, or do I need to directly call into mprotect? If I'm going to be calling mprotect, what's the cleanest way to do that, do I need to declare my own prototype for the function and its flags, or should I write some C code that does the call?

Thanks for your help.

On 31-07-2012 19:23, Maxime Chevalier wrote:
> New to the D language here. I'm working on a tracing JIT compiler and
> will need to allocate chunks of memory that are marked as executable.
>
> Is there a standard way of doing this in D, or do I need to directly
> call into mprotect? If I'm going to be calling mprotect, what's the
> cleanest way to do that, do I need to declare my own prototype for the
> function and its flags, or should I write some C code that does the call?
>
> Thanks for your help.

There's no standard way to do it. You'll have to either use mprotect or just pass the relevant flags when you call mmap.

The functions and constants related to this are all in core.sys.posix.mman.

See also:

* https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71 (and further down)
* https://github.com/lycus/mci/blob/master/src/mci/vm/code.d

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

> The functions and constants related to this are all in core.sys.posix.mman.
>
> See also:
>
> * https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71 (and further down)
> * https://github.com/lycus/mci/blob/master/src/mci/vm/code.d

Thanks. Forgive my ignorance though, but does core.sys.posix.sys.mman ship standard with D compilers?

On Tue, Jul 31, 2012 at 9:36 PM, Maxime Chevalier < maximechevalierb@gmail.com> wrote:

> The functions and constants related to this are all in core.sys.posix.mman.
>>
>> See also:
>>
>> * https://github.com/lycus/mci/**blob/master/src/mci/core/**memory.d#L71<https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71>(and further down)
>> * https://github.com/lycus/mci/**blob/master/src/mci/vm/code.d<https://github.com/lycus/mci/blob/master/src/mci/vm/code.d>
>>
>
> Thanks. Forgive my ignorance though, but does core.sys.posix.sys.mman ship standard with D compilers?
>

It's part of druntime. Druntime is D's runtime library, without which D code above version 2.0 can't work.

-- 
Bye,
Gor Gyolchanyan.

On 31-07-2012 19:36, Maxime Chevalier wrote:
>> The functions and constants related to this are all in
>> core.sys.posix.mman.
>>
>> See also:
>>
>> * https://github.com/lycus/mci/blob/master/src/mci/core/memory.d#L71
>> (and further down)
>> * https://github.com/lycus/mci/blob/master/src/mci/vm/code.d
>
> Thanks. Forgive my ignorance though, but does core.sys.posix.sys.mman
> ship standard with D compilers?

Yep. Anything core.* and std.* is part of druntime and phobos which both ship with any D 2.0 compiler.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

> Yep. Anything core.* and std.* is part of druntime and phobos which both ship with any D 2.0 compiler.

Thanks very much for the quick responses.

My tracing JIT will likely need to have the compiled code write into data structures of the host VM. This will mean the compiled code holds pointers to host data.

How well does the D GC deal with immovable objects. Would it be a problem if I allocated many chunks of immovable data?

Would it be preferable to minimize immovable data by having compiled code refer to an (immovable) table with pointers to (movable) host data?

Also, if you don't mind me asking, which D compiler do you guys prefer? Which one is most widely supported, most up to date?

On 31-07-2012 20:06, Maxime Chevalier wrote:
>> Yep. Anything core.* and std.* is part of druntime and phobos which
>> both ship with any D 2.0 compiler.
>
> Thanks very much for the quick responses.
>
> My tracing JIT will likely need to have the compiled code write into
> data structures of the host VM. This will mean the compiled code holds
> pointers to host data.
>
> How well does the D GC deal with immovable objects. Would it be a
> problem if I allocated many chunks of immovable data?

Can you clarify what you mean by immovable data? I don't think I follow.

In any case, if the compiled code's executable memory regions aren't added as root ranges to D's GC (which is entirely reasonable to not do), then you can keep the data alive by simply keeping pointers to the D GC-managed data in the host environment. D doesn't use a moving collector.

>
> Would it be preferable to minimize immovable data by having compiled
> code refer to an (immovable) table with pointers to (movable) host data?
>
> Also, if you don't mind me asking, which D compiler do you guys prefer?
> Which one is most widely supported, most up to date?

DMD: Latest updates, fixes, and enhancements. Very fast compilation. Not that great optimization.
GDC: Best optimization and fantastic debug info generation. Somewhat slow at compilation.
LDC: Fast compilation and good optimization. Sits somewhere between DMD and GDC.

Note that DMD can only target x86 while LDC and GDC (due to using LLVM and GCC respectively) can target many other 32-bit and 64-bit architectures.

Also, LDC and GDC tend to lag one release behind DMD sometimes, since they rely on the DMD front end code.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

On Tuesday, 31 July 2012 at 17:23:08 UTC, Maxime Chevalier wrote:
> New to the D language here. I'm working on a tracing JIT compiler and will need to allocate chunks of memory that are marked as executable.
>
> Is there a standard way of doing this in D, or do I need to directly call into mprotect? If I'm going to be calling mprotect, what's the cleanest way to do that, do I need to declare my own prototype for the function and its flags, or should I write some C code that does the call?

 The x86 chip it's a simple flag that the OS can set. For use with like UPX, the whole section is marked read, write & execute I believe (since it has to expand it first before it can execute the code; That is of course for loaded memory, not allocating...). I would say check their sources, may prove interesting.

On 07/31/2012 08:06 PM, Maxime Chevalier wrote:
>
> How well does the D GC deal with immovable objects. Would it be a
> problem if I allocated many chunks of immovable data?
>

The current GC in druntime never moves data.

Maxime Chevalier:

> How well does the D GC deal with immovable objects. Would it be a problem if I allocated many chunks of immovable data?

The GC supports flags to define memory as movable or immovable,
but I think at the moment it performs no memory movements.


> Also, if you don't mind me asking, which D compiler do you guys prefer? Which one is most widely supported, most up to date?

DMD has a back-end that produces less efficient binaries compared
to GDC and LDC. But DMD is more up to date (but not a lot), and
it's the most supported, it's the reference implementation. DMD
works well on Windows too, GDC works on Windows too. I think LDC
doesn't work well on Windows.

Bye,
bearophile