On Saturday, 23 March 2013 at 21:24:50 UTC, Jens Mueller wrote:
> Moritz Maxeiner wrote:
>> On Saturday, 23 March 2013 at 16:37:35 UTC, Jens Mueller wrote:
>> >Moritz Maxeiner wrote:
>> >>On Saturday, 23 March 2013 at 10:31:30 UTC, Jens Mueller wrote:
>> >>
>> >>It looks mostly okay to me with one problem: Afaict the code
>> >>enforces the presence of the initialization routines of all
>> >>targets,
>> >>e.g. if one target is missing LLVMInitializeAllTargets will not
>> >>link, as there are undefined references for that missing target,
>> >>but
>> >>LLVM may or may not be compiled with that target so you cannot
>> >>enforce its presence in the bindings. For runtime loading the
>> >>solution I used was to check the function pointer for null; for
>> >>linking you have this problem: When using linking, knowing which
>> >>targets are available happens at link time (when the LLVM
>> >>libraries
>> >>are linked in), which means you cannot use any compile time
>> >>tricks
>> >>for automatic detection of which targets are available.
>> >>The only solution for that problem I can think of would be to
>> >>use
>> >>runtime reflection and check at runtime for each initialiation
>> >>routine if it is a callable function, but afaik D only has
>> >>compile
>> >>time reflection.
>> >
>> >I wonder how they do it in C. Don't you have to set a macro?
>> 
>> Afaict they rely on the fact that when you install llvm on your
>> system you get the {/usr/include/}llvm/Config/Targets.def file, in
>> which is set what targets LLVM was compiled with (it gets generated
>> at LLVM compile-time). Then the Target.h, in which the
>> LLVMInitializeAllTargets function rests, includes that file and does
>> some of that macro-voodoo that makes C/C++ so "lovable" to only
>> create calls for the targets enabled in the Targets.def file when
>> LLVMInitalizeAllTargets gets inlined. Of course, that solution isn't
>> viable because afaik you cannot access filesystem IO functions in D
>> CTFE, meaning the Targets.def file is useless to you. <rant>Hooray
>> for the C/C++ preprocessor, may it die, die, die!</rant>
>
> If I knew the path to the Targets.def file at compile I could load its
> contents and use it.
> http://dlang.org/expression.html#ImportExpression
> Still have to find out how to get the path.
>
> Jens

Oh, I did not know that was possible, thanks for the link!
Can't really help you with the path, though. On Linux it's probably either /usr/include/llvm/Config/Targets.def or /usr/local/include/llvm/Config/Targets.def, but some people might install it on /opt/llvm or other custom paths. You also have a problem when there's only a shared lib available for LLVM and no headers.. No idea about OSX though. And on Windows you'll probably only deal with a DLL and not have the headers available anyway - at least I do (I cross compile the LLVM dll on linux with a mingw64 gcc toolchain, so no headers on windows for me).

Moritz

On Saturday, 23 March 2013 at 21:19:14 UTC, Moritz Maxeiner wrote:
>
> TLDR: Your example should now work, provided you fix what I previously mentioned. You can also look at sample/fibonacci.d which I used instead of your fac to confirm that you gist now works.
>
> - Moritz

Awesome. Indeed, it now fully works (and JIT does work after all! Thanks for showing me how to use that). Thanks for the more interesting example in the README, it's extremely helpful. And also thank you for taking some time to help with the issues I was having.

On Sunday, 24 March 2013 at 01:35:28 UTC, Chris Cain wrote:
> On Saturday, 23 March 2013 at 21:19:14 UTC, Moritz Maxeiner wrote:
>>
>> TLDR: Your example should now work, provided you fix what I previously mentioned. You can also look at sample/fibonacci.d which I used instead of your fac to confirm that you gist now works.
>>
>> - Moritz
>
> Awesome. Indeed, it now fully works (and JIT does work after all! Thanks for showing me how to use that). Thanks for the more interesting example in the README, it's extremely helpful. And also thank you for taking some time to help with the issues I was having.

No problem, writing that fibonacci example forced me to read up Stuff about LLVM I need to know anyway (for making the D API)^^
Just one thing I forgot to mention: When you're using llvm.util.memory.toCString you'll need to take care of the allocated memory yourself, or you'll get memory leaks. The example is a special case as all the c strings need to be kept aroound until program termination, anyway (since LLVM's global context exists until then and all the c strings used by LLVM internally), but that's not the case with all LLVM C functions with c string args.

- Moritz

Moritz Maxeiner wrote:
> On Saturday, 23 March 2013 at 21:24:50 UTC, Jens Mueller wrote:
> >Moritz Maxeiner wrote:
> >>On Saturday, 23 March 2013 at 16:37:35 UTC, Jens Mueller wrote:
> >>>Moritz Maxeiner wrote:
> >>>>On Saturday, 23 March 2013 at 10:31:30 UTC, Jens Mueller wrote:
> >>>>
> >>>>It looks mostly okay to me with one problem: Afaict the code
> >>>>enforces the presence of the initialization routines of all
> >>>>targets,
> >>>>e.g. if one target is missing LLVMInitializeAllTargets will
> >>>>not
> >>>>link, as there are undefined references for that missing
> >>>>target,
> >>>>but
> >>>>LLVM may or may not be compiled with that target so you
> >>>>cannot
> >>>>enforce its presence in the bindings. For runtime loading the
> >>>>solution I used was to check the function pointer for null;
> >>>>for
> >>>>linking you have this problem: When using linking, knowing
> >>>>which
> >>>>targets are available happens at link time (when the LLVM
> >>>>libraries
> >>>>are linked in), which means you cannot use any compile time
> >>>>tricks
> >>>>for automatic detection of which targets are available.
> >>>>The only solution for that problem I can think of would be to
> >>>>use
> >>>>runtime reflection and check at runtime for each
> >>>>initialiation
> >>>>routine if it is a callable function, but afaik D only has
> >>>>compile
> >>>>time reflection.
> >>>
> >>>I wonder how they do it in C. Don't you have to set a macro?
> >>
> >>Afaict they rely on the fact that when you install llvm on your
> >>system you get the {/usr/include/}llvm/Config/Targets.def file,
> >>in
> >>which is set what targets LLVM was compiled with (it gets
> >>generated
> >>at LLVM compile-time). Then the Target.h, in which the
> >>LLVMInitializeAllTargets function rests, includes that file and
> >>does
> >>some of that macro-voodoo that makes C/C++ so "lovable" to only
> >>create calls for the targets enabled in the Targets.def file
> >>when
> >>LLVMInitalizeAllTargets gets inlined. Of course, that solution
> >>isn't
> >>viable because afaik you cannot access filesystem IO functions
> >>in D
> >>CTFE, meaning the Targets.def file is useless to you.
> >><rant>Hooray
> >>for the C/C++ preprocessor, may it die, die, die!</rant>
> >
> >If I knew the path to the Targets.def file at compile I could load
> >its
> >contents and use it.
> >http://dlang.org/expression.html#ImportExpression
> >Still have to find out how to get the path.
> >
> >Jens
> 
> Oh, I did not know that was possible, thanks for the link!
> Can't really help you with the path, though. On Linux it's probably
> either /usr/include/llvm/Config/Targets.def or
> /usr/local/include/llvm/Config/Targets.def, but some people might
> install it on /opt/llvm or other custom paths. You also have a
> problem when there's only a shared lib available for LLVM and no
> headers.. No idea about OSX though. And on Windows you'll probably
> only deal with a DLL and not have the headers available anyway - at
> least I do (I cross compile the LLVM dll on linux with a mingw64 gcc
> toolchain, so no headers on windows for me).

I think passing the path -J should work. But I don't know what to do when there is no Targets.def.

Jens

> I think passing the path -J should work. But I don't know what to do
> when there is no Targets.def.
>
> Jens

Off topic, but can you please ensure not to split the thread like this ?

deadalnix wrote:
> >I think passing the path -J should work. But I don't know what to
> >do
> >when there is no Targets.def.
> >
> >Jens
> 
> Off topic, but can you please ensure not to split the thread like this ?

How do I ensure this?
I'm using the mailman interface.
There are no such problems with digitalsmars.D using the same setup.

Jens

On Monday, 25 March 2013 at 08:54:04 UTC, Jens Mueller wrote:
> deadalnix wrote:
>> >I think passing the path -J should work. But I don't know what to
>> >do
>> >when there is no Targets.def.
>> >
>> >Jens
>> 
>> Off topic, but can you please ensure not to split the thread like
>> this ?
>
> How do I ensure this?
> I'm using the mailman interface.
> There are no such problems with digitalsmars.D using the same setup.
>
> Jens

I don't know. I used to use thunderbird, which worked fine, and now mostly the web forum ( http://forum.dlang.org/ which does work great as well ).

On Sunday, 24 March 2013 at 02:37:52 UTC, Moritz Maxeiner wrote:
> On Sunday, 24 March 2013 at 01:35:28 UTC, Chris Cain wrote:
>> On Saturday, 23 March 2013 at 21:19:14 UTC, Moritz Maxeiner wrote:
>>>
>>> TLDR: Your example should now work, provided you fix what I previously mentioned. You can also look at sample/fibonacci.d which I used instead of your fac to confirm that you gist now works.
>>>
>>> - Moritz
>>
>> Awesome. Indeed, it now fully works (and JIT does work after all! Thanks for showing me how to use that). Thanks for the more interesting example in the README, it's extremely helpful. And also thank you for taking some time to help with the issues I was having.
>
> No problem, writing that fibonacci example forced me to read up Stuff about LLVM I need to know anyway (for making the D API)^^
> Just one thing I forgot to mention: When you're using llvm.util.memory.toCString you'll need to take care of the allocated memory yourself, or you'll get memory leaks. The example is a special case as all the c strings need to be kept aroound until program termination, anyway (since LLVM's global context exists until then and all the c strings used by LLVM internally), but that's not the case with all LLVM C functions with c string args.
>

Question : why did you reorganize the modules ? They don't match LLVM's .h filenames.

Is that intended , if yes, why ?

On Wednesday, 27 March 2013 at 15:29:19 UTC, deadalnix wrote:
> On Sunday, 24 March 2013 at 02:37:52 UTC, Moritz Maxeiner wrote:
>> On Sunday, 24 March 2013 at 01:35:28 UTC, Chris Cain wrote:
>>> On Saturday, 23 March 2013 at 21:19:14 UTC, Moritz Maxeiner wrote:
>>>>
>>>> TLDR: Your example should now work, provided you fix what I previously mentioned. You can also look at sample/fibonacci.d which I used instead of your fac to confirm that you gist now works.
>>>>
>>>> - Moritz
>>>
>>> Awesome. Indeed, it now fully works (and JIT does work after all! Thanks for showing me how to use that). Thanks for the more interesting example in the README, it's extremely helpful. And also thank you for taking some time to help with the issues I was having.
>>
>> No problem, writing that fibonacci example forced me to read up Stuff about LLVM I need to know anyway (for making the D API)^^
>> Just one thing I forgot to mention: When you're using llvm.util.memory.toCString you'll need to take care of the allocated memory yourself, or you'll get memory leaks. The example is a special case as all the c strings need to be kept aroound until program termination, anyway (since LLVM's global context exists until then and all the c strings used by LLVM internally), but that's not the case with all LLVM C functions with c string args.
>>
>
> Question : why did you reorganize the modules ? They don't match LLVM's .h filenames.
>
> Is that intended , if yes, why ?

Yes it is intended to be that way. Regarding the reasons for that decision:

Short answer:

Because in the case of the LLVM C API it makes things a lot easier and simpler to maintain, as well as trivial to add new parts for new LLVM versions.

Long answer:

1) The LLVM C API has headers which have very few lines of actual code (e.g. Linker.h which only contains one enum and one functions), making those into D modules seems wasteful.

2) Over versions of the LLVM C API headers appear and dissappear (e.g. Linker.h exists since 3.2, EnhancedDissassembly.h has been removed in trunk 3.3) and having them all around as D modules makes maintenence a lot more complicated.

3) Having all functions in a single compile time enum makes the runtime loading process a lot easier as you can generate all three steps ( 1. function pointer alias for every C function 2. function pointer variable for every function pointer alias 3. loading of the shared lib symbol into the function pointer for every function pointer) with but a few lines of codes, instead of having to write the names of all C functions three times. And since you can encode more information in that enum (an associative array in this case) adding more function is trivial: Simply add the function to the array in this manner: "NAME" : ["SIGNATURE", "+", "VERSION"] and that's it (and if the function has been removed in said VERSION, change "+" into "-"). Since the MixinMap template (a CTFE version of the map function designed to get an array as its list and create D code for each of the items of said array based on a delegate function f) is used for the three steps described above no further code is needed.

TLDR: Supporting different versions of the LLVM C API takes considerably less effort this way and if there is a disadvantage to this approach big enough to outweigh that I can't see it.

On Wednesday, 27 March 2013 at 21:28:42 UTC, Moritz Maxeiner wrote:
> Yes it is intended to be that way. Regarding the reasons for that decision:
>
> Short answer:
>
> Because in the case of the LLVM C API it makes things a lot easier and simpler to maintain, as well as trivial to add new parts for new LLVM versions.
>
> Long answer:
>
> 1) The LLVM C API has headers which have very few lines of actual code (e.g. Linker.h which only contains one enum and one functions), making those into D modules seems wasteful.
>
> 2) Over versions of the LLVM C API headers appear and dissappear (e.g. Linker.h exists since 3.2, EnhancedDissassembly.h has been removed in trunk 3.3) and having them all around as D modules makes maintenence a lot more complicated.
>
> 3) Having all functions in a single compile time enum makes the runtime loading process a lot easier as you can generate all three steps ( 1. function pointer alias for every C function 2. function pointer variable for every function pointer alias 3. loading of the shared lib symbol into the function pointer for every function pointer) with but a few lines of codes, instead of having to write the names of all C functions three times. And since you can encode more information in that enum (an associative array in this case) adding more function is trivial: Simply add the function to the array in this manner: "NAME" : ["SIGNATURE", "+", "VERSION"] and that's it (and if the function has been removed in said VERSION, change "+" into "-"). Since the MixinMap template (a CTFE version of the map function designed to get an array as its list and create D code for each of the items of said array based on a delegate function f) is used for the three steps described above no further code is needed.
>
> TLDR: Supporting different versions of the LLVM C API takes considerably less effort this way and if there is a disadvantage to this approach big enough to outweigh that I can't see it.

Ok, that sound reasonable.

Second question, why do you do stuff like :

enum Foo {
   FooA,
   FooB,
   FooC,
}

when you would do

enum Foo {
    A,
    B,
    C,
}

?

D enums introduce a scope, when C's don't. So C need to prefix enums entries manually, when in D it isn't required. The C to D translation goes as follow : FooA => Foo.A instead of Foo.FooA .

If the goal isn't to follow LLVM's source as closely as possible, I think this is the way to go.