January 09, 2014
Hi Dan,

On 8 Jan 2014, at 18:39, Dan Olson wrote:
> By the way, llc is very cool.  As I said, I am totally new to llvm
> internals, so am having fun learning what it has in its magic bag of
> tricks.

Glad to hear you are having fun. ;) By the way, are you aware of the dump() functions that LLVM offers on many of its objects (llvm::Type, llvm::Value, …)? They are very useful when debugging inside gdb, as they let you print out the contents of an object in a human-readable form (usually LLVM IR). For example, to track down what exactly causes the assertion failure you were seeing, you could simply run LDC (linked against a debug build of LLVM) in GDB, go up a few levels in the call stack, and then dump the various parameters to see where exactly the bitcast goes wrong. You can even dump entire LLVM functions/modules (e.g. try "p gIR->module->dump()" when debugging LDC), which is sometimes useful to orient oneself when debugging a complex code generation problem.

> So far I cannot find where clang adds the bitcast thing for sjlj eh
> personality.  I will keep hunting.  Any ideas where I would do this in
> ldc?

As Kai mentioned, this is/should really be fixed in LLVM, but in case you are curious: The landing pad instructions are created in createLandingPadInst() (ir/irlandingpad.cpp), I'd just insert a bitcast (e.g. using the DtoBitCast helper) there. You can use gTargetMachine->getMCAsmInfo()->getExceptionHandlingType() to determine which EH style LLVM will lower the instructions too.

Generally, I'd really recommend working with the latest LLVM version when tackling a new port, though. Especially for some of the non-x86 targets, there is usually quite a bit of progress behind the releases, and if you need to patch LLVM at some point, it will take less effort to upstream your changes.

> When it comes to updating the _d_eh_personality for sjlj, I noticed a
> very small difference in the c++ versions ( _gxx_personality_v0 and
> _gxx_personality_sj0).  It is actually one funtion body in
>
> gcc-4.8-20121028/libstdc++-v3/libsupc++/eh_personality.cc
>
> with a little extra code in an ifdef _GLIBCXX_SJLJ_EXCEPTIONS
> conditional.

Yep, I also noticed that when working on ARM EABI EH support a few days back. Let's hope for the best – although the ARM EABI unwinder changes were only a few lines too, but took quite a bit of effort to reverse-engineer.

Best,
David
January 10, 2014
"Kai Nacke" <kai@redstar.de> writes:

> On Thursday, 9 January 2014 at 08:39:26 UTC, Dan Olson wrote:
>>
>> Yes.  It is this one.
>>
>> Assertion failed: (getOperand(0)->getType() ==
>> cast<PointerType>(getOperand(1)->getType())->getElementType() &&
>> "Ptr must be a pointer to Val type!"), function AssertOK, file
>> /Users/dan/projects/ldc/llvm-3.3.src/lib/IR/Instructions.cpp, line
>> 1084.
>
> Hm. Please, could you try it with LLVM 3.4? I am not sure if my fix made it into LLVM 3.3.
>
> Thanks for working on this!!!
>
> Regards,
> Kai

I am grabbing 3.4 now...
January 10, 2014
"David Nadlinger" <code@klickverbot.at> writes:

> Glad to hear you are having fun. ;) By the way, are you aware of the dump() functions that LLVM offers on many of its objects (llvm::Type, llvm::Value, …)? They are very useful when debugging inside gdb, as they let you print out the contents of an object in a human-readable form (usually LLVM IR). For example, to track down what exactly causes the assertion failure you were seeing, you could simply run LDC (linked against a debug build of LLVM) in GDB, go up a few levels in the call stack, and then dump the various parameters to see where exactly the bitcast goes wrong. You can even dump entire LLVM functions/modules (e.g. try "p gIR->module->dump()" when debugging LDC), which is sometimes useful to orient oneself when debugging a complex code generation problem.

Sounds cool.  I have not tried any of that yet.

>
>> So far I cannot find where clang adds the bitcast thing for sjlj eh personality.  I will keep hunting.  Any ideas where I would do this in ldc?
>
> As Kai mentioned, this is/should really be fixed in LLVM, but in case
> you are curious: The landing pad instructions are created in
> createLandingPadInst() (ir/irlandingpad.cpp), I'd just insert a
> bitcast (e.g. using the DtoBitCast helper) there. You can use
> gTargetMachine->getMCAsmInfo()->getExceptionHandlingType() to
> determine which EH style LLVM will lower the instructions too.
>
> Generally, I'd really recommend working with the latest LLVM version when tackling a new port, though. Especially for some of the non-x86 targets, there is usually quite a bit of progress behind the releases, and if you need to patch LLVM at some point, it will take less effort to upstream your changes.

Getting 3.4 now and will see what happens.

>> When it comes to updating the _d_eh_personality for sjlj, I noticed a very small difference in the c++ versions ( _gxx_personality_v0 and _gxx_personality_sj0).  It is actually one funtion body in
>>
>> gcc-4.8-20121028/libstdc++-v3/libsupc++/eh_personality.cc
>>
>> with a little extra code in an ifdef _GLIBCXX_SJLJ_EXCEPTIONS conditional.
>
> Yep, I also noticed that when working on ARM EABI EH support a few days back. Let's hope for the best – although the ARM EABI unwinder changes were only a few lines too, but took quite a bit of effort to reverse-engineer.

Yeah, low level code code can take sometime to get right.  But it can be a master piece that you proudly look at and say, "it all fit in 256 bytes".  Embedded assembly is my favorite.  Now I need to get familiar with ARM assembly.
January 10, 2014
Dan Olson <zans.is.for.cans@yahoo.com> writes:

> "Kai Nacke" <kai@redstar.de> writes:
>
>> On Thursday, 9 January 2014 at 08:39:26 UTC, Dan Olson wrote:
>>>
>>> Yes.  It is this one.
>>>
>>> Assertion failed: (getOperand(0)->getType() ==
>>> cast<PointerType>(getOperand(1)->getType())->getElementType() &&
>>> "Ptr must be a pointer to Val type!"), function AssertOK, file
>>> /Users/dan/projects/ldc/llvm-3.3.src/lib/IR/Instructions.cpp, line
>>> 1084.
>>
>> Hm. Please, could you try it with LLVM 3.4? I am not sure if my fix made it into LLVM 3.3.
>>
>> Thanks for working on this!!!
>>
>> Regards,
>> Kai
>
> I am grabbing 3.4 now...

and assertion passes!  Much more of runtime .d files are turning into .o's.  Not sure how good they are yet, but getting further.

Now I am on to the next assertion that fails.  I know the state of TLS is not good, but I'll toss this out for comment before I dig in again tonight.

Assertion failed: (Subtarget->isTargetELF() && "TLS not implemented for non-ELF targets"), function LowerGlobalTLSAddress, file /Users/dan/projects/ldc/llvm-3.4/lib/Target/ARM/ARMISelLowering.cpp, line 2475.

...

/bin/sh: line 1: 41428 Illegal instruction: 4  /Users/dan/projects/ldc/buildldc-3.4-hacked-armv7/bin/ldc2 --output-o -c -I/Users/dan/projects/ldc/ldc-0.12.1-src/runtime/druntime/src -I/Users/dan/projects/ldc/ldc-0.12.1-src/runtime/druntime/src/gc /Users/dan/projects/ldc/ldc-0.12.1-src/runtime/druntime/src/rt/lifetime.d -of/Users/dan/projects/ldc/buildldc-3.4-hacked-armv7/runtime/src/rt/lifetime.o -w -d -march=thumb -relocation-model=static -O3 -release

Note that I am still working from the ldc-0.12.1-src tar, not git.  Only change I've made so far is in adding flags

-march=thumb -relocation-model=static

because without them, things blow up fast.
January 10, 2014
On Fri, Jan 10, 2014 at 5:14 PM, Dan Olson <zans.is.for.cans@yahoo.com> wrote:
> Now I am on to the next assertion that fails.  I know the state of TLS is not good, but I'll toss this out for comment before I dig in again tonight.
>
> Assertion failed: (Subtarget->isTargetELF() && "TLS not implemented for non-ELF targets"), function LowerGlobalTLSAddress, file /Users/dan/projects/ldc/llvm-3.4/lib/Target/ARM/ARMISelLowering.cpp, line 2475.

D (also the runtime) relies fairly heavily on thread-local storage support. LLVM provides an easy way of creating TLS variables via the thread_local attribute, but support for that is not available on all target platforms – and it seems that Darwin/ARM is one of them (they work on Linux resp. glibc).

I'm not sure what the best way of going forward with this would be. One option might be to just hack LDC to emit all TLS globals as normal globals. This would obviously break threading support, but might be interesting for getting first Hello-World-style programs to run.

The other option, and what we might end up with in the long run, would be to implement a TLS emulation scheme in either LDC or, better, LLVM itself. pthread_{get,set}specific seem to be supported on iOS, which would make an implementation similar to DMD already does on OS X possible (see src/rt/sections_osx.d in the upstream druntime repository). It might be possible to transfer this approach to LDC without touching LLVM, even though implementing it on the LLVM side would certainly be cleaner. Let us know if you want to pursue this direction so we can discuss possible designs.

Best,
David

January 12, 2014
David Nadlinger <code@klickverbot.at> writes:

> I'm not sure what the best way of going forward with this would be. One option might be to just hack LDC to emit all TLS globals as normal globals. This would obviously break threading support, but might be interesting for getting first Hello-World-style programs to run.

I think this is the best option for now.  I want to see how viable D is on iOS and seeing most of the runtime and phobos running on an iPhone would be cool.  Then begin work on the slow, detailed work of TLS, exceptions, and all the other gottchas that pop up when I have a way of running the unittests.

> The other option, and what we might end up with in the long run, would be to implement a TLS emulation scheme in either LDC or, better, LLVM itself. pthread_{get,set}specific seem to be supported on iOS, which would make an implementation similar to DMD already does on OS X possible (see src/rt/sections_osx.d in the upstream druntime repository). It might be possible to transfer this approach to LDC without touching LLVM, even though implementing it on the LLVM side would certainly be cleaner. Let us know if you want to pursue this direction so we can discuss possible designs.
>
> Best,
> David

Thanks much!

January 15, 2014
I finally stubbed and hacked enough places (mostly thread and exception handling) to build druntime and phobos.  The only LDC hack was to disable use of TLS.

I am building everthing with -march thumb.

I linked the resulting druntime into an xcode project to try some things out on an iphone.

However, this call sequence appears often (D caling D).

	mov	lr, pc
	bx	r7

When the function called returns, exectution is no longer in thumb mode because of the way the link register was loaded.  I think it is generated by an llvm fastcc call, but haven't narrowed it down yet.  I will try to create a simple D function to be sure.

I am thinking it should just be a blx r7 call.

I am referring to:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0042e/IHI0042E_aapcs.pdf

Thoughts of where to look for the culprit?

note: I am have been using thumb because -march arm fails an assertion in llvm.

Dan
January 16, 2014
Dan Olson <zans.is.for.cans@yahoo.com> writes:

> However, this call sequence appears often (D caling D).
>
> 	mov	lr, pc
> 	bx	r7
>
> When the function called returns, exectution is no longer in thumb mode because of the way the link register was loaded.  I think it is generated by an llvm fastcc call, but haven't narrowed it down yet.  I will try to create a simple D function to be sure.
>
> I am thinking it should just be a blx r7 call.

Nevermind, adding -mcpu cortex-a8 gives me the right blx call.

I found the clue: http://llvm.org/bugs/show_bug.cgi?id=16572

I would have though configuring with --target=armv7-apple-darwin would have defaulted the cpu to armv7, but apparently it is defaulting to something else.
January 19, 2014
Well, I am kind of stuck and am going to have to start learning LLVM internals to go further.

I can build entire druntime and phobos for iOS if I target thumb 16-bit with llvm-3.4.  However, every function that must handle exceptions ends up with a bad prologue/epilogue.  I get the same problem with clang++, so it is not an ldc issue.  I am pretty sure it is related to sjlj eh.

For example, a c++ function that has a try/catch (-mtriple=thumbv6-darwin):

	.thumb_func	__Z3barv
__Z3barv:
	.cfi_startproc
	.cfi_personality 155, L___gxx_personality_sj0$non_lazy_ptr
Leh_func_begin0:
	.cfi_lsda 16, Lexception0
@ BB#0:                                 @ %entry
	push	{d8, d9, d10, d11, d12, d13, d14, d15, r8, r10, r11, r4, r5, r6, r7, lr}
	add	r7, sp, #12
        ...
	pop	{d8, d9, d10, d11, d12, d13, d14, d15, r8, r10, r11, r4, r5, r6, r7, pc}

The pop and push are incorrect and using otool to disassemble shows what really ends up in the .o, which is a proper instruction, but will not work.

__Z3barv:
00000000	    b420	push	{r5}
00000002	    af03	add	r7, sp, #12
...
00000050	    bc20	pop	{r5}

I think it should be more like:
	push	{r4, r5, r6, r7, lr}
	pop  	{r4, r5, r6, r7, pc}

Perhaps I need to go back to working on targeting armv7 or thumb2.  Its assembly looks better for exception handling, but I get assertion failures from llvm-3.4 on several files in druntime.  Maybe llvm trunk will be better?

Dan


January 20, 2014
Hi Dan!

On Sunday, 19 January 2014 at 00:48:47 UTC, Dan Olson wrote:
> Well, I am kind of stuck and am going to have to start learning LLVM
> internals to go further.
>
> I can build entire druntime and phobos for iOS if I target thumb 16-bit
> with llvm-3.4.  However, every function that must handle exceptions ends
> up with a bad prologue/epilogue.  I get the same problem with clang++,
> so it is not an ldc issue.  I am pretty sure it is related to sjlj eh.
>
> For example, a c++ function that has a try/catch (-mtriple=thumbv6-darwin):
>
> 	.thumb_func	__Z3barv
> __Z3barv:
> 	.cfi_startproc
> 	.cfi_personality 155, L___gxx_personality_sj0$non_lazy_ptr
> Leh_func_begin0:
> 	.cfi_lsda 16, Lexception0
> @ BB#0:                                 @ %entry
> 	push	{d8, d9, d10, d11, d12, d13, d14, d15, r8, r10, r11, r4, r5, r6, r7, lr}
> 	add	r7, sp, #12
>         ...
> 	pop	{d8, d9, d10, d11, d12, d13, d14, d15, r8, r10, r11, r4, r5, r6, r7, pc}
>
> The pop and push are incorrect and using otool to disassemble shows what
> really ends up in the .o, which is a proper instruction, but will not work.
>
> __Z3barv:
> 00000000	    b420	push	{r5}
> 00000002	    af03	add	r7, sp, #12
> ...
> 00000050	    bc20	pop	{r5}
>
> I think it should be more like:
> 	push	{r4, r5, r6, r7, lr}
> 	pop  	{r4, r5, r6, r7, pc}
>
> Perhaps I need to go back to working on targeting armv7 or thumb2.  Its
> assembly looks better for exception handling, but I get assertion failures from
> llvm-3.4 on several files in druntime.  Maybe llvm trunk will be better?
>
> Dan

I think using trunk is always the best option if you port ldc to a new platform. I try to keep ldc in sync with LLVM trunk therefore you should have no problems with using trunk.

If the bug is also present in trunk then you should enter a bug report at http://llvm.org/bugs. It sounds pretty bad.

Could you please post which triple you are suing and on which druntime files you get assertion errors? An assertion in LLVM may also be triggered by bad IR. I like to have a look at it.

Regards,
Kai