Jump to page: 1 2
Thread overview
TLS for iOS - how to proceed
Apr 01, 2014
Dan Olson
Apr 01, 2014
David Nadlinger
Apr 02, 2014
Dan Olson
Apr 02, 2014
Dan Olson
Apr 02, 2014
Joakim
Apr 02, 2014
Dan Olson
Apr 04, 2014
Joakim
Apr 04, 2014
Dan Olson
Apr 05, 2014
Jacob Carlborg
Apr 05, 2014
Dan Olson
Apr 05, 2014
Jacob Carlborg
May 08, 2014
Yuriy
May 08, 2014
Jacob Carlborg
May 10, 2014
Paolo Invernizzi
May 10, 2014
Jacob Carlborg
April 01, 2014
I decided to start a new thread specific to this problem.

I have all these pieces to make TLS work for D on iOS, just sitting on my computer, but they need to go somewhere.  I'd like like to push this up to https://github.com/smolt/ldc.git but I am not sure the best way to proceed.  I am looking for advice.

First, here are the pieces:

1. Mods to LLVM: llvm is modified to emit code to lookup TLS vars address.

2. The tlv code from Apple's dyld open source (threadLocalHelpers.s and threadLocalVariables.c) is dropped into druntime and modified to slightly to work for ARM.  This provides the same API as seen on MacOS x86.

3. D runtime initSections() in sections_ldc.d makes a call to
tlv_initializer() (from threadLocalVariables.c) to set everything up.

Note - LDC compiler code does not change at all for iOS TLS - all handled in LLVM.


Here are the issues:

What is best way to maintain and distribute the LLVM changes?

The tlv runtime support is currently in the form of modified copies of Apple dyld open source.  How do we properly use this (licensing)?  Can it go in druntime or should it go elsewhere?  Should it be reimplemented?

Also, I'd like help making my LLVM changes better.  I didn't do things the proper way (had to cheat with a const_cast).
-- 
Dan
April 01, 2014
Hi Dan,

Great work!

On 1 Apr 2014, at 17:31, Dan Olson wrote:
> What is best way to maintain and distribute the LLVM changes?

For question concerning LLVM development, I found that the best thing usually is to just ask on the llvmdev mailing list or on IRC (#llvm@irc.oftc.net). The devs are usually quick to suggest a solution.

If integration of the changes proves problematic (or simply until a solution is worked out), we could also consider maintaining a patch or a forked repo somewhere on GitHub for people to use when building LDC for iOS. This is certainly not the best solution in the long run, though.

> The tlv runtime support is currently in the form of modified copies of
> Apple dyld open source.  How do we properly use this (licensing)?  Can
> it go in druntime or should it go elsewhere?  Should it be
> reimplemented?

I neither looked at the details of the Apple Open Source license, nor am I a lawyer, but I wouldn't see too much of a problem with including the code in the druntime repository, as long as we don't claim it's Boost licensed.

However, since your work is potentially also useful for other projects targeting iOS, maybe a small "stand-alone" library pulled in as a submodule by the LDC runtime build system would be the better idea?

> Also, I'd like help making my LLVM changes better.  I didn't do things
> the proper way (had to cheat with a const_cast).

Did I miss a link to the changes?

David
April 02, 2014
"David Nadlinger" <code@klickverbot.at> writes:

>> Also, I'd like help making my LLVM changes better.  I didn't do things the proper way (had to cheat with a const_cast).
>
> Did I miss a link to the changes?
>
> David

I did not put up the change anywhere yet.  I just tried to create a gist of the changes, but keep getting a gist website error.  I can't access any of my gists.  Hmm, how about a svn diff until gist recovers?

The new code in ARMTargetLowering::LowerGlobalTLSAddress() for Darwin is
based somewhat on a function of the same name in
llvm/lib/Target/X86/X86ISelLowering.cpp and on pieces from
ARMTargetLowering::LowerGlobalAddressDarwin() and
ARMTargetLowering::LowerToTLSGeneralDynamicModel().

Loading the address of the global GV though didn't work (got internal
error) when using this call I found in LowerGlobalAddressDarwin().

  DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, ARMII::MO_NONLAZY);

apparently because GV is thread local.  So I cheated by temporarily changing GV into a normal global, and then the proper assembly was created.  I really don't know what I am doing, just found a way to get something to work.  My comments begin with // dano

llvm/lib/Target/ARM/ARMISelLowering.cpp

Index: ARMISelLowering.cpp ===================================================================
--- ARMISelLowering.cpp	(revision 199570)
+++ ARMISelLowering.cpp	(working copy)
@@ -2460,9 +2460,12 @@
 SDValue
 ARMTargetLowering::LowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const {
   // TODO: implement the "local dynamic" model
+#if 0
   assert(Subtarget->isTargetELF() &&
          "TLS not implemented for non-ELF targets");
+#endif
   GlobalAddressSDNode *GA = cast<GlobalAddressSDNode>(Op);
+  if (Subtarget->isTargetELF()) {

   TLSModel::Model model = getTargetMachine().getTLSModel(GA->getGlobal());

@@ -2475,6 +2478,61 @@
       return LowerToTLSExecModels(GA, DAG, model);
   }
   llvm_unreachable("bogus TLS model");
+  }
+
+  if (Subtarget->isTargetDarwin()) {
+      EVT PtrVT = getPointerTy();
+      SDLoc dl(Op);
+      const GlobalValue *GV = GA->getGlobal();
+      Reloc::Model RelocM = getTargetMachine().getRelocationModel();
+
+  if (Subtarget->useMovt())
+    ++NumMovwMovt;
+
+  // FIXME: Once remat is capable of dealing with instructions with register
+  // operands, expand this into multiple nodes
+  unsigned Wrapper =
+      RelocM == Reloc::PIC_ ? ARMISD::WrapperPIC : ARMISD::Wrapper;
+
+  // dano: Get an internal compiler error unless I cheat and temp change to
+  // no thread local.  There must be a better way.
+  GlobalVariable *GVar = const_cast<GlobalVariable*>(dyn_cast<GlobalVariable>(GV));
+  GlobalVariable::ThreadLocalMode tlsMode = GVar->getThreadLocalMode();
+  GVar->setThreadLocalMode(GlobalVariable::NotThreadLocal);
+  SDValue G = DAG.getTargetGlobalAddress(GV, dl, PtrVT, 0, ARMII::MO_NONLAZY);
+  SDValue Result = DAG.getNode(Wrapper, dl, PtrVT, G);
+  GVar->setThreadLocalMode(tlsMode);
+
+  if (Subtarget->GVIsIndirectSymbol(GV, RelocM))
+    Result = DAG.getLoad(PtrVT, dl, DAG.getEntryNode(), Result,
+                         MachinePointerInfo::getGOT(), false, false, false, 0);
+
+  // dano: call __tls_get_addr(TLVDescriptor *Result)
+  //   blx ___tls_get_addr
+  // Really want this to be be an indirect call (*Result->thunk)(Result)
+  //   ld r1, [r0]
+  //   blx r1
+  SDValue Chain = DAG.getEntryNode();
+  ArgListTy Args;
+  ArgListEntry Entry;
+  Entry.Node = Result;
+  Entry.Ty = (Type *) Type::getInt32Ty(*DAG.getContext());
+  Args.push_back(Entry);
+
+  TargetLowering::CallLoweringInfo CLI
+      (Chain,
+       (Type *) Type::getInt32Ty(*DAG.getContext()),
+       false, false, false, false,
+       0, CallingConv::C, /*isTailCall=*/false,
+       /*doesNotRet=*/false, /*isReturnValueUsed=*/true,
+       DAG.getExternalSymbol("__tls_get_addr", PtrVT),
+       Args, DAG, dl);
+  std::pair<SDValue, SDValue> CallResult = LowerCallTo(CLI);
+  return CallResult.first;
+  }
+
+  llvm_unreachable("TLS not implemented for this target.");
 }

 SDValue ARMTargetLowering::LowerGlobalAddressELF(SDValue Op,
April 02, 2014
"David Nadlinger" <code@klickverbot.at> writes:

> On 1 Apr 2014, at 17:31, Dan Olson wrote:
>> What is best way to maintain and distribute the LLVM changes?
>
> For question concerning LLVM development, I found that the best thing usually is to just ask on the llvmdev mailing list or on IRC (#llvm@irc.oftc.net). The devs are usually quick to suggest a solution.

Ok, I will check into it when things get stable.

> If integration of the changes proves problematic (or simply until a solution is worked out), we could also consider maintaining a patch or a forked repo somewhere on GitHub for people to use when building LDC for iOS. This is certainly not the best solution in the long run, though.

Hmmm, forking llvm on github seems like the easiest. I had not looked at the github llvm mirror before. That seems like a good way to keep it everything in sync.

>> The tlv runtime support is currently in the form of modified copies of Apple dyld open source.  How do we properly use this (licensing)?  Can it go in druntime or should it go elsewhere?  Should it be reimplemented?
>
> I neither looked at the details of the Apple Open Source license, nor am I a lawyer, but I wouldn't see too much of a problem with including the code in the druntime repository, as long as we don't claim it's Boost licensed.

My quick read of the http://www.opensource.apple.com/apsl/ makes me think compliance would be by documenting any modification from original, keeping the license intact in source, and a prominent notice by executable only versions of the license and how to view the modified source. Not sure how that applies to a library.

> However, since your work is potentially also useful for other projects targeting iOS, maybe a small "stand-alone" library pulled in as a submodule by the LDC runtime build system would be the better idea?

That might be a good idea. Keeps licensing issues, if any, away from druntime or LDC. As a separate lib, it can just be added to the link command as needed. And easily removed if Apple turns tlv on for iPhone SDK.
-- 
Dan
April 02, 2014
Again, nice work.  Do all the unit tests pass now?  If so, you should mention this on D.announce after you get the code up somewhere, as I'm sure others would like to play with it, :) and not everyone follows this newsgroup.

On Wednesday, 2 April 2014 at 07:35:20 UTC, Dan Olson wrote:
> My quick read of the http://www.opensource.apple.com/apsl/ makes me
> think compliance would be by documenting any modification from original,
> keeping the license intact in source, and a prominent notice by
> executable only versions of the license and how to view the modified
> source. Not sure how that applies to a library.

I reread the APSL and yeah, you pretty much got it right.  It applies for any binary, doesn't matter if it's a library or not.  The APSL requires that you provide source for the files or modifications you took from them, but it doesn't care if you compile those files with other source files that are licensed differently, unlike the GPL but similar to the CDDL.  So there's no real problem with combining it with boost libraries like druntime, ie the license is compatible and not viral, but it does pose an additional requirement that anyone using it has to post a notice making the APS-licensed source available.

> That might be a good idea. Keeps licensing issues, if any, away from
> druntime or LDC. As a separate lib, it can just be added to the link
> command as needed. And easily removed if Apple turns tlv on for iPhone
> SDK.

Sounds like a good approach.  If anyone cares enough about the source restrictions of the APSL, maybe they'll reimplement it as boost-licensed source and submit it to druntime someday.
April 02, 2014
"Joakim" <dlang@joakim.airpost.net> writes:

> Again, nice work.  Do all the unit tests pass now?  If so, you should mention this on D.announce after you get the code up somewhere, as I'm sure others would like to play with it, :) and not everyone follows this newsgroup.

No, there are still some failing unittests.  All druntime unittests that should pass with current LDC do pass.  There is one know problem with some of rt.aaA unittest not compiling, so I versioned it out.

The unittests in phobos that fail tend to be mostly be floating point related.  I haven't looked into all the details, but many are comparison options in the unittest that don't match.  For example, std.getopt just fails its test for floating point options because the comparison fails.

Much of it could have to do with crossing compiling on x86 -> ARM.  Kai has an update for cross compiling, so I am kind of holding off investigating float problems until that change is pulled in.   I also noticed that GDC has a math updates for ARM, so some of that might be needed too.

The std.parallism one I think I am sure is just iOS and OSX differences.

Testing 40 std.conv: FAIL std.conv.ConvException@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/conv.d(2663): Range error

Testing 42 std.csv: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/csv.d(499): Assertion failure

Testing 52 std.getopt: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/getopt.d(699): Assertion failure

Testing 56 std.internal.math.errorfunction: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/internal/math/errorfunction.d(218): Assertion failure

Testing 57 std.internal.math.gammafunction: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/internal/math/gammafunction.d(348): Assertion failure

Testing 61 std.mathspecial: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/mathspecial.d(157): Assertion failure

Testing 66 std.numeric: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/numeric.d(1504): Assertion failure

Testing 68 std.parallelism: totalCPUs = 0
FAIL
Memory allocation failed

Testing 69 std.random: FAIL core.exception.AssertError@/Users/dan/projects/ldc/ldc-git/runtime/phobos/std/random.d(2487): 99: 7082 > 5300.
April 04, 2014
On Wednesday, 2 April 2014 at 15:34:42 UTC, Dan Olson wrote:
> "Joakim" <dlang@joakim.airpost.net> writes:
>
>> Again, nice work.  Do all the unit tests pass now?  If so, you should
>> mention this on D.announce after you get the code up somewhere, as I'm
>> sure others would like to play with it, :) and not everyone follows
>> this newsgroup.
>
> No, there are still some failing unittests.  All druntime unittests that should pass with current LDC do pass.  There is one know problem with some of rt.aaA unittest not compiling, so I versioned it out.
>
> The unittests in phobos that fail tend to be mostly be floating point related.  I haven't looked into all the details, but many are comparison options in the unittest that don't match.  For example, std.getopt just fails its test for floating point options because the comparison fails.
>
> Much of it could have to do with crossing compiling on x86 -> ARM.  Kai has an update for cross compiling, so I am kind of holding off investigating float problems until that change is pulled in.   I also noticed that GDC has a math updates for ARM, so some of that might be needed too.

OK, I don't think that should preclude letting people know on D.announce, once you put your source and some build instructions online.  You might get some help with the remaining porting work.

btw, Apple just open-sourced their custom AArch64 backend a week ago:

http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-March/071574.html
https://github.com/llvm-mirror/llvm/tree/master/lib/Target/ARM64

If you have any hardware with their 64-bit A7 chip, it might be interesting to see if you can get D running on 64-bit iOS also. :)
April 04, 2014
"Joakim" <joakim@airpost.net> writes:

> OK, I don't think that should preclude letting people know on D.announce, once you put your source and some build instructions online.  You might get some help with the remaining porting work.

Sounds good, something to work on this weekend to get my github filled with build notes and other goodies.

I am dithering on whether to put in a flag to compile without TLS support with a warning when each thread local is compiled. Then iOS ldc can be build with released llvm-3.4 or 3.5. Only the phobos std.concurrency unittest was fixed by adding TLS support. I have an explicit ThreadLocal type used in Thread/Fiber classes so they don't need TLS to work properly.

Apple recommends that iPhone apps don't make use of threads directly and use Grand Central Dispatch to manage concurrency. It may be that a non-TLS version of D for iOS will work just fine for most uses.

https://developer.apple.com/library/mac/documentation/General/Conceptual/ConcurrencyProgrammingGuide/ConcurrencyandApplicationDesign/ConcurrencyandApplicationDesign.html#//apple_ref/doc/uid/TP40008091-CH100-SW1

> btw, Apple just open-sourced their custom AArch64 backend a week ago:
>
> http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-March/071574.html https://github.com/llvm-mirror/llvm/tree/master/lib/Target/ARM64
>
> If you have any hardware with their 64-bit A7 chip, it might be interesting to see if you can get D running on 64-bit iOS also. :)

I noticed as I was getting ready to upgrade my llvm source tree. I saw the new arm64 target files under the ARM tree that iPhone SDK uses. I don't have any h/w to support it yet but would be fun to try. I did get some older h/w though and was going to see if things compile fine to armv6 too.
-- 
Dan
April 05, 2014
On 2014-04-04 18:57, Dan Olson wrote:

> Apple recommends that iPhone apps don't make use of threads directly and
> use Grand Central Dispatch to manage concurrency. It may be that a
> non-TLS version of D for iOS will work just fine for most uses.

That might be fine for iOS specific code. But libraries not target specifically for iOS my expect TLS to be available, since that's what D supports.

-- 
/Jacob Carlborg
April 05, 2014
Jacob Carlborg <doob@me.com> writes:

> On 2014-04-04 18:57, Dan Olson wrote:
>
>> Apple recommends that iPhone apps don't make use of threads directly and use Grand Central Dispatch to manage concurrency. It may be that a non-TLS version of D for iOS will work just fine for most uses.
>
> That might be fine for iOS specific code. But libraries not target specifically for iOS my expect TLS to be available, since that's what D supports.

Do you think having an option to disable TLS in D is not good then?  I would put our warning for each variable that should have been a thread local.

BTW Jacob, I started using DStep to translate some of the iPhone SDK headers, like CoreFoundation.framework for use in sample iOS apps.  Has anybody already done this and put in in repo somewhere?  I don't want to be redoing what someone else has done.  I am looking forward to when Objective-C support is in D as I am just using objc_msgSend directly now.  I wonder how much work it will be to get into LDC.
-- 
Dan
« First   ‹ Prev
1 2