bitwise <bitwise.pvt@gmail.com> writes:

> On Tue, 30 Jun 2015 16:09:19 -0400, Jacob Carlborg <doob@me.com> wrote:
>
>> On 30/06/15 18:29, Dan Olson wrote:
>>
>>> Not to my knowledge, but please do try.  The D iOS toolchain built by [1] only creates the armv7 libraries, but I have all the pieces staged to support universal libs for running in the x86 iOS Simulator too.  I expect to have that up on github in a week along with prebuilt binaries.
>>>
>>> I am curious to see what App Store says about a submission.  If there are problems, then we can hopefully make fixes.  I will be happy to help
>>
>> The LDC runtime contains at least one private function, to access TLS data.
>>
>
> That reminds me, I have to email tech support again. I tried emailing apple to ask if their private-API guideline applies to their open-source APIs as well. They were shutting down for 2 weeks at the time though.
>
>   Bit

The private APIs being called by osx_tls.c like dyld_enumerate_tlv_storage() are overriden for iOS by using [1].  I can change all dyld prefixes in theadLocalVariable.c to something else and remove all code not needed.  That would leave a call to private API dyld_register_image_state_change_handler() in threadLocalVariables.c, but it looks to me like it could be rewritten to use _dyld_register_func_for_add_image().  Tried it and seems to work.

[1] https://github.com/smolt/iphoneos-apple-support/blob/master/threadLocalVariables.c#L345

On Saturday, 27 June 2015 at 16:13:19 UTC, bitwise wrote:
> Has anyone tried it?
>
>   Bit

are you kidding? it doesn't even work on win 8.1 desktop, can't do dll's etc..
fix that compiler for desktop first.
one of your videos was an eye opener! this compiler isn't usable for any larger project and you people play around.

On 01/07/15 10:06, Dan Olson wrote:

> The private APIs being called by osx_tls.c like
> dyld_enumerate_tlv_storage() are overriden for iOS by using [1].  I can
> change all dyld prefixes in theadLocalVariable.c to something else and
> remove all code not needed.

Right, I forgot that you have implemented those functions yourself. But removing the "dyld" prefix might be a good idea.

> That would leave a call to private API
> dyld_register_image_state_change_handler() in threadLocalVariables.c,
> but it looks to me like it could be rewritten to use
> _dyld_register_func_for_add_image().  Tried it and seems to work.

_dyld_register_func_for_add_image won't work for dynamic libraries. There's no API to unregister the callback. So, if _dyld_register_func_for_add_image is called from a dynamic library then the library is unloaded. The next call to any of the "dl" functions, i.e. "dlopen" will crash.

dyld_register_image_state_change_handler works because it pins the dynamic library, if I recall correctly. It won't unload it.

_dyld_register_func_for_add_image is already used by druntime, perhaps it doesn't matter. I think bitwise is working on adding support for dynamic libraries on OS X. On Linux the compiler is generating a function that behave as the C function with the attribute __attribute__((constructor)). Perhaps that can be used.

Otherwise perhaps look at the implementation of
dyld_register_image_state_change_handler and see if you can manually pin the dynamic library.

-- 
/Jacob Carlborg

On Wednesday, 1 July 2015 at 10:49:03 UTC, jd wrote:
> are you kidding? it doesn't even work on win 8.1 desktop, can't do dll's etc..

I've been making DLL in Windows 8.1 all day.

On Wed, 01 Jul 2015 06:49:01 -0400, jd <jd@matterminds.com> wrote:

> On Saturday, 27 June 2015 at 16:13:19 UTC, bitwise wrote:
>> Has anyone tried it?
>>
>>   Bit
>
> are you kidding? it doesn't even work on win 8.1 desktop, can't do dll's etc..
> fix that compiler for desktop first.

I think you mean usage of DLLs via import library doesn't work right? Pretty sure you can still dynamically load DLLS and use them with GetProcAddress().

> one of your videos was an eye opener! this compiler isn't usable for any larger project and you people play around.

I haven't made any videos.

  Bit

On Wed, 01 Jul 2015 02:40:01 -0400, Dan Olson <gorox@comcast.net> wrote:

> bitwise <bitwise.pvt@gmail.com> writes:
>
>> On Tue, 30 Jun 2015 12:29:17 -0400, Dan Olson <gorox@comcast.net> wrote:
>>
>>> bitwise <bitwise.pvt@gmail.com> writes:
>>>>
>>>> Has anyone tried to submit an app for iOS with D code in it yet? I'm
>>>> going to be putting out a game in the next little while, so I was
>>>> thinking about sticking a D plugin in it to see if it raised any
>>>> eyebrows.
>>>>
>>>>   Bit
>>>
>>> Not to my knowledge, but please do try.  The D iOS toolchain built by
>>> [1] only creates the armv7 libraries, but I have all the pieces staged
>>> to support universal libs for running in the x86 iOS Simulator too.  I
>>> expect to have that up on github in a week along with prebuilt binaries.
>>>
>>> I am curious to see what App Store says about a submission.  If there
>>> are problems, then we can hopefully make fixes.  I will be happy to
>>> help
>>
>> Actually... Apple recently started enforcing the requirement for 64bit
>> support. Unless the code is built for both armv7 and arm64,
>> iTunes/XCode will reject it.
>>
>>   Bit
>
> I guess arm64 just moved up on my TODO list.

:)

  Bit

On Wed, 01 Jul 2015 08:23:18 -0400, Jacob Carlborg <doob@me.com> wrote:

> On 01/07/15 10:06, Dan Olson wrote:
>
>> The private APIs being called by osx_tls.c like
>> dyld_enumerate_tlv_storage() are overriden for iOS by using [1].  I can
>> change all dyld prefixes in theadLocalVariable.c to something else and
>> remove all code not needed.
>
> Right, I forgot that you have implemented those functions yourself. But removing the "dyld" prefix might be a good idea.
>
>> That would leave a call to private API
>> dyld_register_image_state_change_handler() in threadLocalVariables.c,
>> but it looks to me like it could be rewritten to use
>> _dyld_register_func_for_add_image().  Tried it and seems to work.
>
> _dyld_register_func_for_add_image won't work for dynamic libraries. There's no API to unregister the callback. So, if _dyld_register_func_for_add_image is called from a dynamic library then the library is unloaded. The next call to any of the "dl" functions, i.e. "dlopen" will crash.
>
> dyld_register_image_state_change_handler works because it pins the dynamic library, if I recall correctly. It won't unload it.
>
> _dyld_register_func_for_add_image is already used by druntime, perhaps it doesn't matter. I think bitwise is working on adding support for dynamic libraries on OS X. On Linux the compiler is generating a function that behave as the C function with the attribute __attribute__((constructor)). Perhaps that can be used.
>
> Otherwise perhaps look at the implementation of
> dyld_register_image_state_change_handler and see if you can manually pin the dynamic library.
>

I think apple recently added support to iOS for shared libraries as "embedded binaries", but you still can't dynamically load code. So for iOS, the above shouldn't matter.

Section 2.5, 2.7
https://developer.apple.com/app-store/review/guidelines/


> I think bitwise is working on adding support for dynamic libraries on OS X.

Indeed, trying ;)

I've got it partially working. My druntime supports multiple images now and I've got druntime building as a shared library.

Now, I'm working on implementing something like Martin's _dso_registry approach for registering the images. The part I'm having a hard time with is getting my loader code into the object files. Basically, I need to put a small function into each object file as a local comdat symbol. This is a lot more difficult than it first seemed. Martin has done this through manually writing assembly byte codes and whatnot to the object-file. I'm still hoping I will find a simpler way of doing this. I'm reading over the Apple Mach-O docs right now(thoroughly this time). Once I understand the object file format properly I'll give it a try. Martin's busy with the release right now though, so he doesn't have time to help.

  Bit

On 01/07/15 17:03, bitwise wrote:

> Now, I'm working on implementing something like Martin's _dso_registry
> approach for registering the images. The part I'm having a hard time
> with is getting my loader code into the object files. Basically, I need
> to put a small function into each object file as a local comdat symbol.
> This is a lot more difficult than it first seemed. Martin has done this
> through manually writing assembly byte codes and whatnot to the
> object-file. I'm still hoping I will find a simpler way of doing this.

Could you try using C? I know Clang/GCC has a lot of attributes, something like __attribute__((section ("<section name<"))) or similar.

-- 
/Jacob Carlborg

On Wed, 01 Jul 2015 15:04:16 -0400, Jacob Carlborg <doob@me.com> wrote:
> Could you try using C? I know Clang/GCC has a lot of attributes, something like __attribute__((section ("<section name<"))) or similar.

I was thinking about this, but it won't work because we have no control over what linker will be used to link the object files produced by dmd. One idea is to embed the sections-init function as a string in dmd and just compile it with gcc. dmd already uses gcc to do linking on OSX. I don't think it's reasonable though to expect people to know they should compile the extra "entry.o" they find in their object directory when building dylibs.

Another idea is to require a D entry point for D DLLs. So when dmd found a module in which an entry point was defined, it could dump the sections-init function there, the same way that dmd already does for application-main. The user would surely expect to have to link the object file containing the entry point. The problem with this approach, is that again, we have no control over the linker that will be used. This means that there is no way to enforce that an entry point be present in D DLLs. For C/C++, a main entry point is required, and linking an application without it will cause a linker error. I'm not sure if this is built into C/C++ linkers in a standard way, or whether the C runtime library has a reference to the main() symbol, but without fail, a linker error will occur when linking an application without main(). Now, adding a D application main function to a module causes dmd to dump a C application main into the same output file for that module, thus satisfying the requirement for a main entry point. But, the C/C++ runtime/linker does not require any kind of entry point for DLLs on OSX. So there would be no way to enforce that D DLLs have an entry point(that I can think of), and trying to use a dylib which has not had their runtime/sections initialized would result in very strange and unpredictable crashes. I really doubt most coders would be able to figure out that it was because their D DLL main function was missing.

Finally, Martin's solution is to just put the sections-init function in _every_ object file, but as COMDAT. This means that the function definitions will be treated like template definitions, and the linker will just pick the first one it finds at link-time if there are multiple definitions. In order to do this, Martin has basically added code to dmd's backend for linux/freebsd which manually outputs this stuff to every object file.

So the question is, from the point specified above, how do we get the C code in there? One way would be have dmd compile the function and spit out an object file. But then, dmd would have to be able to read that object file, extract it's contents, and reinject it into the D object files. I'm not sure this would be any easier than just outputting the function manually.

I've been thinking about making the function in D and just parsing it like this:
https://github.com/D-Programming-Language/dmd/blob/master/src/mars.c#L233

Once I had the above AST, I could hack it a bit to make it private/local COMDAT, then output it to all the object files. At this point though, I'm not sure the OSX backend for dmd even knows how to put out private/local symbols.

I'm going to revisit this once I've finished familiarizing myself with Mach-O format, but if you have any ideas..... :)

  Bit