I got asked whether there are any porting hints for phobos on other architectures the other day from the debian GCC maintainers.  So I gathered this must be at least a dedicated wiki or article to be written up on the subject. :)

I know there are a few working on porting gdc and associated libraries over to ARM (with my assistance from the compiler side).  So please tell, what are your experiences? Successes?  Failures?  What tips would you give to someone wanting to port to their own architecture?

Regards
Iain

On 08-04-2012 21:08, Iain Buclaw wrote:
> I got asked whether there are any porting hints for phobos on other
> architectures the other day from the debian GCC maintainers. So I
> gathered this must be at least a dedicated wiki or article to be written
> up on the subject. :)
>
> I know there are a few working on porting gdc and associated libraries
> over to ARM (with my assistance from the compiler side). So please tell,
> what are your experiences? Successes? Failures? What tips would you give
> to someone wanting to port to their own architecture?
>
> Regards
> Iain

For the love of god, use D_LP64. I cannot count how many times X86 and X86_64 (and similar pairs) have been misused for this.

Don't rely on extern (D) for inline asm in any capacity. It differs across compilers, architectures, bitnesses, and OSs (this is one seriously stupid aspect of the language).

Not so much for when you're porting, but as a help for others who might have to port platform-specific code you're writing: *Always* include an else block for the unsupported case that static asserts.

-- 
- Alex

Am Sun, 08 Apr 2012 21:08:52 +0200
schrieb "Iain Buclaw" <ibuclaw@ubuntu.com>:

> I got asked whether there are any porting hints for phobos on other architectures the other day from the debian GCC maintainers.  So I gathered this must be at least a dedicated wiki or article to be written up on the subject. :)
> 
> I know there are a few working on porting gdc and associated libraries over to ARM (with my assistance from the compiler side).  So please tell, what are your experiences? Successes? Failures?  What tips would you give to someone wanting to port to their own architecture?
> 
> Regards
> Iain

(This is mostly about porting to a different C library. I don't remember many issues when porting to a different CPU architecture)

Issues I hit with druntime:

* Adapting the core.stdc bindings to something different than the
  currently supported C libraries sucks: The version blocks are
  sometimes completely wrong. For example Android's bionic is a C
  library based on BSD code, but running on Linux. As a result
  sometimes the version(FreeBSD) blocks apply for bionic, but sometimes
  the version(linux) blocks are right. I basically had to rewrite
  the complete core.stdc bindings. This is an issue because druntime
  and phobos do not distinguish between OS/Kernel and C library.

* Wrong constants or macros in the C bindings are very hard to spot -
  you'll only notice those at runtime

* When statically linking the phobos/druntime library you are no warned
  about missing symbols - For shared libraries  -Wl,--no-undefined can
  be used, however, there are some issues with that as well:
  (http://stackoverflow.com/questions/2356168/force-gcc-to-notify-about-undefined-references-in-shared-libraries
  second answer)

* Bionic just implements some functions as macros and never exports
  those as functions (htons, etc). Because of the last point it's easy
  to miss that

Ideally all of the core.stdc bindings should be generated automatically. This is possible if we can run code (using offsetof, alignof, etc) but it's not that easy for cross compilation. I thought about hooking into the GCC C frontend to do that, but I had no time to look at it yet.

* All those issues also apply to phobos, where phobos uses custom C
  bindings / extern(C) declarations.

* I had to edit some stuff in std.stdio (because Android has no wide
  character/fwide support). Templates can be annoying in this case:
  some if(isOutputRange!T) chains hid an error in the IO code, it took
  me some time to find that problem. The reported error was completely
  misleading (cannot put dchar[] into LockingTextWriter or something)

* When adding new, system specific code to a module and using selective
  imports, that may affect other modules (can't remember which compiler
  bug this was). This means that adding an import in one module might
  break another module on another architecture.

* Porting the GC doesn't seem to be too difficult, but some care is
  needed to get stack scanning/TLS scanning right (If you have random
  crashes, it's either the GC not working(probably not scanning
  stack/tls) or fno-section-anchors missing)

* Always use "-fno-section-anchors". It's not needed for simple code,
  but I was chasing a weird bug in derelict, till I realized I didn't
  compile derelict with "-fno-section-anchors".

* Right now, issue 284 is a little annoying. At least unittest and
  phobos/druntime as shared libraries won't work at all till that's
  fixed.

* AFAIK the unittests cannot be run when cross-compiling right now?

* There might be more issues like this one where phobos is checking for
  a wrong status code:
  (https://github.com/D-Programming-Language/phobos/pull/487)

* For systems where long double isn't available, fixing core.stdc.math
  is annoying. I have to implement a proper solution which works
  for all systems without long double.

However, all that considered most issues are when interfacing C. The D code most of the time 'just works'.

On 2012-04-09 11:05, Johannes Pfau wrote:

> * Adapting the core.stdc bindings to something different than the
>    currently supported C libraries sucks: The version blocks are
>    sometimes completely wrong. For example Android's bionic is a C
>    library based on BSD code, but running on Linux. As a result
>    sometimes the version(FreeBSD) blocks apply for bionic, but sometimes
>    the version(linux) blocks are right. I basically had to rewrite
>    the complete core.stdc bindings. This is an issue because druntime
>    and phobos do not distinguish between OS/Kernel and C library.

Is it possible to treat bionic as its own platform:


version (bionic) {}

else version (linux{}

and so on.

-- 
/Jacob Carlborg

On 9 April 2012 10:35, Jacob Carlborg <doob@me.com> wrote:
> On 2012-04-09 11:05, Johannes Pfau wrote:
>
>> * Adapting the core.stdc bindings to something different than the
>>   currently supported C libraries sucks: The version blocks are
>>   sometimes completely wrong. For example Android's bionic is a C
>>   library based on BSD code, but running on Linux. As a result
>>   sometimes the version(FreeBSD) blocks apply for bionic, but sometimes
>>   the version(linux) blocks are right. I basically had to rewrite
>>   the complete core.stdc bindings. This is an issue because druntime
>>   and phobos do not distinguish between OS/Kernel and C library.
>
>
> Is it possible to treat bionic as its own platform:
>
>
> version (bionic) {}
>
> else version (linux{}
>
> and so on.
>

Personally I feel that people porting to specific architectures should maintain their differences in separate files under a /ports directory structure - lets say core.stdc.stdio as a cod example. The version for bionic would be under /ports/bionic/core/stdc/stdio.d, and that is the module that gets compiled into the library when building for bionic. When installing, the build process generates a header file of the bionic version of core.stdc.stdio and puts the file in the correct /include/core/stdc/stdio.di location.

Though it is fine to say using version {} else version {} else static assert(false); when dealing with a small set of architectures.  I feel strongly this is not practical when considering there are 23+ architectures and 12+ platforms that could be in mixed combination. The result would either be lots of code duplications everywhere, or just a wiry long block of spaghetti code.  Every port in one file would (eventually) make it difficult for maintainers IMO.

-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

> Personally I feel that people porting to specific architectures should
> maintain their differences in separate files under a /ports directory
> structure - lets say core.stdc.stdio as a cod example. The version for
> bionic would be under /ports/bionic/core/stdc/stdio.d, and that is the
> module that gets compiled into the library when building for bionic.
> When installing, the build process generates a header file of the
> bionic version of core.stdc.stdio and puts the file in the correct
> /include/core/stdc/stdio.di location.
>
> Though it is fine to say using version {} else version {} else static
> assert(false); when dealing with a small set of architectures.  I feel
> strongly this is not practical when considering there are 23+
> architectures and 12+ platforms that could be in mixed combination.
> The result would either be lots of code duplications everywhere, or
> just a wiry long block of spaghetti code.  Every port in one file
> would (eventually) make it difficult for maintainers IMO.

I agree.  Submitted an enhancement here:  https://d.puremagic.com/issues/show_bug.cgi?id=11666

On 3 December 2013 02:43, Mike <none@none.com> wrote:
>> Personally I feel that people porting to specific architectures should maintain their differences in separate files under a /ports directory structure - lets say core.stdc.stdio as a cod example. The version for bionic would be under /ports/bionic/core/stdc/stdio.d, and that is the module that gets compiled into the library when building for bionic. When installing, the build process generates a header file of the bionic version of core.stdc.stdio and puts the file in the correct /include/core/stdc/stdio.di location.
>>
>> Though it is fine to say using version {} else version {} else static assert(false); when dealing with a small set of architectures.  I feel strongly this is not practical when considering there are 23+ architectures and 12+ platforms that could be in mixed combination. The result would either be lots of code duplications everywhere, or just a wiry long block of spaghetti code.  Every port in one file would (eventually) make it difficult for maintainers IMO.
>
>
> I agree.  Submitted an enhancement here: https://d.puremagic.com/issues/show_bug.cgi?id=11666

Thanks.

My name is Iain.

On Monday, 9 April 2012 at 09:05:25 UTC, Johannes Pfau wrote:
> Am Sun, 08 Apr 2012 21:08:52 +0200
> schrieb "Iain Buclaw" <ibuclaw@ubuntu.com>:
>
>> I got asked whether there are any porting hints for phobos on other architectures the other day from the debian GCC maintainers.  So I gathered this must be at least a dedicated wiki or article to be written up on the subject. :)
>> 
>> I know there are a few working on porting gdc and associated libraries over to ARM (with my assistance from the compiler side).  So please tell, what are your experiences? Successes?  Failures?  What tips would you give to someone wanting to port to their own architecture?
>> 
>> Regards
>> Iain
>
> (This is mostly about porting to a different C library. I don't
> remember many issues when porting to a different CPU architecture)
>
> Issues I hit with druntime:
>
> * Adapting the core.stdc bindings to something different than the
>   currently supported C libraries sucks: The version blocks are
>   sometimes completely wrong. For example Android's bionic is a C
>   library based on BSD code, but running on Linux. As a result
>   sometimes the version(FreeBSD) blocks apply for bionic, but sometimes
>   the version(linux) blocks are right. I basically had to rewrite
>   the complete core.stdc bindings. This is an issue because druntime
>   and phobos do not distinguish between OS/Kernel and C library.
>
> * Wrong constants or macros in the C bindings are very hard to spot -
>   you'll only notice those at runtime
>
> * When statically linking the phobos/druntime library you are no warned
>   about missing symbols - For shared libraries  -Wl,--no-undefined can
>   be used, however, there are some issues with that as well:
>   (http://stackoverflow.com/questions/2356168/force-gcc-to-notify-about-undefined-references-in-shared-libraries
>   second answer)
>
> * Bionic just implements some functions as macros and never exports
>   those as functions (htons, etc). Because of the last point it's easy
>   to miss that
>
> Ideally all of the core.stdc bindings should be generated
> automatically. This is possible if we can run code (using offsetof,
> alignof, etc) but it's not that easy for cross compilation. I thought
> about hooking into the GCC C frontend to do that, but I had no time to
> look at it yet.
>
> * All those issues also apply to phobos, where phobos uses custom C
>   bindings / extern(C) declarations.
>
> * I had to edit some stuff in std.stdio (because Android has no wide
>   character/fwide support). Templates can be annoying in this case:
>   some if(isOutputRange!T) chains hid an error in the IO code, it took
>   me some time to find that problem. The reported error was completely
>   misleading (cannot put dchar[] into LockingTextWriter or something)
>
> * When adding new, system specific code to a module and using selective
>   imports, that may affect other modules (can't remember which compiler
>   bug this was). This means that adding an import in one module might
>   break another module on another architecture.
>
> * Porting the GC doesn't seem to be too difficult, but some care is
>   needed to get stack scanning/TLS scanning right (If you have random
>   crashes, it's either the GC not working(probably not scanning
>   stack/tls) or fno-section-anchors missing)
>
> * Always use "-fno-section-anchors". It's not needed for simple code,
>   but I was chasing a weird bug in derelict, till I realized I didn't
>   compile derelict with "-fno-section-anchors".
>
> * Right now, issue 284 is a little annoying. At least unittest and
>   phobos/druntime as shared libraries won't work at all till that's
>   fixed.
>
> * AFAIK the unittests cannot be run when cross-compiling right now?
>
> * There might be more issues like this one where phobos is checking for
>   a wrong status code:
>   (https://github.com/D-Programming-Language/phobos/pull/487)
>
> * For systems where long double isn't available, fixing core.stdc.math
>   is annoying. I have to implement a proper solution which works
>   for all systems without long double.
>
> However, all that considered most issues are when interfacing C. The D
> code most of the time 'just works'.
Seems like you got pretty far with your Android port: are you planning on submitting any of these patches back upstream?

Am Tue, 03 Dec 2013 12:26:57 +0100
schrieb "Joakim" <joakim@airpost.net>:

> Seems like you got pretty far with your Android port: are you planning on submitting any of these patches back upstream?

At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

On 3 December 2013 16:13, Johannes Pfau <nospam@example.com> wrote:
> Am Tue, 03 Dec 2013 12:26:57 +0100
> schrieb "Joakim" <joakim@airpost.net>:
>
>> Seems like you got pretty far with your Android port: are you planning on submitting any of these patches back upstream?
>
> At some point, probably yes. However, I want to get the 'easy' stuff working first. This means a stable and well-tested ARM/Linux/glibc build should be available first. (It makes more sense this way as we still have/had some ARM codegen bugs in the compiler.)

That need testing / approval. ;-)