Thread overview
[dmd-internals] DragonFlyBSD Support
Aug 19, 2017
Dave MacFarlane
Aug 20, 2017
Jonathan M Davis
August 19, 2017
Hello,

I'm trying to add support for DragonFlyBSD to dmd, but I'm running into an issue that I'm not sure how to move forward on..

I've compiled a version of ldc's ltsmaster branch (after adding support to druntime and phobos), and using it I'm able to compile dmd (after adding support to druntime and phobos in HEAD, rather than ltsmaster..), but the compiled dmd isn't able to link anything. It gets a bunch of errors of the form "error: TLS relocation against invalid instruction" (the complete log of the errors is at https://gist.github.com/driusan/6a04060c234806d3fbb2625759bd55a4)

From what I can tell, the symbols like _d_arrayappendcTX are supposed to be added from dmd, but for some reason aren't (I might be misreading the errors, though..)

Unrelatedly (or relatedly?) I'm not sure how the DMD backend uses the EX_* enums defined in cdefs.h or where the values come from, so it's not obvious what the right value is for DragonFly. The comment says it's the "Executable file format", but at least 8 of them just use elf/elf64 for their executable file format, so I don't see why they're different or if I should just shift over 1 bit and add a new EX_DRAGONFLY64.

(I tried sending a pull request with my runtime/phobos ldc changes but
was told I should send them upstream instead.. but I don't want to
send my runtime changes upstream until I'm sure that they work, and
I'm not as confident in these changes since the compiled dmd can't
link. I'm not sure if the trouble is in the backend or the runtime
since the problem seems to come from linking the runtime..)
- Dave
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals
August 19, 2017
On Saturday, August 19, 2017 23:21:50 Dave MacFarlane via dmd-internals wrote:
> Hello,
>
> I'm trying to add support for DragonFlyBSD to dmd, but I'm running into an issue that I'm not sure how to move forward on..
>
> I've compiled a version of ldc's ltsmaster branch (after adding support to druntime and phobos), and using it I'm able to compile dmd (after adding support to druntime and phobos in HEAD, rather than ltsmaster..), but the compiled dmd isn't able to link anything. It gets a bunch of errors of the form "error: TLS relocation against invalid instruction" (the complete log of the errors is at https://gist.github.com/driusan/6a04060c234806d3fbb2625759bd55a4)
>
> From what I can tell, the symbols like _d_arrayappendcTX are supposed to be added from dmd, but for some reason aren't (I might be misreading the errors, though..)

Well, grepping druntime's source, it's in src/rt/lifetime.d. From my limited understanding of how dmd and druntime interact, I'd expect druntime to be the one generating the actual object code for it, whereas dmd just references it, but it's going to have to be linked in regardless, and normally druntime is part of the library file for Phobos when it's built, meaning that you'd probably have to have Phobos available, even if it's not being used by dmd at the moment. I really don't know how all of that works with cross-compiling, and dmd's switch to D makes all of this more complicated. We really need some sort of guide for it, but I don't know who'd write it. Fortunately, it's not an issue that's likely to come up all that often, but as you'd know better than I would, it's still going to be pretty painful to get a new platform going without it.

- Jonathan M Davis

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals
December 11
On Saturday, 19 August 2017 at 23:21:50 UTC, Dave MacFarlane wrote:
> Hello,
>
> I'm trying to add support for DragonFlyBSD to dmd, but I'm running into an issue that I'm not sure how to move forward on..
>
> I've compiled a version of ldc's ltsmaster branch (after adding support to druntime and phobos), and using it I'm able to compile dmd (after adding support to druntime and phobos in HEAD, rather than ltsmaster..), but the compiled dmd isn't able to link anything. It gets a bunch of errors of the form "error: TLS relocation against invalid instruction" (the complete log of the errors is at https://gist.github.com/driusan/6a04060c234806d3fbb2625759bd55a4)
>
> From what I can tell, the symbols like _d_arrayappendcTX are supposed to be added from dmd, but for some reason aren't (I might be misreading the errors, though..)
>
> Unrelatedly (or relatedly?) I'm not sure how the DMD backend uses the EX_* enums defined in cdefs.h or where the values come from, so it's not obvious what the right value is for DragonFly. The comment says it's the "Executable file format", but at least 8 of them just use elf/elf64 for their executable file format, so I don't see why they're different or if I should just shift over 1 bit and add a new EX_DRAGONFLY64.
>
> (I tried sending a pull request with my runtime/phobos ldc changes but
> was told I should send them upstream instead.. but I don't want to
> send my runtime changes upstream until I'm sure that they work, and
> I'm not as confident in these changes since the compiled dmd can't
> link. I'm not sure if the trouble is in the backend or the runtime
> since the problem seems to come from linking the runtime..)
> - Dave

Hi Dave,

I finished a (initial) port of dmd/druntime and phobos to DragonFlyBSD.

Stacked port:
- I first ported the dmd-cxx branch (this version is using g++) (stage0).
- Because dmd-cxx is not able to compile the current dmd master, i choose a closely related branch (v2.068.2) as the next staging compiler (stage1). In retrospect i should have choosen v2.072.2 as the intermediate.
- The stage1 (v2.068.2) compiler is then able to compile the current master (rev: 49568bf) and run 99% of the unittests. A little more work is needed to get the last couple of unittests working as well.

You can find give it a try:
mkdir dmd-cxx
cd dmd-cxx
git clone -b dmd-cxx https://github.com/dkgroot/dmd.git
git clone -b dmd-cxx https://github.com/dkgroot/druntime.git
git clone -b dmd-cxx https://github.com/dkgroot/phobos.git
cd dmd
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
cd ../druntime
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
cd ../phobos
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
export HOST_DC=`pwd`/install/dragonflybsd/bin64/dmd
cd ..

mkdir v2.068.2
cd v2.068.2
git clone -b dragonflybsd_v2.068.2 https://github.com/dkgroot/dmd.git
git clone -b dragonflybsd_v2.068.2 https://github.com/dkgroot/druntime.git
git clone -b dragonflybsd_v2.068.2 https://github.com/dkgroot/phobos.git
cd dmd
gmake -f posix.mak HOST_CSS=g++ HOST_DC=${HOST_DC}
gmake -f posix.mak HOST_CSS=g++ HOST_DC=${HOST_DC} install
cd ../druntime
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
cd ../phobos
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
export HOST_DMD=`pwd`/install/dragonflybsd/bin64/dmd
cd ..

mkdir master
cd master
git clone https://github.com/dkgroot/dmd.git
git clone https://github.com/dkgroot/druntime.git
git clone https://github.com/dkgroot/phobos.git
cd dmd
gmake -f posix.mak HOST_CSS=g++ HOST_DMD=${HOST_DMD}
gmake -f posix.mak HOST_CSS=g++ HOST_DMD=${HOST_DMD} install
cd ../druntime
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
cd ../phobos
gmake -f posix.mak HOST_CSS=g++
gmake -f posix.mak HOST_CSS=g++ install
cd ..
cd dmd
gmake -f posix.mak HOST_CSS=g++ HOST_DMD=${HOST_DMD} test
cd ../druntime
gmake -f posix.mak HOST_CSS=g++ unittest
cd ../phobos
gmake -f posix.mak HOST_CSS=g++ unittest
cd ..

Maybe you can give it a try.

Question to the Forum:
- Is using v2.068.2 as a stage1 compiler going to be a problem when getting the Pull Request accepted ?
- Is dmd-cxx going to be supported longtime ?
- Is dmd-cxx going to be updated to being able to compile v2.77.2 directly ?

@Dave: getting these changes into druntime and phobos should also making it a lot easier to port the ldc2 version, which was what you were looking for.

Regards,

Diederik
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


December 16
I tried your changes, Diederik, and they run and pass all tests on my DragonFly 5.0.2 system, but I had a quick look and some of the system constants in your code don't match the C system headers (for things that don't affect the compiler or tests.)

For instance, compare the event types in event.d
(https://github.com/dkgroot/druntime/blob/master/src/core/sys/dragonflybsd/sys/event.d)
vs the system header in event.h
(http://gitweb.dragonflybsd.org/dragonfly.git/blob/b479b18f8d1d047fbe99e46084e9aeff702c62f2:/sys/sys/event.h)

I'm not sure if it's worth taking the time to audit them all before you send a PR or if it would be better to get something in and then fix them up after, but either way it seems to be working.

On Mon, Dec 11, 2017 at 2:04 PM, Diederik de Groot via dmd-internals <dmd-internals@puremagic.com> wrote:
> On Saturday, 19 August 2017 at 23:21:50 UTC, Dave MacFarlane wrote:
>>
>> Hello,
>>
>> I'm trying to add support for DragonFlyBSD to dmd, but I'm running into an issue that I'm not sure how to move forward on..
>>
>> I've compiled a version of ldc's ltsmaster branch (after adding support to druntime and phobos), and using it I'm able to compile dmd (after adding support to druntime and phobos in HEAD, rather than ltsmaster..), but the compiled dmd isn't able to link anything. It gets a bunch of errors of the form "error: TLS relocation against invalid instruction" (the complete log of the errors is at https://gist.github.com/driusan/6a04060c234806d3fbb2625759bd55a4)
>>
>> From what I can tell, the symbols like _d_arrayappendcTX are supposed to be added from dmd, but for some reason aren't (I might be misreading the errors, though..)
>>
>> Unrelatedly (or relatedly?) I'm not sure how the DMD backend uses the EX_* enums defined in cdefs.h or where the values come from, so it's not obvious what the right value is for DragonFly. The comment says it's the "Executable file format", but at least 8 of them just use elf/elf64 for their executable file format, so I don't see why they're different or if I should just shift over 1 bit and add a new EX_DRAGONFLY64.
>>
>> (I tried sending a pull request with my runtime/phobos ldc changes but
>> was told I should send them upstream instead.. but I don't want to
>> send my runtime changes upstream until I'm sure that they work, and
>> I'm not as confident in these changes since the compiled dmd can't
>> link. I'm not sure if the trouble is in the backend or the runtime
>> since the problem seems to come from linking the runtime..)
>> - Dave
>
>
> Hi Dave,
>
> I finished a (initial) port of dmd/druntime and phobos to DragonFlyBSD.
>
> Stacked port:
> - I first ported the dmd-cxx branch (this version is using g++) (stage0).
> - Because dmd-cxx is not able to compile the current dmd master, i choose a
> closely related branch (v2.068.2) as the next staging compiler (stage1). In
> retrospect i should have choosen v2.072.2 as the intermediate.
> - The stage1 (v2.068.2) compiler is then able to compile the current master
> (rev: 49568bf) and run 99% of the unittests. A little more work is needed to
> get the last couple of unittests working as well.
>
> You can find give it a try:
> mkdir dmd-cxx
> cd dmd-cxx
> git clone -b dmd-cxx https://github.com/dkgroot/dmd.git
> git clone -b dmd-cxx https://github.com/dkgroot/druntime.git
> git clone -b dmd-cxx https://github.com/dkgroot/phobos.git
> cd dmd
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> cd ../druntime
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> cd ../phobos
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> export HOST_DC=`pwd`/install/dragonflybsd/bin64/dmd
> cd ..
>
> mkdir v2.068.2
> cd v2.068.2
> git clone -b dragonflybsd_v2.068.2 https://github.com/dkgroot/dmd.git
> git clone -b dragonflybsd_v2.068.2 https://github.com/dkgroot/druntime.git
> git clone -b dragonflybsd_v2.068.2 https://github.com/dkgroot/phobos.git
> cd dmd
> gmake -f posix.mak HOST_CSS=g++ HOST_DC=${HOST_DC}
> gmake -f posix.mak HOST_CSS=g++ HOST_DC=${HOST_DC} install
> cd ../druntime
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> cd ../phobos
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> export HOST_DMD=`pwd`/install/dragonflybsd/bin64/dmd
> cd ..
>
> mkdir master
> cd master
> git clone https://github.com/dkgroot/dmd.git
> git clone https://github.com/dkgroot/druntime.git
> git clone https://github.com/dkgroot/phobos.git
> cd dmd
> gmake -f posix.mak HOST_CSS=g++ HOST_DMD=${HOST_DMD}
> gmake -f posix.mak HOST_CSS=g++ HOST_DMD=${HOST_DMD} install
> cd ../druntime
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> cd ../phobos
> gmake -f posix.mak HOST_CSS=g++
> gmake -f posix.mak HOST_CSS=g++ install
> cd ..
> cd dmd
> gmake -f posix.mak HOST_CSS=g++ HOST_DMD=${HOST_DMD} test
> cd ../druntime
> gmake -f posix.mak HOST_CSS=g++ unittest
> cd ../phobos
> gmake -f posix.mak HOST_CSS=g++ unittest
> cd ..
>
> Maybe you can give it a try.
>
> Question to the Forum:
> - Is using v2.068.2 as a stage1 compiler going to be a problem when getting
> the Pull Request accepted ?
> - Is dmd-cxx going to be supported longtime ?
> - Is dmd-cxx going to be updated to being able to compile v2.77.2 directly ?
>
> @Dave: getting these changes into druntime and phobos should also making it a lot easier to port the ldc2 version, which was what you were looking for.
>
> Regards,
>
> Diederik
>
> _______________________________________________
> dmd-internals mailing list
> dmd-internals@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals



-- 
- Dave
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


December 16

On December 16, 2017 7:37:14 PM GMT+01:00, Dave MacFarlane <driusan@gmail.com> wrote:
>I tried your changes, Diederik, and they run and pass all tests on my DragonFly 5.0.2 system, but I had a quick look and some of the system constants in your code don't match the C system headers (for things that don't affect the compiler or tests.)
>
>For instance, compare the event types in event.d
>(https://github.com/dkgroot/druntime/blob/master/src/core/sys/dragonflybsd/sys/event.d)
>vs the system header in event.h
>(http://gitweb.dragonflybsd.org/dragonfly.git/blob/b479b18f8d1d047fbe99e46084e9aeff702c62f2:/sys/sys/event.h)
>
>I'm not sure if it's worth taking the time to audit them all before you send a PR or if it would be better to get something in and then fix them up after, but either way it seems to be working.
>

Hi Dave,

Thanks for testing and checking. Fixing up event.d should not be a problem. I did most of the conversion in the hope that the unittests would catch any mistakes. If you do find any other, do let me know, or send a PR against my repo. I tried to do the conversion carefully, but  because of the amount of files it is quite a bit of a task. Especially having to go through everything twice (bootstrap and master).

I don't mind waiting a little before sending in the PR while we finish this up. Maybe we should add some unittests to catch these types of porting issues (if possible). How did you find this one ?

Regards,

Diederik

_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals


December 16
On Saturday, 16 December 2017 at 18:37:14 UTC, Dave MacFarlane wrote:
> I tried your changes, Diederik, and they run and pass all tests on my DragonFly 5.0.2 system, but I had a quick look and some of the system constants in your code don't match the C system headers (for things that don't affect the compiler or tests.)
>
> For instance, compare the event types in event.d
> (https://github.com/dkgroot/druntime/blob/master/src/core/sys/dragonflybsd/sys/event.d)
> vs the system header in event.h
> (http://gitweb.dragonflybsd.org/dragonfly.git/blob/b479b18f8d1d047fbe99e46084e9aeff702c62f2:/sys/sys/event.h)
>
> I'm not sure if it's worth taking the time to audit them all before you send a PR or if it would be better to get something in and then fix them up after, but either way it seems to be working.
>
> On Mon, Dec 11, 2017 at 2:04 PM, Diederik de Groot via dmd-internals <dmd-internals@puremagic.com> wrote:


Hi Dave,

Thanks for testing and checking. Fixed event.d just now (thanks). I did most of the conversion in the hope that the unittests would catch any mistakes. If you do find any other, do let me know, or send a PR against my repo. I tried to do the conversion carefully, but  because of the amount of files it is quite a bit of a task. Especially having to go through everything twice (bootstrap and master).

I don't mind waiting a little before sending in the PR while we finish this up. Maybe we should add some unittests to catch these types of porting issues (if possible). How did you find this one ?

Regards,

Diederik
_______________________________________________
dmd-internals mailing list
dmd-internals@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-internals