Hi,

I will be developing some hardware and software that will need to be portable across a multitude of systems.

At first, I will develop some of the non-specific infrastructural code base on windows for convenience. These involve the GUI, overall structure, etc.

I think will need to move in to more mobile aspects such as the android and Raspberry Pi which will allow me to start to integrate and work with the future hardware based parts of this system.

At this point I am in the setup stages that will organize and formulate a proper plan of action. I am aware that D works for Windows/Linux/Mac and has some Raspberry Pi and android/arm support with very little FPGA/ASIC/DSP/uC support.

I hope, by the time that I get in to the hardware layers that D will have advanced further down the line on them.

My understanding is that DMD is only applicable for the 3 main platforms, and this is fine for now as that is where I'll start. LDC is starting to progress on ARM/android and I have seen someone write a to use D on FPGA through a transformation process. Some work has been done with some lower level uC's such as the PIC and ARM processors.

Now that GDC is in the works, I assume that will open up even more doors. Because this is a long term project(10+ years) and involves many different systems, subsystems, architectures, and so forth, organization is key for success.  The problem I have with the choice of D over C/C++, at the moment, is in it's knowledge base. Most of the advancements for the different platforms are done by individuals with their own unique goals and motivations in mind. There is very little organized structure to the overall development. Finding information for specific aspects and requirements involves hunting down the appropriate information. Usually it is forum posts and link fishing, usually outdated. Structured documentation is key for efficient and successful progress, which I find D to be lacking in. While library functionality is documented and adequate, the architectural issues and development is not.

Is there any hope of seeing a unified structured interface to the different platforms that D is starting to be developed on rather than having to hunt and peck for the answers?

If not, may I suggest, at a minimum, some type of hierarchy based solution that involves all the major design applications of D with the ability to cross-reference, date, comment, and modify? A sort of wiki but with a bit more hierarchical structure.

D
  Cross-Platform
     Compilers
        ...
        GDC
        ...
     Library
        ...
     ...
  Windows
     Compiler
        ...
        Compiling
            ...
        ...
     Library
        ...
        Compiling
            ...
        Documentation
            ...
            Wi32 API
                ...
                Problems
                ...
                Functions
                ...
                Misc
                ...
                Uncategorized
                ...
            ...
        ...
  ...
  DSP
     ...
     Progress
       ...
     ...


The idea is that one can delve in to the hierarchy and find whatever info they want or add to it if it is not there so that someone else can find it. At some point, most of what D can do is found in the hierarchy, which, as we all know, only requires around O(log2(n)) search time(rather than O(n^m) for googling topics and link fishing).  At some point, this hierarchy becomes very efficient at representing the information we all are searching for rather than a flat random access type of list. The point is to design such an interface and let users like me fill it out and as time goes on,  finding relevant information becomes quicker and quicker. For proper design, meta-information must be used to it's fullest. (Tags, date stamping, ratings, relevancy, cross-references, etc)

Or, perhaps, we already have something this powerful in the works?

On Friday, 23 June 2017 at 19:03:01 UTC, FoxyBrown wrote:
> Hi,
>
> I will be developing some hardware and software that will need to be portable across a multitude of systems.
>
> At first, I will develop some of the non-specific infrastructural code base on windows for convenience. These involve the GUI, overall structure, etc.
>
> I think will need to move in to more mobile aspects such as the android and Raspberry Pi which will allow me to start to integrate and work with the future hardware based parts of this system.
>
> At this point I am in the setup stages that will organize and formulate a proper plan of action. I am aware that D works for Windows/Linux/Mac and has some Raspberry Pi and android/arm support with very little FPGA/ASIC/DSP/uC support.
>
> I hope, by the time that I get in to the hardware layers that D will have advanced further down the line on them.
>
> My understanding is that DMD is only applicable for the 3 main platforms, and this is fine for now as that is where I'll start. LDC is starting to progress on ARM/android and I have seen someone write a to use D on FPGA through a transformation process. Some work has been done with some lower level uC's such as the PIC and ARM processors.
>
> Now that GDC is in the works, I assume that will open up even more doors. Because this is a long term project(10+ years) and involves many different systems, subsystems, architectures, and so forth, organization is key for success.  The problem I have with the choice of D over C/C++, at the moment, is in it's knowledge base. Most of the advancements for the different platforms are done by individuals with their own unique goals and motivations in mind. There is very little organized structure to the overall development. Finding information for specific aspects and requirements involves hunting down the appropriate information. Usually it is forum posts and link fishing, usually outdated. Structured documentation is key for efficient and successful progress, which I find D to be lacking in. While library functionality is documented and adequate, the architectural issues and development is not.
>
> Is there any hope of seeing a unified structured interface to the different platforms that D is starting to be developed on rather than having to hunt and peck for the answers?
>
> If not, may I suggest, at a minimum, some type of hierarchy based solution that involves all the major design applications of D with the ability to cross-reference, date, comment, and modify? A sort of wiki but with a bit more hierarchical structure.
>
> D
>   Cross-Platform
>      Compilers
>         ...
>         GDC
>         ...
>      Library
>         ...
>      ...
>   Windows
>      Compiler
>         ...
>         Compiling
>             ...
>         ...
>      Library
>         ...
>         Compiling
>             ...
>         Documentation
>             ...
>             Wi32 API
>                 ...
>                 Problems
>                 ...
>                 Functions
>                 ...
>                 Misc
>                 ...
>                 Uncategorized
>                 ...
>             ...
>         ...
>   ...
>   DSP
>      ...
>      Progress
>        ...
>      ...
>
>
> The idea is that one can delve in to the hierarchy and find whatever info they want or add to it if it is not there so that someone else can find it. At some point, most of what D can do is found in the hierarchy, which, as we all know, only requires around O(log2(n)) search time(rather than O(n^m) for googling topics and link fishing).  At some point, this hierarchy becomes very efficient at representing the information we all are searching for rather than a flat random access type of list. The point is to design such an interface and let users like me fill it out and as time goes on,  finding relevant information becomes quicker and quicker. For proper design, meta-information must be used to it's fullest. (Tags, date stamping, ratings, relevancy, cross-references, etc)
>
> Or, perhaps, we already have something this powerful in the works?

The ldc wiki page has a rudimentary version of what you're looking for, listing all platforms worked on and some idea of how well they're supported:

https://wiki.dlang.org/LDC

Dmd will likely never target the vast majority of the platforms you list, so it will depend on the gdc and ldc teams doing it.

Also, I'm looking at submitting a pull with my last ldc CMake patch to cross-compile the stdlib for Android/ARM and generalizing it for other platforms too:

https://gist.github.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636

Once that's in, building a cross-compiled stdlib for several platforms should be as simple as setting some flags with the location of a C cross-compiler and any C/linker flags you need, then running the same cmake/make combo that you use to build a native stdlib today.

No modifications are needed to ldc itself for well-supported platforms, as ldc is always a cross-compiler by default.  For example, kinke used the new llvm linker, lld, to cross-compile Windows/x64 binaries from linux/x64 by using the stock ldc compiler.  The instructions for how he did it are linked from the changelog for the latest ldc 1.3 beta2:

https://github.com/ldc-developers/ldc/pull/2142#issuecomment-304472412

Finally, the latest ldc 1.3 betas for the first time include the llvm code-generation target backends for various other CPUs, like ARM or PowerPC, whereas before ldc only bothered to compile in the x86/x64 backends, as those were the only stdlibs it distributed:

[joakim@localhost ~]$ ./ldc2-1.3.0-beta2-linux-x86_64/bin/ldc2 --version
LDC - the LLVM D compiler (1.3.0-beta2):
  based on DMD v2.073.2 and LLVM 4.0.0
  built with LDC - the LLVM D compiler (1.3.0-beta2)
  Default target: x86_64-unknown-linux-gnu
  Host CPU: broadwell                                                        http://dlang.org - http://wiki.dlang.org/LDC

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64

Not very useful right now unless you're doing bare-metal programming on those newly added architectures, because the cross-compiled stdlibs are not included and there's no easy way to generate them, but we could also put out separate tarfiles with the cross-compiled stdlibs (separated only to save space in the compiler download for those who don't want them), just as kinke used the ldc stdlib compiled for Windows/x64 on linux/x64 when cross-compiling.

Just last night, I was thinking about some easy way to let ldc users cross-compile the stdlib themselves from the regular ldc download, and I thought of doing it with dub, the D package manager, since we now distribute dub alongside ldc.  However, I don't know how feasible this is with dub, needs to be looked into.

On Friday, 23 June 2017 at 19:56:17 UTC, Joakim wrote:
> 
> Just last night, I was thinking about some easy way to let ldc users cross-compile the stdlib themselves from the regular ldc download, and I thought of doing it with dub, the D package manager, since we now distribute dub alongside ldc.  However, I don't know how feasible this is with dub, needs to be looked into.

Being able to build druntime/Phobos locally on the user's machine would also help with generating LTO versions of the stdlibs.
I think the first step is making sure that all source is available in our packages. At the moment, druntime/src/rt is not present in the install (for example).

Cheers,
 Johan

On Friday, 23 June 2017 at 19:56:17 UTC, Joakim wrote:
> On Friday, 23 June 2017 at 19:03:01 UTC, FoxyBrown wrote:
>> [...]
>
> The ldc wiki page has a rudimentary version of what you're looking for, listing all platforms worked on and some idea of how well they're supported:
>
> https://wiki.dlang.org/LDC
>

Thanks, it's a start.

> Dmd will likely never target the vast majority of the platforms you list, so it will depend on the gdc and ldc teams doing it.

I'm not too worried about dmd, I am mainly interested in simply a way. D is a nice language, so being able to use it, regardless of which compiler, is the goal.

> Also, I'm looking at submitting a pull with my last ldc CMake patch to cross-compile the stdlib for Android/ARM and generalizing it for other platforms too:
>
> https://gist.github.com/joakim-noah/d74af3cf1355492557a9c56ef1bf2636
>
> Once that's in, building a cross-compiled stdlib for several platforms should be as simple as setting some flags with the location of a C cross-compiler and any C/linker flags you need, then running the same cmake/make combo that you use to build a native stdlib today.
>
> No modifications are needed to ldc itself for well-supported platforms, as ldc is always a cross-compiler by default.  For example, kinke used the new llvm linker, lld, to cross-compile Windows/x64 binaries from linux/x64 by using the stock ldc compiler.  The instructions for how he did it are linked from the changelog for the latest ldc 1.3 beta2:
>
> https://github.com/ldc-developers/ldc/pull/2142#issuecomment-304472412
>
> Finally, the latest ldc 1.3 betas for the first time include the llvm code-generation target backends for various other CPUs, like ARM or PowerPC, whereas before ldc only bothered to compile in the x86/x64 backends, as those were the only stdlibs it distributed:
>
> [joakim@localhost ~]$ ./ldc2-1.3.0-beta2-linux-x86_64/bin/ldc2 --version
> LDC - the LLVM D compiler (1.3.0-beta2):
>   based on DMD v2.073.2 and LLVM 4.0.0
>   built with LDC - the LLVM D compiler (1.3.0-beta2)
>   Default target: x86_64-unknown-linux-gnu
>   Host CPU: broadwell
>            http://dlang.org - http://wiki.dlang.org/LDC
>
>   Registered Targets:
>     aarch64    - AArch64 (little endian)
>     aarch64_be - AArch64 (big endian)
>     arm        - ARM
>     arm64      - ARM64 (little endian)
>     armeb      - ARM (big endian)
>     ppc32      - PowerPC 32
>     ppc64      - PowerPC 64
>     ppc64le    - PowerPC 64 LE
>     thumb      - Thumb
>     thumbeb    - Thumb (big endian)
>     x86        - 32-bit X86: Pentium-Pro and above
>     x86-64     - 64-bit X86: EM64T and AMD64
>
> Not very useful right now unless you're doing bare-metal programming on those newly added architectures, because the cross-compiled stdlibs are not included and there's no easy way to generate them, but we could also put out separate tarfiles with the cross-compiled stdlibs (separated only to save space in the compiler download for those who don't want them), just as kinke used the ldc stdlib compiled for Windows/x64 on linux/x64 when cross-compiling.
>
> Just last night, I was thinking about some easy way to let ldc users cross-compile the stdlib themselves from the regular ldc download, and I thought of doing it with dub, the D package manager, since we now distribute dub alongside ldc.  However, I don't know how feasible this is with dub, needs to be looked into.

Nice to know it's being worked on. I am not really familiar with llvm but it seems like it can be quite useful. I'm I correct that basically llvm is an intermediate representation(sorta like C#/VBS/F#/etc's IR)?

This means that one simply needs a front end that converts the source code to llvm's IR and then a back end that converts the IR to targets binary?


For example, a quick search for llvm and 8051 gives this:

https://github.com/AndreaCorallo/llvm-i8051

Since, I assume(since there is no documentation describing what it does) is a backend compiler for llvm's IR to 8051, we can go from LDC to LLVM to 8051? In theory, of course.

This is all bare metal, of course.

Is GCC effectively the same(although the IR is skipped?)?


Thanks.

On Saturday, 24 June 2017 at 18:16:46 UTC, FoxyBrown wrote:
>
> Nice to know it's being worked on. I am not really familiar with llvm but it seems like it can be quite useful. I'm I correct that basically llvm is an intermediate representation(sorta like C#/VBS/F#/etc's IR)?
>
> This means that one simply needs a front end that converts the source code to llvm's IR and then a back end that converts the IR to targets binary?

Yes indeed. LDC will output the IR when you use the `-output-ll` (human readable) or `-output-bc` (binary representation) compile switches.

> For example, a quick search for llvm and 8051 gives this:
>
> https://github.com/AndreaCorallo/llvm-i8051
>
> Since, I assume(since there is no documentation describing what it does) is a backend compiler for llvm's IR to 8051, we can go from LDC to LLVM to 8051? In theory, of course.

In theory, yes indeed. What may be important is that the LLVM versions match (the LLVM version used to build LDC, and the LLVM version of that 8051 fork). With every new LLVM version, the IR may change in subtle ways...

> Is GCC effectively the same(although the IR is skipped?)?

As far as I know, GCC also uses an IR (it's used for LTO).

-Johan

On Saturday, 24 June 2017 at 18:51:44 UTC, Johan Engelen wrote:
> On Saturday, 24 June 2017 at 18:16:46 UTC, FoxyBrown wrote:
>> [...]
>
> Yes indeed. LDC will output the IR when you use the `-output-ll` (human readable) or `-output-bc` (binary representation) compile switches.
>
>> [...]
>
> In theory, yes indeed. What may be important is that the LLVM versions match (the LLVM version used to build LDC, and the LLVM version of that 8051 fork). With every new LLVM version, the IR may change in subtle ways...
>
>> [...]
>
> As far as I know, GCC also uses an IR (it's used for LTO).
>
> -Johan

Why, then, doesn't anything that targets llvm simply target everything else? D is not just for the ones listed but also for many other platforms? is it simply because no one has implemented the transformations to prove it works?

On Saturday, 24 June 2017 at 19:18:24 UTC, FoxyBrown wrote:
> On Saturday, 24 June 2017 at 18:51:44 UTC, Johan Engelen wrote:
>> On Saturday, 24 June 2017 at 18:16:46 UTC, FoxyBrown wrote:
>>> [...]
>>
>> Yes indeed. LDC will output the IR when you use the `-output-ll` (human readable) or `-output-bc` (binary representation) compile switches.
>>
>>> [...]
>>
>> In theory, yes indeed. What may be important is that the LLVM versions match (the LLVM version used to build LDC, and the LLVM version of that 8051 fork). With every new LLVM version, the IR may change in subtle ways...
>>
>>> [...]
>>
>> As far as I know, GCC also uses an IR (it's used for LTO).
>>
>> -Johan
>
> Why, then, doesn't anything that targets llvm simply target everything else? D is not just for the ones listed but also for many other platforms? is it simply because no one has implemented the transformations to prove it works?

In theory, once you can produce llvm IR, you get every CPU that llvm targets, which grows by the day.  However, being able to target CPUs is not the only aspect of porting, you also have to integrate with the OS.  D's runtime and phobos rely on the C runtime for some basic functions, though there has been some talk of re-implementing them in pure D.

So if you're running on an OS, you also need to wrap several C APIs from that OS, see my pulls for bionic/Android, mostly the bottom five:

https://github.com/dlang/druntime/pulls?q=is%3Apr+author%3Ajoakim-noah+is%3Aclosed

If you're running on bare metal, that's obviously not an issue, but conversely those who've tried it say D doesn't really support stripping out the druntime for those targets.  There has been a renewed focus on this recently, with work on the -betterC flag, about which you can see another forum thread.