The progress here is nice.  I'd like to do similar stuff with LDC but have a couple other projects to finish up first.

More thoughts on supporting systems without threads:

"Mike" <none@none.com> writes:
> On Tuesday, 5 May 2015 at 17:27:05 UTC, Johannes Pfau wrote:
>> I'm not really sure about this. It's trivial to implement but it
>> generates a semantic difference which harms code portability
>> (e.g. if
>> you have code where you want a global variable, don't mark it as
>> __gshared and then move to an environment with threads).

I think this could happen anyway.  I write some D code on linux, but my program doesn't create additional threads so I don't realize I should have used __gshared somewhere.

Sometime later when I use code in program with multi threads, I discover my problem that I should have used __gshared on a var.

> I'm coming around and feeling inclined to agree with this.  One of the things I want to avoid is creatIng an embedded dialect of the D.  I hope D for microcontrollers, kernels and such looks very much like the idiomatic D in the application programming domain.

I don't think it changes D, does it?  I want to try to convince here because I think it opens up a lot of D code for no-thread systems that would otherwise need to be edited and have __gshared added here and there.

The language just says a non-immutable global declaration [without shared or __gshared] lives in thread local storage.  In a single thread environment, a thread local degenerates to a global, doesn't it?

Looking at it another way.  I could design a "TLS" model for a non-threaded system that has the compiler puts all "thread local" vars in section named ".tls" instead of ".data".  Because there is only one thread, there is only one ".tls" section.  But I could then have linker merge .tls and .data sections.  And instructions to reference tls and non-tls vars are identical.

I don't think going the other way is safe.  If a system supports threads but not TLS, then turning D thread local vars into globals is not good (however I did do that so I could get initial work done on iOS before I added TLS to backend).
-- 
Dan

"Mike" <none@none.com> writes:
> I've gotten even further thanks to everyone's help, but LDC is still giving me a little grief.  Take a look at these undefined references (abbreviated for this forum):
>
> (_D10TypeInfo_l6__vtblZ+0x8): undef ref
> `_D6object8TypeInfo8toStringMxFNaNbNfZAya'

-- snip --

> Are there any compiler flags I can add to help trim the fat?

Mike, try creating an empty ldc2.conf file where you compile from (I just tried so it should work). That will override the one that came with your ldc2 installation that has searches the real druntime/phobos before your versions. It looks like there will be an option -conf with 0.16.0 to override the default conf file.

https://github.com/ldc-developers/ldc/issues/879

You can put -v on your command ldc2 command line to see where it is picking up files.

On Tuesday, 5 May 2015 at 02:26:28 UTC, Mike wrote:
> Porting to a New Platform
> ******************************
> The platform-agnostic code in "d" delgates implementation details to the platform-specific code using `extern(C) extern _d_sys_name` "system calls" (for lack of a better term).

You plan to have a sizable API without type safety? Why PAL is not good?

On Thursday, 7 May 2015 at 07:00:58 UTC, Dan Olson wrote:

> Mike, try creating an empty ldc2.conf file where you compile from (I
> just tried so it should work). That will override the one that came with
> your ldc2 installation that has searches the real druntime/phobos before
> your versions. It looks like there will be an option -conf with 0.16.0
> to override the default conf file.
>

That did it!  But I thought defaultLib= did the same thing.  Anyway, Thanks for all your help, Dan.

I now have support for all 3 compilers porting to 2 different platforms (one bare metal microcontroller, one modern OS).  It can't do anything more than "Hello, World!", but it's a start :-)

Mike

On Tuesday, 5 May 2015 at 17:38:38 UTC, Johannes Pfau wrote:

>
> I would probably split the runtime into at least two, probably three
> parts:
>
> * Compiler support library (object.d, gcc/*.d, exception
>   implementation) (module rt/ no module name)
> * Higher level library (portable, micro)
> * Hardware specific library (not portable, avr/ stm/)
>
> I think some basic portability between microcontrollers is useful. The
> compiler support library should be only a few 100 lines of code and
> should be without external dependencies. This way it's also useful for
> very small platforms (8bit) or use-cases where you inject code into
> other processes. It's trivial to port: we should have an ansi-c port
> and native ports should be implementable in 1-2 hours.
>
>
> The higher level library should contain stuff like 'core.time.Duration',
> emplace, allocator framework, Volatile!T, register wrapper types, etc.
> The implementation could be hardware specific, but it should provide a
> common interface (e.g. a delay!Duration function has got a common
> interface but different implementation, malloc is similar).
> Namespace micro => import micro.time; import micro.memory;
> So everything in the micro namespace is portable.
>
>
> The hardware library the provides access to hardware specific
> peripherals. In reality the high-level-library might require the
> hardware library or they could actually be merged. The important part
> is portable vs platform specific module API for user code.

Sorry for the late reply, but I'm still chewing and digesting this.  How do you propose delegating implementation down the supply chain?  I suppose you may have answered that with your git idea below, but did you have something else in mind?

I think that's really what I'm trying to work out with this experiment:  How to delegate the implementation so it's obvious what porters need to do without a lot of explanation, and they can implement just the features they need.

>
> A radically different approach instead of using ports directories is
> using git features: Have a base repository with a master branch which
> only includes the interfaces, probably with static assert wherever
> platform specific code is necessary. Then have AVR/STM32/LPC/...
> branches where you simply implement these functions. (We could also
> have different repositories instead of branches)
>
> + you can modify all code
> + you already start with a common interface
> + changes are easy to compare by comparing git branches
> + changes have descriptions (in their git commits)
> + it's easy to merge further generic changes

I'll have to think about this and give it a try.  I fear, however, that it might be a little too radical for what people are used to, but maybe not.

>
> Phobos and core.stdc should then also be separate libraries.

Agreed.


>> *  Add -fno-rtti compiler switch
>
> This is something I could probably finish up in a few hours next
> weekend.

That would be an immense help and productivity boost.  Thanks for your support.

>
>> *  Add attribute support so programmer can choose which types to generate runtime-time info for.
>
> I can add an @attribute(notypeinfo) in GDC but it'll be implemented in
> the backend. This means we can prevent TypeInfo output but we can't
> reliably warn on TypeInfo usage. You'll get linker errors instead.

Probably a good idea to table that for now.

Mike

On Tuesday, 5 May 2015 at 06:56:52 UTC, Iain Buclaw wrote:
> On 5 May 2015 at 08:39, Dan Olson via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>
>> How about a -disable-tls option so that when there are no threads and
>> thus no TLS, you can compile exising D code as-is.  Otherwise you have
>> to rewrite normal variables to __gshared everywhere (I actually have this
>> option in a ldc fork).
>>
>
> Configure GDC with --disable-tls --disable-threads to get the desired
> behaviour (disabling only TLS just makes codegen fallback to emulated
> thread support).

I already have supplied those options in my toolchain.
But does anyone know if this is advisable when using FreeRTOS (or any other RTOS for that matter) ?
-I'm asking, because I'm not using any RTOS myself, but there are loads of people who do.

On Tuesday, 5 May 2015 at 02:26:28 UTC, Mike wrote:
{snip}
>
> Putting it all Together
> ************************
> Users are not expected to use this code directly.  Rather, I envision toolchain, silicon, and board vendors will use this code as a small, but essential part of, their platform's D programming package.  For example, a package for the ARM Cortex-M family of MCUs might contain the following:
> * arm-none-eabi GDC cross-compiler
> * a library containing compiled code for the "d" folder in this repository
> * a library containing the compiled code for the "ports/arm/cortexm" folders in this repository

I'd like to insert CMSIS here. CMSIS should be shared between all Cortex-M platforms.
My impression is that it's fairly easy to do; CMSIS also provides convenience functions like WFI(), WFE(), ROR(), NOP(), CLZ(), REV(), RBIT(), etc...

Note: It might be a good idea to add these as well: CLO(), CTZ() and CTO()

> * cortex-m core startup files
> * the newlib C library
> * the C library bindings from Deimos
> * multilibs from the GNU toolchain.

{snip}

Am Thu, 07 May 2015 11:52:29 +0000
schrieb "Mike" <none@none.com>:

> On Tuesday, 5 May 2015 at 17:38:38 UTC, Johannes Pfau wrote:
> 
> >
> > I would probably split the runtime into at least two, probably
> > three
> > parts:
> >
> > * Compiler support library (object.d, gcc/*.d, exception
> >   implementation) (module rt/ no module name)
> > * Higher level library (portable, micro)
> > * Hardware specific library (not portable, avr/ stm/)
> >
> > I think some basic portability between microcontrollers is
> > useful. The
> > compiler support library should be only a few 100 lines of code
> > and
> > should be without external dependencies. This way it's also
> > useful for
> > very small platforms (8bit) or use-cases where you inject code
> > into
> > other processes. It's trivial to port: we should have an ansi-c
> > port
> > and native ports should be implementable in 1-2 hours.
> >
> >
> > The higher level library should contain stuff like
> > 'core.time.Duration',
> > emplace, allocator framework, Volatile!T, register wrapper
> > types, etc.
> > The implementation could be hardware specific, but it should
> > provide a
> > common interface (e.g. a delay!Duration function has got a
> > common
> > interface but different implementation, malloc is similar).
> > Namespace micro => import micro.time; import micro.memory;
> > So everything in the micro namespace is portable.
> >
> >
> > The hardware library the provides access to hardware specific
> > peripherals. In reality the high-level-library might require the
> > hardware library or they could actually be merged. The
> > important part
> > is portable vs platform specific module API for user code.
> 
> Sorry for the late reply, but I'm still chewing and digesting this.  How do you propose delegating implementation down the supply chain?  I suppose you may have answered that with your git idea below, but did you have something else in mind?
> 

Mostly the git idea. It makes sense to keep the 'customization' points to a minimum and have well-defined hooks like in newlib and like you suggested. But the git branch way also allows modifying code everywhere, in case the generic hooks don't fit one platform for some reason.

> I think that's really what I'm trying to work out with this experiment:  How to delegate the implementation so it's obvious what porters need to do without a lot of explanation, and they can implement just the features they need.

Let me explain the git idea:

core library; branch 'common':
--------------------------------------------
object.d:
class Object
{

}

void traceLine(string msg)
{
    static assert(false, "Not implemented");
}
--------------------------------------------
branch avr
--------------------------------------------
object.d:
class Object
{

}

version(none) // Not implemented for AVRs right now
{
void traceLine(string msg)
{
    static assert(false, "Not implemented");
}
}
--------------------------------------------
branch nintendo-ds
--------------------------------------------
object.d:
class Object
{
    //Lots of memory, let's add some useful methods
    void toString(scope delegate(const(char)[] buf))
    {
    ...
    }
}

void traceLine(string msg)
{
    asm (print to port xyz...)
}
--------------------------------------------

Porters simply grep for '"Not implemented"' and implement the function
or comment/remove it.
If we now change the interface in the common branch:
void traceLine(string msg) => void traceLine(string msg, int level = 0)

Both branches will get the changes or a merge conflict by simple merging the common branch. This way we can keep a kinda common interface and have lots of customization possibilities.

Note 1: The core lib might be a bad example. It should be mostly portable and have little user-facing code so keeping a common interface is not important here. It's more useful for a high-level library (e.g. keep emplace/allocator interface compatible while still allowing custom malloc or other allocator implementations)

Note 2: Even a simple 'traceLine' function depends on board definitions. So a port targeting many boards could not even implement a 'traceLine' in the corlib. It would have to delegate the implementation to the hardware library* effectively tying the core library to the hardware library. I think this is sometimes unavoidable for low-level code but this should probably be a decision porters make. The 'common' corlib should not depend on other libraries.

*or even user code.

Note 3: It could still make sense to keep all the 'to-be-implemented' functions/hooks in a special module or package. But instead of having ports/arm/ ports/avr we could have one port/*.d with template files as described above and the actual platform specific implementations in branches.

> >
> > A radically different approach instead of using ports
> > directories is
> > using git features: Have a base repository with a master branch
> > which
> > only includes the interfaces, probably with static assert
> > wherever
> > platform specific code is necessary. Then have AVR/STM32/LPC/...
> > branches where you simply implement these functions. (We could
> > also
> > have different repositories instead of branches)
> >
> > + you can modify all code
> > + you already start with a common interface
> > + changes are easy to compare by comparing git branches
> > + changes have descriptions (in their git commits)
> > + it's easy to merge further generic changes
> 
> I'll have to think about this and give it a try.  I fear, however, that it might be a little too radical for what people are used to, but maybe not.

Many newlib ports are not part of official newlib but custom forks AFAIK. There's no big difference here (instead of branches we could also use different repos). By using git we make maintaining custom changes much simpler. But it is indeed a rather radical approach and there could certainly be some drawbacks as well.

(Even if we define formal hooks and a port/ directory structure porters could of course still fork&modify the code instead. They could probably use the techniques described here without our explicit support. But if we decide to go this way, we can simplify _our_ code. No need to have lots of hooks or version(ARM) else version(AVR) else version() or a port/ directory structure. Every port is in its own branch)

On Thursday, 7 May 2015 at 16:12:36 UTC, Johannes Pfau wrote:
{snip}
>
> Let me explain the git idea:

I think it's a great idea.

{snip}
>
> Porters simply grep for '"Not implemented"' and implement the function

This sounds easy.

{snip}
> (Even if we define formal hooks and a port/ directory structure porters
> could of course still fork&modify the code instead. They could probably
> use the techniques described here without our explicit support.

That was the only 'downside' I could think of; eg. if I wanted to develop firmware for both the LPC and STM series, switching branch for my repository all the time would not be ideal. ;)
-But of course, it can be solved easily by making a repository, which contains a "read-only" directory structure of all branches (or something similar).
-Such a repository could be generated automatically and tagged by a script very easily.

{snip}
> Every port is in its own branch

In this case, the files must share the same parent directory, in order to be updated by a merge with master; correct ?

"Mike" <none@none.com> writes:
> That did it!  But I thought defaultLib= did the same thing.  Anyway, Thanks for all your help, Dan.
>
> I now have support for all 3 compilers porting to 2 different platforms (one bare metal microcontroller, one modern OS).  It can't do anything more than "Hello, World!", but it's a start :-)

You may not have h/w to test this on, but support for x86_64 OSX could be added easily. Syscall assembly is same as linux x86_64, just the syscall # is different:

exit rax=0x2000001
write rax=0x2000004

I just tried in a test D program and it worked. Anyway, probably little value to have OSX here as a platform. I think interesting platforms will be no OS, where end user plugs in their write handler to write fd 1,2 to serial port or to go through gdbstub protocol over serial, or ...
-- 
Dan