On Thursday, 30 April 2015 at 00:14:18 UTC, Martin Nowak wrote:

> Wonder if it makes more sense to start from zero and add as few files as
> possible.

When I started I did not know D enough to understand what druntime does. Just picked the easy way. The amount of required changes has got smaller in every release. It is easy to maintain: I get mainstream updates and bug fixes just by patching.
Mike has done some work with an empty runtime and Adam has a chapter in his book. I do not know any real programs made from this base


> Druntime doesn't do much useful stuff for a µC anyhow.
>
> - GC and rt.lifetime (new, arrays)
> - Moduleinfo
> - EH unwind support
> - AA implementation
> - vectorized array ops
> - core.time/sync/thread/demangle

UC programs mostly use static resources but I think 'new' is useful to initialize class objects at runtime. I got exceptions to work but I do not know how much use they have. There should be some time routines.

>
> What might be interesting is this.
>
> - core.bitop
> - maybe core.atomic
> - some gcc simd module
> - libc bindings for core.stdc.math and core.stdc.stdio for printf


Printf is a little tricky. It is actually a file operation to stdout and that is actually a syscall to kernel. In a controller board we may have several devices where we can send data like uart and oled display. (I even have a video output where video signal is generated with just one spi and one timer)
I think java style would be better here like
console = new Formatter (new uartWriter (uart2) )
console.printf("",...)
This way the formatter can be reused and is not bound to any hardware.


I think it is possible to make a minimum bare metal runtime fom scratch. However, there are some requirements:
- A team. One man project may never get ready and it will not receive enough attention. It is also often tailored to personal needs which makes it difficult for others to use.
- An insider. A project from unknown people will just be ignored. (I think we may have one now)
- A project coordinator. I think we will find one.
- Some people who want to work. We may have a couple.

If people thik we should start, I am with.

On Thursday, 30 April 2015 at 00:14:18 UTC, Martin Nowak wrote:

>
> Wonder if it makes more sense to start from zero and add as few files as
> possible.
> Druntime doesn't do much useful stuff for a µC anyhow.
>
> - GC and rt.lifetime (new, arrays)
> - Moduleinfo
> - EH unwind support
> - AA implementation
> - vectorized array ops
> - core.time/sync/thread/demangle
>
> What might be interesting is this.
>
> - core.bitop
> - maybe core.atomic
> - some gcc simd module
> - libc bindings for core.stdc.math and core.stdc.stdio for printf


Starting from zero appeals to my way of thinking.  I've made several attempts at this both on the PC and for microcontrollers, so please allow me to offer my thoughts on the idea:

If starting from zero, I recommend narrowing the scope to simply runtime initialization. IMO, if one can get to main with a small subset of the language ready for use, the rest is mostly icing on the cake.

While this may seem simple to achieve, I think it will raise a few questions that will need answering.

* Can ModuleInfo be leveraged, without introducing overhead, to call module constructors and static constructors?  They might be useful for hardware initialization.
* Is dynamic memory allocation a requirement of D, or a library feature?
* Does D need the C runtime, or can it init on its own?
* Should the C standard library bindings be part of the runtime, or exist as an external Deimos library?  Either way they can still be used by the runtime, I'm just suggesting that they should be encapsulated.
* What will be done about TypeInfo for now?  It's causing me some serious code-bloat problems.  See http://forum.dlang.org/post/quemhwpgijwmqtpxukiv@forum.dlang.org
* Is data and bss initialization part of  the runtime, or delegated to toolchain, silicon, and BSP vendors?

I think runtime initialization is a relatively simple task, but would still present a few challenges, choices, and even a few problems.  However, due to the fact that it's narrow in scope, I suspect it would be a likely success and could put some momentum in the right direction.

Mike

On 04/30/2015 01:30 PM, Mike wrote:
> While this may seem simple to achieve, I think it will raise a few questions that will need answering.
> 
> * Can ModuleInfo be leveraged, without introducing overhead, to call module constructors and static constructors?  They might be useful for hardware initialization.

Currently D sorts modules by initialization order,which requires every moduleinfo to contain an array of imported modules. Quite a lot of RAM for a nice to have feature, so we should drop it or at least provide a -fno-moduleinfo switch.

> * Is dynamic memory allocation a requirement of D, or a library feature?

Definitely a library feature, though many language features, like array appending, won't work without it (delegate closures won't even work without a GC).

> * Does D need the C runtime, or can it init on its own?

It shouldn't need a C runtime.

> * Should the C standard library bindings be part of the runtime, or exist as an external Deimos library?  Either way they can still be used by the runtime, I'm just suggesting that they should be encapsulated.

It doesn't cost you anything to include the bindings in a release, they could be maintained in a separate project if that helps.

> * What will be done about TypeInfo for now?  It's causing me some serious code-bloat problems.  See http://forum.dlang.org/post/quemhwpgijwmqtpxukiv@forum.dlang.org

Implement a -fno-rtti switch for GDC.

> * Is data and bss initialization part of  the runtime, or delegated to toolchain, silicon, and BSP vendors?

We should provide appropriate linker scripts and do the initialization.

On 04/30/2015 08:43 AM, Timo Sintonen wrote:
> Printf is a little tricky. It is actually a file operation to stdout and that is actually a syscall to kernel.

No, you usually have to implement some hook for outputting yourself, e.g. putc or write, printf solely takes care of the formatting.

On Thursday, 30 April 2015 at 06:43:18 UTC, Timo Sintonen wrote:
{snip}
> I think it is possible to make a minimum bare metal runtime from scratch.
{snip}

Yes I think it won't be too difficult.
All are good points that we should have in mind, while deciding on how we proceed.
-Also Mike's input is valuable. I agree with both of you.

Mike, Timo, Martin and Johannes:
Here's what I planned a few week ago ...
1: Get the toolchain working for Cortex-M0, Cortex-M3 and Cortex-M4. (Done, thanks to you).
2: Create a bunch of startup files for different devices as an appetizer. (Done, but ongoing)
3: Create a linker-script suitable for the casual MCU programmer. (Almost done)
4: Create a simple Makefile as an example. (Pending)
5: Create a set of files, which can be shared across all MCUs (for instance containing the *int*_t) types. This could later grow to a bare-metal druntime. (Pending)
6: Create a couple of driver files for one of the STM32 microcontrollers. (Pending)
7: Create a complete example project, similar to those already written in C, which demonstrates that it can be done completely in D.
...
10: Present this to Reinhard Keil, as the ARM Keil tools now use LLVM. I am sure Reinhard will find the D language intriguing. But as he is very busy, I would like to wait until I have a fully functioning example, which shows that D is now ready for microcontrollers.

If step 10 is done too early, it might not be as appealing as if we have a working demo.

Mike, Timo and I all have different "approaches" and "needs".
We can definitely use this to our benefit. D)iversity is great. =D
I have the impression that Martin seems to have a good overview and also knowledge about different MCUs.

On Thursday, 30 April 2015 at 20:54:07 UTC, Martin Nowak wrote:
> On 04/30/2015 08:43 AM, Timo Sintonen wrote:
>> Printf is a little tricky. It is actually a file operation to stdout and that is actually a syscall to kernel.
>
> No, you usually have to implement some hook for outputting yourself, e.g. putc or write, printf solely takes care of the formatting.

My knowledge isn't really deep here, but doesn't printf call vfprintf(stdout, ...) and
vfprintf call vsnprintf followed by fwrite, etc. ?

Thus I would expect the hook to be somewhere in vfprintf ?

On Thursday, 30 April 2015 at 21:08:22 UTC, Jens Bauer wrote:
> On Thursday, 30 April 2015 at 20:54:07 UTC, Martin Nowak wrote:
>> On 04/30/2015 08:43 AM, Timo Sintonen wrote:
>>> Printf is a little tricky. It is actually a file operation to stdout and that is actually a syscall to kernel.
>>
>> No, you usually have to implement some hook for outputting yourself, e.g. putc or write, printf solely takes care of the formatting.
>
> My knowledge isn't really deep here, but doesn't printf call vfprintf(stdout, ...) and
> vfprintf call vsnprintf followed by fwrite, etc. ?
>
> Thus I would expect the hook to be somewhere in vfprintf ?

As Timo said, eventually, what printf needs is the `write` syscall.  The C library needs to be ported to the hardware in question.  That requires implementing all the syscalls the the C library needs:  `write` for printf, `sbrk` for malloc, etc... (http://wiki.osdev.org/Porting_Newlib#newlib.2Flibc.2Fsys.2Fmyos.2Fsyscalls.c) Sometimes the toolchain vendors provide this, sometimes the programmer has to do it.

IMO, C library bindings are quite useful, and an important part of the microcontroller ecosystem, but I don't think it should be a prerequisite for druntime.  I recommend keeping it as a separate project in Deimos, and leave it to the programmer to implement the syscalls if they choose to employ it.

Mike

On Thursday, 30 April 2015 at 20:45:28 UTC, Martin Nowak wrote:

>> 
>> * Can ModuleInfo be leveraged, without introducing overhead, to call
>> module constructors and static constructors?  They might be useful for
>> hardware initialization.
>
> Currently D sorts modules by initialization order,which requires every
> moduleinfo to contain an array of imported modules. Quite a lot of RAM
> for a nice to have feature, so we should drop it or at least provide a
> -fno-moduleinfo switch.

Agreed.  Luckily -fno-emit-moduleinfo is already implemented, but without a ModuleInfo implementation, this will be a required switch to get a successful build.

>> * Is dynamic memory allocation a requirement of D, or a library feature?
>
> Definitely a library feature, though many language features, like array
> appending, won't work without it (delegate closures won't even work
> without a GC).

Good!

>
>> * Does D need the C runtime, or can it init on its own?
>
> It shouldn't need a C runtime.

Good!

>
>> * Should the C standard library bindings be part of the runtime, or
>> exist as an external Deimos library?  Either way they can still be used
>> by the runtime, I'm just suggesting that they should be encapsulated.
>
> It doesn't cost you anything to include the bindings in a release, they
> could be maintained in a separate project if that helps.

Yes, that helps.  There's already a project created in Deimos: https://github.com/D-Programming-Deimos/libc

>
>> * What will be done about TypeInfo for now?  It's causing me some
>> serious code-bloat problems.  See
>> http://forum.dlang.org/post/quemhwpgijwmqtpxukiv@forum.dlang.org
>
> Implement a -fno-rtti switch for GDC.

What about https://issues.dlang.org/show_bug.cgi?id=12270?  ...perhaps in addition to -fno-rtti.

>
>> * Is data and bss initialization part of  the runtime, or delegated to
>> toolchain, silicon, and BSP vendors?
>
> We should provide appropriate linker scripts and do the initialization.

The linker script can give the location of the data and bss segments.  My question is really about whether or not the druntime implementation should assume responsibility for loading the data segment from flash and zeroing bss.  I suppose that should be left to the vendor's BSP, not part of druntime itself.

I guess what I'm trying to articulate is that currently when you download an MCU toolchain, it contains a collection of things from many different vendors (GCC, binutils, newlib, startup files, linker scripts, multilibs, etc...), all in one package.  I recommend, not doing that with this druntime.  druntime should just be the implementation of D language features for microcontrollers.

Once a minimal druntime is created, some other effort can take that druntime and package it with a compiler, linker, startup files, linker scripts, c standard library, debugger, flash programmer, etc... and make a convenient downloadable/installable package for immediate use.

Assuming that is an appropriate strategy, what does the first druntime release look like.
* For example: What will `assert` do?  Is that just a stub to be implemented by the programmer?
* What are druntime's "syscalls" that need to be implemented by the silicon vendor, BSP, and/or programmer (I'm using a newlib port analogy here)?  What does druntime do, and what does it require its users to do?

My mind is currently seeing an object.d with some aliases and some TypeInfo stubs, __entrypoint.d, and a minimal rt/dmain2.d.  That will give you a build and get you to main() for a trivial program, but you'll start getting linker errors, and potentially runtime errors, as you begin to employ more of D's features; even the simple ones like `assert`.

That is one of the things that's making it difficult for me to simply get a working language.  druntime has features that really seem to be more "standard library" features (e.g. `assert`), but they are mingled with the language implementation.  It, therefore, makes it a challenge to get a build, separate concerns, and delegate implementation to users.

Mike

On Thursday, 30 April 2015 at 23:03:48 UTC, Mike wrote:
> On Thursday, 30 April 2015 at 20:45:28 UTC, Martin Nowak wrote:
>
>>> * Is data and bss initialization part of  the runtime, or delegated to toolchain, silicon, and BSP vendors?
>>
>> We should provide appropriate linker scripts and do the initialization.
>
> The linker script can give the location of the data and bss segments.  My question is really about whether or not the druntime implementation should assume responsibility for loading the data segment from flash and zeroing bss.  I suppose that should be left to the vendor's BSP, not part of druntime itself.

How about providing functions for copying data and zeroing bss ?
-In my startup file, I use functions for that, because they're optimized pretty well.
The code is very similar to my optimized #define in C.

Most Cortex-M microcontrollers allow you to set the clock frequency.
It's a good idea to do this before you start copying a large block of data, because that means the startup time is quicker.
The drawback is that ... CMSIS wants to write in the BSS (the CoreClockFrequency variable, for instance), and that's just erased when BSS is cleared - I wish they had just reserved a hardware register for these 4 variables.

Anyway, personally, I like the startup file to contain the data/bss init, because I might want to customize it. My startup files allow you to override Reset_Handler and each exception, which means you don't even have to modify the startup file. That means one less template-file to copy to every project.

> I guess what I'm trying to articulate is that currently when you download an MCU toolchain, it contains a collection of things from many different vendors (GCC, binutils, newlib, startup files, linker scripts, multilibs, etc...), all in one package.  I recommend, not doing that with this druntime.  druntime should just be the implementation of D language features for microcontrollers.

Agree.

> Once a minimal druntime is created, some other effort can take that druntime and package it with a compiler, linker, startup files, linker scripts, c standard library, debugger, flash programmer, etc... and make a convenient downloadable/installable package for immediate use.

Agree.
Linker-scripts can usually be written so they work well with multiple devices from the same vendor. I'm not talking about specific RAM and Flash settings, but the main body of the linker script. Small 3 .. 8 line scripts can then include the body, which provides all the standard stuff. So here I'm talking about a body linker-script could be provided for each vendor (not each device family or each device). If each device family or device requires different attention, then I suggest something like Vendor:DeviceFamily:DeviceSpecific.
Thus it would not be overwhelming writing one linker script per vendor and it would be fairly straight-forward for the casual user to add a new device or even a device family if needed.
I'd also prefer having the linker-script in a location, outside each project (because I have many projects and I hate duplicates and updating each duplicate each time I find something that needs to be changed).

> Assuming that is an appropriate strategy, what does the first druntime release look like.
> * For example: What will `assert` do?  Is that just a stub to be implemented by the programmer?

Most asserts on microcontrollers I've seen are just implemented as while(1){}
-But one could probably also trigger the debugger (BKPT), HardFault or RESET if necessary.
Perhaps the default could be while(1){} and then version(ASSERT_BKPT) could be used to add a breakpoint before that while(1){}. Thus --version=ASSERT_BKPT could be specified on the command-line.
Would it be possible to 'extend' an existing assert; eg. the user might want to be notified via the U(S)ART ?

On Thursday, 30 April 2015 at 21:35:44 UTC, Mike wrote:
> On Thursday, 30 April 2015 at 21:08:22 UTC, Jens Bauer wrote:
>> Thus I would expect the hook to be somewhere in vfprintf ?
>
> As Timo said, eventually, what printf needs is the `write` syscall.  The C library needs to be ported to the hardware in question.  That requires implementing all the syscalls the the C library needs:  `write` for printf, `sbrk` for malloc, etc... (http://wiki.osdev.org/Porting_Newlib#newlib.2Flibc.2Fsys.2Fmyos.2Fsyscalls.c) Sometimes the toolchain vendors provide this, sometimes the programmer has to do it.

Uhm, in that case, why not supply a weakref dummy, eg. functions that can be overridden.
Thus stat, open, fstat, lseek, read, write, isatty, close, link and unlink just do nothing at all.
If the MCU has a file-system, then it can implement those as strong refs.
Same about gettimeofday, getpid, execve, fork, kill - Often there's only a single thread and no possibility for loading and executing a named program.
-So it should be fairly easy to change newlib 'requirements' to 'optional'. :)

I feel like trying this out, perhaps when the new light comes. ;)