Hi!

This post is inspired by: https://forum.dlang.org/thread/scklbdrgjiypknuephfj@forum.dlang.org

On Saturday, 22 February 2020 at 13:20:40 UTC, IGotD- wrote:
> Will there be a future improvement of this so that OS specific support can be moved to separate files and selected on module level instead? For example.
>
> void DSpecificLibraryFunction(...)
> {
>     OSDependentLibraryFunction(...)
> }
>
> so that there are predefined OS specific functions that are selected on module level. Also version() switches where everything is consolidated into one file makes it more difficult to merge.

More specific proposal how to achieve this step by step:

Сommunity may agree to add druntime module "core.sys.abstract" and everywhere in druntime code what references to a specific OSes and architecures (all these thousands of "version(){} else version(){}") will add also obtain code like:

version(Abstract_Platform){ core.sys.abstract.some_call(); }

This "core.sys.abstract" code for every exotic uncommon (or just "every"?) platform will be placed outside of druntime "src" dir and will be used for compilation only if appropriate compiler flag "-I" is set and version=Abstract_Platform is defined.

Then some uncommon platforms can be moved to "abstract" implementation (for purpose of code coverage for this new "abstract subsystem").

And at next step maybe every platforms can be implemented as abstract (and "core.sys.abstract" will be renamed into "core.sys").

For me it looks like it can be done gradually and at the same time not to break anything.

How do you like this idea? Do you see missings/flaws in it?

On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin wrote:
>
> And at next step maybe every platforms can be implemented as abstract (and "core.sys.abstract" will be renamed into "core.sys").
>
> For me it looks like it can be done gradually and at the same time not to break anything.
>
> How do you like this idea? Do you see missings/flaws in it?

This really opens up a second question, with your proposal with an abstract interface we can start to reduce the dependency on the C library. The C library is just there as baggage in order to give basic OS functionality. With an abstract interface we can use the OS primitives directly without going through any C library. This is a huge task but this offers a gradual approach.

We can have the C library as an option through the abstract interface if we still need it.

I see a double benefit of this.

On Tuesday, 25 February 2020 at 00:22:19 UTC, IGotD- wrote:

> This really opens up a second question, with your proposal with an abstract interface we can start to reduce the dependency on the C library. The C library is just there as baggage in order to give basic OS functionality. With an abstract interface we can use the OS primitives directly without going through any C library. This is a huge task but this offers a gradual approach.
>
> We can have the C library as an option through the abstract interface if we still need it.

Yes!

Perhaps  need to introduce option DRUNTIME_LIBC_AVAILABLE for druntime so that we can gradually get rid of all libc's symbols like "errno" inside of new abstract implementations by wrapping some druntime code pieces into version() branches.

On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin wrote:
> Hi!
>
> This post is inspired by: https://forum.dlang.org/thread/scklbdrgjiypknuephfj@forum.dlang.org
>
> [...]

Somewhat related that might be of interest to you.  I wrote my own standard library that doesn't depend on libc, doesn't use druntime and is compatible with betterC.  https://github.com/dragon-lang/mar

On Tuesday, 25 February 2020 at 16:42:04 UTC, Jonathan Marler wrote:
> On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin wrote:
>> Hi!
>>
>> This post is inspired by: https://forum.dlang.org/thread/scklbdrgjiypknuephfj@forum.dlang.org
>>
>> [...]
>
> Somewhat related that might be of interest to you.  I wrote my own standard library that doesn't depend on libc, doesn't use druntime and is compatible with betterC.  https://github.com/dragon-lang/mar

This project is very interesting to me I believe it may be extremely valuable to D as a whole! I have seen it a few months ago, but back then I didn't have much time to check it out.

What is the current status?
Is it self-hosting? E.g. can you build dmd only with it?
What are your goals (in addition to minimal external dependencies)?
How much of Phobos and druntime do you want reimplement or offer alternative features to?

Threading, ranges, containers and algorithms, IO, std.math, etc.?

On Tuesday, 25 February 2020 at 22:39:54 UTC, Petar Kirov [ZombineDev] wrote:
> On Tuesday, 25 February 2020 at 16:42:04 UTC, Jonathan Marler wrote:
>> On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin wrote:
>>> Hi!
>>>
>>> This post is inspired by: https://forum.dlang.org/thread/scklbdrgjiypknuephfj@forum.dlang.org
>>>
>>> [...]
>>
>> Somewhat related that might be of interest to you.  I wrote my own standard library that doesn't depend on libc, doesn't use druntime and is compatible with betterC.  https://github.com/dragon-lang/mar
>
> This project is very interesting to me I believe it may be extremely valuable to D as a whole! I have seen it a few months ago, but back then I didn't have much time to check it out.
>
> What is the current status?
> Is it self-hosting? E.g. can you build dmd only with it?
> What are your goals (in addition to minimal external dependencies)?
> How much of Phobos and druntime do you want reimplement or offer alternative features to?
>
> Threading, ranges, containers and algorithms, IO, std.math, etc.?

I wanted to be able to use D without relying on libc.  One use case I had for this was to write my own rootfs purely in D (https://github.com/marler8997/maros).  I was very glad I did this as it was instrumental in learning how most linux distributions work.

I want my standard library to work with or without GC, druntime, libc, phobos, and betterC.  I want the API to be as similar as possible whether or not I'm using these features.  I want to be able to enable/disable these like features without rewriting my code.  In order to do this, the standard library needs to work with or without them.

Also note that the "mar" library can be used alongside druntime/phobos as well.  It's in its own namespace so there's no issues in using them together.  I was able to gradually port code to use "mar" instead of druntime/phobos piece by piece by referencing both.

Another big feature is that "mar" tracks null-terminated strings in the type system.  Pretty much all OS APIs use null-terminated (aka sentinel) strings, and it seems like a good idea to enforce them at compile-time rather than runtime.

Anyway, there was alot of reasons I made it.  Even if all it becomes is an educational experience for me then I'm still glad I did it, and if other poeple can find more uses for it then even better.

On Tuesday, 25 February 2020 at 01:21:33 UTC, Denis Feklushkin wrote:
> More specific proposal how to achieve this step by step:

> Сommunity may agree to add druntime module "core.sys.abstract" and everywhere in druntime code what references to a specific OSes and architecures (all these thousands of "version(){} else version(){}") will add also obtain code like

Probably I seem to know easiest way:

Many modules are scattered by version() keywords that implement common calls necessary when using any platform.
For brief example look into: core/sync/event.d

We can improve Makefiles (and cmake in case of LDC fork of druntime) to include only needed platform-dependend subdirs what will contain these modules but without "versions", just code for each supported platform.
(Do not confuse with core.sys.platform_name section!)

(And I couldn’t do it myself right now because I don’t know well enough make and cmake systems. I am need "initial sample template" to do things like in it.)

After such replacement it will be very easy to clear out remaining libc calls (if it will be need - all supported platforms actually have libc interfaces), and also support for new arbitrary platforms will be ready - for supporting any new platform will be enough to replace platform-specific section with your own at build stage by build system.

What do you think about this? Is that good idea?

On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin wrote:
> On Tuesday, 25 February 2020 at 01:21:33 UTC, Denis Feklushkin wrote:
>
> Probably I seem to know easiest way:
>
> Many modules are scattered by version() keywords that implement common calls necessary when using any platform.
> For brief example look into: core/sync/event.d
>
> We can improve Makefiles (and cmake in case of LDC fork of druntime) to include only needed platform-dependend subdirs what will contain these modules but without "versions", just code for each supported platform.
> (Do not confuse with core.sys.platform_name section!)
>
> (And I couldn’t do it myself right now because I don’t know well enough make and cmake systems. I am need "initial sample template" to do things like in it.)
>
> After such replacement it will be very easy to clear out remaining libc calls (if it will be need - all supported platforms actually have libc interfaces), and also support for new arbitrary platforms will be ready - for supporting any new platform will be enough to replace platform-specific section with your own at build stage by build system.
>
> What do you think about this? Is that good idea?

If take src/core/sync/semaphore.d (class Semaphore) for example this is an example with a lot of version switches that could be a good candidate for an abstract interface. I'm not sure what type of abstraction it should be, inherited SemaphoreImpl class, mixin member of a name given by OS module, fixed member name?

There is alot of confusion with the OS and clib as OS support functionality is mixed together with clib functionality in the code. This is very confusing. Instead if an abstract OS want to use clib, it should do so separately. For example Windows should use Microsoft Clib, Linux could use a variety of clib like glibc, musl etc. Clib support should be under OS support and not mixed into the generic code.

Another thing I've noticed is how elf code is duplicated for each OS. As elf is a standard we don't need to duplicate this and we can have a generic solution instead. We should have an ABI for this as well.

First I think we need to determine what type of "polymorphism" to use ie. how to select the OS specific implementation, then start to implement an ABI. Much of the ABI is straight forward I think.

BTW, isn't it time to retire make files and go for cmake only? Perhaps OT, I should start a new thread.

On Sunday, 15 March 2020 at 12:08:54 UTC, IGotD- wrote:
> On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin
>> What do you think about this? Is that good idea?
>
> If take src/core/sync/semaphore.d (class Semaphore) for example this is an example with a lot of version switches that could be a good candidate for an abstract interface. I'm not sure what type of abstraction it should be, inherited SemaphoreImpl class, mixin member of a name given by OS module, fixed member name?

It’s definitely not a class because in this case it is not required to provide polymorphism - there is only one platform will be presented in result binary. (If someone occasionally wants to make a multi-platform binaries it will still be more difficult than just several implementations of these base class.)

Just fixed member is preferred, as for me.

>
> There is alot of confusion with the OS and clib as OS support functionality is mixed together with clib functionality in the code. This is very confusing. Instead if an abstract OS want to use clib, it should do so separately. For example Windows should use Microsoft Clib, Linux could use a variety of clib like glibc, musl etc. Clib support should be under OS support and not mixed into the generic code.

My investigation showed me that there is not so much of this libc code. I even think that it will be possible to introduce  temporary abstract libc version(CRuntime_AbstractLibc) to cover this case.

>
> Another thing I've noticed is how elf code is duplicated for each OS. As elf is a standard we don't need to duplicate this and we can have a generic solution instead. We should have an ABI for this as well.

And, probably, not all platforms need ELF support in druntime.

> First I think we need to determine what type of "polymorphism" to use ie. how to select the OS specific implementation, then start to implement an ABI. Much of the ABI is straight forward I think.

Just improve makefiles to be able select needed subdirectories like in
posix.mak/win64.mak and LDC's druntime CMakeLists.txt:

# only include core/sys/ modules matching the platform
file(GLOB_RECURSE DRUNTIME_D_BIONIC  ${RUNTIME_DIR}/src/core/sys/bionic/*.d)
file(GLOB_RECURSE DRUNTIME_D_DARWIN  ${RUNTIME_DIR}/src/core/sys/darwin/*.d)
file(GLOB_RECURSE DRUNTIME_D_DRAGONFLYBSD ${RUNTIME_DIR}/src/core/sys/dragonflybsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_FREEBSD ${RUNTIME_DIR}/src/core/sys/freebsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_LINUX   ${RUNTIME_DIR}/src/core/sys/linux/*.d)
file(GLOB_RECURSE DRUNTIME_D_NETBSD  ${RUNTIME_DIR}/src/core/sys/netbsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_OPENBSD ${RUNTIME_DIR}/src/core/sys/openbsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_POSIX   ${RUNTIME_DIR}/src/core/sys/posix/*.d)
file(GLOB_RECURSE DRUNTIME_D_SOLARIS ${RUNTIME_DIR}/src/core/sys/solaris/*.d)
file(GLOB_RECURSE DRUNTIME_D_WINDOWS ${RUNTIME_DIR}/src/core/sys/windows/*.d)

>
> BTW, isn't it time to retire make files and go for cmake only? Perhaps OT, I should start a new thread.

Do you know a mind state when you recognize a new tool and it starts to seem perfect and you want to use it everywhere and always? So for me now this tool is Meson build system. Therefore, I am biased.

Maybe we should consider switching to it. It is ideal for D in terms of the ability to simplified link programs written in various languages.

In fact, I'm not going to use druntime with existing built-in make/cmake scripts in my Dlang MCU projects because it are too complex and it is very difficult to configure to support dozens of all kinds of options that microcontroller binaries need.

On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin wrote:
> What do you think about this? Is that good idea?

Sounds like what Walter wanted to do in the first place:

https://github.com/dlang/druntime/pull/2616#issuecomment-496312793
https://github.com/dlang/druntime/pull/2616#issuecomment-503503846