On Friday, 24 August 2018 at 22:16:25 UTC, Jonathan M Davis wrote:
> On Friday, August 24, 2018 7:46:57 AM MDT Mike Franklin via Digitalmars-d wrote:
>> On Friday, 24 August 2018 at 13:21:25 UTC, Jonathan M Davis wrote:
>> > I think that you're crazy.
>>
>> No, I just see more potential in D than you do.
>
> To be clear, I'm not calling you crazy in general. I'm calling the idea of bypassing libc to call syscalls directly under any kind of normal circumstances crazy. There is tons of work to be done around here to improve D, and IMHO, reimplementing OS functions just because they're written in C is a total waste of time and an invitation for bugs - in addition to making the druntime code that much less portable, since it bypasses the API layer that was standardized for POSIX systems. It's the kind of thing that's going to cause us way more work, more bugs, and make us that much less compatible with existing libraries. And for what? To _maybe_ get slightly better performance (which you probably won't get)? I honestly think that trying to bypass libc to talk to the kernel directly is actively worse than just using libc much as it would be great if we somehow lived in a world where every library we used was written in D. But the reality of the matter is that there is a _lot_ out there already written in C where it simply makes no sense to try to replace it. We're always going to need to interoperate with C unless we somehow convince all of the C developers to at least switch to -betterC (which obviously isn't happening).
>
> - Jonathan M Davis

You're underestimating the benefits. It's not just to be eventually slightly faster. It makes @safe versions possible, this in turn avoids a lot of @trusted calls, so reduces review effort. It allows also to develop own kernels (for maybe new hardware) without needing a c-toolchain an it makes D more self contained. There are certainly more advantages. And if you don't like it, the c stuff remains there for you to use.

On 8/24/18 6:16 PM, Jonathan M Davis wrote:
> On Friday, August 24, 2018 7:46:57 AM MDT Mike Franklin via Digitalmars-d
> wrote:
>> On Friday, 24 August 2018 at 13:21:25 UTC, Jonathan M Davis wrote:
>>> I think that you're crazy.
>>
>> No, I just see more potential in D than you do.
> 
> To be clear, I'm not calling you crazy in general. I'm calling the idea of
> bypassing libc to call syscalls directly under any kind of normal
> circumstances crazy. There is tons of work to be done around here to improve
> D, and IMHO, reimplementing OS functions just because they're written in C
> is a total waste of time and an invitation for bugs - in addition to making
> the druntime code that much less portable, since it bypasses the API layer
> that was standardized for POSIX systems.

Let me say that I both agree with Jonathan and with Mike.

I think we should reduce Phobos dependence on the user-library part of libc, while at the same time, not re-inventing how the OS bindings are called. For example, using Martin's std.io library instead of <stdio.h>.

I really don't want to see dlang have to maintain posix system calls on all supported OSes when that's already being done for us.

Windows makes this simpler -- the system calls are separate from the C runtime. It would be nice if Posix systems were that way, but it's both silly to reinvent the system calls (they are on every OS anyways, and in shared-library form), and a maintenance nightmare.

For platforms that DON'T have an OS abstraction, or it's split out from the user library part of libc, it would be perfectly acceptable to write a shim there if needed. I'd be surprised if it's not already present in C form.

-Steve

On Friday, August 24, 2018 4:44:31 PM MDT Dominikus Dittes Scherkl via Digitalmars-d wrote:
> You're underestimating the benefits. It's not just to be eventually slightly faster. It makes @safe versions possible, this in turn avoids a lot of @trusted calls, so reduces review effort. It allows also to develop own kernels (for maybe new hardware) without needing a c-toolchain an it makes D more self contained. There are certainly more advantages. And if you don't like it, the c stuff remains there for you to use.

It doesn't reduce the number of @trusted calls at all. Best case, you're talking about using @trusted with syscall instead of a function like stat, and for many of the C functions, it's perfectly reasonable to just mark their bindings as @trusted, eliminating the need to use @trusted yourself entirely. You're not reducing the number of uses of @trusted. All you get out of it is that then you've written the OS code using @system rather than relying on the C programmers to do their job when writing basic OS stuff. You then might catch a problem writing that code that someone writing the same code in C wouldn't have caught as easily, but that's it. You're bypassing a _heavily_ used and tested piece of code written by experts just because you want to be able to have @safe verify it, or because you want to avoid it simply because it's C. And because of the prevalence of pointers to local addresses in such code, there's a pretty good chance that a lot of it will have to be hand-vetted and marked with @trusted anyway instead of being able to take advantage of @safe.

And if someone wants to write an OS in D, then fine. They can do it. There's nothing about our current approach that stops them. As I understand it, there have already been a couple of projects to do exactly that, but you're not going to replace the major OSes with D any time soon (or likely ever), and the vast majority of D code is going to be interacting with those OSes - most of which provide their public APIs via C (many via the same POSIX API).

By using libc like everyone else, we get to take advantage of that work and work with a more portable API, risking fewer bugs in the process. Right now, we don't have to test all of the bindings in druntime to death, because they're just bindings, and we can rely on the libc guys to have done their job, whereas we would then be doing their jobs if we insisted on bypassing libc. It's a maintenance nightmare for little to no benefit. I don't want to have to deal with it as a maintainer, and I don't want programs that I write to be bypassing libc using a custom implementation just because someone decided that they didn't like the fact that it was in C instead of D.

- Jonathan M Davis

On Friday, 24 August 2018 at 22:42:19 UTC, Walter Bright wrote:
> On 8/24/2018 12:42 PM, tide wrote:
>> Some problems require new features like how taking the address of a member function without an object returns a function pointer, but requires a delegate where C++ has member function pointers, D just has broken unusable code. Or old features that were implemented poorly (C++ mangling for example).
>
> How to do member function pointers in D:
>
> https://www.digitalmars.com/articles/b68.html

struct SomeStruct
{
    void foo() {
        // use SomeStruct
    }
}


void broken()
{
    void function() foo = &SomeStruct.foo;
    foo(); // runtime error, isn't actually safe uses wrong calling convention as well
}

Not really lack of feature so much as there exists broken code. This has been valid code for god knows how long. At some point it was usable in @safe, but it looks you can't take an address of a member function without "this" as well in safe anymore.

On Friday, 24 August 2018 at 22:52:07 UTC, Steven Schveighoffer wrote:

> I really don't want to see dlang have to maintain posix system calls on all supported OSes when that's already being done for us.
>
> Windows makes this simpler -- the system calls are separate from the C runtime. It would be nice if Posix systems were that way, but it's both silly to reinvent the system calls (they are on every OS anyways, and in shared-library form), and a maintenance nightmare.

Keep in mind that we only need to implement the system calls that we need.  I haven't looked into Phobos, and probably never will.  My interest is mostly in druntime.  At this time, I think we only need 2:  `sbrk` and `mmap` for `malloc`. I don't consider that much of a maintenance burden, and `malloc` and friends are my least concern at the moment.

We're disproportionately leveraging libc in druntime; there are only a few things needed from libc for druntime, and I think I can demonstrate benefit writing them in D (or if someone else wants to, please do, I may never even get to it).

If I even stick around in the D community long enough to pursue this, this change it'll be incremental and I'll demonstrate benefit each step of the way.

Mike

On Friday, 24 August 2018 at 11:55:47 UTC, Petar Kirov [ZombineDev] wrote:
> One of the things that makes Go successful is the quality/ease of use of its toolchain. They have full cross-compilation support out of the box because they don't rely on anything from the C toolchain (libc, linker, etc.). They implement implement everything themselves, from the syscall layer

This is something that has caused breakage on newer versions of macos, and broadly makes it difficult to port to new OSes.  Something that might be worth pursuing, though, is implementing some of the things in core.stdc in pure d, like printf or strcmp, but printf *would* ultimately forward to the actual libc fwrite.

On Friday, 24 August 2018 at 19:26:40 UTC, Walter Bright wrote:
> On 8/24/2018 6:04 AM, Chris wrote:
>> For about a year I've had the feeling that D is moving too fast and going nowhere at the same time. D has to slow down and get stable. D is past the experimental stage. Too many people use it for real world programming and programmers value and _need_ both stability and consistency.
>
> Every programmer who says this also demands new (and breaking) features.

Heh, thought this proggit comment thread was funny given this complaint, some C++ users feel it's moving too fast now:

"In the last few years it has basically become a different language, the feature creep is insane. I stopped caring about new features since C++11, and progressively used the language less and less."

Another user:

"I remember being really excited about C++11 - and I think it really did add some much needed features. But it's been getting more and more out of hand since then..."
https://www.reddit.com/r/programming/comments/99rnuq/comment/e4q8iqn

On 8/24/2018 4:22 PM, tide wrote:
> struct SomeStruct
> {
>      void foo() {
>          // use SomeStruct
>      }
> }
> 
> 
> void broken()
> {
>      void function() foo = &SomeStruct.foo;
>      foo(); // runtime error, isn't actually safe uses wrong calling convention as well
> }
> 
> Not really lack of feature so much as there exists broken code. This has been valid code for god knows how long. At some point it was usable in @safe, but it looks you can't take an address of a member function without "this" as well in safe anymore.


That's because it isn't safe. But being able to take the address is important for system work.

On Friday, August 24, 2018 6:54:38 PM MDT Joakim via Digitalmars-d wrote:
> On Friday, 24 August 2018 at 19:26:40 UTC, Walter Bright wrote:
> > On 8/24/2018 6:04 AM, Chris wrote:
> >> For about a year I've had the feeling that D is moving too fast and going nowhere at the same time. D has to slow down and get stable. D is past the experimental stage. Too many people use it for real world programming and programmers value and _need_ both stability and consistency.
> >
> > Every programmer who says this also demands new (and breaking)
> > features.
>
> Heh, thought this proggit comment thread was funny given this complaint, some C++ users feel it's moving too fast now:
>
> "In the last few years it has basically become a different language, the feature creep is insane. I stopped caring about new features since C++11, and progressively used the language less and less."
>
> Another user:
>
> "I remember being really excited about C++11 - and I think it really did add some much needed features. But it's been getting more and more out of hand since then..." https://www.reddit.com/r/programming/comments/99rnuq/comment/e4q8iqn

LOL. Yeah. Basically, we all somehow want stuff to be completely stable and never break any of our code, but we also want new stuff that improves the language - which frequently requires breaking existing code (and even if it doesn't require breaking existing code, adding features always risks breaking existing ones - even simply fixing bugs risks introducing new ones). There are better and worse ways to handle change, but I think that we're pretty much all fundamentally conflicted in what we want. You simply can't have everything stay the same and have it change at the same time, and yet, that's basically what everyone wants. Pretty much the only way to get around that would be to have a perfect language with no bugs, and that's obviously not happening even if we could all agree on what the "perfect" language would entail (which we're clearly not going to do).

- Jonathan M Davis

On 8/24/2018 6:34 AM, Shachar Shemesh wrote:
> No, unlike what I suggest, that doesn't work without carefully reviewing every single place you put it to see whether the constructor actually supports destructing a partially constructed object.

All D objects are default-initialized before the constructor sees it (unlike C++). A destructor should be able to handle a default-initialized object.