On 23/08/18 15:03, Walter Bright wrote:
>> So you will excuse me, but I don't think this bug is being taken as seriously as I think it should.
> 
> It is a serious problem. (There are workarounds available, like using scope(failure).)

I don't think you understand how unworkable this workaround is.

struct A {
  this(something);
  ~this();
}

struct B {
  A a;

  this(something) {
    a = A(something);
    // Safeguard against 14246
    scope(failure) destroy(a);

    // more code
  }
}

struct C {
  A a;

  this(something) {
    a = A(something);
    // Also needs a safeguard against 14246
    scope(failure) destroy(a);

    // more code
  }
}

struct D {
  this(something) {
    // Do nothing
  }
}

struct E {
  B b;
  D d;

  this(something) {
    b = B(something);
    // B doesn't even have an explicit destructor, but we are supposed to look at its implementation and understand that it contains A, so:
    scope(failure) destroy(b);

    d = D(something);
    // D doesn't even contain A, but it might one day, so:
    scope(failure) destroy(d);
  }
}


The chances of this scheme actually working without errors are, more or less, zero.

On Friday, 24 August 2018 at 09:46:08 UTC, Jonathan M Davis wrote:

> For any kind of normal operating system, you _have_ to use libc. It's part of the OS. Some pieces could be done without it, but on the whole, you use libc if you want to talk to the OS. That's just life. The only exceptions I'm aware of to that are embedded devices, and my understanding is that if anything, such devices are more and more likely to run a fullblown OS, making it that much less likely that you'd ever do anything without libc.

That is not true.  You can write your own system calls.  Here's "hello world" with no libc:

---object.d
module object;

alias immutable(char)[] string;

private long __d_sys_write(long arg1, in void* arg2, long arg3)
{
    long result;

    asm
    {
        mov RAX, 1;
        mov RDI, arg1;
        mov RSI, arg2;
        mov RDX, arg3;
        syscall;
    }

    return result;
}

void write(string text)
{
    __d_sys_write(2, text.ptr, text.length);
}

private void __d_sys_exit(long arg1)
{
    asm
    {
        mov RAX, 60;
        mov RDI, arg1;
        syscall;
    }
}

extern void main();
private extern(C) void _start()
{
    main();
    __d_sys_exit(0);
}

---main.d
module main;

void main()
{
    write("Hello, World\n");
}

$dmd -c -lib -conf= object.d main.d -of=main.o
$ld main.o -o main
$size main
   text    data     bss     dec     hex filename
    176       0       0     176      b0 main
$main
Hello, World

You just need to re-implement what you need in D.  For Phobos, that might be a big job (or maybe not, I haven't looked into it).  For druntime, it's not so bad -- it's mostly just memcpy, memcmp, malloc, free, and maybe a couple of others.  Those are not trivial implementations, but it's not out of reach, and I think there is opportunity with, with CTFE, templates, __traits, inline asm, and a number of other D features, to do even better than C, and make it more type- and memory-safe while we're at it.  Implementing those building-blocks in D, would be good for the language, would solve a number of issues I'm currently having with druntime updates, and would be a fun project for those that are interested in that kind of stuff.

Mike

On Friday, 24 August 2018 at 10:16:34 UTC, Shachar Shemesh wrote:
> On 23/08/18 15:03, Walter Bright wrote:
>>> So you will excuse me, but I don't think this bug is being taken as seriously as I think it should.
>> 
>> It is a serious problem. (There are workarounds available, like using scope(failure).)
>
> I don't think you understand how unworkable this workaround is.

I think Walter was talking more about "scope (failure) destroy(this)" at the top of all your structs? I don't know if it has some gotchas, though (as I don't use RAII in D...).

On Friday, August 24, 2018 4:18:31 AM MDT Mike Franklin via Digitalmars-d wrote:
> On Friday, 24 August 2018 at 09:46:08 UTC, Jonathan M Davis wrote:
> > For any kind of normal operating system, you _have_ to use libc. It's part of the OS. Some pieces could be done without it, but on the whole, you use libc if you want to talk to the OS. That's just life. The only exceptions I'm aware of to that are embedded devices, and my understanding is that if anything, such devices are more and more likely to run a fullblown OS, making it that much less likely that you'd ever do anything without libc.
>
> That is not true.  You can write your own system calls.  Here's "hello world" with no libc:

Linux is the only OS I'm aware of that considers the syscall layer to be something that anything outside the OS would normally call. Other OSes consider libc to be part of the OS. In theory, you could call the syscalls directly in the BSDs (and probably on Mac OS), but the expectation is that you're going to use libc. Calling them directly would be way more error-prone, since you'd basically have to reimplement portions of libc and have to deal with any changes they make which normally would be hidden by libc. You're basically trying to bypass the OS' public API if you're trying to bypass libc. And of course, that's definitely not how things are done on Windows.

Honestly, I don't see how it's at all reasonable to be trying to access syscalls directly rather than using libc under any kind of normal circumstances - especially if you're not on Linux.

- Jonathan M Davis

On Friday, August 24, 2018 4:08:52 AM MDT Shachar Shemesh via Digitalmars-d wrote:
> On 24/08/18 10:43, FeepingCreature wrote:
> > Have you tried to use the excellent Dustmite tool? It's never failed to reduce a bug for me.
>
> Dustmite might be excellent. I wouldn't know. It cannot swallow the Weka code base.

Dustmite can be fantastic, but it's far from a silver bullet. It's often difficult to give it stopping criteria in a way that gets you what you want, and you don't have to have all that much code before it takes hours to finish (one of my coworkers was recently trying to use dustmite to find an issue, and he had it running for days). I would expect that with a code base of any real size, you would have to be able to take a modular piece of it and test it directly in order to have much hope of it getting the job done.

- Jonathan M Davis

On Friday, 24 August 2018 at 09:46:08 UTC, Jonathan M Davis wrote:
> On Friday, August 24, 2018 2:46:06 AM MDT Dave Jones via Digitalmars-d wrote:
>> On Friday, 24 August 2018 at 04:50:34 UTC, Mike Franklin wrote:
>> > On Friday, 24 August 2018 at 04:12:42 UTC, Jonathan M Davis wrote:
>> >
>> >
>> > It's not a problem for Phobos to depend on the C standard library.  My goals have to do with making D, the language, freestanding (a.k.a nimble-D).
>>
>> If the poster feature for D in the upcoming years is memory safety then how can Walter seriously consider continued dependency on libc?
>
> For any kind of normal operating system, you _have_ to use libc. It's part of the OS. Some pieces could be done without it, but on the whole, you use libc if you want to talk to the OS. That's just life. The only exceptions I'm aware of to that are embedded devices, and my understanding is that if anything, such devices are more and more likely to run a fullblown OS, making it that much less likely that you'd ever do anything without libc.

Another notable exception is WebAssembly. Others include the whole distroless container trend going down to uni-kernels for use as a slim base for micro services. Why ship a container with full libc, if you're only going to use a handful of syscalls? That's simply a larger than necessary attack surface.

> Sure, we don't need to call C functions like strcmp, but if you want to do things like I/O, you have to use the OS' API, and that's libc. And yes, that means that we depend on code that's not vetted via @safe, but at this point, the major OSes are written in C, and they present a C API. So, accessing that functionality means depending on the OS devs to have gotten it right, much as it would be nice if it were verified with something like @safe. The same is going to happen with plenty of existing libraries that are almost certainly not going to have replacements in D (e.g. postgres, ffmpeg, etc). We're never going to completely escape the @safety issues introduced by C. Ultimately, the best that we can do is make sure that the actual D code is @safe as long as any libraries it uses are @safe.
>
> - Jonathan M Davis

One of the things that makes Go successful is the quality/ease of use of its toolchain. They have full cross-compilation support out of the box because they don't rely on anything from the C toolchain (libc, linker, etc.). They implement implement everything themselves, from the syscall layer - up the whole stack (e.g. https://golang.org/pkg/syscall , https://golang.org/pkg/os/ ). Rust is also taking the same path. In recent times it seems that every new systems programming language is trying to prove that while it can interact with existing C libraries effortlessly, it has to be fully independent in order to be taken seriously. You can't replace C/C++ if you depend on them.
One small data point: changes to the C toolchain on Linux broke the druntime's backtrace support several times. If we didn't depend on them this probably wouldn't have happened.

On Friday, 24 August 2018 at 11:15:21 UTC, Jonathan M Davis wrote:

> Linux is the only OS I'm aware of that considers the syscall layer to be something that anything outside the OS would normally call.

I think Linux considers system calls the OS API.

> Other OSes consider libc to be part of the OS.

Not Windows.  Windows has it's own API, and when interfacing with the OS, I never use libc.

> In theory, you could call the syscalls directly in the BSDs (and probably on Mac OS), but the expectation is that you're going to use libc.

That's the expectation for application programming, not systems programming.

> Calling them directly would be way more error-prone, since you'd basically have to reimplement portions of libc and have to deal with any changes they make which normally would be hidden by libc.

Most of what we need is already implemented in the language, druntime, and Phobos.  We just need a few fundamental building blocks that are currently implemented in C and a few calls into the OS APIs (syscalls for linux, Window API for Windows, not sure about Mac...probably syscalls).  And just like DMD benefited from being written in D, so would those building-blocks, and the code that calls into them.

> You're basically trying to bypass the OS' public API if you're trying to bypass libc.

No I'm trying to bypass libc and use the OS API directly.

> Honestly, I don't see how it's at all reasonable to be trying to access syscalls directly rather than using libc under any kind of normal circumstances - especially if you're not on Linux.

I think it'd be nice if D were freestanding and portable without requiring libraries written in other languages.  Purity, safety, CTFE, introspection, etc... all the way down.

Mike

On Friday, 24 August 2018 at 09:52:20 UTC, Walter Bright wrote:
> On 8/24/2018 1:45 AM, Trass3r wrote:
>> Are you referring to http://wg21.link/P0709 ?
>
> Yes. (please don't use link shorteners, they tend to go poof)
>
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r1.pdf

I expect it to always point to the latest revision. Not sure if it's an official WG21 service though.

Anyway, very interesting paper and approach.
I'm eager to see how this will work out.

On Wednesday, 22 August 2018 at 11:59:37 UTC, Paolo Invernizzi wrote:
> Just found by chance, if someone is interested [1] [2].
>
> /Paolo

After having seen all the discussions around Mihails post in these days, I'm puzzled by one fact.

There was no discussions around one paragraph:

"You can't assume there is any control over how declared vision documents get executed in practice. You can't trust any promises from language authors because they don't keep any track of those."

I think that this is one of the central points of the post, so why?

/Paolo

On Friday, 24 August 2018 at 12:25:58 UTC, Paolo Invernizzi wrote:
> On Wednesday, 22 August 2018 at 11:59:37 UTC, Paolo Invernizzi wrote:
>> Just found by chance, if someone is interested [1] [2].
>>
>> /Paolo
>
> After having seen all the discussions around Mihails post in these days, I'm puzzled by one fact.
>
> There was no discussions around one paragraph:
>
> "You can't assume there is any control over how declared vision documents get executed in practice. You can't trust any promises from language authors because they don't keep any track of those."
>
> I think that this is one of the central points of the post, so why?
>
> /Paolo

I think I touched it indirectly. There is every time "the feature" that will make D most popular language in the world, be it safety (which will kill C) or betterC (which will kill C?), RC instead of GC or whatever. A lot of work is done, but after some time everyone loses interest and the written code becomes a mess. Look how are Phobos containers implemented, there is no consistent memory model.