Jump to page: 1 2 3
Thread overview
nogc v0.5.0 - DIP1008 works!
May 24, 2019
Atila Neves
May 24, 2019
Seb
May 24, 2019
ag0aep6g
May 24, 2019
Atila Neves
May 24, 2019
ag0aep6g
May 24, 2019
Atila Neves
May 24, 2019
ag0aep6g
May 24, 2019
ag0aep6g
May 24, 2019
Meta
May 27, 2019
Atila Neves
May 27, 2019
Paolo Invernizzi
May 27, 2019
Atila Neves
May 27, 2019
Paolo Invernizzi
May 27, 2019
ag0aep6g
May 27, 2019
Atila Neves
May 27, 2019
ag0aep6g
May 27, 2019
Atila Neves
May 27, 2019
ag0aep6g
May 29, 2019
Valeriy Fedotov
May 29, 2019
ag0aep6g
May 24, 2019
Mike Franklin
May 24, 2019
I'd been holding off on announcing this until DIP1008 actually got implemented, and now it has:

https://code.dlang.org/packages/nogc

This dub package has a @nogc version of `std.conv.text` (which probably isn't as good yet) that, instead of returning a `string` returns an `automem.vector.Vector` of char. This handles managing memory allocation for the exception message itself in `NoGcException`, which does what it says on the tin. Confused? Here's some code:


@safe @nogc unittest {
    import nogc;
    import std.algorithm: equal;

    int a = 1, b = 2;
    try
        enforce(a == b, a, " was not equal to ", b);
    catch(NoGcException e) {
        assert(equal(e.msg, "1 was not equal to 2"));
    }

    try
        throw new NoGcException(42, " foobar ", 33.3);
    catch(NoGcException e) {
        assert(equal(e.msg, "42 foobar 33.300000"));
        assert(e.file == __FILE__);
        assert(e.line == __LINE__ - 4);
    }
}

It doesn't leak memory either, as proved by ldc's asan.

May 24, 2019
On Friday, 24 May 2019 at 11:41:12 UTC, Atila Neves wrote:
> I'd been holding off on announcing this until DIP1008 actually got implemented, and now it has:
>
> [...]

Awesome!!
Now we just need to get to compile Druntime and Phobos with -preview=dip1008, s.t. we can enable it by default :)


See e.g. https://github.com/dlang/dmd/pull/8508
May 24, 2019
On 24.05.19 13:41, Atila Neves wrote:
> I'd been holding off on announcing this until DIP1008 actually got implemented, and now it has:
> 
> https://code.dlang.org/packages/nogc

You've got safety violations:

----
/+ dub.sdl:
    name "test"
    dependency "nogc" version="~>0.5.0"
+/

import core.stdc.stdio: puts;

struct S1
{
    S2 s2;
    this(ref const S1 src) const @nogc @system { this.s2 = src.s2; }
}

struct S2
{
    this(ref const S2 src) const @nogc @system { puts("@system 1"); }
}

struct Z
{
    char* stringz() const @nogc @system
    {
        puts("@system 2");
        return null;
    }
}

struct UnsafeAllocator
{
    import std.experimental.allocator.mallocator: Mallocator;
    enum instance = UnsafeAllocator.init;
    void deallocate(void[] bytes) @nogc @system
    {
        puts("@system 3");
        Mallocator.instance.deallocate(bytes);
    }
    void[] allocate(size_t sz) @nogc @system
    {
        puts("@system 4");
        return Mallocator.instance.allocate(sz);
    }
}

void main() @safe @nogc
{
    import nogc: BUFFER_SIZE, text;
    S1 a;
    Z* z;
    auto t = text!(BUFFER_SIZE, UnsafeAllocator)(a, z);
}
----

All of the `puts` lines are executed. That should not be possible in @safe code. You're applying @trusted too liberally.
May 24, 2019
On Friday, 24 May 2019 at 12:32:45 UTC, ag0aep6g wrote:
> On 24.05.19 13:41, Atila Neves wrote:
>> [...]
>
> You've got safety violations:
>
> ----
> /+ dub.sdl:
>     name "test"
>     dependency "nogc" version="~>0.5.0"
> +/
>
> import core.stdc.stdio: puts;
>
> struct S1
> {
>     S2 s2;
>     this(ref const S1 src) const @nogc @system { this.s2 = src.s2; }
> }
>
> struct S2
> {
>     this(ref const S2 src) const @nogc @system { puts("@system 1"); }
> }
>
> struct Z
> {
>     char* stringz() const @nogc @system
>     {
>         puts("@system 2");
>         return null;
>     }
> }
>
> struct UnsafeAllocator
> {
>     import std.experimental.allocator.mallocator: Mallocator;
>     enum instance = UnsafeAllocator.init;
>     void deallocate(void[] bytes) @nogc @system
>     {
>         puts("@system 3");
>         Mallocator.instance.deallocate(bytes);
>     }
>     void[] allocate(size_t sz) @nogc @system
>     {
>         puts("@system 4");
>         return Mallocator.instance.allocate(sz);
>     }
> }
>
> void main() @safe @nogc
> {
>     import nogc: BUFFER_SIZE, text;
>     S1 a;
>     Z* z;
>     auto t = text!(BUFFER_SIZE, UnsafeAllocator)(a, z);
> }
> ----
>
> All of the `puts` lines are executed. That should not be possible in @safe code. You're applying @trusted too liberally.

Thanks for this. I think the only violation is calling `stringz` on `Z`, and that was due to a poorly designed DbI check on being able to call `stringz`. Allocating generally isn't @system, and freeing is ok to trust since vector is taking care of it for us. I've pushed a fix.
May 24, 2019
On Friday, 24 May 2019 at 13:13:12 UTC, Atila Neves wrote:
> Thanks for this. I think the only violation is calling `stringz` on `Z`, and that was due to a poorly designed DbI check on being able to call `stringz`. Allocating generally isn't @system, and freeing is ok to trust since vector is taking care of it for us. I've pushed a fix.

I think you're missing the point. When your function is marked as @safe or @trusted, then any execution of a user-provided @system function is a safety violation.

My `puts`s might not do any harm, but they could just as well be buffer overflows.
May 24, 2019
On Friday, 24 May 2019 at 13:30:05 UTC, ag0aep6g wrote:
> On Friday, 24 May 2019 at 13:13:12 UTC, Atila Neves wrote:
>> Thanks for this. I think the only violation is calling `stringz` on `Z`, and that was due to a poorly designed DbI check on being able to call `stringz`. Allocating generally isn't @system, and freeing is ok to trust since vector is taking care of it for us. I've pushed a fix.
>
> I think you're missing the point. When your function is marked as @safe or @trusted, then any execution of a user-provided @system function is a safety violation.
>
> My `puts`s might not do any harm, but they could just as well be buffer overflows.

Could you please give an example of how @system allocator code could do that?
May 24, 2019
On 24.05.19 18:19, Atila Neves wrote:
> On Friday, 24 May 2019 at 13:30:05 UTC, ag0aep6g wrote:
[...]
>> My `puts`s might not do any harm, but they could just as well be buffer overflows.
> 
> Could you please give an example of how @system allocator code could do that?

Sure. You just write beyond some buffer instead of calling `puts`:

----
char[3] buf;
char[3] foo = "foo";
char[3] bar = "bar";

struct UnsafeAllocator
{
    import std.experimental.allocator.mallocator: Mallocator;
    static instance = UnsafeAllocator.init;
    size_t i;
    void deallocate(void[] bytes) @nogc @system
    {
        buf.ptr[i .. i + 3] = '!';
        Mallocator.instance.deallocate(bytes);
    }
    void[] allocate(size_t sz) @nogc @system
    {
        buf.ptr[i .. i + 3] = '!';
        return Mallocator.instance.allocate(sz);
    }
}

void main() @safe @nogc
{
    {
        import nogc: BUFFER_SIZE, text;
        UnsafeAllocator.instance.i = 8;
            /* greater than buf.length, whoops */
        auto t = text!(BUFFER_SIZE, UnsafeAllocator)(42);
        assert(foo == "foo"); /* fails */
        UnsafeAllocator.instance.i = 16;
            /* also greater than buf.length, whoops again */
    }
    assert(bar == "bar"); /* fails */
}
----

You just can't trust user-provided @system code. It doesn't matter if it's allocator code or whatever.
May 24, 2019
On 24.05.19 18:51, ag0aep6g wrote:
> char[3] buf;
[...]
>          buf.ptr[i .. i + 3] = '!';
[...]
>          UnsafeAllocator.instance.i = 8;
>              /* greater than buf.length, whoops */

Actually, anything greater than zero is a whoops, of course.
May 24, 2019
On Friday, 24 May 2019 at 16:51:11 UTC, ag0aep6g wrote:
> On 24.05.19 18:19, Atila Neves wrote:
>> On Friday, 24 May 2019 at 13:30:05 UTC, ag0aep6g wrote:
> [...]
>>> My `puts`s might not do any harm, but they could just as well be buffer overflows.
>> 
>> Could you please give an example of how @system allocator code could do that?
>
> Sure. You just write beyond some buffer instead of calling `puts`:
>
> ----
> char[3] buf;
> char[3] foo = "foo";
> char[3] bar = "bar";
>
> struct UnsafeAllocator
> {
>     import std.experimental.allocator.mallocator: Mallocator;
>     static instance = UnsafeAllocator.init;
>     size_t i;
>     void deallocate(void[] bytes) @nogc @system
>     {
>         buf.ptr[i .. i + 3] = '!';
>         Mallocator.instance.deallocate(bytes);
>     }
>     void[] allocate(size_t sz) @nogc @system
>     {
>         buf.ptr[i .. i + 3] = '!';
>         return Mallocator.instance.allocate(sz);
>     }
> }
>
> void main() @safe @nogc
> {
>     {
>         import nogc: BUFFER_SIZE, text;
>         UnsafeAllocator.instance.i = 8;
>             /* greater than buf.length, whoops */
>         auto t = text!(BUFFER_SIZE, UnsafeAllocator)(42);
>         assert(foo == "foo"); /* fails */
>         UnsafeAllocator.instance.i = 16;
>             /* also greater than buf.length, whoops again */
>     }
>     assert(bar == "bar"); /* fails */
> }
> ----
>
> You just can't trust user-provided @system code. It doesn't matter if it's allocator code or whatever.

That's right. If you are wrapping code that is provided by a third party, you should not mark any code as @trusted that makes calls to the third party library. By doing this, you are saying "any third party code I call is memory safe (source: dude just trust me)". That may work in the case where this third party code is set in stone and has been hand-audited by either you or the maintainers (ideally both), but you're accepting any implementation through a template argument. Doing this is extremely dangerous, because you're making memory safety promises about every single Allocator implementation in existence, in the present AND for the future.

What you have to do is leave the functions that make these calls unmarked (no @system, @trusted OR @safe), and allow the compiler to infer it based on the whether the third party implementation is @system/@trusted/@safe. That's the only sane way I can think of to do this.
May 24, 2019
On Friday, 24 May 2019 at 11:41:12 UTC, Atila Neves wrote:
> I'd been holding off on announcing this until DIP1008 actually got implemented, and now it has:
>
> https://code.dlang.org/packages/nogc
>
> This dub package has a @nogc version of `std.conv.text` (which probably isn't as good yet) that, instead of returning a `string` returns an `automem.vector.Vector` of char. This handles managing memory allocation for the exception message itself in `NoGcException`, which does what it says on the tin. Confused? Here's some code:

Ok, so exceptions don't rely on the GC anymore.  That's super cool.  However, they are still classes.  So does that mean they also need RTTI (i.e. TypeInfo)?  BetterC builds and some of use trying to use D in a pay-as-you-go fashion intentionally eliminate RTTI from the runtime.  Is there any way we can take this a bit further to no longer require RTTI?  Do exceptions even necessarily need to be classes?

Thanks,
Mike
« First   ‹ Prev
1 2 3