Thread overview | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
August 06, 2018 GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Hello, I have this struct: struct S{ uint kind; void[N] data_; } Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data. If is GC disabled then program run fine. But when is GC enabled then it fail randomly. If the definition of S look like this: struct S{ void[N] data_; uint kind; } then program run fine with GC.enable. Whats the problem? Something with alignment? |
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to vit | On 8/6/18 2:22 PM, vit wrote: > Hello, > I have this struct: > > struct S{ > uint kind; > void[N] data_; define "N" > > } > > Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data. > If is GC disabled then program run fine. But when is GC enabled then it fail randomly. how does it fail? -Steve |
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Monday, 6 August 2018 at 18:28:11 UTC, Steven Schveighoffer wrote:
> On 8/6/18 2:22 PM, vit wrote:
>> Hello,
>> I have this struct:
>>
>> struct S{
>> uint kind;
>> void[N] data_;
>
> define "N"
>
>>
>> }
>>
>> Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data.
>> If is GC disabled then program run fine. But when is GC enabled then it fail randomly.
>
> how does it fail?
>
> -Steve
private auto sizeOf(T)(){return T.sizeof;}
struct ExprImpl(Ts...){
enum N = max(staticMap!(sizeOf, Ts));
invariant(kind_ != 0);
uint kind_ = 0;
void[N] data_;
this(T)(auto ref T x){/+emplace T to data_ and change kind_ to something != 0+/}
}
Ts == structs
data change without triggering invariant after allocation in other part of program.
|
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to vit | On Monday, 6 August 2018 at 18:22:24 UTC, vit wrote:
> Hello,
> I have this struct:
>
> struct S{
> uint kind;
> void[N] data_;
>
> }
>
> Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data.
> If is GC disabled then program run fine. But when is GC enabled then it fail randomly.
>
> If the definition of S look like this:
>
> struct S{
> void[N] data_;
> uint kind;
> }
>
> then program run fine with GC.enable.
>
> Whats the problem? Something with alignment?
Probably. Try something like:
struct S
{
uint kind;
align((void *).alignof) void[N] data_;
}
And see if it solves the problem.
|
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to nkm1 | On Monday, 6 August 2018 at 19:17:58 UTC, nkm1 wrote:
> On Monday, 6 August 2018 at 18:22:24 UTC, vit wrote:
>> Hello,
>> I have this struct:
>>
>> struct S{
>> uint kind;
>> void[N] data_;
>>
>> }
>>
>> Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data.
>> If is GC disabled then program run fine. But when is GC enabled then it fail randomly.
>>
>> If the definition of S look like this:
>>
>> struct S{
>> void[N] data_;
>> uint kind;
>> }
>>
>> then program run fine with GC.enable.
>>
>> Whats the problem? Something with alignment?
>
> Probably. Try something like:
>
> struct S
> {
> uint kind;
> align((void *).alignof) void[N] data_;
> }
>
> And see if it solves the problem.
align((void *).alignof) work, thanks.
|
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to vit | On 8/6/18 2:59 PM, vit wrote: > On Monday, 6 August 2018 at 18:28:11 UTC, Steven Schveighoffer wrote: >> On 8/6/18 2:22 PM, vit wrote: >>> Hello, >>> I have this struct: >>> >>> struct S{ >>> uint kind; >>> void[N] data_; >> >> define "N" >> >>> >>> } >>> >>> Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data. >>> If is GC disabled then program run fine. But when is GC enabled then it fail randomly. >> >> how does it fail? >> > > > private auto sizeOf(T)(){return T.sizeof;} Hm... wouldn't enum sizeOf(T) = T.sizeof work better? > > struct ExprImpl(Ts...){ > enum N = max(staticMap!(sizeOf, Ts)); This is clever! > > invariant(kind_ != 0); > uint kind_ = 0; > void[N] data_; > > this(T)(auto ref T x){/+emplace T to data_ and change kind_ to something != 0+/} > } > > Ts == structs > > > > data change without triggering invariant after allocation in other part of program. Most definitely this is alignment problem. Here is what I *think* is happening: 1. You are constructing one of these structs, and storing a pointer as the T type. 2. You are on a 64-bit CPU. 3. The pointer is misaligned on the CPU, so when the GC scans this struct to see if it's pointing at anything, it sees one half as the kind_ value, and the other half is half of the pointer. 4. It misses the object being pointed at by the T inside the struct, and collects it, leaving a dangling pointer. 5. Memory corruption. when you put the void[N] member *first*, it can properly align the item (most cases where the compiler is placing data, it starts out aligned) but this does not guarantee you have proper alignment, as void[N] has no alignment constraints. I'd recommend instead, changing the uint kind_ to a size_t. This not only aligns the void[N] to size_t size, which should put any pointers in the right place, but it also makes sure the entire struct is aligned. -Steve |
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On 8/6/18 3:43 PM, Steven Schveighoffer wrote: > On 8/6/18 2:59 PM, vit wrote: >> On Monday, 6 August 2018 at 18:28:11 UTC, Steven Schveighoffer wrote: >>> On 8/6/18 2:22 PM, vit wrote: >>>> Hello, >>>> I have this struct: >>>> >>>> struct S{ >>>> uint kind; >>>> void[N] data_; >>> >>> define "N" >>> >>>> >>>> } >>>> >>>> Instances of struct S are allocated by standard GC new and S.data_ can contain pointers/ranges to GC allocated data. >>>> If is GC disabled then program run fine. But when is GC enabled then it fail randomly. >>> >>> how does it fail? >>> >> >> >> private auto sizeOf(T)(){return T.sizeof;} > > Hm... wouldn't enum sizeOf(T) = T.sizeof work better? > >> >> struct ExprImpl(Ts...){ >> enum N = max(staticMap!(sizeOf, Ts)); > > This is clever! > >> >> invariant(kind_ != 0); >> uint kind_ = 0; >> void[N] data_; >> >> this(T)(auto ref T x){/+emplace T to data_ and change kind_ to something != 0+/} >> } >> >> Ts == structs >> >> >> >> data change without triggering invariant after allocation in other part of program. > > Most definitely this is alignment problem. > > Here is what I *think* is happening: > > 1. You are constructing one of these structs, and storing a pointer as the T type. > 2. You are on a 64-bit CPU. > 3. The pointer is misaligned on the CPU, so when the GC scans this struct to see if it's pointing at anything, it sees one half as the kind_ value, and the other half is half of the pointer. > 4. It misses the object being pointed at by the T inside the struct, and collects it, leaving a dangling pointer. > 5. Memory corruption. > > when you put the void[N] member *first*, it can properly align the item (most cases where the compiler is placing data, it starts out aligned) but this does not guarantee you have proper alignment, as void[N] has no alignment constraints. > > I'd recommend instead, changing the uint kind_ to a size_t. This not only aligns the void[N] to size_t size, which should put any pointers in the right place, but it also makes sure the entire struct is aligned. BTW, is there a reason you aren't just using Algebraic? https://dlang.org/phobos/std_variant.html#.Algebraic -Steve |
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steven Schveighoffer | On Monday, 6 August 2018 at 19:56:03 UTC, Steven Schveighoffer wrote:
>
> BTW, is there a reason you aren't just using Algebraic?
>
> https://dlang.org/phobos/std_variant.html#.Algebraic
>
> -Steve
primarily visit for Algebraic isn't pure, @nogc, @safe, nothrow.
|
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to vit | On Monday, 6 August 2018 at 20:22:36 UTC, vit wrote: > On Monday, 6 August 2018 at 19:56:03 UTC, Steven Schveighoffer wrote: >> >> BTW, is there a reason you aren't just using Algebraic? >> >> https://dlang.org/phobos/std_variant.html#.Algebraic >> >> -Steve > > primarily visit for Algebraic isn't pure, @nogc, @safe, nothrow. I wrote the 'sumtype' package to solve this exact problem: https://code.dlang.org/packages/sumtype |
August 06, 2018 Re: GC and void[N] in struct | ||||
---|---|---|---|---|
| ||||
Posted in reply to Paul Backus | On Monday, 6 August 2018 at 21:23:36 UTC, Paul Backus wrote: > On Monday, 6 August 2018 at 20:22:36 UTC, vit wrote: >> On Monday, 6 August 2018 at 19:56:03 UTC, Steven Schveighoffer wrote: >>> >>> BTW, is there a reason you aren't just using Algebraic? >>> >>> https://dlang.org/phobos/std_variant.html#.Algebraic >>> >>> -Steve >> >> primarily visit for Algebraic isn't pure, @nogc, @safe, nothrow. > > I wrote the 'sumtype' package to solve this exact problem: > > https://code.dlang.org/packages/sumtype I'm using simpler (and less powerful) version with different visit/visitor syntax: struct Foo{ string foo; } struct Bar1{ string bar; } struct Bar2{ string bar; } void main()pure nothrow @safe @nogc{ Variant!(true, Foo, Bar1, Bar2) var; ///Nullable == true static visit(T)(auto ref const(T) x, string def = ""){ import std.experimental.all; static if(is(T == Foo)){ return x.foo; } else static if(false || is(T == Bar1) || is(T == Bar2) ){ return x.bar; } else static if(is(T == typeof(null))){ return def; } else static assert(0, "no impl"); } assert(var.isa!null); assert(var.visitor!visit("null") == "null"); var = Foo("foo"); assert(var.isa!Foo); assert(var.visitor!visit == "foo"); var = Bar1("bar1"); assert(var.isa!Bar1); assert(var.visitor!visit == "bar1"); var = Bar2("bar2"); assert(var.isa!Bar2); assert(var.visitor!visit == "bar2"); var = null; assert(var.isa!null); auto var2 = Variant!(false, Foo, Bar1, Bar2)(Foo("foo"));///Nullable == false assert(var2.visitor!visit == "foo"); assert(var2.as!Foo.foo == "foo"); ///var2 = null; //error, variant is not null } full code: https://dpaste.dzfl.pl/d83ecca23694 |
Copyright © 1999-2021 by the D Language Foundation