Jump to page: 1 2
Thread overview
January 07

I was doing a completely functional code, then I noticed some variable changing abruptly to a strange value. After the first run in a function, the value completely changed, I've reduced and found that I was able to actually modify .init.

struct MyTest
{
    string[] dirs = ["source"];
}

void DoIt(ref MyTest a )
{
    a.dirs[0] = null;
}

void main()
{
    MyTest t;
    DoIt(t);
    MyTest t2;
    assert(t2.dirs == MyTest.init.dirs); //Crashes
}

Looks like everyone knew this bug existed, but it caused me waste 4 hours trying to find when did it happen. I can't understand why this has not been fixed yet, but this leads to extremely erratic behaviour, which I'm going to do an ugly workaround.

Since we are on a year of stability, I think getting this kind of bug fixed could be a priority.

January 08
On Sunday, 7 January 2024 at 23:12:30 UTC, Hipreme wrote:
> ...

For what it seems this happens only with arrays?

This happened with for example int[] a; but when I tried with other types (Non-arrays) like (string, int) this problem didn't occur.

Since like in this example string arrays are pointers, I think something was messed-up with the address.

Matheus.
January 08
On Monday, 8 January 2024 at 04:14:48 UTC, matheus wrote:
> ...
> Since like in this example string arrays are pointers, I think something was messed-up with the address.
>

Correction: Something is messing-up with the address.

Matheus.


January 07
On Sunday, January 7, 2024 9:14:48 PM MST matheus via Digitalmars-d wrote:
> On Sunday, 7 January 2024 at 23:12:30 UTC, Hipreme wrote:
> > ...
>
> For what it seems this happens only with arrays?
>
> This happened with for example int[] a; but when I tried with
> other types (Non-arrays) like (string, int) this problem didn't
> occur.
>
> Since like in this example string arrays are pointers, I think something was messed-up with the address.
>
> Matheus.

Well, to an extent, the problem here is simply that the member variable is a type with mutable indirections. The dynamic array itself is just a pointer and a length, and mutating what it points to like the OP's example doesn't actually mutate the init value. In order to avoid that sort of mutation, initializing the struct would need to give you a deep copy rather than a shallow copy. Other member variables with mutable indirections would have similar problems. For instance, all of the assertions in this code pass:

class C
{
    string foo = "foo";
}

struct S
{
    C c = new C;
}

void main()
{
    S s1;
    assert(s1.c.foo == "foo");
    assert(s1.c.foo == S.init.c.foo);

    S s2;
    s2.c.foo = "bar";
    assert(s2.c.foo == "bar");

    assert(s1.c.foo == "bar");
    assert(S.init.c.foo == "bar");

    assert((cast(void*)s1.c) is cast(void*)s2.c);
    assert((cast(void*)s1.c) is cast(void*)S.init.c);

    S s3;
    s3.c = new C;
    assert(s3.c.foo == "foo");

    assert(s1.c.foo == "bar");
    assert(s2.c.foo == "bar");
    assert(S.init.c.foo == "bar");

    assert((cast(void*)s1.c) is cast(void*)s2.c);
    assert((cast(void*)s1.c) is cast(void*)S.init.c);

    assert((cast(void*)s3.c) !is cast(void*)S.init.c);
}

The init value itself is never mutated, but what it points to is. Someone could interpret it like the init value had mutated, because what it's pointed to changed, and therefore, accessing the values via init then results in different values that were there at the start of the program, but technically, init itself never changed.

As such, the OP should expect that mutating values in the array would affect any other variable when it's default-initialized, and I don't think that it's realistic for it to work any other way, because that requires making a deep copy of the init value rather than a shallow copy, and that's not something that can be done in the general case, because D doesn't have a mechanism for that.

However, in spite of all of that, something weird is going on with the OP's example, because while variables that are created after the element of the array has been mutated see that mutation, the init value itself does not. So, it would appear that the array in the init value itself somehow ends up pointing to a different block of memory than the array does when the struct is default initilaized. This example shows the same problem

struct S
{
    int[] arr = [1, 2, 3];
}

void main()
{
    S s1;
    assert(s1.arr == [1, 2, 3]);
    assert(s1.arr == S.init.arr);

    S s2;
    s2.arr[0] = 42;
    assert(s2.arr == [42, 2, 3]);

    assert(s1.arr == [42, 2, 3]);
    assert(S.init.arr == [1, 2, 3]);
}

and if I were to change main to

void main()
{
    import std.stdio;
    writeln(S.init.arr.ptr);

    S s1;
    writeln(s1.arr.ptr);

    S s2;
    writeln(s2.arr.ptr);

    writeln(S.init.arr.ptr);
}

running it on my computer results in

C378D421000
2C3F08
2C3F08
C378D421010

So, for some reason, the ptr of the array in the init value has a different address from the one in the struct's after they've been default-initialized, but the struct's get the same pointer value.

This is in stark contrast to my previous example where the member variable that was a class reference ended up with the address being the same for both the init value and the default-initialized structs. So, it would appear that the compiler or runtime is doing something different with dynamic arrays than it is with classes, and the behavior definitely seems wrong, because it means that a default-initialized struct does not match its init value - and this without worrying about mutating anything. e.g. this assertion fails when it should never fail

struct S
{
    int[] arr = [1, 2, 3];
}

void main()
{
    S s;
    assert(s is S.init);
}

So, I would say that there is definitely a bug here with regards to init values when a struct has a member variable which is a dynamic array.

However, ultimately, that's not really what the OP seems to be complaining about. Rather, he seems to be complaining about the fact that if you mutate a member variable with mutable indirections which were not null in the init value, then the changes to those mutable indirections are visible via the init value, and I don't think that that's something that's ever going to change, because it would require that default initialization do a deep copy rather than a shallow copy. Obviously, it can be surprising if it's not something that you've run into or thought through before, but D doesn't have a generic way to do deep copies, and you wouldn't necessarily want a deep copy in all cases anyway, so even if we could make it do a deep copy, that wouldn't necessarily be a good change for the language as a whole, though it would certainly fix the issue that the OP ran into.

Arguably, what the OP needs is a default constructor rather than an init value, but that's not the way that D is designed, and it would be problematic with some of its features to use default construction instead (though this is far from the only case where not having default construction for structs can be annoying).

- Jonathan M Davis



January 07
On Sunday, January 7, 2024 10:22:20 PM MST Jonathan M Davis via Digitalmars-d wrote:
> So, I would say that there is definitely a bug here with regards to init values when a struct has a member variable which is a dynamic array.

https://issues.dlang.org/show_bug.cgi?id=24324

- Jonathan M Davis



January 08
On Monday, 8 January 2024 at 05:22:20 UTC, Jonathan M Davis wrote:
> On Sunday, January 7, 2024 9:14:48 PM MST matheus via Digitalmars-d wrote:
>> On Sunday, 7 January 2024 at 23:12:30 UTC, Hipreme wrote:
>> > ...
>>
>> For what it seems this happens only with arrays?
>>
>> This happened with for example int[] a; but when I tried with
>> other types (Non-arrays) like (string, int) this problem didn't
>> occur.
>>
>> Since like in this example string arrays are pointers, I think something was messed-up with the address.
>>
>> Matheus.
>
> Well, to an extent, the problem here is simply that the member variable is a type with mutable indirections. The dynamic array itself is just a pointer and a length, and mutating what it points to like the OP's example doesn't actually mutate the init value. In order to avoid that sort of mutation, initializing the struct would need to give you a deep copy rather than a shallow copy. Other member variables with mutable indirections would have similar problems. For instance, all of the assertions in this code pass:
>
> class C
> {
>     string foo = "foo";
> }
>
> struct S
> {
>     C c = new C;
> }
>
> void main()
> {
>     S s1;
>     assert(s1.c.foo == "foo");
>     assert(s1.c.foo == S.init.c.foo);
>
>     S s2;
>     s2.c.foo = "bar";
>     assert(s2.c.foo == "bar");
>
>     assert(s1.c.foo == "bar");
>     assert(S.init.c.foo == "bar");
>
>     assert((cast(void*)s1.c) is cast(void*)s2.c);
>     assert((cast(void*)s1.c) is cast(void*)S.init.c);
>
>     S s3;
>     s3.c = new C;
>     assert(s3.c.foo == "foo");
>
>     assert(s1.c.foo == "bar");
>     assert(s2.c.foo == "bar");
>     assert(S.init.c.foo == "bar");
>
>     assert((cast(void*)s1.c) is cast(void*)s2.c);
>     assert((cast(void*)s1.c) is cast(void*)S.init.c);
>
>     assert((cast(void*)s3.c) !is cast(void*)S.init.c);
> }
>
> The init value itself is never mutated, but what it points to is. Someone could interpret it like the init value had mutated, because what it's pointed to changed, and therefore, accessing the values via init then results in different values that were there at the start of the program, but technically, init itself never changed.
>
> As such, the OP should expect that mutating values in the array would affect any other variable when it's default-initialized, and I don't think that it's realistic for it to work any other way, because that requires making a deep copy of the init value rather than a shallow copy, and that's not something that can be done in the general case, because D doesn't have a mechanism for that.
>
> However, in spite of all of that, something weird is going on with the OP's example, because while variables that are created after the element of the array has been mutated see that mutation, the init value itself does not. So, it would appear that the array in the init value itself somehow ends up pointing to a different block of memory than the array does when the struct is default initilaized. This example shows the same problem
>
> struct S
> {
>     int[] arr = [1, 2, 3];
> }
>
> void main()
> {
>     S s1;
>     assert(s1.arr == [1, 2, 3]);
>     assert(s1.arr == S.init.arr);
>
>     S s2;
>     s2.arr[0] = 42;
>     assert(s2.arr == [42, 2, 3]);
>
>     assert(s1.arr == [42, 2, 3]);
>     assert(S.init.arr == [1, 2, 3]);
> }
>
> and if I were to change main to
>
> void main()
> {
>     import std.stdio;
>     writeln(S.init.arr.ptr);
>
>     S s1;
>     writeln(s1.arr.ptr);
>
>     S s2;
>     writeln(s2.arr.ptr);
>
>     writeln(S.init.arr.ptr);
> }
>
> running it on my computer results in
>
> C378D421000
> 2C3F08
> 2C3F08
> C378D421010
>
> So, for some reason, the ptr of the array in the init value has a different address from the one in the struct's after they've been default-initialized, but the struct's get the same pointer value.
>
> This is in stark contrast to my previous example where the member variable that was a class reference ended up with the address being the same for both the init value and the default-initialized structs. So, it would appear that the compiler or runtime is doing something different with dynamic arrays than it is with classes, and the behavior definitely seems wrong, because it means that a default-initialized struct does not match its init value - and this without worrying about mutating anything. e.g. this assertion fails when it should never fail
>
> struct S
> {
>     int[] arr = [1, 2, 3];
> }
>
> void main()
> {
>     S s;
>     assert(s is S.init);
> }
>
> So, I would say that there is definitely a bug here with regards to init values when a struct has a member variable which is a dynamic array.
>
> However, ultimately, that's not really what the OP seems to be complaining about. Rather, he seems to be complaining about the fact that if you mutate a member variable with mutable indirections which were not null in the init value, then the changes to those mutable indirections are visible via the init value, and I don't think that that's something that's ever going to change, because it would require that default initialization do a deep copy rather than a shallow copy. Obviously, it can be surprising if it's not something that you've run into or thought through before, but D doesn't have a generic way to do deep copies, and you wouldn't necessarily want a deep copy in all cases anyway, so even if we could make it do a deep copy, that wouldn't necessarily be a good change for the language as a whole, though it would certainly fix the issue that the OP ran into.
>
> Arguably, what the OP needs is a default constructor rather than an init value, but that's not the way that D is designed, and it would be problematic with some of its features to use default construction instead (though this is far from the only case where not having default construction for structs can be annoying).
>
> - Jonathan M Davis

Force default values to be typed immutable?
January 08

On Sunday, 7 January 2024 at 23:12:30 UTC, Hipreme wrote:

>
struct MyTest
{
    string[] dirs = ["source"];
}

I have argued that initializers of non-static members that allocate should be an error [1]. Mike and Steven agree [2, 3].

[1] https://forum.dlang.org/post/ogvubzgprghefclgluce@forum.dlang.org
[2] https://forum.dlang.org/post/wvrasioewzbqrqsufwxd@forum.dlang.org
[3] https://forum.dlang.org/post/t7vm2o$p4q$1@digitalmars.com

I will add pointers to https://issues.dlang.org/show_bug.cgi?id=24324.

-- Bastiaan.

January 08

On Monday, 8 January 2024 at 11:53:52 UTC, Bastiaan Veelo wrote:

>

On Sunday, 7 January 2024 at 23:12:30 UTC, Hipreme wrote:

>
struct MyTest
{
    string[] dirs = ["source"];
}

I have argued that initializers of non-static members that allocate should be an error [1]. Mike and Steven agree [2, 3].

[1] https://forum.dlang.org/post/ogvubzgprghefclgluce@forum.dlang.org
[2] https://forum.dlang.org/post/wvrasioewzbqrqsufwxd@forum.dlang.org
[3] https://forum.dlang.org/post/t7vm2o$p4q$1@digitalmars.com

I will add pointers to https://issues.dlang.org/show_bug.cgi?id=24324.

-- Bastiaan.

I have 2 thoughts on how it could be solved:

  • Make it an error
  • Make a copy of a shared instance

Currently, the solution for that is doing a copy of a shared instance anyway. If it impacts the performance, that is on the user. Having this erratic behavior is the only thing that can't continue. I've lost a plenty of time in a big project.

And no, almost no one has the entire spec in mind. Even more to getting those particular behaviors. The spec can be precise, but it doesn't mean it is correct to work like that.
People will continue falling on the same stone, early or late.

January 08
On Monday, January 8, 2024 3:57:52 AM MST FeepingCreature via Digitalmars-d wrote:
> > However, ultimately, that's not really what the OP seems to be complaining about. Rather, he seems to be complaining about the fact that if you mutate a member variable with mutable indirections which were not null in the init value, then the changes to those mutable indirections are visible via the init value, and I don't think that that's something that's ever going to change, because it would require that default initialization do a deep copy rather than a shallow copy. Obviously, it can be surprising if it's not something that you've run into or thought through before, but D doesn't have a generic way to do deep copies, and you wouldn't necessarily want a deep copy in all cases anyway, so even if we could make it do a deep copy, that wouldn't necessarily be a good change for the language as a whole, though it would certainly fix the issue that the OP ran into.
> >
> > Arguably, what the OP needs is a default constructor rather than an init value, but that's not the way that D is designed, and it would be problematic with some of its features to use default construction instead (though this is far from the only case where not having default construction for structs can be annoying).
>
> Force default values to be typed immutable?

That would require that there be a general way to copy an immutable value to a mutable one, because the object must be mutable after it's been default-initialized.

Also, if any sort of deep-copying occurs, then you end up in the bizarre situation where code like

S s;
assert(s is S.init);

fails, which is also problematic.

Bastiaan's suggestion to make such cases illegal is probably the only sane way to prevent these kind of issues, though I think that it would have to be an error to have mutable indirections with an allocation rather than illegal to point to any allocations; otherwise, stuff like SysTime's member which is Rebindable(immutable TimeZone) becomes illegal.

- Jonathan M Davis



January 08
On Sun, Jan 07, 2024 at 11:12:30PM +0000, Hipreme via Digitalmars-d wrote:
> I was doing a completely functional code, then I noticed some variable changing abruptly to a strange value. After the first run in a function, the value completely changed, I've reduced and found that I was able to actually modify `.init`.
> 
> ```d
> struct MyTest
> {
>     string[] dirs = ["source"];
> }

Using initializer syntax to initialize mutable array members is generally not recommended, because it does not do what you expect. It does NOT allocate a new array per instance of MyTest; instead, it stores the array once in the program's preinit data segment, and sets the default value of .dirs to point to that instance.  This is generally harmless if the array is immutable, but if the data is mutable, you're probably in for a surprise.

If you want every instance of the struct to have a fresh copy of the array, use a ctor instead.

(Whether the current behaviour should be changed is up for debate, though. This definitely isn't the first time users have run into this. In fact just today somebody else asked the same question on D.learn. So it's definitely in the territory of "does not do the expected thing", which is an indication that the default behaviour was poorly chosen.)


T

-- 
If blunt statements had a point, they wouldn't be blunt...
« First   ‹ Prev
1 2