Hello,

I'm a little confused about what is actually happening when I try to pass a reference, returned by a method that produces the object (associative array), to a setter method which expects a reference. What seems to be happening is that it simply does nothing, as if the setter method is never called.

I'll try to provide some code to highlight the signatures.

The setter method being called on the reference returned by the method that produces the array:

agent.beliefs = combination( ... ); // should receive a reference, and pass this to the setter method.

Combination is a function point to one of two functions, whose signatures look like:

static auto ref consensus( ... )

and the agent's setter method looks like the following:

void beliefs(ref double[int] beliefs)

Now obviously if I remove ref in the above signature, everything works fine, but I am surprised I wasn't getting any kind of warning about the reference because the setter function just doesn't run at all. I tried checking if (beliefs is null) but perhaps this isn't the correct way to check if an associative array's reference is no longer reachable?

Some explanation would be great, thanks!

On Friday, 6 July 2018 at 13:13:43 UTC, Michael wrote:
> static auto ref consensus( ... )

`auto ref` infers the return type from the return statement [1]. So it's not necessarily returning a ref type.

However, I don't think this matters if the only concern you have is that the setter function actually modifies the passed object.

>
> and the agent's setter method looks like the following:
>
> void beliefs(ref double[int] beliefs)
>
> Now obviously if I remove ref in the above signature, everything works fine, but I am surprised I wasn't getting any kind of warning about the reference because the setter function just doesn't run at all. I tried checking if (beliefs is null) but perhaps this isn't the correct way to check if an associative array's reference is no longer reachable?
>
> Some explanation would be great, thanks!

This works for me:

    auto create()
    {
        string[int] dict;
        dict[2] = "hello";
        return dict;
    }

    void modifyNoRef(string[int] m)
    {
         writeln("Address not ref: ", &m);
         m[0] = "modified";
    }

    void modifyRef(ref string[int] m)
    {
         writeln("Address ref: ", &m);
         m[1] = "modified";
    }

    unittest
    {
        auto r = create();
        writeln("Address r: ", &r);
        assert(r.keys.length == 1);
        modifyNoRef(r);
        assert(r.keys.length == 2);	
        modifyRef(r);
        assert(r.keys.length == 3);	
    }

So either with ref or not as parameter storage class the assoc. array is modified. Note the address in the "ref" one is the same as "r" in the unittest.

[1]: https://dlang.org/spec/function.html#auto-ref-functions

On Friday, 6 July 2018 at 14:11:42 UTC, Timoses wrote:
> On Friday, 6 July 2018 at 13:13:43 UTC, Michael wrote:
>> static auto ref consensus( ... )
>
> `auto ref` infers the return type from the return statement [1]. So it's not necessarily returning a ref type.
>
> However, I don't think this matters if the only concern you have is that the setter function actually modifies the passed object.
>
>>
>> and the agent's setter method looks like the following:
>>
>> void beliefs(ref double[int] beliefs)
>>
>> Now obviously if I remove ref in the above signature, everything works fine, but I am surprised I wasn't getting any kind of warning about the reference because the setter function just doesn't run at all. I tried checking if (beliefs is null) but perhaps this isn't the correct way to check if an associative array's reference is no longer reachable?
>>
>> Some explanation would be great, thanks!
>
> This works for me:
>
>     auto create()
>     {
>         string[int] dict;
>         dict[2] = "hello";
>         return dict;
>     }
>
>     void modifyNoRef(string[int] m)
>     {
>          writeln("Address not ref: ", &m);
>          m[0] = "modified";
>     }
>
>     void modifyRef(ref string[int] m)
>     {
>          writeln("Address ref: ", &m);
>          m[1] = "modified";
>     }
>
>     unittest
>     {
>         auto r = create();
>         writeln("Address r: ", &r);
>         assert(r.keys.length == 1);
>         modifyNoRef(r);
>         assert(r.keys.length == 2);	
>         modifyRef(r);
>         assert(r.keys.length == 3);	
>     }
>
> So either with ref or not as parameter storage class the assoc. array is modified. Note the address in the "ref" one is the same as "r" in the unittest.
>
> [1]: https://dlang.org/spec/function.html#auto-ref-functions

Aah yes I had forgotten that I had set it as auto ref. However, even when I set that function to always return a copy of the associative array, it didn't change anything -- the issue always seems to be with the setter method expecting a reference.

So then, supposing it's returning a copy of the object. Given that my setter method was still expecting a reference, is there a reason why it doesn't even invoke the setter unless I change the parameter from expecting a reference to expecting a copy of the object? It's not like the function failed in any way, it just wouldn't run the function at all. Not even while I was printing something right at the top.

On Friday, 6 July 2018 at 14:11:42 UTC, Timoses wrote:
>
> This works for me:
>
>     auto create()
>     {
>         string[int] dict;
>         dict[2] = "hello";
>         return dict;
>     }
>
>     void modifyNoRef(string[int] m)
>     {
>          writeln("Address not ref: ", &m);
>          m[0] = "modified";
>     }
>
>     void modifyRef(ref string[int] m)
>     {
>          writeln("Address ref: ", &m);
>          m[1] = "modified";
>     }
>
>     unittest
>     {
>         auto r = create();
>         writeln("Address r: ", &r);
>         assert(r.keys.length == 1);
>         modifyNoRef(r);
>         assert(r.keys.length == 2);	
>         modifyRef(r);
>         assert(r.keys.length == 3);	
>     }
>
> So either with ref or not as parameter storage class the assoc. array is modified. Note the address in the "ref" one is the same as "r" in the unittest.
>
> [1]: https://dlang.org/spec/function.html#auto-ref-functions


I'm wondering, seeing as it works in the example given above, but not in my case, if this is a weird edge-case with setter member functions?

What I wanted to do was have the "Agent" object set its internal variable to point to the newly created associative array but instead it just seems to go right past the setter function.

On 07/06/2018 07:36 AM, Michael wrote:
> but not in
> my case, if this is a weird edge-case with setter member functions?

This is all very interesting but I'm dying to see the code. :) Can you change Timoses's code to demonstrate your case?

Ali

On Friday, 6 July 2018 at 14:50:39 UTC, Ali Çehreli wrote:
> On 07/06/2018 07:36 AM, Michael wrote:
> > but not in
> > my case, if this is a weird edge-case with setter member
> functions?
>
> This is all very interesting but I'm dying to see the code. :) Can you change Timoses's code to demonstrate your case?
>
> Ali

I'm just trying to do that now.

Here is what I have in terms of code:

class Agent
{
    private
    {
        double[int] mDict;
    }

    // Setter: copy
    void beliefs(ref double[int] dict)
    {
        import std.stdio : writeln;
        writeln("Setter function.");
        this.mDict = dict;
    }

    // Getter
    auto ref beliefs()
    {
        return this.mDict;
    }
}

class Operator
{
    static auto ref create()
    {
        double[int] dict;
        dict[2] = 1.0;
        dict[1] = 0.0;
        return dict;
    }
}

unittest
{
    import std.stdio : writeln;

    Agent a = new Agent();

    a.beliefs = Operator.create();
    assert(a.beliefs.keys.length == 2);

    writeln(a.beliefs);
}


and here is the output WITH "ref" in beliefs's signature before double[int] dict:

➜  dempshaf git:(master) ✗ rdmd -I../ -unittest -main bug_test.d
[2:1, 1:0]

and without "ref":

➜  dempshaf git:(master) ✗ rdmd -I../ -unittest -main bug_test.d
Setter function.
[2:1, 1:0]


I'm surprised it's showing the correct value of the associative array, but hopefully this shows my point that it's not printing for some reason, when the parameter is ref.

On Friday, 6 July 2018 at 15:14:01 UTC, Michael wrote:
> On Friday, 6 July 2018 at 14:50:39 UTC, Ali Çehreli wrote:
>> [...]
>
> I'm just trying to do that now.
>
> Here is what I have in terms of code:
>
> [...]

This is definitely to do with my use of the setter syntax, which maybe I am misunderstanding? Because if I change it to a normal function call like so:

a.beliefs(Operator.create());

then it complains if I use ref, and doesn't complain if I don't.

On Friday, 6 July 2018 at 15:14:01 UTC, Michael wrote:
> class Agent
> {
>     private
>     {
>         double[int] mDict;
>     }
>
>     // Setter: copy
>     void beliefs(ref double[int] dict)
>     {
>         import std.stdio : writeln;
>         writeln("Setter function.");
>         this.mDict = dict;
>     }
>
>     // Getter
>     auto ref beliefs()
>     {
>         return this.mDict;
>     }
> }
>
> class Operator
> {
>     static auto ref create()
>     {
>         double[int] dict;
>         dict[2] = 1.0;
>         dict[1] = 0.0;
>         return dict;
>     }
> }

Throw in a writeln statement in getter and be amazed : D

    class Agent
    {
        private
        {
            double[int] mDict;
        }

        // Setter: copy
        void beliefs(ref double[int] dict)
        {
            import std.stdio : writeln;
            writeln("Setter function.");
            this.mDict = dict;
        }

        // Getter
        auto ref beliefs()
        {
            writeln("getter");
            return this.mDict;
        }
    }

    class Operator
    {
        static auto ref create()
        {
            double[int] dict;
            dict[2] = 1.0;
            dict[1] = 0.0;
            return dict;
        }
    }

    unittest
    {
        import std.stdio : writeln;

        Agent a = new Agent();

        writeln("Statement 1");
        a.beliefs = Operator.create();
        writeln("Statement 2");
        assert(a.beliefs.keys.length == 2);

        writeln("Statement 3");
        writeln(a.beliefs);
    }

If you remove the ref then "Statement 1" will use the setter method. Otherwise it calls the "auto ref" function to get a reference of 'mDict' and assign it a value. Essentially

        // Getter
        auto ref beliefs()
        {
            return this.mDict;
        }

makes your "private" variable not private any more, since you are leaking a reference to it. So any program calling "a.beliefs" can get a reference to your private data and modify it.

My guess is that (if ref is removed from the setter) it serves as a @property function:
https://dlang.org/spec/function.html#property-functions
So this allows the function to work in a statement like:
a.beliefs = <Whatever>

And apparently it is prioritized over the "ref beliefs()" function.


I guess it is nicer to actually use the "setter" function, as it allows you to inspect what the private data will be assigned to and take appropriate measures, whereas with the "ref beliefs()" method the private data is open for any manipulation out of your control.

On Friday, 6 July 2018 at 15:37:25 UTC, Timoses wrote:
> On Friday, 6 July 2018 at 15:14:01 UTC, Michael wrote:
>> class Agent
>> {
>>     private
>>     {
>>         double[int] mDict;
>>     }
>>
>>     // Setter: copy
>>     void beliefs(ref double[int] dict)
>>     {
>>         import std.stdio : writeln;
>>         writeln("Setter function.");
>>         this.mDict = dict;
>>     }
>>
>>     // Getter
>>     auto ref beliefs()
>>     {
>>         return this.mDict;
>>     }
>> }
>>
>> class Operator
>> {
>>     static auto ref create()
>>     {
>>         double[int] dict;
>>         dict[2] = 1.0;
>>         dict[1] = 0.0;
>>         return dict;
>>     }
>> }
>
> Throw in a writeln statement in getter and be amazed : D
>
>     class Agent
>     {
>         private
>         {
>             double[int] mDict;
>         }
>
>         // Setter: copy
>         void beliefs(ref double[int] dict)
>         {
>             import std.stdio : writeln;
>             writeln("Setter function.");
>             this.mDict = dict;
>         }
>
>         // Getter
>         auto ref beliefs()
>         {
>             writeln("getter");
>             return this.mDict;
>         }
>     }
>
>     class Operator
>     {
>         static auto ref create()
>         {
>             double[int] dict;
>             dict[2] = 1.0;
>             dict[1] = 0.0;
>             return dict;
>         }
>     }
>
>     unittest
>     {
>         import std.stdio : writeln;
>
>         Agent a = new Agent();
>
>         writeln("Statement 1");
>         a.beliefs = Operator.create();
>         writeln("Statement 2");
>         assert(a.beliefs.keys.length == 2);
>
>         writeln("Statement 3");
>         writeln(a.beliefs);
>     }
>
> If you remove the ref then "Statement 1" will use the setter method. Otherwise it calls the "auto ref" function to get a reference of 'mDict' and assign it a value. Essentially
>
>         // Getter
>         auto ref beliefs()
>         {
>             return this.mDict;
>         }
>
> makes your "private" variable not private any more, since you are leaking a reference to it. So any program calling "a.beliefs" can get a reference to your private data and modify it.
>
> My guess is that (if ref is removed from the setter) it serves as a @property function:
> https://dlang.org/spec/function.html#property-functions
> So this allows the function to work in a statement like:
> a.beliefs = <Whatever>
>
> And apparently it is prioritized over the "ref beliefs()" function.
>
>
> I guess it is nicer to actually use the "setter" function, as it allows you to inspect what the private data will be assigned to and take appropriate measures, whereas with the "ref beliefs()" method the private data is open for any manipulation out of your control.

How strange! Nice find though, thanks for your help with this, I really appreciate it as this had been bugging me for a while.

So essentially, what is happening is, if the setter method expects a reference, then because the create() function isn't returning a reference but a copy, it actually does the following:

a.beliefs = Operator.create();

receives an object from create(), while a.beliefs retrieves a reference to agent.mBeliefs, and then effectively sets this reference to point to the object copied by create()?

Also, yes, I am using the setter method to play around with the precision of the double values, and do some normalising. I always want it to access the setter method, but I had hoped that, given it's an associative array, that the creation of the object in create() would simply create it on the heap and I could pass back a reference. It seems that I was incorrect, as if I set create() to explicitly return a reference, it complains about returning a reference to a local variable. Any advice on the best way to pass it as a reference?

On Friday, 6 July 2018 at 15:33:18 UTC, Michael wrote:
>
> This is definitely to do with my use of the setter syntax, which maybe I am misunderstanding? Because if I change it to a normal function call like so:
>
> a.beliefs(Operator.create());
>
> then it complains if I use ref, and doesn't complain if I don't.

You can try removing the "auto" from the Operator class method

// Error: returning dict escapes a reference to local variable dict
false
        static ref create()
        {
            double[int] dict;
            dict[2] = 1.0;
            dict[1] = 0.0;
            return dict;
        }

That indicates that 'dict' is actually returned by value (at least the reference). This sounds a bit confusing.
AFAIK, in above code a reference 'dict' is created pointing towards memory that stores that dictionary entries. The reference itself is a "pointer"(?) which will "die" when running out of the method's scope. This pointer lives on the function's stack. So returning this "pointer" (essentially a value type, e.g. int) will return it by value.

You could return a reference like so:

    class Operator
    {
        static double[int] hdict;
        static this()
        {
             hdict[2] = 1.0;
             hdict[1] = 0.0;
        }
        static ref create()
        {
            return hdict;
        }
    }

then

> a.beliefs(Operator.create());

should work on the "ref" setter.