On Monday, 14 August 2017 at 19:22:23 UTC, Steven Schveighoffer wrote:
> On 8/13/17 11:40 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
>> On Saturday, August 12, 2017 18:57:44 Arek via Digitalmars-d-learn wrote:
>>> I have the folowing problem:
>>> I like to envelope the class object in struct to control the
>>> destruction moment and then send this object to another
>>> thread/fiber (or task, cause I use vibe-d).
>>>
>>> I can't find any method to make it working. Any ideas?
>> 
>> Unfortunately, send and receive do not currently work with shared because of
>> issues with Variant, which they use internally.
>
> This can't be a correct statement. This is the whole point of shared.
>
> -Steve

First of all, I'm not native English speaker, so forgive me possible misunderstanding

In my opinion the whole problem of 'shared' is that when I use send/receive operation (I believe it's should be something similar to go's channels or erlang messages) I do not want to share the memory so any "shared" concept is useless.

What I really would like to get is possibility to make:

// in first thread
Unique!MyObject msg = new MyObject()
send(msg.release);

// in other thread:
auto msg = receiveOnly!(Unique!MyObject)();


My object disappears in "send" operation and its new instance is constructed on receive.
If I can ensure the uniqueness of the object, there is no need to "share" it or synchronize the access.

But now such operation is impossible and the compiler forces me to cope with shared if MyObject has any references to other areas of memory.

Anyway, thanks for all replies.

Arek

On Monday, August 14, 2017 15:22:23 Steven Schveighoffer via Digitalmars-d- learn wrote:
> On 8/13/17 11:40 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
> > On Saturday, August 12, 2017 18:57:44 Arek via Digitalmars-d-learn
wrote:
> >> I have the folowing problem:
> >> I like to envelope the class object in struct to control the
> >> destruction moment and then send this object to another
> >> thread/fiber (or task, cause I use vibe-d).
> >>
> >> I can't find any method to make it working. Any ideas?
> >
> > Unfortunately, send and receive do not currently work with shared because of issues with Variant, which they use internally.
>
> This can't be a correct statement. This is the whole point of shared.

What's incorrect about it? It's a longstanding issue that because Variant can't contain shared data, send and receive do not work with shared. You can send and receive mutable data with no indirections, and AFAIK, you can send immutable data (though the OP is apparently having problems with that, so I guess that that doesn't work completely, though I definitely recall doing so previously), but you get compilation errors if you try to send shared data. For instance, this code

import std.concurrency;

void main()
{
    static void func(Tid parent)
    {
        auto received = receiveOnly!(shared int[string]);
    }

    shared int[string] aa;

    auto tid = spawn(&func, thisTid);
    send(tid, aa);
}

will fail to compile with

/usr/local/include/dmd/std/variant.d(625): Error: function
core.stdc.string.memcpy (return scope void* s1, scope const(void*) s2, ulong
n) is not callable using argument types (ubyte[32]*, shared(int[string])*,
ulong)
/usr/local/include/dmd/std/variant.d(625): Error: function
core.stdc.string.memcpy (return scope void* s1, scope const(void*) s2, ulong
n) is not callable using argument types (ubyte[32]*, shared(int)*, ulong)
/usr/local/include/dmd/std/variant.d(424): Error: template instance
std.variant.VariantN!32LU.VariantN.opAssign!(shared(int)) error
instantiating
/usr/local/include/dmd/std/variant.d(645):        instantiated from here:
handler!(shared(int[string]))
/usr/local/include/dmd/std/variant.d(559):        instantiated from here:
opAssign!(shared(int[string]))
/usr/local/include/dmd/std/concurrency.d(101):        instantiated from
here: __ctor!(shared(int[string]))
/usr/local/include/dmd/std/concurrency.d(606):        ... (2 instantiations,
-v to show) ...
/usr/local/include/dmd/std/concurrency.d(576):        instantiated from
here: _send!(shared(int[string]))
q.d(13):        instantiated from here: send!(shared(int[string]))

whereas if you use something like int, it has no problem. It won't even compile if you give it a shared int, and that arguably should just be converting to and from int, since shared int has no indirections.

There are at least two bugs reported on the issue:

https://issues.dlang.org/show_bug.cgi?id=13262 https://issues.dlang.org/show_bug.cgi?id=14893

I suspect that the solution involves making it so that std.concurrency uses something other than Variant to store the data being sent, but regardless, no one has fixed the issue yet. So, while send and receive work fine for simple message, they're fairly crippled when it comes to complex data and have been for their entire existence AFAIK.

- Jonathan M Davis

On Monday, 14 August 2017 at 21:27:48 UTC, Jonathan M Davis wrote:
> On Monday, August 14, 2017 15:22:23 Steven Schveighoffer via Digitalmars-d- learn wrote:
>> On 8/13/17 11:40 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
>> > On Saturday, August 12, 2017 18:57:44 Arek via Digitalmars-d-learn
> wrote:
>> >> I have the folowing problem:
>> >> I like to envelope the class object in struct to control the
>> >> destruction moment and then send this object to another
>> >> thread/fiber (or task, cause I use vibe-d).
>> >>
>> >> I can't find any method to make it working. Any ideas?
>> >
>> > Unfortunately, send and receive do not currently work with shared because of issues with Variant, which they use internally.
>>
>> This can't be a correct statement. This is the whole point of shared.
>
> What's incorrect about it? It's a longstanding issue that because Variant can't contain shared data, send and receive do not work with shared. You can send and receive mutable data with no indirections, and AFAIK, you can send immutable data (though the OP is apparently having problems with that, so I guess that that doesn't work completely, though I definitely recall doing so previously), but you get compilation errors if you try to send shared data. For instance, this code
>
> import std.concurrency;
>
> void main()
> {
>     static void func(Tid parent)
>     {
>         auto received = receiveOnly!(shared int[string]);
>     }
>
>     shared int[string] aa;
>
>     auto tid = spawn(&func, thisTid);
>     send(tid, aa);
> }
>

I've found some simple workaround for this problem:

import std.stdio;
import std.concurrency;

struct Envelope(T) if (is(T == class)) // for simplicity of this example, only classes
{
	shared(T)[] obj;

	this(shared T o)
	{
		this.obj = [o];
	}

	T get() @property nothrow @nogc
	{
		return cast() obj[0];
	}
}

class A
{

}

void consumer()
{
	auto r = receiveOnly!(Envelope!(A))();
	writeln("Got: ", typeof(r).stringof);
}

void main()
{
	auto cons = spawn(&consumer);
	auto o = Envelope!A(new A());
	send(cons, o);
}

Shared object can be encapsulated in the array. In case of other (non-class) types the pointer can be used, and get() should return ref to the pointed object (after stripping off the shared qualifier).

send() could encapsulate itself shared objectes.

Arek

On Monday, 14 August 2017 at 20:13:28 UTC, Arek wrote:
> If I can ensure the uniqueness of the object, there is no need to "share" it or synchronize the access.

You use manually managed multithreading, that's why you need shared. And because compiler can't verify uniqueness, you are requested to do it manually by casting.

On Monday, 14 August 2017 at 22:22:58 UTC, Arek wrote:
> I've found some simple workaround for this problem:
>
> import std.stdio;
> import std.concurrency;
>
> struct Envelope(T) if (is(T == class)) // for simplicity of this example, only classes
> {
> 	shared(T)[] obj;
>
> 	this(shared T o)
> 	{
> 		this.obj = [o];
> 	}
>
> 	T get() @property nothrow @nogc
> 	{
> 		return cast() obj[0];
> 	}
> }
>
> class A
> {
>
> }
>
> void consumer()
> {
> 	auto r = receiveOnly!(Envelope!(A))();
> 	writeln("Got: ", typeof(r).stringof);
> }
>
> void main()
> {
> 	auto cons = spawn(&consumer);
> 	auto o = Envelope!A(new A());
> 	send(cons, o);
> }
>
> Shared object can be encapsulated in the array. In case of other (non-class) types the pointer can be used, and get() should return ref to the pointed object (after stripping off the shared qualifier).

Rather like this:

struct Sendable(T)
{
	shared T o;
	alias o this;
}

import std.concurrency;

class A
{
	int method() shared;
}

void consumer()
{
	shared A a = receiveOnly!(Sendable!(A))();
}

void producer()
{
	auto cons = spawn(&consumer);
	shared A a = new shared A();
	send(cons, Sendable!A(a));
}

Well, no wrapper is actually needed here:

class A
{
	int method() shared;
}

void consumer()
{
	shared a = receiveOnly!(shared A)();
}

void producer()
{
	auto cons = spawn(&consumer);
	send(cons, new shared A());
}

On 8/14/17 5:27 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
> On Monday, August 14, 2017 15:22:23 Steven Schveighoffer via Digitalmars-d-
> learn wrote:
>> On 8/13/17 11:40 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
>>> On Saturday, August 12, 2017 18:57:44 Arek via Digitalmars-d-learn
> wrote:
>>>> I have the folowing problem:
>>>> I like to envelope the class object in struct to control the
>>>> destruction moment and then send this object to another
>>>> thread/fiber (or task, cause I use vibe-d).
>>>>
>>>> I can't find any method to make it working. Any ideas?
>>>
>>> Unfortunately, send and receive do not currently work with shared
>>> because of issues with Variant, which they use internally.
>>
>> This can't be a correct statement. This is the whole point of shared.
> 
> What's incorrect about it? It's a longstanding issue that because Variant
> can't contain shared data, send and receive do not work with shared.

The implementation details aren't important. From the documentation (and no, this documentation is not wrong):

From spawn:
args must not have unshared aliasing. In other words, all arguments to fn must either be shared or immutable or have no pointer indirection. This is necessary for enforcing isolation among threads.

From send:
As with std.concurrency.spawn, T must not have unshared aliasing.

So clearly passing shared pointers or things containing shared pointers should work fine.

As I was building code to test, I found that it does actually work for shared int pointers:

import std.concurrency;
import std.typecons;
import std.stdio;
import core.thread;

void threadfunc()
{
    bool done = false;
    while(!done)
    {  receive(
                  (shared(int)*foo) {*foo = 5; done = true;},
                  (Variant v) {}
                  );
    }
}

shared int f;

void main()
{
    auto tid = spawn(&threadfunc);
    tid.send(&f);
    Thread.sleep(1.seconds);
    writeln(f);
}


No error, completes as expected, and outputs 5.

So it looks like this is really a straight up bug and has nothing to do with the shared type qualifier. It is documented as working, and does work in some cases.

I think if the shared item is not a reference, it is doing something different, and this is incompatible with something in the implementation. Indeed, if you attempt to send unshared references, you get an assert. However, if you send a shared int (not a pointer), you get a bunch of compiler errors. Clearly the implementation expects it to work, as it doesn't fail the logical checks.

-Steve

https://github.com/dlang/phobos/blob/master/std/variant.d#L623
memcpy(&store, cast(const(void*)) &rhs, rhs.sizeof);

should be ok to cast unconditionally

On 8/15/17 10:42 AM, Kagamin wrote:
> https://github.com/dlang/phobos/blob/master/std/variant.d#L623
> memcpy(&store, cast(const(void*)) &rhs, rhs.sizeof);
> 
> should be ok to cast unconditionally

Agreed, the T value being copied has already been copied onto the stack, so it's not really shared.

However, I'm not sure about the postblit being called afterward. Does a postblit need to be marked shared in order to work for shared types?

I think also I understand why it's working for shared int* but not shared int -- IFTI automatically infers tail-modified for such things to cut down on instantiations. tail-modified means the head is not shared (which is in this case more accurate). This means the cast is not necessary, since you are copying unqualified data.

-Steve

On Tuesday, 15 August 2017 at 10:37:08 UTC, Kagamin wrote:
> Well, no wrapper is actually needed here:
>
> class A
> {
> 	int method() shared;
> }
>
> void consumer()
> {
> 	shared a = receiveOnly!(shared A)();
> }
>
> void producer()
> {
> 	auto cons = spawn(&consumer);
> 	send(cons, new shared A());
> }

Yes, but this doesn't compile:

import std.stdio;
import std.concurrency;

struct A
{
	int t;
	int r;
	int method() shared
	{
		return 0;
	}
}

void consumer()
{
	shared a = receiveOnly!(shared A)();
}

void main()
{
	auto cons = spawn(&consumer);
	send(cons, shared A());
}

This very simple code also doesn't compile:

shared struct S
{
	int i;

	~this()
	{
	}
}

void main()
{
	shared s = shared S();
}

In general, shared structs with postblit and destructor make problems.

Arek