The following code compiles but doesn't work as expected:

     import std.stdio;
     import std.concurrency;

     class A
     {
         this() immutable {}
     }

     void main()
     {
         auto tid = spawn( &fooBar, thisTid );
         while(true)
         {
             receive(
                 (Variant any) {
                     writeln( "Received a variant" );
                     writeln( "Received ", any );
                 }
             );
         }
     }

     void fooBar( Tid masterTid )
     {
         scope(failure) writeln( "fooBar failed" );
         scope(success) writeln( "fooBar succeeded" );
         scope(exit) writeln( "fooBar exiting" );
         try
         {
             immutable A b = new immutable A();
             masterTid.send( 42 ); // This works
             masterTid.send( b );  // This doesn't
         }
         catch( Exception e )
         {
             writeln( "Exception received" );
         }
     }


I see this in the console:

     fooBar exiting
     fooBar failed
     Received a variant
     Received 42

(then it just hangs)

I'm especially puzzled by:
  * Sending an int as a message works but not an immutable object.
Wasn't this  (safely sharing objects across threads) one of the
basic use cases for the immutable type qualifier?
  * scope(failure) failed but my exception handler didn't catch
anything. How is this possible? What could cause that?
Assertions/abort?

There seemed to be a 3-year-old ticket on this issue
(http://d.puremagic.com/issues/show_bug.cgi?id=5538) with very
little activity, which is a bit surprising given how much
emphasis is given to this feature (the D homepage mentions "D
offers an innovative approach to concurrency, featuring true
immutable data, message passing, no sharing by default, and
controlled mutable sharing across threads", and TDPL devotes a
whole chapter on it). This thread
(http://forum.dlang.org/thread/kgk8hc$12fa$1@digitalmars.com)
also says std.concurrency is "very buggy".

If I can't use std.concurrency, is there any other safe
alternative for multithreaded programming with D?

Thanks,
A.

On 2013-11-25 06:03:27 +0000, Antoche said:

> The following code compiles but doesn't work as expected:
> 
>       import std.stdio;
>       import std.concurrency;
> 
>       class A
>       {
>           this() immutable {}
>       }
> 
>       void main()
>       {
>           auto tid = spawn( &fooBar, thisTid );
>           while(true)
>           {
>               receive(
>                   (Variant any) {
>                       writeln( "Received a variant" );
>                       writeln( "Received ", any );
>                   }
>               );
>           }
>       }
> 
>       void fooBar( Tid masterTid )
>       {
>           scope(failure) writeln( "fooBar failed" );
>           scope(success) writeln( "fooBar succeeded" );
>           scope(exit) writeln( "fooBar exiting" );
>           try
>           {
>               immutable A b = new immutable A();
>               masterTid.send( 42 ); // This works
>               masterTid.send( b );  // This doesn't
>           }
>           catch( Exception e )
>           {
>               writeln( "Exception received" );
>           }
>       }
> 
> 
> I see this in the console:
> 
>       fooBar exiting
>       fooBar failed
>       Received a variant
>       Received 42
> 
> (then it just hangs)
> 
> I'm especially puzzled by:
>    * Sending an int as a message works but not an immutable object.
> Wasn't this  (safely sharing objects across threads) one of the
> basic use cases for the immutable type qualifier?
>    * scope(failure) failed but my exception handler didn't catch
> anything. How is this possible? What could cause that?
> Assertions/abort?
> 
> There seemed to be a 3-year-old ticket on this issue
> (http://d.puremagic.com/issues/show_bug.cgi?id=5538) with very
> little activity, which is a bit surprising given how much
> emphasis is given to this feature (the D homepage mentions "D
> offers an innovative approach to concurrency, featuring true
> immutable data, message passing, no sharing by default, and
> controlled mutable sharing across threads", and TDPL devotes a
> whole chapter on it). This thread
> (http://forum.dlang.org/thread/kgk8hc$12fa$1@digitalmars.com)
> also says std.concurrency is "very buggy".
> 
> If I can't use std.concurrency, is there any other safe
> alternative for multithreaded programming with D?
> 
> Thanks,
> A.


There is a bug with the internals of send/receive.   It has to be able to copy the reference to a another place while sending --  and it seems it can't do that because it's immutable.     I don't think many people are using it.    I personally did not realize the bug was from 2011.  I think DMD has been fixed enough to make a patch to phobos now.  I'm going to ping this bug report.

However,  even CONSTRUCTING immutable objects right now is very difficult.   It's arguably "correct" but, seems impossible to actually be able to make something of use without many many idups.  It's kind of in the same boat as shared.

What I am currently doing, is casting to shared, and then back.  I tried making my classes shared, and also immutable as you have.   Nothing else seems to work:

> class A
>       {
>           this() {}
>       }
> 
>       void main()
>       {
>           auto tid = spawn( &fooBar, thisTid );
>           while(true)
>           {
>               receive(
		   (shared A _m)
	     	   {
			auto m = cast(A)_m;
			//Do stuff.
		   },
>                   (Variant any) {
>                       writeln( "Received a variant" );
>                       writeln( "Received ", any );
>                   }
>               );
>           }
>       }
> 
>       void fooBar( Tid masterTid )
>       {
>           scope(failure) writeln( "fooBar failed" );
>           scope(success) writeln( "fooBar succeeded" );
>           scope(exit) writeln( "fooBar exiting" );
>           try
>           {
>               immutable A b = new A();
>               masterTid.send( 42 ); // This works
>               masterTid.send( cast(shared) b );  // Should work
>           }
>           catch( Exception e )
>           {
>               writeln( "Exception received" );
>           }
>       }

-Shammah

Sending immutable classes currently does not work because of https://d.puremagic.com/issues/show_bug.cgi?id=7069 (and has never worked despite being intended).

On Monday, 25 November 2013 at 11:48:06 UTC, Shammah Chancellor wrote:
> On 2013-11-25 06:03:27 +0000, Antoche said:
>
>> The following code compiles but doesn't work as expected:
>> 
>>      import std.stdio;
>>      import std.concurrency;
>> 
>>      class A
>>      {
>>          this() immutable {}
>>      }
>> 
>>      void main()
>>      {
>>          auto tid = spawn( &fooBar, thisTid );
>>          while(true)
>>          {
>>              receive(
>>                  (Variant any) {
>>                      writeln( "Received a variant" );
>>                      writeln( "Received ", any );
>>                  }
>>              );
>>          }
>>      }
>> 
>>      void fooBar( Tid masterTid )
>>      {
>>          scope(failure) writeln( "fooBar failed" );
>>          scope(success) writeln( "fooBar succeeded" );
>>          scope(exit) writeln( "fooBar exiting" );
>>          try
>>          {
>>              immutable A b = new immutable A();
>>              masterTid.send( 42 ); // This works
>>              masterTid.send( b );  // This doesn't
>>          }
>>          catch( Exception e )
>>          {
>>              writeln( "Exception received" );
>>          }
>>      }
>> 
>> 
>> I see this in the console:
>> 
>>      fooBar exiting
>>      fooBar failed
>>      Received a variant
>>      Received 42
>> 
>> (then it just hangs)
>> 
>> I'm especially puzzled by:
>>   * Sending an int as a message works but not an immutable object.
>> Wasn't this  (safely sharing objects across threads) one of the
>> basic use cases for the immutable type qualifier?
>>   * scope(failure) failed but my exception handler didn't catch
>> anything. How is this possible? What could cause that?
>> Assertions/abort?
>> 
>> There seemed to be a 3-year-old ticket on this issue
>> (http://d.puremagic.com/issues/show_bug.cgi?id=5538) with very
>> little activity, which is a bit surprising given how much
>> emphasis is given to this feature (the D homepage mentions "D
>> offers an innovative approach to concurrency, featuring true
>> immutable data, message passing, no sharing by default, and
>> controlled mutable sharing across threads", and TDPL devotes a
>> whole chapter on it). This thread
>> (http://forum.dlang.org/thread/kgk8hc$12fa$1@digitalmars.com)
>> also says std.concurrency is "very buggy".
>> 
>> If I can't use std.concurrency, is there any other safe
>> alternative for multithreaded programming with D?
>> 
>> Thanks,
>> A.
>
>
> There is a bug with the internals of send/receive.   It has to be able to copy the reference to a another place while sending --  and it seems it can't do that because it's immutable.     I don't think many people are using it.    I personally did not realize the bug was from 2011.  I think DMD has been fixed enough to make a patch to phobos now.  I'm going to ping this bug report.
>
> However,  even CONSTRUCTING immutable objects right now is very difficult.   It's arguably "correct" but, seems impossible to actually be able to make something of use without many many idups.  It's kind of in the same boat as shared.
>
> What I am currently doing, is casting to shared, and then back.
>  I tried making my classes shared, and also immutable as you have.   Nothing else seems to work:
>
>> class A
>>      {
>>          this() {}
>>      }
>> 
>>      void main()
>>      {
>>          auto tid = spawn( &fooBar, thisTid );
>>          while(true)
>>          {
>>              receive(
> 		   (shared A _m)
> 	     	   {
> 			auto m = cast(A)_m;
> 			//Do stuff.
> 		   },
>>                  (Variant any) {
>>                      writeln( "Received a variant" );
>>                      writeln( "Received ", any );
>>                  }
>>              );
>>          }
>>      }
>> 
>>      void fooBar( Tid masterTid )
>>      {
>>          scope(failure) writeln( "fooBar failed" );
>>          scope(success) writeln( "fooBar succeeded" );
>>          scope(exit) writeln( "fooBar exiting" );
>>          try
>>          {
>>              immutable A b = new A();
>>              masterTid.send( 42 ); // This works
>>              masterTid.send( cast(shared) b );  // Should work
>>          }
>>          catch( Exception e )
>>          {
>>              writeln( "Exception received" );
>>          }
>>      }
>
> -Shammah

Thanks for the suggestion. My problem with casting is that it results in undefined behaviour, so I have no guarantee whatsoever what the program is actually going to do. I could simply not use immutable at all, and use shared instead, but then I might as well stick to C++ and volatile. The whole point of using D was trying to write safer code with equivalent performance.

I see std.parallelism seems to offer some features that might compensate for the broken std.concurrency. Is it working better or is it as broken as std.concurrency? Is there any other way to write safe multithreaded programs?

Finally, can anyone explained why the scope(failure) is reached? Looking at the stacktrace in gdb at this point gives me no indication of what's happening.

On 2013-11-25 14:08:53 +0000, Dicebot said:

> Sending immutable classes currently does not work because of https://d.puremagic.com/issues/show_bug.cgi?id=7069 (and has never worked despite being intended).

Can you send immutable struct references?

On 2013-11-25 20:55:15 +0000, Antoche said:

> On Monday, 25 November 2013 at 11:48:06 UTC, Shammah Chancellor wrote:
>> On 2013-11-25 06:03:27 +0000, Antoche said:
>> 
>>> The following code compiles but doesn't work as expected:
>>> 
>>> import std.stdio;
>>> import std.concurrency;
>>> 
>>> class A
>>> {
>>> this() immutable {}
>>> }
>>> 
>>> void main()
>>> {
>>> auto tid = spawn( &fooBar, thisTid );
>>> while(true)
>>> {
>>> receive(
>>> (Variant any) {
>>> writeln( "Received a variant" );
>>> writeln( "Received ", any );
>>> }
>>> );
>>> }
>>> }
>>> 
>>> void fooBar( Tid masterTid )
>>> {
>>> scope(failure) writeln( "fooBar failed" );
>>> scope(success) writeln( "fooBar succeeded" );
>>> scope(exit) writeln( "fooBar exiting" );
>>> try
>>> {
>>> immutable A b = new immutable A();
>>> masterTid.send( 42 ); // This works
>>> masterTid.send( b );  // This doesn't
>>> }
>>> catch( Exception e )
>>> {
>>> writeln( "Exception received" );
>>> }
>>> }
>>> 
>>> 
>>> I see this in the console:
>>> 
>>> fooBar exiting
>>> fooBar failed
>>> Received a variant
>>> Received 42
>>> 
>>> (then it just hangs)
>>> 
>>> I'm especially puzzled by:
>>> * Sending an int as a message works but not an immutable object.
>>> Wasn't this  (safely sharing objects across threads) one of the
>>> basic use cases for the immutable type qualifier?
>>> * scope(failure) failed but my exception handler didn't catch
>>> anything. How is this possible? What could cause that?
>>> Assertions/abort?
>>> 
>>> There seemed to be a 3-year-old ticket on this issue
>>> (http://d.puremagic.com/issues/show_bug.cgi?id=5538) with very
>>> little activity, which is a bit surprising given how much
>>> emphasis is given to this feature (the D homepage mentions "D
>>> offers an innovative approach to concurrency, featuring true
>>> immutable data, message passing, no sharing by default, and
>>> controlled mutable sharing across threads", and TDPL devotes a
>>> whole chapter on it). This thread
>>> (http://forum.dlang.org/thread/kgk8hc$12fa$1@digitalmars.com)
>>> also says std.concurrency is "very buggy".
>>> 
>>> If I can't use std.concurrency, is there any other safe
>>> alternative for multithreaded programming with D?
>>> 
>>> Thanks,
>>> A.
>> 
>> 
>> There is a bug with the internals of send/receive.   It has to be able to copy the reference to a another place while sending --  and it seems it can't do that because it's immutable.     I don't think many people are using it.    I personally did not realize the bug was from 2011.  I think DMD has been fixed enough to make a patch to phobos now.  I'm going to ping this bug report.
>> 
>> However,  even CONSTRUCTING immutable objects right now is very difficult.   It's arguably "correct" but, seems impossible to actually be able to make something of use without many many idups.  It's kind of in the same boat as shared.
>> 
>> What I am currently doing, is casting to shared, and then back.
>> I tried making my classes shared, and also immutable as you have.   Nothing else seems to work:
>> 
>>> class A
>>> {
>>> this() {}
>>> }
>>> 
>>> void main()
>>> {
>>> auto tid = spawn( &fooBar, thisTid );
>>> while(true)
>>> {
>>> receive(
>> 		   (shared A _m)
>> 	     	   {
>> 			auto m = cast(A)_m;
>> 			//Do stuff.
>> 		   },
>>> (Variant any) {
>>> writeln( "Received a variant" );
>>> writeln( "Received ", any );
>>> }
>>> );
>>> }
>>> }
>>> 
>>> void fooBar( Tid masterTid )
>>> {
>>> scope(failure) writeln( "fooBar failed" );
>>> scope(success) writeln( "fooBar succeeded" );
>>> scope(exit) writeln( "fooBar exiting" );
>>> try
>>> {
>>> immutable A b = new A();
>>> masterTid.send( 42 ); // This works
>>> masterTid.send( cast(shared) b );  // Should work
>>> }
>>> catch( Exception e )
>>> {
>>> writeln( "Exception received" );
>>> }
>>> }
>> 
>> -Shammah
> 
> Thanks for the suggestion. My problem with casting is that it results in undefined behaviour, so I have no guarantee whatsoever what the program is actually going to do. I could simply not use immutable at all, and use shared instead, but then I might as well stick to C++ and volatile. The whole point of using D was trying to write safer code with equivalent performance.
> 
> I see std.parallelism seems to offer some features that might compensate for the broken std.concurrency. Is it working better or is it as broken as std.concurrency? Is there any other way to write safe multithreaded programs?
> 
> Finally, can anyone explained why the scope(failure) is reached? Looking at the stacktrace in gdb at this point gives me no indication of what's happening.


Try wrapping it in a Try/catch and see.    Sending non-value types right now seems to be a bit frustrating.   Unless you *need* polymorphism for your messages I would suggest sending immutable struct references or something.   Or you can also use the idea I suggested and cast(const) to ensure you are not messing with the data again afterwards.    I am going to ping this issue very heavily though, and possibly start working on std.concurrency myself.   The current state of the library is unacceptable.

-Shammah

On Monday, 25 November 2013 at 23:00:41 UTC, Shammah Chancellor wrote:
> On 2013-11-25 14:08:53 +0000, Dicebot said:
>
>> Sending immutable classes currently does not work because of https://d.puremagic.com/issues/show_bug.cgi?id=7069 (and has never worked despite being intended).
>
> Can you send immutable struct references?

If you mean mutable pointer to immutable struct - yes, this should work (Did not try that). But any fully immutable type will fail because of Variant implementation issue.

By the way this works:

import std.stdio;
import std.concurrency;

class A
{
    this() immutable {}

    // this line
    string toString() const { return "aaa"; }
}

But it will still fail if you try to receive by class type explicitly instead of Variant.


std.parallelism is much more mature in my opinion, but std.concurrency is mostly broken when it comes to qualifiers and needs some serious bug-fixing rampage.

On Monday, 25 November 2013 at 20:55:18 UTC, Antoche wrote:
> Thanks for the suggestion. My problem with casting is that it results in undefined behaviour,

Casting is defined, modifications afterwards are not.

> Finally, can anyone explained why the scope(failure) is reached? Looking at the stacktrace in gdb at this point gives me no indication of what's happening.

An assert error wouldn't get picked up by your catch while still triggering scope(failure)

catch(Throwable) to grab anything.