Hi,

I am getting totally confused by Exceptions and Multi-threading with std.concurrency.
I have a simple setup:
    - the main thread spawns several child-threads
    - each child-thread performs a task
    - each child-thread sends their result back to the main thread and finishes.
    - the main thread uses receiveOnly to get the message.

To spawn the child-threads I use spawn, not spawnLinked. Nevertheless, in all runs, at some point the program finishes with an uncaught LinkTerminated exception!
I am confused, because I thought, that LinkTerminated can only happen if the child-threads are started with spawnLinked. In small test programs, I don't get LinkTerminated exceptions if a child thread crashes. So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".

My second question: Given that spawn works as documented and expected: How can I report an Error or an Exception thrown in a child-thread? Should I wrap the whole function that is spawned in a try-catch statement to print any exception manually?

Thanks for any clarification!

Stephan

On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
> Hi,
>
> I am getting totally confused by Exceptions and Multi-threading with std.concurrency.
> I have a simple setup:
>     - the main thread spawns several child-threads
>     - each child-thread performs a task
>     - each child-thread sends their result back to the main thread and finishes.
>     - the main thread uses receiveOnly to get the message.
>
> To spawn the child-threads I use spawn, not spawnLinked. Nevertheless, in all runs, at some point the program finishes with an uncaught LinkTerminated exception!
> I am confused, because I thought, that LinkTerminated can only happen if the child-threads are started with spawnLinked. In small test programs, I don't get LinkTerminated exceptions if a child thread crashes. So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".
> Thanks for any clarification!

I have had the same problem, especially when creating large numbers of threads (several hundred)

On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
> So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".

To track this down, what about looking at the std.concurrency source and placing a breakpoint at the appropriate line in MessageBox.get()?

David

On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:
> On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
>> So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".
>
> To track this down, what about looking at the std.concurrency source and placing a breakpoint at the appropriate line in MessageBox.get()?
>
> David

Yeah, I've been playing with std.concurrency too lately, and getting random bugs. I'll try to formalize them into formal requests.

BTW: Question: Is "LinkTerminated" a priority message? I've been avoiding using SpawnLinked because I've been unable to effectively end my workers. Maybe I just suck though...

On Jan 24, 2013, at 5:58 AM, "monarch_dodra" <monarchdodra@gmail.com> wrote:

> On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:
>> On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
>>> So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".
>> 
>> To track this down, what about looking at the std.concurrency source and placing a breakpoint at the appropriate line in MessageBox.get()?
>> 
>> David
> 
> Yeah, I've been playing with std.concurrency too lately, and getting random bugs. I'll try to formalize them into formal requests.
> 
> BTW: Question: Is "LinkTerminated" a priority message? I've been avoiding using SpawnLinked because I've been unable to effectively end my workers. Maybe I just suck though...

It's a control message. These are messages generated automatically by std.concurrency rather than sent by the user. They live in the normal message queue and so are processed when receive doesn't find a match earlier in the queue. The idea is that if an owner sends a spawned thread a bunch of messages, the spawned thread should have an opportunity to process those messages before receiving an OwnerTerminated message.  Making them priority messages would make designing predictable algorithms difficult.

On Thursday, 24 January 2013 at 17:12:49 UTC, Sean Kelly wrote:
> On Jan 24, 2013, at 5:58 AM, "monarch_dodra" <monarchdodra@gmail.com> wrote:
>
>> On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:
>>> On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
>>>> So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".
>>> 
>>> To track this down, what about looking at the std.concurrency source and placing a breakpoint at the appropriate line in MessageBox.get()?
>>> 
>>> David
>> 
>> Yeah, I've been playing with std.concurrency too lately, and getting random bugs. I'll try to formalize them into formal requests.
>> 
>> BTW: Question: Is "LinkTerminated" a priority message? I've been avoiding using SpawnLinked because I've been unable to effectively end my workers. Maybe I just suck though...
>
> It's a control message. These are messages generated automatically by std.concurrency rather than sent by the user. They live in the normal message queue and so are processed when receive doesn't find a match earlier in the queue. The idea is that if an owner sends a spawned thread a bunch of messages, the spawned thread should have an opportunity to process those messages before receiving an OwnerTerminated message.  Making them priority messages would make designing predictable algorithms difficult.

But still, LinkTerminated needs to be thrown somewhere. And without spawnLinked I can't see why it should be thrown anywhere.
I am trying to use a different setup now, with a fewer number of threads. Maybe that'll work.

Stephan

On Jan 24, 2013, at 9:58 AM, Stephan <stephan_schiffels@mac.com> wrote:

> On Thursday, 24 January 2013 at 17:12:49 UTC, Sean Kelly wrote:
>> On Jan 24, 2013, at 5:58 AM, "monarch_dodra" <monarchdodra@gmail.com> wrote:
>> 
>>> On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:
>>>> On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
>>>>> So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".
>>>> To track this down, what about looking at the std.concurrency source and placing a breakpoint at the appropriate line in MessageBox.get()?
>>>> David
>>> Yeah, I've been playing with std.concurrency too lately, and getting random bugs. I'll try to formalize them into formal requests.
>>> BTW: Question: Is "LinkTerminated" a priority message? I've been avoiding using SpawnLinked because I've been unable to effectively end my workers. Maybe I just suck though...
>> 
>> It's a control message. These are messages generated automatically by std.concurrency rather than sent by the user. They live in the normal message queue and so are processed when receive doesn't find a match earlier in the queue. The idea is that if an owner sends a spawned thread a bunch of messages, the spawned thread should have an opportunity to process those messages before receiving an OwnerTerminated message.  Making them priority messages would make designing predictable algorithms difficult.
> 
> But still, LinkTerminated needs to be thrown somewhere. And without spawnLinked I can't see why it should be thrown anywhere.
> I am trying to use a different setup now, with a fewer number of threads. Maybe that'll work.

It shouldn't be thrown unless you spawn with spawnLinked.  The code is pretty straightforward.  If you can come up with a repro of unexpected behavior, please file a bug report.

On Thursday, 24 January 2013 at 17:12:49 UTC, Sean Kelly wrote:
> On Jan 24, 2013, at 5:58 AM, "monarch_dodra" <monarchdodra@gmail.com> wrote:
>
>> On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:
>>> On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:
>>>> So my first question is: Where does the LinkTerminated Exception come from, when the only functions from std.concurrency that I use are "spawn" and "receiveOnly".
>>> 
>>> To track this down, what about looking at the std.concurrency source and placing a breakpoint at the appropriate line in MessageBox.get()?
>>> 
>>> David
>> 
>> Yeah, I've been playing with std.concurrency too lately, and getting random bugs. I'll try to formalize them into formal requests.
>> 
>> BTW: Question: Is "LinkTerminated" a priority message? I've been avoiding using SpawnLinked because I've been unable to effectively end my workers. Maybe I just suck though...
>
> It's a control message. These are messages generated automatically by std.concurrency rather than sent by the user. They live in the normal message queue and so are processed when receive doesn't find a match earlier in the queue. The idea is that if an owner sends a spawned thread a bunch of messages, the spawned thread should have an opportunity to process those messages before receiving an OwnerTerminated message.  Making them priority messages would make designing predictable algorithms difficult.

OK. TY.

I'll try to reproduce, but I'm 90% sure my reduced code was this:

//----
void worker(Tid owner)
{
  owner.send(1);
}

void main()
{
    spawnLinked(&worker, thisTid);
    receive(
        (int a){}
    );
}
//----
And it (50% of the time) terminated in a LinkTerminated exception being thrown.

Any quick thoughts? I'll investigate on my end depending on your explanation.

On Friday, 25 January 2013 at 01:13:59 UTC, monarch_dodra wrote:
>
> I'll try to reproduce, but I'm 90% sure my reduced code was this:
>
> //----
> void worker(Tid owner)
> {
>   owner.send(1);
> }
>
> void main()
> {
>     spawnLinked(&worker, thisTid);
>     receive(
>         (int a){}
>     );
> }
> //----
> And it (50% of the time) terminated in a LinkTerminated exception being thrown.
>
> Any quick thoughts? I'll investigate on my end depending on your explanation.


Nope. That code works 100% for me.

Stephan

On Friday, 25 January 2013 at 07:33:57 UTC, Stephan wrote:
> On Friday, 25 January 2013 at 01:13:59 UTC, monarch_dodra wrote:
>> [SNIP]
>
>
> Nope. That code works 100% for me.
>
> Stephan

OK. I'm getting a better hang at the semantics of how message passing works.

I think I found I've identified the one of the issues I was running into though. I don't have much experience with MPI, so I don't know if this is a bug, or a user issue. In any case, I'm getting inconsistent behavior.

The basic premise of my problem is that one of my workers had a full mail box, while being set to "block". If it dies *while* the owner is blocked for sending, then the sender is blocked... forever.

//----
import std.concurrency, std.stdio;

void worker()
{
    thisTid.setMaxMailboxSize(10, OnCrowding.block);
    Thread.sleep(msecs(1000));
}
void main()
{
    auto wid = spawn(&worker);
    foreach ( i ; 0 .. 20 )
    {
        writeln(i);
        wid.send(i);
    }
}
//----

Basically, the worker sleeps for a second, the mailbox fills. The owner waits. The worker then dies, but the owner never wakes :(

The *inconsistent* behavior I'm getting though is that if I comment the "writeln(i)", then the program doesn't hang.

Say... what exactly happens when a message is sent to a dead thread? Seems like it just silently disappears...