View mode: basic / threaded / horizontal-split · Log in · Help
May 19, 2012
std.concurrency.send
Multithreading in D confuses me more and more.

import std.concurrency;
import std.stdio;
shared Tid tid;
void main() {
  send(cast(Tid)tid, "Hello, World");
}
void worker() {
   writeln(receiveOnly!string);
}
shared static this() {
  tid = cast(shared)spawn(&worker);
}

I hate these explicit casts. It is impossible sharing anything 
between threads without these ugly casts from/to shared. Seems 
like something wrong in program design when I'm forced to use 
explicit casts. But I don't understand what is it exactly.

For example. I need create mutable object in one thread and send 
to another. I don't need to share this object, just create, send 
and forget. But I have no idea how make this without using shared 
attribute and casting to/from it.
May 19, 2012
Re: std.concurrency.send
On Saturday, 19 May 2012 at 13:26:20 UTC, japplegame wrote:
> Multithreading in D confuses me more and more.
>
> import std.concurrency;
> import std.stdio;
> shared Tid tid;
> void main() {
>   send(cast(Tid)tid, "Hello, World");
> }
> void worker() {
>    writeln(receiveOnly!string);
> }
> shared static this() {
>   tid = cast(shared)spawn(&worker);
> }
>
> I hate these explicit casts. It is impossible sharing anything 
> between threads without these ugly casts from/to shared. Seems 
> like something wrong in program design when I'm forced to use 
> explicit casts. But I don't understand what is it exactly.

You don't need to mark Tids as shared.

> For example. I need create mutable object in one thread and 
> send to another. I don't need to share this object, just 
> create, send and forget. But I have no idea how make this 
> without using shared attribute and casting to/from it.

If you originally create it as shared, you don't need to do the 
casting.
May 19, 2012
Re: std.concurrency.send
> You don't need to mark Tids as shared.
Okay. I'm writting logger. Logger is global object and it is
running in its own separate thread (for example, writting logs to
remote database).
My application has several threads and all of them want to log
something. How to share this global logger between threads? I
think the simplest way is to share logger's thread tid and other
thread can send logs via this shared tid.
> If you originally create it as shared, you don't need to do the 
> casting.
Yes. I don't need to cast to shared, but inside thread I get
shared object and can't store/call/pass it without casting away
that shared attribute. Or I should make shared everyting that
have deal with shared object.
I'am trying to follow Safe D concept, but it forbids casting away
shared.
May 20, 2012
Re: std.concurrency.send
On Saturday, 19 May 2012 at 21:13:14 UTC, japplegame wrote:
>> You don't need to mark Tids as shared.
> Okay. I'm writting logger. Logger is global object and it is
> running in its own separate thread (for example, writting logs 
> to
> remote database).
> My application has several threads and all of them want to log
> something. How to share this global logger between threads? I
> think the simplest way is to share logger's thread tid and other
> thread can send logs via this shared tid.

public:
void startLogger(LogConstructorArgs args) {
    loggerTid = spawn(&loggerThread, args);
}

void log(string msg, OtherOptions oo) {
    loggerTid.send(LogMsg(msg, oo));
}

void stopLogger() {
    loggerTid.send(QuitMsg());
}

private:
Tid loggerTid;

struct LogMsg {
    string msg;
    OtherOptions oo;
}

struct QuitMsg {}

void loggerThread(LogConstructorArgs args) {
    Logger lg = new Logger(args);
    bool cont = true;
    while(cont) {
        receive((LogMsg lm) { lg.log(lm.msg, lm.oo); },
                (QuitMsg qm) { cont = false; });
    }
}

>> If you originally create it as shared, you don't need to do 
>> the casting.
> Yes. I don't need to cast to shared, but inside thread I get
> shared object and can't store/call/pass it without casting away
> that shared attribute. Or I should make shared everyting that
> have deal with shared object.
> I'am trying to follow Safe D concept, but it forbids casting 
> away
> shared.

If you are passing objects between threads, make it shared. This 
might seem annoying, but in general you should try to shift your 
thinking into having thread-local objects and communicating via 
structs.

But when you use global/singleton objects (any case where there's 
one instance of the class), convert it into a thread, FROM

class <name> {
    this(<cons args>) {
        <constructor>
    }

    void <method1>() {
        // ...
    }

    int <method2>() {
        int result;
        // ...
        return result;
    }

    <private fields> // you encapsulate and have no public 
fields, right?
}

TO

void <name>Thread(<cons args>) {
    <private fields>
    <constructor>
    bool cont = true;
    while(cont) {
        receive(
            (<method1>Msg) {
                // ...
            },
            (Tid r, <method2>Msg) {
                int result;
                // ...
                r.send(<method2>ReturnMsg(result));
            }
            (QuitMsg qm) {cont = false;});
    }
}
May 20, 2012
Re: std.concurrency.send
> public:
> void startLogger(LogConstructorArgs args) {
>     loggerTid = spawn(&loggerThread, args);
> }
>
> void log(string msg, OtherOptions oo) {
>     loggerTid.send(LogMsg(msg, oo));
> }
>
> void stopLogger() {
>     loggerTid.send(QuitMsg());
> }
>
> private:
> Tid loggerTid;
>
> struct LogMsg {
>     string msg;
>     OtherOptions oo;
> }
>
> struct QuitMsg {}
>
> void loggerThread(LogConstructorArgs args) {
>     Logger lg = new Logger(args);
>     bool cont = true;
>     while(cont) {
>         receive((LogMsg lm) { lg.log(lm.msg, lm.oo); },
>                 (QuitMsg qm) { cont = false; });
>     }
> }
I don't understand. In this way I should call 
startLogger/stopLogger in every application thread because 
loggerTid is thread related and every application thread has its 
own instanse of loggerTid. Instead N+1 threads (N application and 
1 logger) we will get 2*N threads (N application and N loggers). 
Also we should synchronize Logger class because many instances of 
them will run concurrently. This is terrible.
> If you are passing objects between threads, make it shared. 
> This might seem annoying, but in general you should try to 
> shift your thinking into having thread-local objects and 
> communicating via structs.
>
> But when you use global/singleton objects (any case where 
> there's one instance of the class), convert it into a thread, 
> FROM
>
> class <name> {
>     this(<cons args>) {
>         <constructor>
>     }
>
>     void <method1>() {
>         // ...
>     }
>
>     int <method2>() {
>         int result;
>         // ...
>         return result;
>     }
>
>     <private fields> // you encapsulate and have no public 
> fields, right?
> }
>
> TO
>
> void <name>Thread(<cons args>) {
>     <private fields>
>     <constructor>
>     bool cont = true;
>     while(cont) {
>         receive(
>             (<method1>Msg) {
>                 // ...
>             },
>             (Tid r, <method2>Msg) {
>                 int result;
>                 // ...
>                 r.send(<method2>ReturnMsg(result));
>             }
>             (QuitMsg qm) {cont = false;});
>     }
> }
Intresting idea. I will try it. Thanks.
May 20, 2012
Re: std.concurrency.send
On Sunday, 20 May 2012 at 04:09:50 UTC, japplegame wrote:
>> public:
>> void startLogger(LogConstructorArgs args) {
>>    loggerTid = spawn(&loggerThread, args);
>> }
>>
>> void log(string msg, OtherOptions oo) {
>>    loggerTid.send(LogMsg(msg, oo));
>> }
>>
>> void stopLogger() {
>>    loggerTid.send(QuitMsg());
>> }
>>
>> private:
>> Tid loggerTid;
>>
>> struct LogMsg {
>>    string msg;
>>    OtherOptions oo;
>> }
>>
>> struct QuitMsg {}
>>
>> void loggerThread(LogConstructorArgs args) {
>>    Logger lg = new Logger(args);
>>    bool cont = true;
>>    while(cont) {
>>        receive((LogMsg lm) { lg.log(lm.msg, lm.oo); },
>>                (QuitMsg qm) { cont = false; });
>>    }
>> }
> I don't understand. In this way I should call 
> startLogger/stopLogger in every application thread because 
> loggerTid is thread related and every application thread has 
> its own instanse of loggerTid. Instead N+1 threads (N 
> application and 1 logger) we will get 2*N threads (N 
> application and N loggers). Also we should synchronize Logger 
> class because many instances of them will run concurrently. 
> This is terrible.

Tids aren't threads, they're thread-IDs, so you wouldn't be 
creating threads.

Though you're right in that tids would need to be shared (for 
some reason I assumed that child thread TLS storage was copied 
from its parents). In general, this shows an important principle 
of D-ish concurrent programming: avoid globals.

In your situation, here's what I'd do:

struct Logger {
    void start() { loggerTid = spawn(&loggerThread, args); }
    void stop() { loggerTid.send(QuitMsg()); }
    void log(string msg) { loggerTid.send(LogMsg(msg)); }
    Tid loggerTid;
}

Construct loggerThread like the class-to-thread I show below, so 
you don't need a Logger class, and make sure to start() at the 
beginning and stop() at the end, and pass your Logger struct 
around.

This may seem like an inconvenience compared to a shared 
synchronized object, but here's an interesting thought: once a 
working thread sends a message to the logger thread to log, it 
can continue with its work. This makes it more efficient than 
having to wait for the logging to finish before continuing with 
work, which is what you'd do with synchronized(this). You are 
only doing synchronized(queue) in the background.

What D is doing is making it easier to do it the right/efficient 
way (message passing), by making the usually simple way (shared 
objects) inconvenient.

Note: unfortunately, I'm not sure if the Logger will work due to 
this bug:
http://d.puremagic.com/issues/show_bug.cgi?id=4957
Hopefully it's fixed soon :(

>> If you are passing objects between threads, make it shared. 
>> This might seem annoying, but in general you should try to 
>> shift your thinking into having thread-local objects and 
>> communicating via structs.
>>
>> But when you use global/singleton objects (any case where 
>> there's one instance of the class), convert it into a thread, 
>> FROM
>>
>> class <name> {
>>    this(<cons args>) {
>>        <constructor>
>>    }
>>
>>    void <method1>() {
>>        // ...
>>    }
>>
>>    int <method2>() {
>>        int result;
>>        // ...
>>        return result;
>>    }
>>
>>    <private fields> // you encapsulate and have no public 
>> fields, right?
>> }
>>
>> TO
>>
>> void <name>Thread(<cons args>) {
>>    <private fields>
>>    <constructor>
>>    bool cont = true;
>>    while(cont) {
>>        receive(
>>            (<method1>Msg) {
>>                // ...
>>            },
>>            (Tid r, <method2>Msg) {
>>                int result;
>>                // ...
>>                r.send(<method2>ReturnMsg(result));
>>            }
>>            (QuitMsg qm) {cont = false;});
>>    }
>> }
> Intresting idea. I will try it. Thanks.

I like to thing of this idea as the "thread-object" paradigm. 
I've considered using D's great templating ability to allow you 
to make a class and have it be converted to a thread-object, but 
the __traits(getMember) stuff is confusing.

NMS
May 24, 2012
Re: std.concurrency.send
On May 19, 2012, at 2:13 PM, japplegame wrote:

>> You don't need to mark Tids as shared.
> Okay. I'm writting logger. Logger is global object and it is
> running in its own separate thread (for example, writting logs to
> remote database).
> My application has several threads and all of them want to log
> something. How to share this global logger between threads? I
> think the simplest way is to share logger's thread tid and other
> thread can send logs via this shared tid.

Maybe use register().


>> If you originally create it as shared, you don't need to do the casting.
> Yes. I don't need to cast to shared, but inside thread I get
> shared object and can't store/call/pass it without casting away
> that shared attribute. Or I should make shared everyting that
> have deal with shared object.
> I'am trying to follow Safe D concept, but it forbids casting away
> shared.

I'll admit that send() has only spotty support for shared.  This should be fixed.
Top | Discussion index | About this forum | D home