Multithreading in D confuses me more and more.

import std.concurrency;
import std.stdio;
shared Tid tid;
void main() {
  send(cast(Tid)tid, "Hello, World");
}
void worker() {
   writeln(receiveOnly!string);
}
shared static this() {
  tid = cast(shared)spawn(&worker);
}

I hate these explicit casts. It is impossible sharing anything between threads without these ugly casts from/to shared. Seems like something wrong in program design when I'm forced to use explicit casts. But I don't understand what is it exactly.

For example. I need create mutable object in one thread and send to another. I don't need to share this object, just create, send and forget. But I have no idea how make this without using shared attribute and casting to/from it.

On Saturday, 19 May 2012 at 13:26:20 UTC, japplegame wrote:
> Multithreading in D confuses me more and more.
>
> import std.concurrency;
> import std.stdio;
> shared Tid tid;
> void main() {
>   send(cast(Tid)tid, "Hello, World");
> }
> void worker() {
>    writeln(receiveOnly!string);
> }
> shared static this() {
>   tid = cast(shared)spawn(&worker);
> }
>
> I hate these explicit casts. It is impossible sharing anything between threads without these ugly casts from/to shared. Seems like something wrong in program design when I'm forced to use explicit casts. But I don't understand what is it exactly.

You don't need to mark Tids as shared.

> For example. I need create mutable object in one thread and send to another. I don't need to share this object, just create, send and forget. But I have no idea how make this without using shared attribute and casting to/from it.

If you originally create it as shared, you don't need to do the casting.

> You don't need to mark Tids as shared.
Okay. I'm writting logger. Logger is global object and it is
running in its own separate thread (for example, writting logs to
remote database).
My application has several threads and all of them want to log
something. How to share this global logger between threads? I
think the simplest way is to share logger's thread tid and other
thread can send logs via this shared tid.
> If you originally create it as shared, you don't need to do the casting.
Yes. I don't need to cast to shared, but inside thread I get
shared object and can't store/call/pass it without casting away
that shared attribute. Or I should make shared everyting that
have deal with shared object.
I'am trying to follow Safe D concept, but it forbids casting away
shared.

On Saturday, 19 May 2012 at 21:13:14 UTC, japplegame wrote:
>> You don't need to mark Tids as shared.
> Okay. I'm writting logger. Logger is global object and it is
> running in its own separate thread (for example, writting logs to
> remote database).
> My application has several threads and all of them want to log
> something. How to share this global logger between threads? I
> think the simplest way is to share logger's thread tid and other
> thread can send logs via this shared tid.

public:
void startLogger(LogConstructorArgs args) {
    loggerTid = spawn(&loggerThread, args);
}

void log(string msg, OtherOptions oo) {
    loggerTid.send(LogMsg(msg, oo));
}

void stopLogger() {
    loggerTid.send(QuitMsg());
}

private:
Tid loggerTid;

struct LogMsg {
    string msg;
    OtherOptions oo;
}

struct QuitMsg {}

void loggerThread(LogConstructorArgs args) {
    Logger lg = new Logger(args);
    bool cont = true;
    while(cont) {
        receive((LogMsg lm) { lg.log(lm.msg, lm.oo); },
                (QuitMsg qm) { cont = false; });
    }
}

>> If you originally create it as shared, you don't need to do the casting.
> Yes. I don't need to cast to shared, but inside thread I get
> shared object and can't store/call/pass it without casting away
> that shared attribute. Or I should make shared everyting that
> have deal with shared object.
> I'am trying to follow Safe D concept, but it forbids casting away
> shared.

If you are passing objects between threads, make it shared. This might seem annoying, but in general you should try to shift your thinking into having thread-local objects and communicating via structs.

But when you use global/singleton objects (any case where there's one instance of the class), convert it into a thread, FROM

class <name> {
    this(<cons args>) {
        <constructor>
    }

    void <method1>() {
        // ...
    }

    int <method2>() {
        int result;
        // ...
        return result;
    }

    <private fields> // you encapsulate and have no public fields, right?
}

TO

void <name>Thread(<cons args>) {
    <private fields>
    <constructor>
    bool cont = true;
    while(cont) {
        receive(
            (<method1>Msg) {
                // ...
            },
            (Tid r, <method2>Msg) {
                int result;
                // ...
                r.send(<method2>ReturnMsg(result));
            }
            (QuitMsg qm) {cont = false;});
    }
}

> public:
> void startLogger(LogConstructorArgs args) {
>     loggerTid = spawn(&loggerThread, args);
> }
>
> void log(string msg, OtherOptions oo) {
>     loggerTid.send(LogMsg(msg, oo));
> }
>
> void stopLogger() {
>     loggerTid.send(QuitMsg());
> }
>
> private:
> Tid loggerTid;
>
> struct LogMsg {
>     string msg;
>     OtherOptions oo;
> }
>
> struct QuitMsg {}
>
> void loggerThread(LogConstructorArgs args) {
>     Logger lg = new Logger(args);
>     bool cont = true;
>     while(cont) {
>         receive((LogMsg lm) { lg.log(lm.msg, lm.oo); },
>                 (QuitMsg qm) { cont = false; });
>     }
> }
I don't understand. In this way I should call startLogger/stopLogger in every application thread because loggerTid is thread related and every application thread has its own instanse of loggerTid. Instead N+1 threads (N application and 1 logger) we will get 2*N threads (N application and N loggers). Also we should synchronize Logger class because many instances of them will run concurrently. This is terrible.
> If you are passing objects between threads, make it shared. This might seem annoying, but in general you should try to shift your thinking into having thread-local objects and communicating via structs.
>
> But when you use global/singleton objects (any case where there's one instance of the class), convert it into a thread, FROM
>
> class <name> {
>     this(<cons args>) {
>         <constructor>
>     }
>
>     void <method1>() {
>         // ...
>     }
>
>     int <method2>() {
>         int result;
>         // ...
>         return result;
>     }
>
>     <private fields> // you encapsulate and have no public fields, right?
> }
>
> TO
>
> void <name>Thread(<cons args>) {
>     <private fields>
>     <constructor>
>     bool cont = true;
>     while(cont) {
>         receive(
>             (<method1>Msg) {
>                 // ...
>             },
>             (Tid r, <method2>Msg) {
>                 int result;
>                 // ...
>                 r.send(<method2>ReturnMsg(result));
>             }
>             (QuitMsg qm) {cont = false;});
>     }
> }
Intresting idea. I will try it. Thanks.

On Sunday, 20 May 2012 at 04:09:50 UTC, japplegame wrote:
>> public:
>> void startLogger(LogConstructorArgs args) {
>>    loggerTid = spawn(&loggerThread, args);
>> }
>>
>> void log(string msg, OtherOptions oo) {
>>    loggerTid.send(LogMsg(msg, oo));
>> }
>>
>> void stopLogger() {
>>    loggerTid.send(QuitMsg());
>> }
>>
>> private:
>> Tid loggerTid;
>>
>> struct LogMsg {
>>    string msg;
>>    OtherOptions oo;
>> }
>>
>> struct QuitMsg {}
>>
>> void loggerThread(LogConstructorArgs args) {
>>    Logger lg = new Logger(args);
>>    bool cont = true;
>>    while(cont) {
>>        receive((LogMsg lm) { lg.log(lm.msg, lm.oo); },
>>                (QuitMsg qm) { cont = false; });
>>    }
>> }
> I don't understand. In this way I should call startLogger/stopLogger in every application thread because loggerTid is thread related and every application thread has its own instanse of loggerTid. Instead N+1 threads (N application and 1 logger) we will get 2*N threads (N application and N loggers). Also we should synchronize Logger class because many instances of them will run concurrently. This is terrible.

Tids aren't threads, they're thread-IDs, so you wouldn't be creating threads.

Though you're right in that tids would need to be shared (for some reason I assumed that child thread TLS storage was copied from its parents). In general, this shows an important principle of D-ish concurrent programming: avoid globals.

In your situation, here's what I'd do:

struct Logger {
    void start() { loggerTid = spawn(&loggerThread, args); }
    void stop() { loggerTid.send(QuitMsg()); }
    void log(string msg) { loggerTid.send(LogMsg(msg)); }
    Tid loggerTid;
}

Construct loggerThread like the class-to-thread I show below, so you don't need a Logger class, and make sure to start() at the beginning and stop() at the end, and pass your Logger struct around.

This may seem like an inconvenience compared to a shared synchronized object, but here's an interesting thought: once a working thread sends a message to the logger thread to log, it can continue with its work. This makes it more efficient than having to wait for the logging to finish before continuing with work, which is what you'd do with synchronized(this). You are only doing synchronized(queue) in the background.

What D is doing is making it easier to do it the right/efficient way (message passing), by making the usually simple way (shared objects) inconvenient.

Note: unfortunately, I'm not sure if the Logger will work due to this bug:
http://d.puremagic.com/issues/show_bug.cgi?id=4957
Hopefully it's fixed soon :(

>> If you are passing objects between threads, make it shared. This might seem annoying, but in general you should try to shift your thinking into having thread-local objects and communicating via structs.
>>
>> But when you use global/singleton objects (any case where there's one instance of the class), convert it into a thread, FROM
>>
>> class <name> {
>>    this(<cons args>) {
>>        <constructor>
>>    }
>>
>>    void <method1>() {
>>        // ...
>>    }
>>
>>    int <method2>() {
>>        int result;
>>        // ...
>>        return result;
>>    }
>>
>>    <private fields> // you encapsulate and have no public fields, right?
>> }
>>
>> TO
>>
>> void <name>Thread(<cons args>) {
>>    <private fields>
>>    <constructor>
>>    bool cont = true;
>>    while(cont) {
>>        receive(
>>            (<method1>Msg) {
>>                // ...
>>            },
>>            (Tid r, <method2>Msg) {
>>                int result;
>>                // ...
>>                r.send(<method2>ReturnMsg(result));
>>            }
>>            (QuitMsg qm) {cont = false;});
>>    }
>> }
> Intresting idea. I will try it. Thanks.

I like to thing of this idea as the "thread-object" paradigm. I've considered using D's great templating ability to allow you to make a class and have it be converted to a thread-object, but the __traits(getMember) stuff is confusing.

NMS

On May 19, 2012, at 2:13 PM, japplegame wrote:

>> You don't need to mark Tids as shared.
> Okay. I'm writting logger. Logger is global object and it is
> running in its own separate thread (for example, writting logs to
> remote database).
> My application has several threads and all of them want to log
> something. How to share this global logger between threads? I
> think the simplest way is to share logger's thread tid and other
> thread can send logs via this shared tid.

Maybe use register().


>> If you originally create it as shared, you don't need to do the casting.
> Yes. I don't need to cast to shared, but inside thread I get
> shared object and can't store/call/pass it without casting away
> that shared attribute. Or I should make shared everyting that
> have deal with shared object.
> I'am trying to follow Safe D concept, but it forbids casting away
> shared.

I'll admit that send() has only spotty support for shared.  This should be fixed.