On Sat, Aug 24, 2013 at 3:26 PM, bearophile <bearophileHUGS@lycos.com>wrote:

> David Nadlinger:
>
>
>  It's a cute idea, but not really practical, I'm afraid – Goroutines are
>> managed by a scheduler in the Go runtime library, whereas D threads directly map to OS threads.
>>
>
> Can't Rory McGuire add a scheduler to his code? How much code does it take?
>
> Bye,
> bearophile
>

I imagine that it will be fine on Linux because threads truly are lightweight on Linux, but its not going to be great on windows.

Go's scheduler is basically the same as the way vibe.d works. Fibers. I haven't tried it but I could use fibers to do the same sort of thing, though it would be easiest to use in Vibe.d because even in go you have to do something that will wait/sleep in order to have your program advance unless you set GOPROCS to  a value greater than 1. Go's scheduler is not preemptive so there are the equivalent of Fiber.yield() spread throughout the standard library, I think.

I'm not sure how threads + fibers would have to interact in such a system.

On Saturday, 24 August 2013 at 20:03:58 UTC, Rory McGuire wrote:
> On Sat, Aug 24, 2013 at 3:26 PM, bearophile <bearophileHUGS@lycos.com>wrote:
>
>> David Nadlinger:
>>
>>
>>  It's a cute idea, but not really practical, I'm afraid – Goroutines are
>>> managed by a scheduler in the Go runtime library, whereas D threads
>>> directly map to OS threads.
>>>
>>
>> Can't Rory McGuire add a scheduler to his code? How much code does it take?
>>
>> Bye,
>> bearophile
>>
>
> I imagine that it will be fine on Linux because threads truly are
> lightweight on Linux, but its not going to be great on windows.

Funny, I always thought otherwise, because Windows only has threads.

Processes are just a means of grouping threads on Windows, as there
isn't the distinction between threads and processes that UNIX systems used to make.

Then again, I lost track how the performance on Linux systems changed across the whole Processes -> LinuxThreads -> NPTL -> Posix Threads evolution.

--
Paulo

On 24 Aug 2013 23:55, "Paulo Pinto" <pjmp@progtools.org> wrote:
>
> Funny, I always thought otherwise, because Windows only has threads.
>
> Processes are just a means of grouping threads on Windows, as there isn't the distinction between threads and processes that UNIX systems
used to make.
>
> Then again, I lost track how the performance on Linux systems changed
across the whole Processes -> LinuxThreads -> NPTL -> Posix Threads evolution.
>
> --
> Paulo

Just did a quick search to see if I was right. I'm not sure if it's still the case but it appears that is actually the scheduler that causes windows to be much slower than Linux.

Ideally one would want a hybrid of threads and fibres anyway. I wonder how much of the standard library would need to change.

On Sat, Aug 24, 2013 at 3:26 PM, bearophile <bearophileHUGS@lycos.com>wrote:

> David Nadlinger:
>
>
>  It's a cute idea, but not really practical, I'm afraid – Goroutines are
>> managed by a scheduler in the Go runtime library, whereas D threads directly map to OS threads.
>>
>
> Can't Rory McGuire add a scheduler to his code? How much code does it take?
>
> Bye,
> bearophile
>


Here is a basic co-operative scheduler based version of go!(F)  that uses a channel to keep things simple, its co-op so Fiber.yield has to be called at some point. If the channel detects its in a Fiber it calls yield if there is nothing available in it:

void go(alias F)() {
    scheduler._ = new Fiber(F);
}

shared chan!Fiber scheduler; // channel contains Fibers waiting for their
time slice
static this () {
    if (scheduler is null) {
        scheduler = makeChan!Fiber(100);

        // create the workers
        auto goprocs = environment.get("GOPROCS");
        int num_threads = 1;
        if (goprocs != null) {
            num_threads = to!int(goprocs);
        }
        foreach (i; 0..num_threads) {
            // create threads that process the live fibers
            auto t = new Thread(() {
                for (;;) {
                    auto fiber = scheduler._;
                    fiber.call();
                    if (fiber.state != Fiber.State.TERM) {
                        scheduler._ (fiber);
                    }
                }
                });
            t.start();
        }

    }
}

Just need to figure out a way to detect when main() has exited.

On 08/26/2013 12:55 AM, Rory McGuire wrote:
>
> shared chan!Fiber scheduler; // channel contains Fibers waiting for
> their time slice
> static this () {
>      if (scheduler is null) {


You want 'shared static this' instead to avoid a race condition.

Awesome thanks, thought what I did there was dodgy. It was really weird when this() started multiple schedulers at least now I see the obvious reason.

BTW: gist is at: https://gist.github.com/rjmcguire/6336931

Could someone point me at the correct linked list to use inside the channel. I'd prefer to use a range container of some kind if it exists in the std lib. I tried SList and had a bad experience hence the custom implementation.



On Mon, Aug 26, 2013 at 1:21 AM, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 08/26/2013 12:55 AM, Rory McGuire wrote:
>
>>
>> shared chan!Fiber scheduler; // channel contains Fibers waiting for
>> their time slice
>> static this () {
>>      if (scheduler is null) {
>>
>
>
> You want 'shared static this' instead to avoid a race condition.
>

I've made the scheduler a bit more inteligent and the channel implementation is now backed by a lock free queue. https://github.com/rjmcguire/goport/blob/tip/wrapper2.d for a example using libev

https://github.com/rjmcguire/goport/blob/tip/goroutine.d for the "go" routines

https://github.com/rjmcguire/goport/blob/tip/channel.d channel implementation, the channel still has a read lock because I didn't like looping wasting cpu cycles/battery, I may change this to use the same loop as the select(...) expression with a timeout that backs off.

https://github.com/rjmcguire/goport/blob/tip/concurrentlinkedqueue.d queue implementation


there is a problem with the file EV handler in that it can get spurios READ events and I can't seem to set it to non-blocking.

The select(...) implementation and usage is quite interesting because it works very much like Go's select statement.


On Mon, Aug 26, 2013 at 1:28 AM, Rory McGuire <rjmcguire@gmail.com> wrote:

> Awesome thanks, thought what I did there was dodgy. It was really weird when this() started multiple schedulers at least now I see the obvious reason.
>
> BTW: gist is at: https://gist.github.com/rjmcguire/6336931
>
> Could someone point me at the correct linked list to use inside the channel. I'd prefer to use a range container of some kind if it exists in the std lib. I tried SList and had a bad experience hence the custom implementation.
>
>
>
> On Mon, Aug 26, 2013 at 1:21 AM, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 08/26/2013 12:55 AM, Rory McGuire wrote:
>>
>>>
>>> shared chan!Fiber scheduler; // channel contains Fibers waiting for
>>> their time slice
>>> static this () {
>>>      if (scheduler is null) {
>>>
>>
>>
>> You want 'shared static this' instead to avoid a race condition.
>>
>
>

On Aug 24, 2013, at 1:59 AM, David <d@dav1d.de> wrote:

> Daemonic Threads often end with a segfault, so if your main thread exists, the other threads will probably segfault.

By default, sure.  But with daemon threads you really want to have some kind of shutdown mechanism inside a static dtor somewhere.  The goal is more to have threads that don't implicitly block app shutdown.