On Sat, 2015-03-28 at 11:16 +0000, ketmar via Digitalmars-d-announce wrote:
> […]
> 
> hm. yes, that was "coroutines on steroids".

But that's the point isn't it:

1. Processes are too heavyweight, invent threads.
2. We have masses of cores, let's map threads to cores via the kernel.
3. Processes and threads are too heavyweight, invent lightweight
threads (aka fibres, sort of).

It could be argued that it is all just co-routines underneath, but I think that would be missing the point that we have 55 years more experience of doing these things since that single processor operating system model was created. We really should be doing this all a lot better these days.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Sat, 2015-03-28 at 13:27 +0000, via Digitalmars-d-announce wrote:
> […]
> 
> Nah. Cooperative multitasking is a sorry excuse that belongs to the 80s. This should be as transparent as possible. You cannot insert "yield" into an external library.

1960s.

Software developers have spent 50+ years trying to use 1960s operating system concurrency techniques for all of software development.  It's time there was some innovation and less conservatism in software concurrency and parallelism.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Sat, 2015-03-28 at 12:52 +0100, Sönke Ludwig via Digitalmars-d-announce wrote:
> […]
> 
> You can access TLS from an event callback just as easy as from a fiber.
[…]

TLS is the evil here. Anyone working with TLS is either writing an operating system or doing it wrong.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 28 March 2015 at 13:52:54 UTC, ketmar wrote:
> then you have to switch to some functional language, preferably one that
> does cps transformations in the compiler. as you told somewhere else, D
> is too broad to be restricted to this.

Fibers is really not a system level thing. So I don't think D MUST have them as it is claimed that D is meant to be a system level language. I rate D against C/C++/Rust, but not Go per se. I see that Bearophile sometimes express that D should be be pitted with Java/C# rather than C/C++/Rust, but that does not fit for what is claimed for D…

The bottom line is: if you decide to make fibers a language/runtime feature you have to make them work well across the board because you are evaluated against other languages that provide them. If it is just an "extras" library features, then you can make do with "basic features", nobody expects more.

Am 28.03.2015 um 13:32 schrieb "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>":
> On Saturday, 28 March 2015 at 11:52:34 UTC, Sönke Ludwig wrote:
>> You can access TLS from an event callback just as easy as from a fiber.
>
> Yes, but it is much easier to verify that you don't hold onto references
> to TLS if get rid of arbitrary call stacks when moving to a new thread.

It's not mainly about holding references to TLS data, but about program correctness. You store something in a TLS variable and the next time you read it, there is something different in it. This is not only an issue for your own code, but also for external libraries that you have no control or even insight of.

Apart from that, what is stopping you to hold such references implicitly in a callback closure?

>
>> And why can't you do the same with fibers and schedule the fibers
>> accordingly?
>
> You could, but that's even more work since you then need to encode
> progress in a way the scheduler can use to estimate when the task can
> complete and when it must complete.

The fiber part is purely additive. Anything you can to to schedule events in an event based programming model, you can do in a fiber backed one, too. You just have the additional state of the fiber that gets carried around, nothing more.

>
>> It's you who brought up the randomization argument. Tasks are assigned
>> to a more or less random thread that is currently in the scheduling
>> phase, so that your constructed situation is simply *highly* unlikely.
>
> I don't understand how randomization can help you here.

Your constructed case will simply not happen in practice.

>
>> They *do* get balanced over threads, just like requests get balanced
>> over instances by the load balancer, even if requests
>
> A good load balancer measure back-pressure (load information from the
> instance) and fire up new instances.

That depends a lot on your infrastructure, but is irrelevant to the point. Tasks get distributed among threads with a sufficiently large number of tasks (like it would happen for a busy web service), you'll have a high load on all threads, so it simply doesn't matter if you move tasks between threads.

If you have a low number of requests you may be able to avoid some bad corner cases, but only if you did something stupid in the first place, like mixing long CPU computations without any yield() calls with I/O processing tasks in the same thread (since you seem like a smart person I'll leave it up to you construct cases where moving between threads doesn't help either).

>
>> are not moved between instances. But IMO it doesn't make sense to go
>> further with this argument with some actual benchmarks. It's not at
>> all as clear as you'd like what the effects on overall performance and
>> on average/~maximum latency are in practice for different applications.
>
> This is something you can do on paper. A language feature should support
> a wide set of applications.

*I* can't do that on paper. I invite you to do it and then we can verify your claims with actual measurements. If you don't, this is nothing more than hot air.

Am 28.03.2015 um 15:33 schrieb Russel Winder via Digitalmars-d-announce:
> On Sat, 2015-03-28 at 12:52 +0100, Sönke Ludwig via Digitalmars-d-announce wrote:
>> […]
>>
>> You can access TLS from an event callback just as easy as from a
>> fiber.
> […]
>
> TLS is the evil here. Anyone working with TLS is either writing an
> operating system or doing it wrong.
>

As long as we are talking about a closed system that works exclusively on this fiber based concurrency model, I completely agree with you (fiber local storage would be fine, though).

But we have the "unfortunate" situation that the language is not an isolated ecosystem. There are many C libraries that do thread-specific things in one way or another, or worse, make use of ordinary global variables without any protection against data races, and we simply cannot ignore that.

On Sat, 28 Mar 2015 14:28:00 +0000, Russel Winder via Digitalmars-d-announce wrote:

> It could be argued that it is all just co-routines underneath, but I think that would be missing the point that we have 55 years more experience of doing these things since that single processor operating system model was created. We really should be doing this all a lot better these days.

yet current CPUs are still the same as 50 years before, that is the problem. ;-)

On Sat, 2015-03-28 at 17:57 +0000, ketmar via Digitalmars-d-announce wrote:
> On Sat, 28 Mar 2015 14:28:00 +0000, Russel Winder via Digitalmars-d-announce wrote:
> 
> > It could be argued that it is all just co-routines underneath, but
> > I
> > think that would be missing the point that we have 55 years more
> > experience of doing these things since that single processor
> > operating
> > system model was created. We really should be doing this all a lot
> > better these days.
> 
> yet current CPUs are still the same as 50 years before, that is the problem. ;-)

I'd suggest that a Intel x86_64 of 2015 bears only a passing relationship to an IBM 360 of the 1960s.

It is true that hardware design has been constrained by a weird constraint that no-one has investigated alternative architectures to the register/CPU that software people insist is the only way forward.

With all the transistors available per mm² these days, it is about
time we investigated alternate, implicitly parallel ways of working.
Intel had a go a few years ago with various alternative dataflow based
architectures, but they were told by the software people that they had
no future because software inertia was more important than innovation.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 28 March 2015 at 17:57:35 UTC, ketmar wrote:
> On Sat, 28 Mar 2015 14:28:00 +0000, Russel Winder via
> Digitalmars-d-announce wrote:
>
>> It could be argued that it is all just co-routines underneath, but I
>> think that would be missing the point that we have 55 years more
>> experience of doing these things since that single processor operating
>> system model was created. We really should be doing this all a lot
>> better these days.
>
> yet current CPUs are still the same as 50 years before, that is the
> problem. ;-)

heavily disagree

On Saturday, 28 March 2015 at 18:39:47 UTC, Russel Winder wrote:
> On Sat, 2015-03-28 at 17:57 +0000, ketmar via Digitalmars-d-announce wrote:
>> On Sat, 28 Mar 2015 14:28:00 +0000, Russel Winder via
>> Digitalmars-d-announce wrote:
>> 
>> > It could be argued that it is all just co-routines underneath, but I
>> > think that would be missing the point that we have 55 years more
>> > experience of doing these things since that single processor operating
>> > system model was created. We really should be doing this all a lot
>> > better these days.
>> 
>> yet current CPUs are still the same as 50 years before, that is the problem. ;-)
>
> I'd suggest that a Intel x86_64 of 2015 bears only a passing
> relationship to an IBM 360 of the 1960s.
>
> It is true that hardware design has been constrained by a weird
> constraint that no-one has investigated alternative architectures to
> the register/CPU that software people insist is the only way forward.
>
> With all the transistors available per mm² these days, it is about
> time we investigated alternate, implicitly parallel ways of working.
> Intel had a go a few years ago with various alternative dataflow based
> architectures, but they were told by the software people that they had
> no future because software inertia was more important than innovation.
> 

Thoughts on mill architecture?