January 20, 2013
On Sun, Jan 20, 2013 at 5:27 PM, deadalnix <deadalnix@gmail.com> wrote:

>> This can be implemented a lot better looking in D. (My quick hack already looks better.) But I think we should first build a general-purpose DSEL library.
>
>
> DSEL ?

Domain-Specific Embedded Language, I guess.

Timon is just making a distinction between internal DSL (hence DSEL),
that you 'embed' in your code and external DSL (like build files...)
that are, well, in external files.
January 20, 2013
On Sun, 20 Jan 2013 03:45:00 +0100
Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 01/19/2013 10:46 PM, Nick Sabalausky wrote:
> > On Sat, 19 Jan 2013 09:45:25 +0100
> > "qznc" <qznc@go.to> wrote:
> >
> >> On Friday, 18 January 2013 at 17:59:36 UTC, Nick Sabalausky wrote:
> >>> Then there's C/C++ which has libs that offer what are known as
> >>> "stackless fibers". These utilize the preprocessor, along with
> >>> switch
> >>> and goto, to accomplish the same thing that (AIUI) C# does for
> >>> its
> >>> coroutines: It lets the user write a normal coroutine, with a
> >>> normal
> >>> yield, which then gets trivially rewritten behind-the-scenes
> >>> into an
> >>> event loop (with NO actual fibers involved). I'm not sure to
> >>> what
> >>> extent this would be possible in D. If it is, the lack of
> >>> preprocessor
> >>> would probably make it much less nice-looking to use than the
> >>> C/C++/C#
> >>> versions. (Not that I'd want a preprocessor.)
> >>
> >> Is this also known as protothreads?
> >>
> >> http://dunkels.com/adam/pt/
> >
> > Yea. In fact, that's the exact same lib I've used.
> >
> 
> This can be implemented a lot better looking in D. (My quick hack already looks better.) But I think we should first build a general-purpose DSEL library.

If we need to resort to using D-as-a-DSL-inside-D to get decent coroutines, then that just further proves the need for a real coroutine support in the language.


January 21, 2013
On Friday, 18 January 2013 at 17:59:36 UTC, Nick Sabalausky wrote:
> Fibers: Too much performance overhead to be a general solution. Only
> good for, as an example, heavily I/O-bound stuff.

Where does the overhead come from? Is the overhead from using fibers the only problem for implementing coroutines?

I ask, because if except for the overhead, fibers are a good general solution, then it makes sense to determine if anything can be done to lessen the overhead before trying to implement yet another solution.

--rt

January 21, 2013
On Sunday, 20 January 2013 at 20:19:56 UTC, Nick Sabalausky wrote:
> If we need to resort to using D-as-a-DSL-inside-D to get decent
> coroutines, then that just further proves the need for a real coroutine
> support in the language.

A good part of the job is done with Fibers. The biggest problem I see is module constructor and switching from TLS to fiber local storage.
January 21, 2013
On Mon, 21 Jan 2013 03:27:49 +0100
"deadalnix" <deadalnix@gmail.com> wrote:

> On Sunday, 20 January 2013 at 20:19:56 UTC, Nick Sabalausky wrote:
> > If we need to resort to using D-as-a-DSL-inside-D to get decent
> > coroutines, then that just further proves the need for a real
> > coroutine
> > support in the language.
> 
> A good part of the job is done with Fibers. The biggest problem I see is module constructor and switching from TLS to fiber local storage.

That's no good for general-purpose, then. Fibers may be light-weight compared to threads, but (as I learned when *I* made a fiber-based solution last year) fibers still have way to much overhead to be a *general-purpose* approach the the matter. For more info, see jerro's posts here:

http://forum.dlang.org/thread/jno6o5$qtb$1@digitalmars.com

So like I already said a few posts back, *ALL* the currently-possible solutions in D have issues:

opApply: Not a range; awkward to use

Manually-Created Input Range: Event-loop-style instead of procedural. Yuck.

Real Fibers: Too slow to be general-purpose.

Stackless "fibers": Requires gross syntactical contortions much like opApply does.

January 21, 2013
On Mon, 21 Jan 2013 01:18:48 +0100
"Rob T" <alanb@ucora.com> wrote:

> On Friday, 18 January 2013 at 17:59:36 UTC, Nick Sabalausky wrote:
> > Fibers: Too much performance overhead to be a general solution.
> > Only
> > good for, as an example, heavily I/O-bound stuff.
> 
> Where does the overhead come from? Is the overhead from using fibers the only problem for implementing coroutines?
> 

Real fibers have their own call stacks, so creating them and switching contexts between them on every iteration can never be made as fast as a simple for-loop or event-loop (both of which are computationally trivial). Stackless "fibers" OTOH can do this easily since they literally *are* trivial switch-driven event-loops under the hood, but just written in a cleaner way.

> I ask, because if except for the overhead, fibers are a good general solution, then it makes sense to determine if anything can be done to lessen the overhead before trying to implement yet another solution.
> 

Yea, they *would* be a good general solution, and I'm sure the overhead can be decreased, but AIUI I don't think it's even theoretically possible for them to optimized enough to compete with any of the other approaches as a general-purpose thing.

The other stuff can be optimized down as far as being little more than an increment per iteration. But AIUI, using a real fiber would require two context-switches per iteration, so I'm not sure that can ever compete.

January 21, 2013
On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky wrote:
> Stackless "fibers": Requires gross syntactical contortions much like
> opApply does.

Can you explain more what a stackless fiber is ? From the linked posted above I did really understood, as the example code clearly call functions, which require stack.
January 21, 2013
On 01/21/2013 10:08 AM, deadalnix wrote:
> On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky wrote:
>> Stackless "fibers": Requires gross syntactical contortions much like
>> opApply does.
>
> Can you explain more what a stackless fiber is ? From the linked posted
> above I did really understood, as the example code clearly call
> functions, which require stack.

A stackless fiber does not have the execution stack as part of its context. (Therefore it cannot yield from nested function calls.)
January 21, 2013
On Monday, 21 January 2013 at 09:18:34 UTC, Timon Gehr wrote:
> On 01/21/2013 10:08 AM, deadalnix wrote:
>> On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky wrote:
>>> Stackless "fibers": Requires gross syntactical contortions much like
>>> opApply does.
>>
>> Can you explain more what a stackless fiber is ? From the linked posted
>> above I did really understood, as the example code clearly call
>> functions, which require stack.
>
> A stackless fiber does not have the execution stack as part of its context. (Therefore it cannot yield from nested function calls.)

Ho yeah, I looked at the implementation. It is mostly made of macro that create a function with a big switch in it.

Why can't this be done in D ? What are the major problems ?
January 21, 2013
On Mon, 21 Jan 2013 11:48:58 +0100
"deadalnix" <deadalnix@gmail.com> wrote:

> On Monday, 21 January 2013 at 09:18:34 UTC, Timon Gehr wrote:
> > On 01/21/2013 10:08 AM, deadalnix wrote:
> >> On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky wrote:
> >>> Stackless "fibers": Requires gross syntactical contortions
> >>> much like
> >>> opApply does.
> >>
> >> Can you explain more what a stackless fiber is ? From the
> >> linked posted
> >> above I did really understood, as the example code clearly call
> >> functions, which require stack.
> >
> > A stackless fiber does not have the execution stack as part of its context. (Therefore it cannot yield from nested function calls.)
> 
> Ho yeah, I looked at the implementation. It is mostly made of macro that create a function with a big switch in it.
> 
> Why can't this be done in D ? What are the major problems ?

I don't know whether or not it can be done in D. But, if it can be done, it would definitely require awkward syntax. Probably more awkward than the preprocessor-based syntax of the C/C++ version. (Not that I want a preprocessor in D.)

Maybe you could do it by sticking the whole coroutine into a string literal that gets ripped apart and reassembled by a CTFE D parser, but that would just be so clumsy and error-prone, and frankly far more complex than should really be necessary, that I'd call it more of a kludge than a solution. And really, if I have to write D code inside a string literal to use it, that alone indicates that we're looking down the wrong path.