View mode: basic / threaded / horizontal-split · Log in · Help
January 20, 2013
Re: Internal and external iteration, fibers
On Sun, Jan 20, 2013 at 5:27 PM, deadalnix <deadalnix@gmail.com> wrote:

>> This can be implemented a lot better looking in D. (My quick hack already
>> looks better.) But I think we should first build a general-purpose DSEL
>> library.
>
>
> DSEL ?

Domain-Specific Embedded Language, I guess.

Timon is just making a distinction between internal DSL (hence DSEL),
that you 'embed' in your code and external DSL (like build files...)
that are, well, in external files.
January 20, 2013
Re: Internal and external iteration, fibers
On Sun, 20 Jan 2013 03:45:00 +0100
Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 01/19/2013 10:46 PM, Nick Sabalausky wrote:
> > On Sat, 19 Jan 2013 09:45:25 +0100
> > "qznc" <qznc@go.to> wrote:
> >
> >> On Friday, 18 January 2013 at 17:59:36 UTC, Nick Sabalausky wrote:
> >>> Then there's C/C++ which has libs that offer what are known as
> >>> "stackless fibers". These utilize the preprocessor, along with
> >>> switch
> >>> and goto, to accomplish the same thing that (AIUI) C# does for
> >>> its
> >>> coroutines: It lets the user write a normal coroutine, with a
> >>> normal
> >>> yield, which then gets trivially rewritten behind-the-scenes
> >>> into an
> >>> event loop (with NO actual fibers involved). I'm not sure to
> >>> what
> >>> extent this would be possible in D. If it is, the lack of
> >>> preprocessor
> >>> would probably make it much less nice-looking to use than the
> >>> C/C++/C#
> >>> versions. (Not that I'd want a preprocessor.)
> >>
> >> Is this also known as protothreads?
> >>
> >> http://dunkels.com/adam/pt/
> >
> > Yea. In fact, that's the exact same lib I've used.
> >
> 
> This can be implemented a lot better looking in D. (My quick hack 
> already looks better.) But I think we should first build a 
> general-purpose DSEL library.

If we need to resort to using D-as-a-DSL-inside-D to get decent
coroutines, then that just further proves the need for a real coroutine
support in the language.
January 21, 2013
Re: Internal and external iteration, fibers
On Friday, 18 January 2013 at 17:59:36 UTC, Nick Sabalausky wrote:
> Fibers: Too much performance overhead to be a general solution. 
> Only
> good for, as an example, heavily I/O-bound stuff.

Where does the overhead come from? Is the overhead from using 
fibers the only problem for implementing coroutines?

I ask, because if except for the overhead, fibers are a good 
general solution, then it makes sense to determine if anything 
can be done to lessen the overhead before trying to implement yet 
another solution.

--rt
January 21, 2013
Re: Internal and external iteration, fibers
On Sunday, 20 January 2013 at 20:19:56 UTC, Nick Sabalausky wrote:
> If we need to resort to using D-as-a-DSL-inside-D to get decent
> coroutines, then that just further proves the need for a real 
> coroutine
> support in the language.

A good part of the job is done with Fibers. The biggest problem I 
see is module constructor and switching from TLS to fiber local 
storage.
January 21, 2013
Re: Internal and external iteration, fibers
On Mon, 21 Jan 2013 03:27:49 +0100
"deadalnix" <deadalnix@gmail.com> wrote:

> On Sunday, 20 January 2013 at 20:19:56 UTC, Nick Sabalausky wrote:
> > If we need to resort to using D-as-a-DSL-inside-D to get decent
> > coroutines, then that just further proves the need for a real 
> > coroutine
> > support in the language.
> 
> A good part of the job is done with Fibers. The biggest problem I 
> see is module constructor and switching from TLS to fiber local 
> storage.

That's no good for general-purpose, then. Fibers may be light-weight
compared to threads, but (as I learned when *I* made a fiber-based
solution last year) fibers still have way to much overhead to be a
*general-purpose* approach the the matter. For more info, see jerro's
posts here:

http://forum.dlang.org/thread/jno6o5$qtb$1@digitalmars.com

So like I already said a few posts back, *ALL* the currently-possible
solutions in D have issues:

opApply: Not a range; awkward to use

Manually-Created Input Range: Event-loop-style instead of
procedural. Yuck.

Real Fibers: Too slow to be general-purpose.

Stackless "fibers": Requires gross syntactical contortions much like
opApply does.
January 21, 2013
Re: Internal and external iteration, fibers
On Mon, 21 Jan 2013 01:18:48 +0100
"Rob T" <alanb@ucora.com> wrote:

> On Friday, 18 January 2013 at 17:59:36 UTC, Nick Sabalausky wrote:
> > Fibers: Too much performance overhead to be a general solution. 
> > Only
> > good for, as an example, heavily I/O-bound stuff.
> 
> Where does the overhead come from? Is the overhead from using 
> fibers the only problem for implementing coroutines?
> 

Real fibers have their own call stacks, so creating them and switching
contexts between them on every iteration can never be made as fast as a
simple for-loop or event-loop (both of which are computationally
trivial). Stackless "fibers" OTOH can do this easily since they
literally *are* trivial switch-driven event-loops under the hood, but
just written in a cleaner way.

> I ask, because if except for the overhead, fibers are a good 
> general solution, then it makes sense to determine if anything 
> can be done to lessen the overhead before trying to implement yet 
> another solution.
> 

Yea, they *would* be a good general solution, and I'm sure the overhead
can be decreased, but AIUI I don't think it's even theoretically
possible for them to optimized enough to compete with any of the other
approaches as a general-purpose thing.

The other stuff can be optimized down as far as being little more than
an increment per iteration. But AIUI, using a real fiber would require
two context-switches per iteration, so I'm not sure that can ever
compete.
January 21, 2013
Re: Internal and external iteration, fibers
On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky wrote:
> Stackless "fibers": Requires gross syntactical contortions much 
> like
> opApply does.

Can you explain more what a stackless fiber is ? From the linked 
posted above I did really understood, as the example code clearly 
call functions, which require stack.
January 21, 2013
Re: Internal and external iteration, fibers
On 01/21/2013 10:08 AM, deadalnix wrote:
> On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky wrote:
>> Stackless "fibers": Requires gross syntactical contortions much like
>> opApply does.
>
> Can you explain more what a stackless fiber is ? From the linked posted
> above I did really understood, as the example code clearly call
> functions, which require stack.

A stackless fiber does not have the execution stack as part of its 
context. (Therefore it cannot yield from nested function calls.)
January 21, 2013
Re: Internal and external iteration, fibers
On Monday, 21 January 2013 at 09:18:34 UTC, Timon Gehr wrote:
> On 01/21/2013 10:08 AM, deadalnix wrote:
>> On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky 
>> wrote:
>>> Stackless "fibers": Requires gross syntactical contortions 
>>> much like
>>> opApply does.
>>
>> Can you explain more what a stackless fiber is ? From the 
>> linked posted
>> above I did really understood, as the example code clearly call
>> functions, which require stack.
>
> A stackless fiber does not have the execution stack as part of 
> its context. (Therefore it cannot yield from nested function 
> calls.)

Ho yeah, I looked at the implementation. It is mostly made of 
macro that create a function with a big switch in it.

Why can't this be done in D ? What are the major problems ?
January 21, 2013
Re: Internal and external iteration, fibers
On Mon, 21 Jan 2013 11:48:58 +0100
"deadalnix" <deadalnix@gmail.com> wrote:

> On Monday, 21 January 2013 at 09:18:34 UTC, Timon Gehr wrote:
> > On 01/21/2013 10:08 AM, deadalnix wrote:
> >> On Monday, 21 January 2013 at 08:27:28 UTC, Nick Sabalausky 
> >> wrote:
> >>> Stackless "fibers": Requires gross syntactical contortions 
> >>> much like
> >>> opApply does.
> >>
> >> Can you explain more what a stackless fiber is ? From the 
> >> linked posted
> >> above I did really understood, as the example code clearly call
> >> functions, which require stack.
> >
> > A stackless fiber does not have the execution stack as part of 
> > its context. (Therefore it cannot yield from nested function 
> > calls.)
> 
> Ho yeah, I looked at the implementation. It is mostly made of 
> macro that create a function with a big switch in it.
> 
> Why can't this be done in D ? What are the major problems ?

I don't know whether or not it can be done in D. But, if it can be
done, it would definitely require awkward syntax. Probably more awkward
than the preprocessor-based syntax of the C/C++ version. (Not that I
want a preprocessor in D.)

Maybe you could do it by sticking the whole coroutine into a string
literal that gets ripped apart and reassembled by a CTFE D parser, but
that would just be so clumsy and error-prone, and frankly far more
complex than should really be necessary, that I'd call it more of a
kludge than a solution. And really, if I have to write D code inside a
string literal to use it, that alone indicates that we're looking down
the wrong path.
1 2 3 4 5 6
Top | Discussion index | About this forum | D home