Walter,

I've just been going back over my old (and now fairly crusty) coroutine implementation, and realised that macros could make these a *major* feature of D.

Ideally, I'd like to see a syntax like this:

> coro showfiles(char[] msg)(char[], char[]) // <-- input,output types
> {
>     char[] path;
>     while( (path = yield) != null ) // <-- yield is a protected method
>     {
>         writefln("%s: %s", msg, path);
>     }
> }

Which a macro could then unwind into something ugly that the user doesn't have to see:

> class showfiles : CoroutineT!(char[], char[], char[])
> {
>     mixin CoroutineMixin!(char[], char[], char[]);
>     void run(char[] msg)
>     {
>         char[] path;
>         while( (path = this.yield) != null )
>         {
>             writefln("%s: %s", msg, path);
>         }
>     }
> }

Which is basically how it looks now.  Then, using it is nice and clean:

> scope sf = showfiles("Hi there");
> sf("file1");
> sf("file2");

I've tried to come up with other syntaxes using current constructs, and I can't really get any better than having the user define a subclass, mixin a template, and then implement a run method.  It just feels... unwieldy.

Will the sort of syntax in the first example be possible?  If not, do you have any thoughts on how to improve the above kind of construct (either now or with D 2.0)?

	-- Daniel

-- 
int getRandomNumber()
{
    return 4; // chosen by fair dice roll.
              // guaranteed to be random.
}

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/

Daniel Keep wrote:
> Walter,
> 
> I've just been going back over my old (and now fairly crusty) coroutine
> implementation, and realised that macros could make these a *major*
> feature of D.
> 
> Ideally, I'd like to see a syntax like this:
> 
>> coro showfiles(char[] msg)(char[], char[]) // <-- input,output types
>> {
>>     char[] path;
>>     while( (path = yield) != null ) // <-- yield is a protected method
>>     {
>>         writefln("%s: %s", msg, path);
>>     }
>> }
> [...]

I thought Tango had Fibers?

Dave

David B. Held wrote:
> Daniel Keep wrote:
>> Walter,
>>
>> I've just been going back over my old (and now fairly crusty) coroutine implementation, and realised that macros could make these a *major* feature of D.
>>
>> Ideally, I'd like to see a syntax like this:
>>
>>> coro showfiles(char[] msg)(char[], char[]) // <-- input,output types
>>> {
>>>     char[] path;
>>>     while( (path = yield) != null ) // <-- yield is a protected method
>>>     {
>>>         writefln("%s: %s", msg, path);
>>>     }
>>> }
>> [...]
> 
> I thought Tango had Fibers?
> 
> Dave

Cooperative multithreading does not a coroutine make.  Fibers, IIRC, are just stack-based threads.  Coroutines are *entirely different*, although in this particular case they are implemented *using* stack threads.

In any case, I have a working library right now... it's just that the syntax to use them is rather ugly and a bit messy.

	-- Daniel

-- 
int getRandomNumber()
{
    return 4; // chosen by fair dice roll.
              // guaranteed to be random.
}

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/

Daniel Keep wrote:
> 
> David B. Held wrote:
>> Daniel Keep wrote:
>>> Walter,
>>>
>>> I've just been going back over my old (and now fairly crusty) coroutine
>>> implementation, and realised that macros could make these a *major*
>>> feature of D.
>>>
>>> Ideally, I'd like to see a syntax like this:
>>>
>>>> coro showfiles(char[] msg)(char[], char[]) // <-- input,output types
>>>> {
>>>>     char[] path;
>>>>     while( (path = yield) != null ) // <-- yield is a protected method
>>>>     {
>>>>         writefln("%s: %s", msg, path);
>>>>     }
>>>> }
>>> [...]
>> I thought Tango had Fibers?
>>
>> Dave
> 
> Cooperative multithreading does not a coroutine make.  Fibers, IIRC, are
> just stack-based threads.  Coroutines are *entirely different*, although
> in this particular case they are implemented *using* stack threads.
> 
> In any case, I have a working library right now... it's just that the
> syntax to use them is rather ugly and a bit messy.

For what it's worth, Mikola Lysenko has coroutines as a part of his StackThreads implementation, and it should be pretty easy to adapt them to use Tango's Fibers.  The page is here: http://www.assertfalse.com/projects.shtml


Sean

Sean Kelly wrote:
> 
> For what it's worth, Mikola Lysenko has coroutines as a part of his StackThreads implementation, and it should be pretty easy to adapt them to use Tango's Fibers.  The page is here: http://www.assertfalse.com/projects.shtml

My fault.  It seems that's the one you wrote :-p


Sean

Daniel Keep wrote:
> 
> [...]
> Cooperative multithreading does not a coroutine make.  Fibers, IIRC, are
> just stack-based threads.  Coroutines are *entirely different*, although
> in this particular case they are implemented *using* stack threads.
> [...]

Wikipedia seems to think they are essentially the same thing:

http://en.wikipedia.org/wiki/Fiber_%28computer_science%29

Fibers are less than stack-based threads because they are cooperative. Whereas in a coroutine you write "return", in a fiber you write "yield".  Otherwise, I believe they are closely isomorphic.

Dave

Daniel Keep wrote:
> Will the sort of syntax in the first example be possible?  If not, do
> you have any thoughts on how to improve the above kind of construct
> (either now or with D 2.0)?

I'm not understanding your example. Where is the macro?

Walter Bright wrote:
> Daniel Keep wrote:
>> Will the sort of syntax in the first example be possible?  If not, do you have any thoughts on how to improve the above kind of construct (either now or with D 2.0)?
> 
> I'm not understanding your example. Where is the macro?

I want "coro" to be the macro.  Basically, I want it to look like a function, except it has a second argument list, where you can stick in the input/output types.

You said before that you can do specialisations on syntax, so I suppose it might be something like...

macro coro(name : name(typelist)(arglist){block}, typelist, arglist, block)
{
    // blah
}

At this stage, I'm just trying to describe the kind of syntax I'd like to be able to make, and then you say "you can't do that, but you can do this", and then I say "well what if we changed this little bit here", and so forth until either one of us goes mad or we come to some sort of middle-ground :P

Plus, I figure the more potential use-cases we show you, the better the end result will be :)

	-- Daniel

-- 
int getRandomNumber()
{
    return 4; // chosen by fair dice roll.
              // guaranteed to be random.
}

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/

David B. Held wrote:
> Daniel Keep wrote:
>>
>> [...]
>> Cooperative multithreading does not a coroutine make.  Fibers, IIRC, are
>> just stack-based threads.  Coroutines are *entirely different*, although
>> in this particular case they are implemented *using* stack threads.
>> [...]
> 
> Wikipedia seems to think they are essentially the same thing:
> 
> http://en.wikipedia.org/wiki/Fiber_%28computer_science%29
> 
> Fibers are less than stack-based threads because they are cooperative.
> Whereas in a coroutine you write "return", in a fiber you write "yield".
>  Otherwise, I believe they are closely isomorphic.
> 
> Dave

Hmm... looks that way.  I think mine are probably closer to generators or channels than strictly coroutines, the difference being that Fibers just do a transfer of control to the given fiber, whereas mine work more like a function call.

Actually, thinking about it, I'm not entirely sure if mine classify as coroutines or not... oh bother :P

	-- Daniel

-- 
int getRandomNumber()
{
    return 4; // chosen by fair dice roll.
              // guaranteed to be random.
}

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/

Walter Bright wrote:
> Don Clugston wrote:
>> Walter Bright wrote:
>>> You'll be able to do things like:
>>>  >>
>>>  >> macro print(arg)
>>>  >> {
>>>  >> writefln(__FILE__, __LINE__, arg);
>>>  >> }
>>
>> This looks great already. However, I don't see any specific "abstract syntax tree" syntax in this. Is that still a work-in-progress?
> 
> The beauty of it is that (a+b), before semantic analysis, is an AST! So there's no funky new grammar to learn.

Fantastic! Does this mean that you just use arg.stringof to get the expression? And 'arg' to evaluate it?

>> An observation:
>> My version of vector operation expression templates is mostly working now, using the existing mixins. It deals with ASTs by constructing a string representing the operations to be performed, and a tuple containing all of the parameters. The operations string is of the form
>> "g+=((a+b)*c)+((d*e)+f)", where a, b, c... correspond to T[0], T[1], T[2]... of the tuple T.
>>
>> A compile-time function converts the string into postfix, and then another CFTE function converts that into optimised x87 asm, which is mixed in.
> 
> Wow! I think the macros can help by obviating the need for the user to do the mixin manually.

Actually usage is OK already. Here's an example:
------
    auto p = Vec([1.0, 2, 18]);
    auto q = Vec([3.5L, 1.1, 3.8]);
    auto r = Vec([17.0f, 28.1, 1]);
    q -= ((p+r)*18.0L*314.1L - (p-r))* 35;
    real d = dot(r, p + 3.7*r);
------
And the only reason for Vec() is because otherwise you can't create array operations. In the first stage, the Vector structs returned by Vec() are removed, and only built-in real[], double[], and float[] vectors are put into the tuple.
BTW the generated asm is really good quality; in each case, it generates the best code I can write by hand.

>> Unexpectedly, I found that it's actually very nice to have flattened tuples. If an AST could recognise that two parameters are the same, it could avoid duplication in the tuple, something that a tree can't easily do:
>> eg   somevec+= othervec + somevec*3;
>> could become "T[1]+=T[0]+(T[1]*3)" which allows better code generation in the final stage.
> 
> There's a way to do this using specialization:
> 
>     macro foo(somevec : somevec+=othervec+somevec, othervec)
> 
> which will only match for parameters of the form:
> 
>     foo( a += b + a);
> 
>     somevec => a
>     othervec => b
> 
> It works analogously to how you can match type patterns in templates using specializations.

Now that could be useful...
Are we going to able to write macros for member functions, and operator overloads?