On 09/25/2016 01:06 PM, Chris Wright wrote:
> On Sun, 25 Sep 2016 01:36:57 -0400, Nick Sabalausky wrote:
>> The "turning functions inside-out" effect of continuation passing is
>> exactly what input ranges suffer from[1] - and is exactly what stackless
>> coroutines *avoid*.
>
> The compiler inverts them for you.
>

Basically, yea. Soooooo much nicer than writing/maintaining inverted code by hand!

>> You may want to give C's protobuf or C#'s yield-based generator
>> functions a try. Yes, true, they're not as convenient as actual fibers,
>> but there's a LOT they let you do painlessly (far more painlessly than
>> writing input ranges[1]) without the fiber overhead - think opApply but
>> without the complete and total incompatibility with functions that
>> operate on ranges.
>
> Yeah, I really want something transparent.
>

Ah. Yea, totally transparent they are not. But I find them to be a solid enough abstraction that I have no issue whatsoever.

Keep in mind though, fibers and threads aren't really all that transparent either, there's a bunch of behind-the-scenes magic there too. Saving/restoring register state. Your threads will just *stop* at arbitrary points while some other code halfway across the codebase is executed. Etc. But they're solid enough as abstractions that we prefer the abstraction to dealing with the underlying mess transparently.

The same applies to nearly all programming constructs, even something as basic as function calls. Bunch of opaque hidden magic there. But it pretty much *just works* so we rarely need/want the transparency.

>> [1] Note I mean actually writing an input range itself. *Using* an input
>> range is wonderfully pleasant. But writing them turns all my logic
>> inside out, somewhat like Node.js-style callback hell (or ActionScript2
>> back in my day).
>
> Agreed. It's relatively easy to compose ranges, which is nice, but
> whenever I have to write one from scratch, I get a little sad.
>

That's why some solid abstraction magic would be very, very nice! :)

On 2016-09-25 20:16, Nick Sabalausky wrote:

> Resumable Function:
> -------------------
> This sounds like a vague general term to me. I would think it casually
> refers to any function that can be suspended/resumed: A Fiber-based
> coroutine, a stackless coroutine, or whatever else may or may not exist.

I think this is the term used in now in C++. It looks like it will have a lower level API for suspending and resuming functions. Then it's possible to build on top of this different concurrency constructs as libraries, like coroutines, stackless threads, goroutines, async/await and so on.

> I don't see why not, but there would be one notable drawback: You could
> only have one instance existing at a time (per thread).

But how does it work in languages which don't pass in the state explicitly, like with "yield" in C#?

Or async/await? I guess one doesn't call the function multiple times in the same way instead it's done automatically under the hood, passing in the state which might be stored in the "Task" that is returned?

> I'd have to look it up and refresh my memory. I just remember D has more
> restrictions regarding jumping to labels than C has. Something like:
>
> ------------------
> State myCoroutine(State s) {
>     yield "hello";
>
>     if(foo) {
>         yield " world";
>     }
>
>     yield "!";
> }
> ------------------
>
> A Protothreads-like preprocessing approach that used switch/case would
> turn that into:
>
> ------------------
> State myCoroutine(State s) {
> switch(s.jumpLabel){
>     case 0:
>     // Start converted code: //////////////
>
>     //yield "hello"; -> return+case:
>     return State(1, "hello");
>     case 1:
>
>     if(int foo = getFoo()) {
>         //yield " world"; ->
>         return State(2, " world"); return+case:
>         case 2:
>     }
>
>     //yield "!"; -> return+case:
>     return State(3, "!");
>     case 3:
>
>     // End converted code //////////////
>     break;
> }
> }
> ------------------
>
> That "if" really messed things up. AIUI, I don't think D will allow
> things like that.

The above code compiles, after removing the extra "return+case:".

-- 
/Jacob Carlborg

On 09/26/2016 02:42 AM, Jacob Carlborg wrote:
> On 2016-09-25 20:16, Nick Sabalausky wrote:
>> I don't see why not, but there would be one notable drawback: You could
>> only have one instance existing at a time (per thread).
>
> But how does it work in languages which don't pass in the state
> explicitly, like with "yield" in C#?
>
> Or async/await? I guess one doesn't call the function multiple times in
> the same way instead it's done automatically under the hood, passing in
> the state which might be stored in the "Task" that is returned?
>

I don't know about async/await, because all my C# work was with versions of C# that predated those. But as for C#'s yield functions, yes, that's my understanding: It's hidden magic automatically inserted by the compiler.

As a side note, that actually makes D's opApply a very interesting case, really:

Instead of doing like the other implementations and "yielding" by removing the top function from the callstack (ie "returning"), opApply does the oppposite: It yields by pushing whatever function it's yielding *to*, *onto* the callstack. So its state is stored on the stack, no need for magic. The downside is, that's exactly what makes it impossible for opApply to be used as a range (at least without introducing a true fiber to save/restore the callstack): popFront *must* return in order for the user's algorithm to call front and obtain the current element - because that's how the range interface works.

>
> The above code compiles, after removing the extra "return+case:".
>

Really? I may have to look into this more then.

Hmm, well for starters, according to <http://dlang.org/spec/statement.html#GotoStatement>:
"It is illegal for a GotoStatement to be used to skip initializations."

Although if that's true, I don't know why that example compiled. Maybe because "foo" wasn't used from "case 2:" onward? Maybe it failed to notice the initialization since it was in an if condition? Or maybe the compiler forgot to implement the same "can't skip initialization" check for switch/case?

On 2016-09-26 16:31, Nick Sabalausky wrote:

> Really? I may have to look into this more then.

Yes, try compiling it next time ;)

> Hmm, well for starters, according to
> <http://dlang.org/spec/statement.html#GotoStatement>:
> "It is illegal for a GotoStatement to be used to skip initializations."
>
> Although if that's true, I don't know why that example compiled. Maybe
> because "foo" wasn't used from "case 2:" onward? Maybe it failed to
> notice the initialization since it was in an if condition? Or maybe the
> compiler forgot to implement the same "can't skip initialization" check
> for switch/case?

It's a bug [1]. But it seems that if State is replaced with a plain int, the compiler will initialize the variable, even if it's later jumped over. So I assume, that when it works as it supposed to, it will be possible to jump like in C, but the compiler will still initialize the variables.

[1] https://issues.dlang.org/show_bug.cgi?id=16549

-- 
/Jacob Carlborg

On Saturday, 24 September 2016 at 08:03:15 UTC, Martin Nowak wrote:
> On Friday, 23 September 2016 at 20:57:49 UTC, Nick Sabalausky wrote:
>> were rejected because it was deemed both easy enough and preferable to get these features by modifying DMD to add behind-the-scenes AST magic to "assert".
>>
>> So...umm...yea...whatever happened to that beefed-up "assert" feature?
>
> assertPred!"=="(a, b);
> assertPred!"!"(a);
> assertPred!(std.range.equal)(a, b);

assertPred!"==" looks like it asserts not equal, which is the 1st problem with this, assertPred!"!=" looks even worse. I considered going down this route but... ugh.

>
> Seems to do most of what DIP83 does w/ expensive feature design, and compiler implementation work.
> Also http://code.dlang.org/packages/unit-threaded comes with a couple of test comparators, though it follows ruby rspec's bad idea of giving every comparator a name (which has to be learnt and documented).

I didn't want to give every comparator a name, this was the least bad option from my point of view. I'd rather `assert` be magical. Or, be able to have AST macros to get around the limitations of how operator overloading works in D. Don't get me wrong, the reason why it works the way it does is awesome in general, but limiting in this case in particular. I ended up implementing this:

foo.should == bar

But unfortunately it only works for equality. This is also possible:

foo.should.not == bar

But I can't do the same with any of the other operators AFAICR so I went with consistency even though the equality check is the most common.


Atila

On Saturday, 24 September 2016 at 14:08:30 UTC, Nick Sabalausky wrote:
> On 09/24/2016 04:03 AM, Martin Nowak wrote:
>> [...]
>
> Yea, incidentally, I just started using unit-threaded for the first time this week, and so far, for the most part, I really quite like it a lot. But those comparator functions give me bad flashbacks of old-school Java.
>
> Hmm, I wonder if Atila would be amenable to a more assertPred-like function in unit-threaded. Maybe a `should!"=="(leftSide, rightSide)` would fit in well. Or better yet, also adding in something like that that trick used in Andre's really nice recently proposed "dump" function (or whatever the name of it was), where it also prints out the actual argument provided in addition to the argument's value.

See my answer to Martin. "!" next to "==" just looks wrong.

Atila

On 2016-09-26 16:31, Nick Sabalausky wrote:

> I don't know about async/await, because all my C# work was with versions
> of C# that predated those. But as for C#'s yield functions, yes, that's
> my understanding: It's hidden magic automatically inserted by the compiler.

This talk [1] from cppcon 2016, C++ Coroutines: Under the covers, gave some really nice insight how they implement coroutines in C++. It basically works like in Protothreads. The secret how the state is handled, I believe, is that all coroutines need to return a specific type that, I'm guessing, contains the state. Like generator<T> or task<T>. This return value is later used to resume the function. Directly calling the coroutine again will start a new invocation with a new state.

I'm guessing it works the same in C# [2]. The state is hidden in the IEnumerable that is returned, which will later be used in a foreach loop which will resume the coroutine.

[1] https://www.youtube.com/watch?v=bzAbe1VzhKk&index=51&list=PLHTh1InhhwT7J5jl4vAhO1WvGHUUFgUQH

[2] https://msdn.microsoft.com/en-us/library/9k7k7cf0.aspx

-- 
/Jacob Carlborg