The notion of output range has been a tad vague in the past; up until now a range that wanted to qualify as an output range had to define a method called put.

That definition is awkward though. For example, the std.algorithm.copy() primitive should work for an output range, but also for some other ranges that allow assignment to front.

In related news, there's been this burning desire regarding a lightweight output range defined as simply a delegate that accepts const(char)[]. That range is an output range all right, and the way you output to it is by simply calling the delegate!

All of the three mechanisms above are desirable for certain types. So imagine this dialog (paraphrased from http://www.youtube.com/watch?v=MrTsuvykUZk):

Guy 1: We need a good output range interface, and we have three good candidates. We should make every one an output range.

Guy 2: What do you mean, every one?

Guy 1: E-V-E-R-Y O-N-E!!!!

So, I defined isOutputRange!R to yield true if at least one of these conditions above is met: put() definition, input range with assignable front, delegate.

Please refer to this code and this doc:

http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227

http://erdani.com/d/phobos/std_range.html

This confers a ton of flexibility to programmers who need to define output ranges. I've also reworked std.format to use put(r, e) so now it works with all output ranges seamlessly.

Any thoughts would be appreciated. Thanks!


Andrei

On 07/11/2010 08:17 PM, Andrei Alexandrescu wrote:
[snip]

Forgot to mention one detail: now the free function std.range.put() serves as a convenient dispatcher for any kind of output range.

Andrei

== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> The notion of output range has been a tad vague in the past; up until
> now a range that wanted to qualify as an output range had to define a
> method called put.
> That definition is awkward though. For example, the std.algorithm.copy()
> primitive should work for an output range, but also for some other
> ranges that allow assignment to front.
> In related news, there's been this burning desire regarding a
> lightweight output range defined as simply a delegate that accepts
> const(char)[]. That range is an output range all right, and the way you
> output to it is by simply calling the delegate!
> All of the three mechanisms above are desirable for certain types. So
> imagine this dialog (paraphrased from
> http://www.youtube.com/watch?v=MrTsuvykUZk):
> Guy 1: We need a good output range interface, and we have three good
> candidates. We should make every one an output range.
> Guy 2: What do you mean, every one?
> Guy 1: E-V-E-R-Y O-N-E!!!!
> So, I defined isOutputRange!R to yield true if at least one of these
> conditions above is met: put() definition, input range with assignable
> front, delegate.
> Please refer to this code and this doc:
> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
> http://erdani.com/d/phobos/std_range.html
> This confers a ton of flexibility to programmers who need to define
> output ranges. I've also reworked std.format to use put(r, e) so now it
> works with all output ranges seamlessly.
> Any thoughts would be appreciated. Thanks!
> Andrei

== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> The notion of output range has been a tad vague in the past; up until
> now a range that wanted to qualify as an output range had to define a
> method called put.
> That definition is awkward though. For example, the std.algorithm.copy()
> primitive should work for an output range, but also for some other
> ranges that allow assignment to front.
> In related news, there's been this burning desire regarding a
> lightweight output range defined as simply a delegate that accepts
> const(char)[]. That range is an output range all right, and the way you
> output to it is by simply calling the delegate!
> All of the three mechanisms above are desirable for certain types. So
> imagine this dialog (paraphrased from
> http://www.youtube.com/watch?v=MrTsuvykUZk):
> Guy 1: We need a good output range interface, and we have three good
> candidates. We should make every one an output range.
> Guy 2: What do you mean, every one?
> Guy 1: E-V-E-R-Y O-N-E!!!!
> So, I defined isOutputRange!R to yield true if at least one of these
> conditions above is met: put() definition, input range with assignable
> front, delegate.
> Please refer to this code and this doc:
> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
> http://erdani.com/d/phobos/std_range.html
> This confers a ton of flexibility to programmers who need to define
> output ranges. I've also reworked std.format to use put(r, e) so now it
> works with all output ranges seamlessly.
> Any thoughts would be appreciated. Thanks!
> Andrei

Nice.  I had quietly wondered why input ranges with assignable front aren't also output ranges, but never gotten around to asking.  Two comments:

1.  What happens if you run out of space in the input range variety?  I guess it throws?

2.  Would there be a "standard" way of signaling how much stuff was written to the input range variety?  I guess that since functions that output their results via output ranges would usually return void, they could return an integer instead indicating how much stuff was written.

3.  While we're on the subject of improving output ranges, I was just thinking before I read your post that it would be nice to have a Tee type in std.range, since outputting to exactly one output range is kind of inflexible.  Such a type would itself be an output range.  It would take in N output ranges where N > 1 as instantiation parameters, and pass any input received to all of the underlying ranges.

The use case I thought of for this is when I'm generating tons of data through a monte carlo simulation and don't want to store it all in memory.  Both Summary in dstats and Histogram in dflplot can be used as output ranges.  I'd love to write a function that outputs results to an output range and use Tee to get both summary statistics and a histogram.

On 07/11/2010 08:34 PM, dsimcha wrote:
> 1.  What happens if you run out of space in the input range variety?  I guess it
> throws?

It's up to the range. Most will throw.

> 2.  Would there be a "standard" way of signaling how much stuff was written to the
> input range variety?  I guess that since functions that output their results via
> output ranges would usually return void, they could return an integer instead
> indicating how much stuff was written.

There is no standard way at the moment. I think not throwing implies all passed data has been written.

> 3.  While we're on the subject of improving output ranges, I was just thinking
> before I read your post that it would be nice to have a Tee type in std.range,
> since outputting to exactly one output range is kind of inflexible.  Such a type
> would itself be an output range.  It would take in N output ranges where N>  1 as
> instantiation parameters, and pass any input received to all of the underlying
> ranges.

Yah, tee would be great.

> The use case I thought of for this is when I'm generating tons of data through a
> monte carlo simulation and don't want to store it all in memory.  Both Summary in
> dstats and Histogram in dflplot can be used as output ranges.  I'd love to write a
> function that outputs results to an output range and use Tee to get both summary
> statistics and a histogram.

... and checkpoint all that to a file.


Andrei

On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu < SeeWebsiteForEmail@erdani.org> wrote:

> In related news, there's been this burning desire regarding a lightweight
> output range defined as simply a delegate that accepts const(char)[]. That
> range is an output range all right, and the way you output to it is by
> simply calling the delegate!
> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>



>
> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>
>
It's still early where I live, but... For the callable case, why just accepting E[] instead of any range with E as element? Though, thinking about it, I right now have no idea how to put that into a template constraint, given only R and E.

Hmm...

On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
> wrote:
>
>     In related news, there's been this burning desire regarding a
>     lightweight output range defined as simply a delegate that accepts
>     const(char)[]. That range is an output range all right, and the way
>     you output to it is by simply calling the delegate!
>     <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>
>     http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>
>
> It's still early where I live, but... For the callable case, why just
> accepting E[] instead of any range with E as element? Though, thinking
> about it, I right now have no idea how to put that into a template
> constraint, given only R and E.
>
> Hmm...

Good point. I haven't thought of it that way - I used arrays as a lingua franca buffer. Your suggestion is interesting. I see a risk of infinite regression in writing the constraint, but the idea warrants more discussion.

Andrei

On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>> wrote:
>>
>>     In related news, there's been this burning desire regarding a
>>     lightweight output range defined as simply a delegate that accepts
>>     const(char)[]. That range is an output range all right, and the way
>>     you output to it is by simply calling the delegate!
>>     <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>
>>     http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>
>>
>> It's still early where I live, but... For the callable case, why just
>> accepting E[] instead of any range with E as element? Though, thinking
>> about it, I right now have no idea how to put that into a template
>> constraint, given only R and E.
>>
>> Hmm...
>
> Good point. I haven't thought of it that way - I used arrays as a lingua franca buffer. Your suggestion is interesting. I see a risk of infinite regression in writing the constraint, but the idea warrants more discussion.

Wait, isn't a delegate that accepts a type T an output range of type T?  Why does the argument have to be an array/range?

For example, a delegate that accepts a string is a range of strings, is it not?  Or do you consider it a range of immutable(char)?

For example, I'd expect to be able to use a push_back delegate on an array-type of integers as an output range.

-Steve

On 07/12/2010 07:44 AM, Steven Schveighoffer wrote:
> On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>>> wrote:
>>>
>>> In related news, there's been this burning desire regarding a
>>> lightweight output range defined as simply a delegate that accepts
>>> const(char)[]. That range is an output range all right, and the way
>>> you output to it is by simply calling the delegate!
>>> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>>
>>>
>>> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>>
>>>
>>>
>>> It's still early where I live, but... For the callable case, why just
>>> accepting E[] instead of any range with E as element? Though, thinking
>>> about it, I right now have no idea how to put that into a template
>>> constraint, given only R and E.
>>>
>>> Hmm...
>>
>> Good point. I haven't thought of it that way - I used arrays as a
>> lingua franca buffer. Your suggestion is interesting. I see a risk of
>> infinite regression in writing the constraint, but the idea warrants
>> more discussion.
>
> Wait, isn't a delegate that accepts a type T an output range of type T?
> Why does the argument have to be an array/range?

Efficiency - see the doFormat disaster.

> For example, a delegate that accepts a string is a range of strings, is
> it not? Or do you consider it a range of immutable(char)?

Good call. Probably I need to handle ranges of arrays differently. (The problem has already come up, but I punted on it.)

> For example, I'd expect to be able to use a push_back delegate on an
> array-type of integers as an output range.

Would it hurt to define push_back for arrays too? The thing is, if you can output an array you can always output one occasional element. But if you only know how to output an element, having the client side output arrays in a loop can be quite slow.


Andrei

On Mon, 12 Jul 2010 10:41:51 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 07/12/2010 07:44 AM, Steven Schveighoffer wrote:
>> On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>>>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>>>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>>>> wrote:
>>>>
>>>> In related news, there's been this burning desire regarding a
>>>> lightweight output range defined as simply a delegate that accepts
>>>> const(char)[]. That range is an output range all right, and the way
>>>> you output to it is by simply calling the delegate!
>>>> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>>>
>>>>
>>>> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>>>
>>>>
>>>>
>>>> It's still early where I live, but... For the callable case, why just
>>>> accepting E[] instead of any range with E as element? Though, thinking
>>>> about it, I right now have no idea how to put that into a template
>>>> constraint, given only R and E.
>>>>
>>>> Hmm...
>>>
>>> Good point. I haven't thought of it that way - I used arrays as a
>>> lingua franca buffer. Your suggestion is interesting. I see a risk of
>>> infinite regression in writing the constraint, but the idea warrants
>>> more discussion.
>>
>> Wait, isn't a delegate that accepts a type T an output range of type T?
>> Why does the argument have to be an array/range?
>
> Efficiency - see the doFormat disaster.

Yes, I'm not saying that a delegate that accepts a range of T cannot be an output range of T, I just wondered why it *has* to be a range.

>
>> For example, a delegate that accepts a string is a range of strings, is
>> it not? Or do you consider it a range of immutable(char)?
>
> Good call. Probably I need to handle ranges of arrays differently. (The problem has already come up, but I punted on it.)

It seems to me to be similar to appending -- you can append an element or an array of elements.  But appending the individual element makes sense in a lot of cases, despite performance.

If output ranges were restricted to deal only with stream data (i.e. char) then I'd agree only accepting an array of data is a good idea.

>> For example, I'd expect to be able to use a push_back delegate on an
>> array-type of integers as an output range.
>
> Would it hurt to define push_back for arrays too? The thing is, if you can output an array you can always output one occasional element. But if you only know how to output an element, having the client side output arrays in a loop can be quite slow.

If I always have to do something like this in order to append a single element:

put(r, (&elem)[0..1]);

Then the concept of output ranges is much less attractive to me.

In some cases, appending a single element is all that works.  For example, a linked list could be an output range, and passing it an array is not going to be any more optimal than passing individual elements.

-Steve

On 07/12/2010 09:59 AM, Steven Schveighoffer wrote:
> On Mon, 12 Jul 2010 10:41:51 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 07/12/2010 07:44 AM, Steven Schveighoffer wrote:
>>> On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org> wrote:
>>>
>>>> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>>>>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>>>>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>>>>> wrote:
>>>>>
>>>>> In related news, there's been this burning desire regarding a
>>>>> lightweight output range defined as simply a delegate that accepts
>>>>> const(char)[]. That range is an output range all right, and the way
>>>>> you output to it is by simply calling the delegate!
>>>>> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>>>>
>>>>>
>>>>>
>>>>> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> It's still early where I live, but... For the callable case, why just
>>>>> accepting E[] instead of any range with E as element? Though, thinking
>>>>> about it, I right now have no idea how to put that into a template
>>>>> constraint, given only R and E.
>>>>>
>>>>> Hmm...
>>>>
>>>> Good point. I haven't thought of it that way - I used arrays as a
>>>> lingua franca buffer. Your suggestion is interesting. I see a risk of
>>>> infinite regression in writing the constraint, but the idea warrants
>>>> more discussion.
>>>
>>> Wait, isn't a delegate that accepts a type T an output range of type T?
>>> Why does the argument have to be an array/range?
>>
>> Efficiency - see the doFormat disaster.
>
> Yes, I'm not saying that a delegate that accepts a range of T cannot be
> an output range of T, I just wondered why it *has* to be a range.
>
>>
>>> For example, a delegate that accepts a string is a range of strings, is
>>> it not? Or do you consider it a range of immutable(char)?
>>
>> Good call. Probably I need to handle ranges of arrays differently.
>> (The problem has already come up, but I punted on it.)
>
> It seems to me to be similar to appending -- you can append an element
> or an array of elements. But appending the individual element makes
> sense in a lot of cases, despite performance.
>
> If output ranges were restricted to deal only with stream data (i.e.
> char) then I'd agree only accepting an array of data is a good idea.
>
>>> For example, I'd expect to be able to use a push_back delegate on an
>>> array-type of integers as an output range.
>>
>> Would it hurt to define push_back for arrays too? The thing is, if you
>> can output an array you can always output one occasional element. But
>> if you only know how to output an element, having the client side
>> output arrays in a loop can be quite slow.
>
> If I always have to do something like this in order to append a single
> element:
>
> put(r, (&elem)[0..1]);

No, the library does that. Look here:

http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L306

Andrei