View mode: basic / threaded / horizontal-split · Log in · Help
July 12, 2010
Overhauling the notion of output range
The notion of output range has been a tad vague in the past; up until 
now a range that wanted to qualify as an output range had to define a 
method called put.

That definition is awkward though. For example, the std.algorithm.copy() 
primitive should work for an output range, but also for some other 
ranges that allow assignment to front.

In related news, there's been this burning desire regarding a 
lightweight output range defined as simply a delegate that accepts 
const(char)[]. That range is an output range all right, and the way you 
output to it is by simply calling the delegate!

All of the three mechanisms above are desirable for certain types. So 
imagine this dialog (paraphrased from 
http://www.youtube.com/watch?v=MrTsuvykUZk):

Guy 1: We need a good output range interface, and we have three good 
candidates. We should make every one an output range.

Guy 2: What do you mean, every one?

Guy 1: E-V-E-R-Y O-N-E!!!!

So, I defined isOutputRange!R to yield true if at least one of these 
conditions above is met: put() definition, input range with assignable 
front, delegate.

Please refer to this code and this doc:

http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227

http://erdani.com/d/phobos/std_range.html

This confers a ton of flexibility to programmers who need to define 
output ranges. I've also reworked std.format to use put(r, e) so now it 
works with all output ranges seamlessly.

Any thoughts would be appreciated. Thanks!


Andrei
July 12, 2010
Re: Overhauling the notion of output range
On 07/11/2010 08:17 PM, Andrei Alexandrescu wrote:
[snip]

Forgot to mention one detail: now the free function std.range.put() 
serves as a convenient dispatcher for any kind of output range.

Andrei
July 12, 2010
Re: Overhauling the notion of output range
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> The notion of output range has been a tad vague in the past; up until
> now a range that wanted to qualify as an output range had to define a
> method called put.
> That definition is awkward though. For example, the std.algorithm.copy()
> primitive should work for an output range, but also for some other
> ranges that allow assignment to front.
> In related news, there's been this burning desire regarding a
> lightweight output range defined as simply a delegate that accepts
> const(char)[]. That range is an output range all right, and the way you
> output to it is by simply calling the delegate!
> All of the three mechanisms above are desirable for certain types. So
> imagine this dialog (paraphrased from
> http://www.youtube.com/watch?v=MrTsuvykUZk):
> Guy 1: We need a good output range interface, and we have three good
> candidates. We should make every one an output range.
> Guy 2: What do you mean, every one?
> Guy 1: E-V-E-R-Y O-N-E!!!!
> So, I defined isOutputRange!R to yield true if at least one of these
> conditions above is met: put() definition, input range with assignable
> front, delegate.
> Please refer to this code and this doc:
> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
> http://erdani.com/d/phobos/std_range.html
> This confers a ton of flexibility to programmers who need to define
> output ranges. I've also reworked std.format to use put(r, e) so now it
> works with all output ranges seamlessly.
> Any thoughts would be appreciated. Thanks!
> Andrei

== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> The notion of output range has been a tad vague in the past; up until
> now a range that wanted to qualify as an output range had to define a
> method called put.
> That definition is awkward though. For example, the std.algorithm.copy()
> primitive should work for an output range, but also for some other
> ranges that allow assignment to front.
> In related news, there's been this burning desire regarding a
> lightweight output range defined as simply a delegate that accepts
> const(char)[]. That range is an output range all right, and the way you
> output to it is by simply calling the delegate!
> All of the three mechanisms above are desirable for certain types. So
> imagine this dialog (paraphrased from
> http://www.youtube.com/watch?v=MrTsuvykUZk):
> Guy 1: We need a good output range interface, and we have three good
> candidates. We should make every one an output range.
> Guy 2: What do you mean, every one?
> Guy 1: E-V-E-R-Y O-N-E!!!!
> So, I defined isOutputRange!R to yield true if at least one of these
> conditions above is met: put() definition, input range with assignable
> front, delegate.
> Please refer to this code and this doc:
> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
> http://erdani.com/d/phobos/std_range.html
> This confers a ton of flexibility to programmers who need to define
> output ranges. I've also reworked std.format to use put(r, e) so now it
> works with all output ranges seamlessly.
> Any thoughts would be appreciated. Thanks!
> Andrei

Nice.  I had quietly wondered why input ranges with assignable front aren't also
output ranges, but never gotten around to asking.  Two comments:

1.  What happens if you run out of space in the input range variety?  I guess it
throws?

2.  Would there be a "standard" way of signaling how much stuff was written to the
input range variety?  I guess that since functions that output their results via
output ranges would usually return void, they could return an integer instead
indicating how much stuff was written.

3.  While we're on the subject of improving output ranges, I was just thinking
before I read your post that it would be nice to have a Tee type in std.range,
since outputting to exactly one output range is kind of inflexible.  Such a type
would itself be an output range.  It would take in N output ranges where N > 1 as
instantiation parameters, and pass any input received to all of the underlying
ranges.

The use case I thought of for this is when I'm generating tons of data through a
monte carlo simulation and don't want to store it all in memory.  Both Summary in
dstats and Histogram in dflplot can be used as output ranges.  I'd love to write a
function that outputs results to an output range and use Tee to get both summary
statistics and a histogram.
July 12, 2010
Re: Overhauling the notion of output range
On 07/11/2010 08:34 PM, dsimcha wrote:
> 1.  What happens if you run out of space in the input range variety?  I guess it
> throws?

It's up to the range. Most will throw.

> 2.  Would there be a "standard" way of signaling how much stuff was written to the
> input range variety?  I guess that since functions that output their results via
> output ranges would usually return void, they could return an integer instead
> indicating how much stuff was written.

There is no standard way at the moment. I think not throwing implies all 
passed data has been written.

> 3.  While we're on the subject of improving output ranges, I was just thinking
> before I read your post that it would be nice to have a Tee type in std.range,
> since outputting to exactly one output range is kind of inflexible.  Such a type
> would itself be an output range.  It would take in N output ranges where N>  1 as
> instantiation parameters, and pass any input received to all of the underlying
> ranges.

Yah, tee would be great.

> The use case I thought of for this is when I'm generating tons of data through a
> monte carlo simulation and don't want to store it all in memory.  Both Summary in
> dstats and Histogram in dflplot can be used as output ranges.  I'd love to write a
> function that outputs results to an output range and use Tee to get both summary
> statistics and a histogram.

... and checkpoint all that to a file.


Andrei
July 12, 2010
Re: Overhauling the notion of output range
On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu <
SeeWebsiteForEmail@erdani.org> wrote:

> In related news, there's been this burning desire regarding a lightweight
> output range defined as simply a delegate that accepts const(char)[]. That
> range is an output range all right, and the way you output to it is by
> simply calling the delegate!
> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>



>
> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>
>
It's still early where I live, but... For the callable case, why just
accepting E[] instead of any range with E as element? Though, thinking about
it, I right now have no idea how to put that into a template constraint,
given only R and E.

Hmm...
July 12, 2010
Re: Overhauling the notion of output range
On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
> wrote:
>
>     In related news, there's been this burning desire regarding a
>     lightweight output range defined as simply a delegate that accepts
>     const(char)[]. That range is an output range all right, and the way
>     you output to it is by simply calling the delegate!
>     <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>
>     http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>
>
> It's still early where I live, but... For the callable case, why just
> accepting E[] instead of any range with E as element? Though, thinking
> about it, I right now have no idea how to put that into a template
> constraint, given only R and E.
>
> Hmm...

Good point. I haven't thought of it that way - I used arrays as a lingua 
franca buffer. Your suggestion is interesting. I see a risk of infinite 
regression in writing the constraint, but the idea warrants more discussion.

Andrei
July 12, 2010
Re: Overhauling the notion of output range
On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail@erdani.org> wrote:

> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>> wrote:
>>
>>     In related news, there's been this burning desire regarding a
>>     lightweight output range defined as simply a delegate that accepts
>>     const(char)[]. That range is an output range all right, and the way
>>     you output to it is by simply calling the delegate!
>>     <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>
>>     http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>
>>
>> It's still early where I live, but... For the callable case, why just
>> accepting E[] instead of any range with E as element? Though, thinking
>> about it, I right now have no idea how to put that into a template
>> constraint, given only R and E.
>>
>> Hmm...
>
> Good point. I haven't thought of it that way - I used arrays as a lingua  
> franca buffer. Your suggestion is interesting. I see a risk of infinite  
> regression in writing the constraint, but the idea warrants more  
> discussion.

Wait, isn't a delegate that accepts a type T an output range of type T?   
Why does the argument have to be an array/range?

For example, a delegate that accepts a string is a range of strings, is it  
not?  Or do you consider it a range of immutable(char)?

For example, I'd expect to be able to use a push_back delegate on an  
array-type of integers as an output range.

-Steve
July 12, 2010
Re: Overhauling the notion of output range
On 07/12/2010 07:44 AM, Steven Schveighoffer wrote:
> On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>>> wrote:
>>>
>>> In related news, there's been this burning desire regarding a
>>> lightweight output range defined as simply a delegate that accepts
>>> const(char)[]. That range is an output range all right, and the way
>>> you output to it is by simply calling the delegate!
>>> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>>
>>>
>>> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>>
>>>
>>>
>>> It's still early where I live, but... For the callable case, why just
>>> accepting E[] instead of any range with E as element? Though, thinking
>>> about it, I right now have no idea how to put that into a template
>>> constraint, given only R and E.
>>>
>>> Hmm...
>>
>> Good point. I haven't thought of it that way - I used arrays as a
>> lingua franca buffer. Your suggestion is interesting. I see a risk of
>> infinite regression in writing the constraint, but the idea warrants
>> more discussion.
>
> Wait, isn't a delegate that accepts a type T an output range of type T?
> Why does the argument have to be an array/range?

Efficiency - see the doFormat disaster.

> For example, a delegate that accepts a string is a range of strings, is
> it not? Or do you consider it a range of immutable(char)?

Good call. Probably I need to handle ranges of arrays differently. (The 
problem has already come up, but I punted on it.)

> For example, I'd expect to be able to use a push_back delegate on an
> array-type of integers as an output range.

Would it hurt to define push_back for arrays too? The thing is, if you 
can output an array you can always output one occasional element. But if 
you only know how to output an element, having the client side output 
arrays in a loop can be quite slow.


Andrei
July 12, 2010
Re: Overhauling the notion of output range
On Mon, 12 Jul 2010 10:41:51 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail@erdani.org> wrote:

> On 07/12/2010 07:44 AM, Steven Schveighoffer wrote:
>> On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>>>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>>>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>>>> wrote:
>>>>
>>>> In related news, there's been this burning desire regarding a
>>>> lightweight output range defined as simply a delegate that accepts
>>>> const(char)[]. That range is an output range all right, and the way
>>>> you output to it is by simply calling the delegate!
>>>> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>>>
>>>>
>>>> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>>>
>>>>
>>>>
>>>> It's still early where I live, but... For the callable case, why just
>>>> accepting E[] instead of any range with E as element? Though, thinking
>>>> about it, I right now have no idea how to put that into a template
>>>> constraint, given only R and E.
>>>>
>>>> Hmm...
>>>
>>> Good point. I haven't thought of it that way - I used arrays as a
>>> lingua franca buffer. Your suggestion is interesting. I see a risk of
>>> infinite regression in writing the constraint, but the idea warrants
>>> more discussion.
>>
>> Wait, isn't a delegate that accepts a type T an output range of type T?
>> Why does the argument have to be an array/range?
>
> Efficiency - see the doFormat disaster.

Yes, I'm not saying that a delegate that accepts a range of T cannot be an  
output range of T, I just wondered why it *has* to be a range.

>
>> For example, a delegate that accepts a string is a range of strings, is
>> it not? Or do you consider it a range of immutable(char)?
>
> Good call. Probably I need to handle ranges of arrays differently. (The  
> problem has already come up, but I punted on it.)

It seems to me to be similar to appending -- you can append an element or  
an array of elements.  But appending the individual element makes sense in  
a lot of cases, despite performance.

If output ranges were restricted to deal only with stream data (i.e. char)  
then I'd agree only accepting an array of data is a good idea.

>> For example, I'd expect to be able to use a push_back delegate on an
>> array-type of integers as an output range.
>
> Would it hurt to define push_back for arrays too? The thing is, if you  
> can output an array you can always output one occasional element. But if  
> you only know how to output an element, having the client side output  
> arrays in a loop can be quite slow.

If I always have to do something like this in order to append a single  
element:

put(r, (&elem)[0..1]);

Then the concept of output ranges is much less attractive to me.

In some cases, appending a single element is all that works.  For example,  
a linked list could be an output range, and passing it an array is not  
going to be any more optimal than passing individual elements.

-Steve
July 12, 2010
Re: Overhauling the notion of output range
On 07/12/2010 09:59 AM, Steven Schveighoffer wrote:
> On Mon, 12 Jul 2010 10:41:51 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 07/12/2010 07:44 AM, Steven Schveighoffer wrote:
>>> On Mon, 12 Jul 2010 02:21:25 -0400, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org> wrote:
>>>
>>>> On 07/12/2010 12:45 AM, Philippe Sigaud wrote:
>>>>> On Mon, Jul 12, 2010 at 03:17, Andrei Alexandrescu
>>>>> <SeeWebsiteForEmail@erdani.org <mailto:SeeWebsiteForEmail@erdani.org>>
>>>>> wrote:
>>>>>
>>>>> In related news, there's been this burning desire regarding a
>>>>> lightweight output range defined as simply a delegate that accepts
>>>>> const(char)[]. That range is an output range all right, and the way
>>>>> you output to it is by simply calling the delegate!
>>>>> <http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227>
>>>>>
>>>>>
>>>>>
>>>>> http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L227
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> It's still early where I live, but... For the callable case, why just
>>>>> accepting E[] instead of any range with E as element? Though, thinking
>>>>> about it, I right now have no idea how to put that into a template
>>>>> constraint, given only R and E.
>>>>>
>>>>> Hmm...
>>>>
>>>> Good point. I haven't thought of it that way - I used arrays as a
>>>> lingua franca buffer. Your suggestion is interesting. I see a risk of
>>>> infinite regression in writing the constraint, but the idea warrants
>>>> more discussion.
>>>
>>> Wait, isn't a delegate that accepts a type T an output range of type T?
>>> Why does the argument have to be an array/range?
>>
>> Efficiency - see the doFormat disaster.
>
> Yes, I'm not saying that a delegate that accepts a range of T cannot be
> an output range of T, I just wondered why it *has* to be a range.
>
>>
>>> For example, a delegate that accepts a string is a range of strings, is
>>> it not? Or do you consider it a range of immutable(char)?
>>
>> Good call. Probably I need to handle ranges of arrays differently.
>> (The problem has already come up, but I punted on it.)
>
> It seems to me to be similar to appending -- you can append an element
> or an array of elements. But appending the individual element makes
> sense in a lot of cases, despite performance.
>
> If output ranges were restricted to deal only with stream data (i.e.
> char) then I'd agree only accepting an array of data is a good idea.
>
>>> For example, I'd expect to be able to use a push_back delegate on an
>>> array-type of integers as an output range.
>>
>> Would it hurt to define push_back for arrays too? The thing is, if you
>> can output an array you can always output one occasional element. But
>> if you only know how to output an element, having the client side
>> output arrays in a loop can be quite slow.
>
> If I always have to do something like this in order to append a single
> element:
>
> put(r, (&elem)[0..1]);

No, the library does that. Look here:

http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/range.d#L306

Andrei
« First   ‹ Prev
1 2 3 4
Top | Discussion index | About this forum | D home