View mode: basic / threaded / horizontal-split · Log in · Help
October 05, 2012
OutputRange should be infinite?
A good while ago, I ran into some issues regarding output ranges. 
(reference 
http://forum.dlang.org/thread/xyvnifnetythvrhtcexm@forum.dlang.org)

The gist of the problem is that with "put" an OutputRange that 
accepts a T will accept a Range!T, and a Range(Range!T) and a 
Range!(Range(Range!T)) add infinitum.

This all works nice and well, provided the output range does not 
ever become empty => is infinite. However, this is currently not 
the case, and this code will blow in your face:

//--------
auto a = new int[](1);
auto b = new int[](2);
assert(isOutputRange!(typeof(a), typeof(b)));
if(!a.empty)
   put(a, b); //Nope
//--------
auto a = new int[](10);
auto b = new int[][](3, 5);
assert(isOutputRange!(typeof(a), typeof(b)));
if(a.length > b.length)
    put(a, b); //Nope
//--------

I had made a "formal request" to deprecate this feature: 
http://forum.dlang.org/thread/xgncorvzlbtcmaxjuvkz@forum.dlang.org 
. I had not come back to this since, but I *have* been thinking 
about it since. I think my request was wrong.

However, I that the "isOutputRange" definition should require 
infinite-ness, as mentioned by others.

The "fun" part of OutputRange is that delegates, or as a general 
rule, any object that implements "put", or in some way shape or 
form, accepts put(range, stuff) is considered OutputRange.

To enforce infiniteness, I'd like to add this to the requirement 
of output range:
*Must meet one of these two criteria:
**isInifite!Range
or
**Does not define "range.empty"
  //notion of infiniteness by default: delegates etc...

This actually has some very very low impact in phobos: The only 
OutputRanges ever used by phobos are appenders/delegates/printers 
anyways.

Also, it does not actually prevent writing put(dynamicArray, [1, 
2]): The dynamic array will seize to match the "isOutputRange", 
but that don't mean the function "put" won't work on it anymore. 
Nuance ho!

The *only* function that is really impacted is "copy". However, 
arguably, it was wrong to use it with OutputRanges to begin with. 
copy(r1, r2) and put(r1, r2) are NOT the same semantic, and 
should not be used as such.

I'd like to try to push for this change. Would this be a lost 
cause, or does the community feel this is indeed the way to go?
October 06, 2012
Re: OutputRange should be infinite?
On Fri, 05 Oct 2012 11:15:44 -0400, monarch_dodra <monarchdodra@gmail.com>  
wrote:

> However, I that the "isOutputRange" definition should require  
> infinite-ness, as mentioned by others.

No, this is very wrong.  A slice is an output range, but is finite.

If you are putting something that is larger into something that is smaller  
and cannot be extended, I would expect an error.  You don't?

This cannot be changed, as the fundamental target for an in-memory output  
range is a slice.

> To enforce infiniteness, I'd like to add this to the requirement of  
> output range:
> *Must meet one of these two criteria:
> **isInifite!Range
> or
> **Does not define "range.empty"
>    //notion of infiniteness by default: delegates etc...
>
> This actually has some very very low impact in phobos: The only  
> OutputRanges ever used by phobos are appenders/delegates/printers  
> anyways.

Just because it isn't *used* by phobos (and I doubt the statement above)  
doesn't mean that it's not a worthwhile part of the API.  Phobos is a  
utility library, not a complete program.

For instance, there is nothing in Phobos that uses  
std.container.RedBlackTree (at least that I know of), but that doesn't  
mean it doesn't have value.

-Steve
October 06, 2012
Re: OutputRange should be infinite?
Am Fri, 05 Oct 2012 17:15:44 +0200
schrieb "monarch_dodra" <monarchdodra@gmail.com>:

> A good while ago, I ran into some issues regarding output ranges. 
> (reference 
> http://forum.dlang.org/thread/xyvnifnetythvrhtcexm@forum.dlang.org)
> 
> The gist of the problem is that with "put" an OutputRange that 
> accepts a T will accept a Range!T, and a Range(Range!T) and a 
> Range!(Range(Range!T)) add infinitum.
> 
> This all works nice and well, provided the output range does not 
> ever become empty => is infinite. However, this is currently not 
> the case, and this code will blow in your face:
> 
> //--------
> auto a = new int[](1);
> auto b = new int[](2);
> assert(isOutputRange!(typeof(a), typeof(b)));
> if(!a.empty)
>     put(a, b); //Nope
> //--------
> auto a = new int[](10);
> auto b = new int[][](3, 5);
> assert(isOutputRange!(typeof(a), typeof(b)));
> if(a.length > b.length)
>      put(a, b); //Nope
> //--------

Couldn't we just fix std.range.put to check for an 'empty' property?
October 06, 2012
Re: OutputRange should be infinite?
On Saturday, 6 October 2012 at 05:24:06 UTC, Steven Schveighoffer
wrote:
> On Fri, 05 Oct 2012 11:15:44 -0400, monarch_dodra 
> <monarchdodra@gmail.com> wrote:
>
>> However, I that the "isOutputRange" definition should require 
>> infinite-ness, as mentioned by others.
>
> No, this is very wrong.  A slice is an output range, but is 
> finite.

A slice is an input range and can safely be used as such. What is
the merit of *also* defining it as an output range? Why even
bother with defining "OutputRange" if it just means "InputRange"
+ "functions"?

> If you are putting something that is larger into something that 
> is smaller and cannot be extended, I would expect an error.  
> You don't?

Yes, but as shown the semantics of "put" are basically: "Cram
*anything* you want inside of me. I can take it".

As evidenced by my two examples, this is clearly not the case,
and, even worse, the developer has _no way_ of knowing this.

> [SNIP]
> -Steve

Long story short, the *only* reason to ever use the "OutputRange"
interface over the "InputRange" interface is:
*When cramming things into a delegate. (which are/should be
infinite by design)
*When cramming things into an input range, but not caring about
capacity.

I'm just saying, "put" is convenient and all, and I have no plan
to have it changed. Users can use it at their own discretion if 
they want to use it on an InputRange, and at their own risk.

However, I really don't like having a range tell me "yeah, I'm an 
Output Range", just to choke on the first call to put.
October 06, 2012
Re: OutputRange should be infinite?
On Saturday, 6 October 2012 at 05:24:06 UTC, Steven Schveighoffer
wrote:
> On Fri, 05 Oct 2012 11:15:44 -0400, monarch_dodra 
> <monarchdodra@gmail.com> wrote:
>
>> However, I that the "isOutputRange" definition should require 
>> infinite-ness, as mentioned by others.
>
> No, this is very wrong.  A slice is an output range, but is 
> finite.

A slice is an input range and can safely be used as such. What is
the merit of *also* defining it as an output range? Why even
bother with defining "OutputRange" if it just means "InputRange"
+ "functions"?

> If you are putting something that is larger into something that 
> is smaller and cannot be extended, I would expect an error.  
> You don't?

Yes, but as shown the semantics of "put" are basically: "Cram
*anything* you want inside of me. I can take it".

As evidenced by my two examples, this is clearly not the case,
and, even worse, the developer has _no way_ of knowing this.

> [SNIP]
> -Steve

Long story short, the *only* reason to ever use the "OutputRange"
interface over the "InputRange" interface is:
*When cramming things into a delegate. (which are/should be
infinite by design)
*When cramming things into an input range, but not caring about
capacity.

I'm just saying, "put" is convenient and all, and I have no plan
to have it changed. Users can use it at their own discretion if
they want to use it on an InputRange, and at their own risk.

However, I really don't like having a range tell me "yeah, I'm an
Output Range", just to choke on the first call to put.
October 06, 2012
Re: OutputRange should be infinite?
On Saturday, 6 October 2012 at 08:00:42 UTC, Johannes Pfau wrote:
> Am Fri, 05 Oct 2012 17:15:44 +0200
> schrieb "monarch_dodra" <monarchdodra@gmail.com>:
>
>> [SNIP]
>
> Couldn't we just fix std.range.put to check for an 'empty' 
> property?

Well, the issue (imo) is not put's implementation: as Steven 
Schveighoffer said, cramming too big into too small is wrong 
(logic error).

The problem (I think), is that once a range verifies the 
isOutputRange criteria, the user should be able to call "put" 
without (too much) worries.
October 06, 2012
Re: OutputRange should be infinite?
On 06-Oct-12 12:13, monarch_dodra wrote:
> On Saturday, 6 October 2012 at 08:00:42 UTC, Johannes Pfau wrote:
>> Am Fri, 05 Oct 2012 17:15:44 +0200
>> schrieb "monarch_dodra" <monarchdodra@gmail.com>:
>>
>>> [SNIP]
>>
>> Couldn't we just fix std.range.put to check for an 'empty' property?
>
> Well, the issue (imo) is not put's implementation: as Steven
> Schveighoffer said, cramming too big into too small is wrong (logic error).
>
> The problem (I think), is that once a range verifies the isOutputRange
> criteria, the user should be able to call "put" without (too much) worries.

Not possible. The only thing isOutputRange serves is that putting stuff 
into X is sensible in one of many ways (delegates, own put, input range 
with assignable elements).
Any run-time properties such as lengths and maximums are out of 
isOutputRange business. And in the end one can still run out of 
supposedly "infinite" things like RAM, disk space etc.

-- 
Dmitry Olshansky
October 06, 2012
Re: OutputRange should be infinite?
On Saturday, 6 October 2012 at 08:51:02 UTC, Dmitry Olshansky 
wrote:
> On 06-Oct-12 12:13, monarch_dodra wrote:
>> On Saturday, 6 October 2012 at 08:00:42 UTC, Johannes Pfau 
>> wrote:
>>> Am Fri, 05 Oct 2012 17:15:44 +0200
>>> schrieb "monarch_dodra" <monarchdodra@gmail.com>:
>>>
>>>> [SNIP]
>>>
>>> Couldn't we just fix std.range.put to check for an 'empty' 
>>> property?
>>
>> Well, the issue (imo) is not put's implementation: as Steven
>> Schveighoffer said, cramming too big into too small is wrong 
>> (logic error).
>>
>> The problem (I think), is that once a range verifies the 
>> isOutputRange
>> criteria, the user should be able to call "put" without (too 
>> much) worries.
>
> Not possible. The only thing isOutputRange serves is that 
> putting stuff into X is sensible in one of many ways 
> (delegates, own put, input range with assignable elements).
> Any run-time properties such as lengths and maximums are out of 
> isOutputRange business. And in the end one can still run out of 
> supposedly "infinite" things like RAM, disk space etc.

Yes, but that would be an exception that also holds true for 
"isInfiniteRange", but only happens under "exceptional" 
circumstances.

Hence the (too much) above.
October 09, 2012
Re: OutputRange should be infinite?
On Sat, 06 Oct 2012 04:07:41 -0400, monarch_dodra <monarchdodra@gmail.com>  
wrote:

> On Saturday, 6 October 2012 at 05:24:06 UTC, Steven Schveighoffer
> wrote:
>> On Fri, 05 Oct 2012 11:15:44 -0400, monarch_dodra  
>> <monarchdodra@gmail.com> wrote:
>>
>>> However, I that the "isOutputRange" definition should require  
>>> infinite-ness, as mentioned by others.
>>
>> No, this is very wrong.  A slice is an output range, but is finite.
>
> A slice is an input range and can safely be used as such. What is
> the merit of *also* defining it as an output range? Why even
> bother with defining "OutputRange" if it just means "InputRange"
> + "functions"?

try doing this on a unix system

cat /dev/zero > ~/zeros

And see if the output file zeros is infinite :)

Even an appender is finite when you run out of memory.

Do you think output files make bad output ranges, even though they are  
finite?

An output range is nothing but an interface to a storage location.   
Whether the storage location is infinite or not is up to the location, the  
output range has to support both infinite and finite targets.

What you are proposing would make it illegal to use std.algorithm.copy on  
*any* memory-based construct, or else have it blissfully succeed by  
throwing away any extra data.  Neither of these situations are tenable.

>> If you are putting something that is larger into something that is  
>> smaller and cannot be extended, I would expect an error.  You don't?
>
> Yes, but as shown the semantics of "put" are basically: "Cram
> *anything* you want inside of me. I can take it".

No, definitely not.  An output range can take input, but must obviously be  
able to say "I'm full".

> As evidenced by my two examples, this is clearly not the case,
> and, even worse, the developer has _no way_ of knowing this.

I'm not against defining a standard way to say "I'm full", but proposing  
it *can't* say that is not the solution.  Clearly, we could do better in  
defining a standard way to test for fullness (full property akin to  
empty?).  Even so, putting into a non-full range could generate an error.

> However, I really don't like having a range tell me "yeah, I'm an Output  
> Range", just to choke on the first call to put.

What about an input range that is immediately empty?  These are corner  
cases, but certainly valid.

-Steve
October 09, 2012
Re: OutputRange should be infinite?
On Tuesday, 9 October 2012 at 13:22:28 UTC, Steven Schveighoffer 
wrote:
> [SNIP]

I tend to disagree with your examples, because, you are mixing 
the notion of run-time failure with logic error.

For example: "new" New can fail. And you don't know unless you 
try.
But new will throw an exception to tell you it failed..

An appender, as you say, is finite in memory, and will end up 
throwing an exception, yes. You also have a chance to try to 
catch it and react.

Over-putting into a finite slice, on the other end, will 
*assert*. Game over. It is a catch 22: You can't know unless you 
try, you crash if you do.

> I'm not against defining a standard way to say "I'm full", but 
> proposing it *can't* say that is not the solution.  Clearly, we 
> could do better in defining a standard way to test for fullness 
> (full property akin to empty?).  Even so, putting into a 
> non-full range could generate an error.

Hum... I'm just kind of wondering here: Couldn't we simply have 
put throw an actual exception? Something along the lines of 
"OutputRangFullException"? That would be a pretty good compromise.

Performance wise, I don't think there'd be any real toll: 
delegates/functions don't have empty anyways, so it would just be 
a matter of catch-rethrow. As for input ranges, well, I think it 
would be safer anyways if they checked and threw, rather than 
blindly over pop and crash...

Didn't fully think this threw yet (just thought of it typing), 
but I thought I'd throw it out there.

>> However, I really don't like having a range tell me "yeah, I'm 
>> an Output Range", just to choke on the first call to put.
>
> What about an input range that is immediately empty?  These are 
> corner cases, but certainly valid.

Wouldn't "empty" simply answer "true" before even starting? At 
least it is being honest.

> -Steve

Thanks for debating.
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home