On Tuesday, October 25, 2011 15:55 Walter Bright wrote:
> On 10/25/2011 3:40 PM, Piotr Szturmaj wrote:
> > I share your opinion. I was thinking about such filter concept for std.crypto.cipher (TBD), but I will also try to convert current hash function code to ranges.
> > 
> > Thanks for pointing that out.
> 
> Andrei and I have pretty much failed at articulating this vision for Phobos. We need to get our act together.

Ranges in general are not something that is comunicated very well. They're not in TDPL, and none of the online documentation discusses them in detail. You pretty much only learn them from reading Phobos' documentation and using Phobos or from discussing it with people who know about them already. And there's no overall plan or design for Phobos articulated _anywhere_ that I'm aware of, so between those two facts, there's really nothing to get such a vision across to anyone other than word of mouth (though I suppose that it's more "word of keyboard" in many cases).

- Jonathan M Davis

On Tue, Oct 25, 2011 at 5:12 PM, Jonathan M Davis <jmdavisProg@gmx.com>wrote:

> On Tuesday, October 25, 2011 15:55 Walter Bright wrote:
> > On 10/25/2011 3:40 PM, Piotr Szturmaj wrote:
> > > I share your opinion. I was thinking about such filter concept for std.crypto.cipher (TBD), but I will also try to convert current hash function code to ranges.
> > >
> > > Thanks for pointing that out.
> >
> > Andrei and I have pretty much failed at articulating this vision for Phobos. We need to get our act together.
>
> Ranges in general are not something that is comunicated very well. They're
> not
> in TDPL, and none of the online documentation discusses them in detail. You
> pretty much only learn them from reading Phobos' documentation and using
> Phobos or from discussing it with people who know about them already. And
> there's no overall plan or design for Phobos articulated _anywhere_ that
> I'm
> aware of, so between those two facts, there's really nothing to get such a
> vision across to anyone other than word of mouth (though I suppose that
> it's
> more "word of keyboard" in many cases).
>
> - Jonathan M Davis
>

I learned about them from Andrei's boostcon talk.  Not exactly the first place you'd look for information about D but the talk was very informative.

Regards,
Brad Anderson

On Tuesday, October 25, 2011 16:16 Brad Anderson wrote:
> On Tue, Oct 25, 2011 at 5:12 PM, Jonathan M Davis
<jmdavisProg@gmx.com>wrote:
> > On Tuesday, October 25, 2011 15:55 Walter Bright wrote:
> > > On 10/25/2011 3:40 PM, Piotr Szturmaj wrote:
> > > > I share your opinion. I was thinking about such filter concept for std.crypto.cipher (TBD), but I will also try to convert current hash function code to ranges.
> > > > 
> > > > Thanks for pointing that out.
> > > 
> > > Andrei and I have pretty much failed at articulating this vision for Phobos. We need to get our act together.
> > 
> > Ranges in general are not something that is comunicated very well.
> > They're not
> > in TDPL, and none of the online documentation discusses them in detail.
> > You pretty much only learn them from reading Phobos' documentation and
> > using Phobos or from discussing it with people who know about them
> > already. And there's no overall plan or design for Phobos articulated
> > _anywhere_ that I'm
> > aware of, so between those two facts, there's really nothing to get such
> > a vision across to anyone other than word of mouth (though I suppose
> > that it's
> > more "word of keyboard" in many cases).
> > 
> > - Jonathan M Davis
> 
> I learned about them from Andrei's boostcon talk. Not exactly the first place you'd look for information about D but the talk was very informative.

That and an article that Andrei wrote on ranges a while back which was not D- specific are the only online sources on ranges that I'm aware of, and to my knowledge, neither of them are referenced on D's site. Ranges are a powerful concept, but they're not well known, in part because they're relatively new - particularly when it comes to being used in a major library, let alone the standard library for a language. So, they definitely need some explaining.

I was working on article on ranges a while back, but tabled it due to bugs related to std.container.Array rendering my examples unworkable. I should probably get back to working on that and see if those issues still exist or whether I can better work around them in the article. But the fact that dynamic arrays are actually a very poor example of ranges in some regards (given the fact that people tend to think of them as containers even though they really aren't in D) complicates things in a way that I'd really like to be able to explain some of the concepts using a real container rather than arrays.

- Jonathan M Davis

If you want a bit of history:

http://www.digitalmars.com/d/archives/digitalmars/D/announce/RFC_on_range_design_for_D2_12922.html#N12922 http://www.digitalmars.com/d/archives/digitalmars/D/announce/Revised_RFC_on_range_design_for_D2_13211.html

http://web.archive.org/web/20090112000313/http://ssli.ee.washington.edu/~aalexand/d/tmp/std_range.html

That last link (although outdated) is probably more descriptive than
what we have now.

Walter Bright wrote:
> On 10/25/2011 3:40 PM, Piotr Szturmaj wrote:
>> I share your opinion. I was thinking about such filter concept for
>> std.crypto.cipher (TBD), but I will also try to convert current hash
>> function
>> code to ranges.
>>
>> Thanks for pointing that out.
>
> Andrei and I have pretty much failed at articulating this vision for
> Phobos. We need to get our act together.

Personally, I learned ranges from Phobos docs/src and Andrei's article about them. In the beginning I was surprised that I could not find any "range howto", yet on this NG some range designs were considered obvious. But I finally (somewhat) understood and used them in my postgres client implementation. And I like them.

I wish there was some article about ranges and their uses. Some tips and tricks thing for users and guidelines for range writers.

But let's get back to std.crypto. I planned to add input ranges to function put, because it already forms an output range. I thought that std.range.put() would serve the need if UFCS will be supported. But, as I guess, the best method is to support input ranges directly.

On 10/25/2011 4:12 PM, Jonathan M Davis wrote:
> Ranges in general are not something that is comunicated very well. They're not
> in TDPL, and none of the online documentation discusses them in detail. You
> pretty much only learn them from reading Phobos' documentation and using
> Phobos or from discussing it with people who know about them already. And
> there's no overall plan or design for Phobos articulated _anywhere_ that I'm
> aware of, so between those two facts, there's really nothing to get such a
> vision across to anyone other than word of mouth (though I suppose that it's
> more "word of keyboard" in many cases).

Yup. Guilty as charged.

On 10/25/2011 5:22 PM, Piotr Szturmaj wrote:
> But let's get back to std.crypto. I planned to add input ranges to function put,
> because it already forms an output range. I thought that std.range.put() would
> serve the need if UFCS will be supported. But, as I guess, the best method is to
> support input ranges directly.

An easy test is that if the interface takes a T[] as input, consider a range instead. Ditto for output. If an interface takes a File as input, it's a red flag that something is wrong.

> 
> An easy test is that if the interface takes a T[] as input, consider a range instead. Ditto for output. If an interface takes a File as input, it's a red flag that something is wrong.

I have a question about ranges that occurred to me when I was composing a MySQL result set into a random access range.

To do that I have to provide the save capability defined by

R r1;
R r2 = r1.save;

Would it harm the use of ranges as elements of operation sequences if the type of the entity that got saved was not the same as the original range provider type. Something like:

R r1;
S r2 = r1.save;
r1.restore(r2);

For entities with a good deal of state, that implement a range, storing the whole state, and more significantly, restoring it, may be non- trivial, but saving and restoring the state of the range may be quite a lightweight affair.

I'm also puzzled by the semantics of the random access range.

If I restrict myself to indexing into the range, then things are fine.

auto x = r[0];
x = r[10];
x = r[0];

But if I slip in a few popFront's, then presumably x = r[0] will give me a different result.

This just makes me slightly uneasy.

Steve

On Wednesday, October 26, 2011 06:28:52 Steve Teale wrote:
> > An easy test is that if the interface takes a T[] as input, consider a range instead. Ditto for output. If an interface takes a File as input, it's a red flag that something is wrong.
> 
> I have a question about ranges that occurred to me when I was composing a MySQL result set into a random access range.
> 
> To do that I have to provide the save capability defined by
> 
> R r1;
> R r2 = r1.save;
> 
> Would it harm the use of ranges as elements of operation sequences if the type of the entity that got saved was not the same as the original range provider type. Something like:
> 
> R r1;
> S r2 = r1.save;
> r1.restore(r2);
> 
> For entities with a good deal of state, that implement a range, storing the whole state, and more significantly, restoring it, may be non- trivial, but saving and restoring the state of the range may be quite a lightweight affair.

It's likely to cause issues if the type returned by save differs from the original type. From your description, it sounds like the range is over something, and it's that something which you don't want to copy. If that's the case, then the range just doesn't contain that data. Rather, it's a view into the data, so saving it just save that view.

The fact that arrays are currently the most used example of ranges definitely makes them more confusing IMHO, since people often think of ranges as containers, whereas ranges aren't. And really, dynamic arrays in D _aren't_ containers. No dynamic array actually owns its elements. The runtime owns it, and dynamic arrays are just ranges over that data.

It's easier to understand when you think about a container, such as vector type (e.g. std.container.Array) or a linked list. The container holds the data. The container is _not_ a range. A range is a view of that data. So, popping elements off of the range has no effect on the original container - the same goes with many range-based functions. As long as the elements in the range aren't mutated or rearranged, the elements in the container are unaltered. So, calling save on a range just gives you a copy of that view of the container, allowing you to alter the range and still have that original view of the container.

> I'm also puzzled by the semantics of the random access range.
> 
> If I restrict myself to indexing into the range, then things are fine.
> 
> auto x = r[0];
> x = r[10];
> x = r[0];
> 
> But if I slip in a few popFront's, then presumably x = r[0] will give me a different result.
> 
> This just makes me slightly uneasy.

r[0] _is_ r.front, so of course popFront would change what r[0] is. You're consuming the range as you pop elements off of its front. So, if you do _anything_ which alters a range (rather than its elements), at least some of (if not all of) the indices in a random access range change. The reason that this is making you uneasy is likely at least partially because of the fact that you're probably thinking about arrays and how they're used rather than thinking more abstractly about ranges. Arrays are really a bad example of ranges, in spite of the fact that they are unfortunately, our most common example of ranges at this point.

- Jonathan M Davis

On 26.10.2011 10:28, Steve Teale wrote:
>>
>> An easy test is that if the interface takes a T[] as input, consider a
>> range instead. Ditto for output. If an interface takes a File as input,
>> it's a red flag that something is wrong.
>
> I have a question about ranges that occurred to me when I was composing a
> MySQL result set into a random access range.
>
> To do that I have to provide the save capability defined by
>
> R r1;
> R r2 = r1.save;
>
> Would it harm the use of ranges as elements of operation sequences if the
> type of the entity that got saved was not the same as the original range
> provider type. Something like:
>
> R r1;
> S r2 = r1.save;
> r1.restore(r2);

Yay! Range with 'restore' strikes again.
Seriously, last time something about range was discussed I noticed this very same thing - full copy is costly, there are ranges that can restore their state using some special tiny savepoint object.
Though currently this kind of thing is not supported nor expected, plus there are cases where copy needs to be made.

>
> For entities with a good deal of state, that implement a range, storing
> the whole state, and more significantly, restoring it, may be non-
> trivial, but saving and restoring the state of the range may be quite a
> lightweight affair.
>
> I'm also puzzled by the semantics of the random access range.
>
> If I restrict myself to indexing into the range, then things are fine.
>
> auto x = r[0];
> x = r[10];
> x = r[0];
>
> But if I slip in a few popFront's, then presumably x = r[0] will give me
> a different result.
>
> This just makes me slightly uneasy.
>
> Steve


-- 
Dmitry Olshansky