I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:

import std.range, std.algorithm;

void main() {
     const foo = map!"a ^^ 2"([1,2,3,4,5]);
     auto myRetro = retro(foo);
}

Right now, I don't even get to the point of instantiating retro because I get the following encrypted Klingon related to its constraints:

test9.d(5): Error: template std.range.retro(R) if
(isBidirectionalRange!(R)) does not match any function template declaration
test9.d(5): Error: template std.range.retro(R) if
(isBidirectionalRange!(R)) cannot deduce template function from argument
types !()(const(Map!(result,int[])))

This error occurs because front(), popFront(), et al. aren't callable on const(Map) objects.  This is pretty much a blocker for fixing the deeper issues.  Does it sound reasonable to everyone that is*Range, where * is forward, bidirectional, etc. should operate on the shallowly Unqual'd version of the type, or is there some good reason I'm missing why this shouldn't be the case?

David Simcha <dsimcha at gmail.com> wrote:
> I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:
> 
> import std.range, std.algorithm;
> 
> void main() {
>      const foo = map!"a ^^ 2"([1,2,3,4,5]);
>      auto myRetro = retro(foo);
> }

Could you elaborate why?

Ranges with const elements must be common; but what's the point of const ranges?  Const random access ranges are usable, but other const ranges seem to be useless since popFront and popBack can't be const.


Shin

On 08/12/2010 12:05 AM, Shin Fujishiro wrote:
> David Simcha<dsimcha at gmail.com>  wrote:
>> I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:
>>
>> import std.range, std.algorithm;
>>
>> void main() {
>>       const foo = map!"a ^^ 2"([1,2,3,4,5]);
>>       auto myRetro = retro(foo);
>> }
>
> Could you elaborate why?
>
> Ranges with const elements must be common; but what's the point of const ranges?  Const random access ranges are usable, but other const ranges seem to be useless since popFront and popBack can't be const.

I think the main argument is that currently most of std.algorithm doesn't work with const arrays, which have a simple "tail-const" version. const(T[]) is implicitly convertible to const(T)[].

That doesn't apply to most other ranges, which don't have an obvious "tail-const" version.

David, I think we need to think through a bit more before proceeding. The way I assume you want to proceed is to essentially add a special function signature for each algorithm and have it forward to the peeled version. Perhaps we could look at a simpler solution, e.g. one that would involve a language change.


Andrei

On Thu, 12 Aug 2010, Shin Fujishiro wrote:

> David Simcha <dsimcha at gmail.com> wrote:
> > I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:
> > 
> > import std.range, std.algorithm;
> > 
> > void main() {
> >      const foo = map!"a ^^ 2"([1,2,3,4,5]);
> >      auto myRetro = retro(foo);
> > }
> 
> Could you elaborate why?
> 
> Ranges with const elements must be common; but what's the point of const ranges?  Const random access ranges are usable, but other const ranges seem to be useless since popFront and popBack can't be const.
> 
> 
> Shin
> _______________________________________________

The case I ran into that I think is a useful one is being able to use const ranges as input to various algorithms, map and filter being the most obvious choices.  In my case I was filtering out a few specific records of a range of records.

Later,
Brad

----- Original Message ----

> From: Shin Fujishiro <rsinfu at gmail.com>
> 
> David Simcha <dsimcha at gmail.com> wrote:
> > I'm  looking to go on a major debugging spree and make std.range work with  const ranges.  For example, the following should work:
> > 
> >  import std.range, std.algorithm;
> > 
> > void main() {
> >       const foo = map!"a ^^ 2"([1,2,3,4,5]);
> >       auto myRetro = retro(foo);
> > }
> 
> Could you elaborate  why?
> 
> Ranges with const elements must be common; but what's the point  of const ranges?  Const random access ranges are usable, but other  const ranges seem to be useless since popFront and popBack can't be  const.

This seems to point at a larger issue.

With the de-facto range T[], we get an implicit cast to a range of const elements, const(T)[].

But we cannot get this with any user-defined type.  I think we will need to address this eventually.  It's one of the main reasons dcollections is not const-aware, because I don't want to create several types of ranges/cursors, and then all the duplicated functions that go with it.  I'd rather use inout and have it designate the return type.

I think Janice proposed something a long long time ago that allowed a template parameter to represent a const factor of a type, a-la:

SList(V)
{
    struct node
    {
       V value;
       node *next;
    }
    node *head;

    struct range(const C)
    {
        C(node)* cur;
        @property C(V) front() {return cur.value;}
        void popFront() {next = cur.next;}
        @property bool empty() {return next !is null;}
    }

    range!(inout) opSlice() inout {range!(inout) result;  result.cur = head;
return result;}
}

So basically, the way it works is C is substituted for a const factor (const, immutable, or nothing).  And more importantly, the compiler allows implicit conversion from a range!(???) to a range!(const).  Also, we can apply inout rules.  (Note, I don't know what to put as a const parameter for mutable, since that's not a keyword, hence the ???).

Does this seem complex?  Yes.  But it's not as complex/hard to read/hard to implement as 3 copies of all functions with 3 different range types.  It's akin to the savings/clarity of inout.

What I'm unsure about is if we need to do it this way.  I don't like using a user-defined parameter as the const factor, it makes it #1 hard to read, and #2 every person's const factor symbol may be different.  Most likely, you will have one const template parameter.  Do we need to support multiple parameters?  inout avoids requiring templates, because the code generation is identical.  Can we do something similar to inout?

We're going to need something like this to avoid rediculous code repetition for any type that uses custom range types.  Essentially what we need to duplicate is the "apply const only to one member" feature of builtin arrays.

-Steve

When const ranges are passed by value to a function, they can safely receive an implicit shallow Unqual.  In many cases this makes them ranges of const elements that is useful.

On 8/12/2010 1:05 AM, Shin Fujishiro wrote:
> David Simcha<dsimcha at gmail.com>  wrote:
> 
>> I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:
>>
>> import std.range, std.algorithm;
>>
>> void main() {
>>       const foo = map!"a ^^ 2"([1,2,3,4,5]);
>>       auto myRetro = retro(foo);
>> }
>> 
> Could you elaborate why?
>
> Ranges with const elements must be common; but what's the point of const ranges?  Const random access ranges are usable, but other const ranges seem to be useless since popFront and popBack can't be const.
>
>
> Shin
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>
> 

The problem is Unqual is good with arrays but does not know what to do with user-defined types. For some range R, Unqual!R does not do anything interesting.

Andrei

David Simcha wrote:
> When const ranges are passed by value to a function, they can safely receive an implicit shallow Unqual.  In many cases this makes them ranges of const elements that is useful.
> 
> On 8/12/2010 1:05 AM, Shin Fujishiro wrote:
>> David Simcha<dsimcha at gmail.com>  wrote:
>> 
>>> I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:
>>>
>>> import std.range, std.algorithm;
>>>
>>> void main() {
>>>       const foo = map!"a ^^ 2"([1,2,3,4,5]);
>>>       auto myRetro = retro(foo);
>>> }
>>> 
>> Could you elaborate why?
>>
>> Ranges with const elements must be common; but what's the point of const ranges?  Const random access ranges are usable, but other const ranges seem to be useless since popFront and popBack can't be const.
>>
>>
>> Shin
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
>>
>> 
> 
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos

But that's the point.  Most ranges will be usable with const + shallow Unqual because they keep track of the iteration state w/o any indirection..  Those that aren't won't pass is*Range if they're const.

On 8/12/2010 9:58 AM, Andrei Alexandrescu wrote:
> The problem is Unqual is good with arrays but does not know what to do with user-defined types. For some range R, Unqual!R does not do anything interesting.
>
> Andrei
>
> David Simcha wrote:
>> When const ranges are passed by value to a function, they can safely receive an implicit shallow Unqual.  In many cases this makes them ranges of const elements that is useful.
>>
>> On 8/12/2010 1:05 AM, Shin Fujishiro wrote:
>>> David Simcha<dsimcha at gmail.com>  wrote:
>>>> I'm looking to go on a major debugging spree and make std.range work with const ranges.  For example, the following should work:
>>>>
>>>> import std.range, std.algorithm;
>>>>
>>>> void main() {
>>>>       const foo = map!"a ^^ 2"([1,2,3,4,5]);
>>>>       auto myRetro = retro(foo);
>>>> }
>>> Could you elaborate why?
>>>
>>> Ranges with const elements must be common; but what's the point of const ranges?  Const random access ranges are usable, but other const ranges seem to be useless since popFront and popBack can't be const.
>>>
>>>
>>> Shin
>>> _______________________________________________
>>> phobos mailing list
>>> phobos at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/phobos
>>>
>>
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos
>

On Thu, Aug 12, 2010 at 1:43 AM, Andrei Alexandrescu <andrei at erdani.com>wrote:

>
> I think the main argument is that currently most of std.algorithm doesn't
> work with const arrays, which have a simple "tail-const" version. const(T[])
> is implicitly convertible to const(T)[].
>
> That doesn't apply to most other ranges, which don't have an obvious "tail-const" version.
>
> David, I think we need to think through a bit more before proceeding. The way I assume you want to proceed is to essentially add a special function signature for each algorithm and have it forward to the peeled version. Perhaps we could look at a simpler solution, e.g. one that would involve a language change.
>

Fair enough.  If you think this might be better solved at the language level, that's a worthwhile discussion to have.  I do believe, though, that * most* ranges besides T[] do have an obvious "tail-const" version, since in practice most ranges are structs that have the iteration state stored inline and only use indirection to store the payload, if anywhere.

At any rate, I think this is a must-solve problem.  Despite its theoretical beauty, I find D's const/immutable system to be utterly useless (and I've made a serious attempt to use it in a real multithreaded program) for all but the simplest cases in practice, for three reasons:

1.   It's too hard to create non-trivial immutable data structures, especially without relying on unchecked casts during construction.

2.  So many things in Phobos (even things as simple as std.math.pow() before I recently fixed it) behave incorrectly when given const/immutable data. This also applies to other libraries I use, including ones that I'm the main author of, so I'm just as guilty of it as anyone.  Given that noone, including me, seems to be able to get this right in generic code, perhaps this does point to the need for a language-level solution.

3.  inout is currently so bug-ridden it's not even funny.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/phobos/attachments/20100812/898f4466/attachment.html>

Tail-const is doable, but does not enjoy the implicit conversion that T[] has.

Without the implicit conversion, tail-const relies on unsafe casting, and that sucks.

I agree with Andrei we need a language solution.  The question really is not what the solution should do, that much is clear -- apply const to a subset of the members (either all references or members designated by some identifier). The question is, what does the syntax look like.  That was the major stumbling block that flipped Walter's switch to "tail-const doesn't work".

I think we should concentrate on structs and not class references, since class references have no syntax that separates the reference from the data.  At least with structs, you can identify the parts to apply const to.  We have a somewhat clunky solution in Rebindable for classes, so it would be nice to include them, but past experience has shown that to be a big rat hole.

-Steve

>
>From: David Simcha <dsimcha at gmail.com>
>To: Discuss the phobos library for D <phobos at puremagic.com>
>Sent: Thu, August 12, 2010 11:28:24 AM
>Subject: Re: [phobos] is*Range + Qualified Types
>
>
>
>
>On Thu, Aug 12, 2010 at 1:43 AM, Andrei Alexandrescu <andrei at erdani.com> wrote:
>
>
>>
I think the main argument is that currently most of std.algorithm doesn't work with const arrays, which have a simple "tail-const" version. const(T[]) is implicitly convertible to const(T)[].
>>
>>That doesn't apply to most other ranges, which don't have an obvious "tail-const" version.
>>
>>David, I think we need to think through a bit more before proceeding. The way I
>
>>assume you want to proceed is to essentially add a special function signature for each algorithm and have it forward to the peeled version. Perhaps we could

>>look at a simpler solution, e.g. one that would involve a language change.
>>

Fair enough.  If you think this might be better solved at the language level, that's a worthwhile discussion to have.  I do believe, though, that most ranges besides T[] do have an obvious "tail-const" version, since in practice most ranges are structs that have the iteration state stored inline and only use indirection to store the payload, if anywhere.

At any rate, I think this is a must-solve problem.  Despite its theoretical beauty, I find D's const/immutable system to be utterly useless (and I've made a

serious attempt to use it in a real multithreaded program) for all but the simplest cases in practice, for three reasons:

1.   It's too hard to create non-trivial immutable data structures, especially without relying on unchecked casts during construction.

2.  So many things in Phobos (even things as simple as std.math.pow() before I recently fixed it) behave incorrectly when given const/immutable data.  This also applies to other libraries I use, including ones that I'm the main author of, so I'm just as guilty of it as anyone.  Given that noone, including me, seems to be able to get this right in generic code, perhaps this does point to the need for a language-level solution.

3.  inout is currently so bug-ridden it's not even funny.