I noticed that the code below doesn't work, and I was wondering if it's by design (hopefully not):

    immutable SEP = ", ";
    ["a", "b"].join(SEP);

The fact that SEP is immutable(char[]) instead of immutable(char)[] shouldn't break the function.

On Sunday 10 July 2011 21:09:27 Mehrdad wrote:
> I noticed that the code below doesn't work, and I was wondering if it's
> by design (hopefully not):
> 
>      immutable SEP = ", ";
>      ["a", "b"].join(SEP);
> 
> The fact that SEP is immutable(char[]) instead of immutable(char)[]
> shouldn't break the function.

It most definitely breaks the function, and it's a limitation of templates. Templates are instantiated with the exact type that they're given, so the compiler tries to instantiate join with immutable(char[]), but join _can't_ work with immutable(char[]), because it needs a mutable range. immutable ranges are worthless. If the compiler were smart enough to realize that it could instantiate join with immutable(char)[] and it would work, then you could use immutable(char[]), but since it isn't that smart, it doesn't work. The same problem happens with static arrays. They can't be used as ranges, so even though they'd work if the compiler picked a dynamic range as the type for the function, they don't work, because the compiler isn't that smart.

The problem may be fixed at some point, but as it stands, it just doesn't work to use immutable arrays with range-based functions.

- Jonathan M Davis

On 7/10/2011 9:17 PM, Jonathan M Davis wrote:
> On Sunday 10 July 2011 21:09:27 Mehrdad wrote:
>> I noticed that the code below doesn't work, and I was wondering if it's
>> by design (hopefully not):
>>
>>       immutable SEP = ", ";
>>       ["a", "b"].join(SEP);
>>
>> The fact that SEP is immutable(char[]) instead of immutable(char)[]
>> shouldn't break the function.
> It most definitely breaks the function, and it's a limitation of templates.
> Templates are instantiated with the exact type that they're given, so the
> compiler tries to instantiate join with immutable(char[]), but join _can't_
> work with immutable(char[]), because it needs a mutable range. immutable
> ranges are worthless. If the compiler were smart enough to realize that it
> could instantiate join with immutable(char)[] and it would work, then you
> could use immutable(char[]), but since it isn't that smart, it doesn't work.
> The same problem happens with static arrays. They can't be used as ranges, so
> even though they'd work if the compiler picked a dynamic range as the type for
> the function, they don't work, because the compiler isn't that smart.
>
> The problem may be fixed at some point, but as it stands, it just doesn't work
> to use immutable arrays with range-based functions.
>
> - Jonathan M Davis
Hm... interesting, I hope it's fixed sometime then. :)
Thanks for the info!

Jonathan M Davis wrote:
> On Sunday 10 July 2011 21:09:27 Mehrdad wrote:
>> I noticed that the code below doesn't work, and I was wondering if it's
>> by design (hopefully not):
>>
>>      immutable SEP = ", ";
>>      ["a", "b"].join(SEP);
>>
>> The fact that SEP is immutable(char[]) instead of immutable(char)[]
>> shouldn't break the function.
>
> It most definitely breaks the function, and it's a limitation of templates. Templates are instantiated with the exact type that they're given, so the compiler tries to instantiate join with immutable(char[]), but join _can't_ work with immutable(char[]), because it needs a mutable range. immutable ranges are worthless. If the compiler were smart enough to realize that it could instantiate join with immutable(char)[] and it would work, then you could use immutable(char[]), but since it isn't that smart, it doesn't work. The same problem happens with static arrays. They can't be used as ranges, so even though they'd work if the compiler picked a dynamic range as the type for the function, they don't work, because the compiler isn't that smart.
>
> The problem may be fixed at some point, but as it stands, it just doesn't work to use immutable arrays with range-based functions.
>
> - Jonathan M Davis

There is no such thing as an immutable range because the range abstraction is
based on mutation.
This does not make sense, because intuitively, immutable(char[]) is a range type
just as immutable(char)[] or char[] is a range type.
It is quite odd that a data structure can only be a D range if it may be modified...

It is not really a compiler issue, but one of library design. Maybe adding a wrapper range to Phobos to iterate over immutable 'ranges' would be an improvement to the current situation.

Cheers,
-Timon

On 2011-07-11 09:06, Timon Gehr wrote:
> Jonathan M Davis wrote:
> > On Sunday 10 July 2011 21:09:27 Mehrdad wrote:
> >> I noticed that the code below doesn't work, and I was wondering if it's
> >> 
> >> by design (hopefully not):
> >> immutable SEP = ", ";
> >> ["a", "b"].join(SEP);
> >> 
> >> The fact that SEP is immutable(char[]) instead of immutable(char)[]
> >> shouldn't break the function.
> > 
> > It most definitely breaks the function, and it's a limitation of templates. Templates are instantiated with the exact type that they're given, so the compiler tries to instantiate join with immutable(char[]), but join _can't_ work with immutable(char[]), because it needs a mutable range. immutable ranges are worthless. If the compiler were smart enough to realize that it could instantiate join with immutable(char)[] and it would work, then you could use immutable(char[]), but since it isn't that smart, it doesn't work. The same problem happens with static arrays. They can't be used as ranges, so even though they'd work if the compiler picked a dynamic range as the type for the function, they don't work, because the compiler isn't that smart.
> > 
> > The problem may be fixed at some point, but as it stands, it just doesn't work to use immutable arrays with range-based functions.
> > 
> > - Jonathan M Davis
> 
> There is no such thing as an immutable range because the range abstraction
> is based on mutation.
> This does not make sense, because intuitively, immutable(char[]) is a range
> type just as immutable(char)[] or char[] is a range type.
> It is quite odd that a data structure can only be a D range if it may be
> modified...
> 
> It is not really a compiler issue, but one of library design. Maybe adding a wrapper range to Phobos to iterate over immutable 'ranges' would be an improvement to the current situation.

??? A range is only of any value if you can process it. That generally requires calling popFront on it. That mutates it. So, sure, you can an immutable range, but it's useless, because you can't process it. And why should that be surprising? There are plenty of things that are essentially useless if they're immutable. Take a stream for instance. Like a range, it's altered as you use. So, you can't have an immutable stream - or if you did, it would be useless. Immutability can be very useful, but not being able to mutate something can really get in the way of doing anything with it. The same goes for const.

There has been some discussion in the past of trying to find a way to convert an immutable or const range to a tail-immutable or tail-const range (similar to how you can pass immutable(char[]) to a function which takes immutable(char)[] and have it work), but the language doesn't currently provide any way to do that, and it wouldn't necessarily make sense for all range types anyway. It's highly dependent on how their implemented and what they're actually iterating over.

It's definitely true that the template situation could use some improvement to better deal with immutable arrays, and an improvement to the type system to better handle tail-const for ranges where applicable would be desirable, but the fact remains that not everything can be fully const or immutable and be useful. It's just a fact of how const and immutable work. There's stuff that needs to be mutable, and when you make it so that you can't mutate them, they're useless.

- Jonathan M Davis

Jonathan M Davis wrote:
> On 2011-07-11 09:06, Timon Gehr wrote:
> > Jonathan M Davis wrote:
> > > On Sunday 10 July 2011 21:09:27 Mehrdad wrote:
> > >> I noticed that the code below doesn't work, and I was wondering if it's
> > >>
> > >> by design (hopefully not):
> > >> immutable SEP = ", ";
> > >> ["a", "b"].join(SEP);
> > >>
> > >> The fact that SEP is immutable(char[]) instead of immutable(char)[]
> > >> shouldn't break the function.
> > >
> > > It most definitely breaks the function, and it's a limitation of templates. Templates are instantiated with the exact type that they're given, so the compiler tries to instantiate join with immutable(char[]), but join _can't_ work with immutable(char[]), because it needs a mutable range. immutable ranges are worthless. If the compiler were smart enough to realize that it could instantiate join with immutable(char)[] and it would work, then you could use immutable(char[]), but since it isn't that smart, it doesn't work. The same problem happens with static arrays. They can't be used as ranges, so even though they'd work if the compiler picked a dynamic range as the type for the function, they don't work, because the compiler isn't that smart.
> > >
> > > The problem may be fixed at some point, but as it stands, it just doesn't work to use immutable arrays with range-based functions.
> > >
> > > - Jonathan M Davis
> >
> > There is no such thing as an immutable range because the range abstraction
> > is based on mutation.
> > This does not make sense, because intuitively, immutable(char[]) is a range
> > type just as immutable(char)[] or char[] is a range type.
> > It is quite odd that a data structure can only be a D range if it may be
> > modified...
> >
> > It is not really a compiler issue, but one of library design. Maybe adding a wrapper range to Phobos to iterate over immutable 'ranges' would be an improvement to the current situation.
>
> ??? A range is only of any value if you can process it. That generally requires calling popFront on it. That mutates it. So, sure, you can an immutable range, but it's useless, because you can't process it.

import std.range;
void main(){
    int[] arr;
    auto a=cast(immutable)map!"a"(arr);
    assert(!isInputRange!(typeof(a))); // this passes

}

So no, you cannot have an immutable range because it cannot fulfill the range interface.

> And why should that be surprising?

This is surprising:
immutable SEP = ", ";

["a","b"].join(SEP);
// -- nope! Guess what? That is because your array is not a range! Why? Easy! It
cannot be changed! See?
// -- I don't care, but this should work... I'll implement the function myself!

immutable SEP = ", ";
string sep=SEP;
["a","b"].join(sep); //sure thing

> There are plenty of things that are essentially
> useless if they're immutable. Take a stream for instance. Like a range, it's
> altered as you use. So, you can't have an immutable stream - or if you did, it
> would be useless.

You can. You can even have immutable IO if you choose the right abstractions ;).

> Immutability can be very useful, but not being able to
> mutate something can really get in the way of doing anything with it. The same
> goes for const.

You can always use it's value to create others.

>
> There has been some discussion in the past of trying to find a way to convert
> an immutable or const range to a tail-immutable or tail-const range (similar
> to how you can pass immutable(char[]) to a function which takes
> immutable(char)[] and have it work),
> but the language doesn't currently
> provide any way to do that, and it wouldn't necessarily make sense for all
> range types anyway. It's highly dependent on how their implemented and what
> they're actually iterating over.

I completely agree. It would be unnecessarily complicated (impossible?) to make all ranges work as immutable that would meaningfully be able to. Also it would give almost no benefit.

But immutable(T[]) should clearly be accepted where immutable(T)[] would be. I've
run across that problem myself
It is just annoying, even more so if you do not actually need the generosity
provided by those functions at all.

>
> It's definitely true that the template situation could use some improvement to better deal with immutable arrays, and an improvement to the type system to better handle tail-const for ranges where applicable would be desirable, but the fact remains that not everything can be fully const or immutable and be useful. It's just a fact of how const and immutable work. There's stuff that needs to be mutable, and when you make it so that you can't mutate them, they're useless.
>
> - Jonathan M Davis

Nothing needs to be mutable. It is possible to work with immutable values only,
but it is no always efficient.
Having an immutable range interface would be quite easy. popFront would just have
to return the next range instead of changing the current one.


Cheers,
-Timon

On 2011-07-11 13:30, Timon Gehr wrote:
> Jonathan M Davis wrote:
> > On 2011-07-11 09:06, Timon Gehr wrote:
> > > Jonathan M Davis wrote:
> > > > On Sunday 10 July 2011 21:09:27 Mehrdad wrote:
> > > >> I noticed that the code below doesn't work, and I was wondering if it's
> > > >> 
> > > >> by design (hopefully not):
> > > >> immutable SEP = ", ";
> > > >> ["a", "b"].join(SEP);
> > > >> 
> > > >> The fact that SEP is immutable(char[]) instead of immutable(char)[]
> > > >> shouldn't break the function.
> > > > 
> > > > It most definitely breaks the function, and it's a limitation of templates. Templates are instantiated with the exact type that they're given, so the compiler tries to instantiate join with immutable(char[]), but join _can't_ work with immutable(char[]), because it needs a mutable range. immutable ranges are worthless. If the compiler were smart enough to realize that it could instantiate join with immutable(char)[] and it would work, then you could use immutable(char[]), but since it isn't that smart, it doesn't work. The same problem happens with static arrays. They can't be used as ranges, so even though they'd work if the compiler picked a dynamic range as the type for the function, they don't work, because the compiler isn't that smart.
> > > > 
> > > > The problem may be fixed at some point, but as it stands, it just doesn't work to use immutable arrays with range-based functions.
> > > > 
> > > > - Jonathan M Davis
> > > 
> > > There is no such thing as an immutable range because the range
> > > abstraction is based on mutation.
> > > This does not make sense, because intuitively, immutable(char[]) is a
> > > range type just as immutable(char)[] or char[] is a range type.
> > > It is quite odd that a data structure can only be a D range if it may
> > > be modified...
> > > 
> > > It is not really a compiler issue, but one of library design. Maybe adding a wrapper range to Phobos to iterate over immutable 'ranges' would be an improvement to the current situation.
> > 
> > ??? A range is only of any value if you can process it. That generally requires calling popFront on it. That mutates it. So, sure, you can an immutable range, but it's useless, because you can't process it.
> 
> import std.range;
> void main(){
> int[] arr;
> auto a=cast(immutable)map!"a"(arr);
> assert(!isInputRange!(typeof(a))); // this passes
> 
> }
> 
> So no, you cannot have an immutable range because it cannot fulfill the range interface.
> 
> > And why should that be surprising?
> 
> This is surprising:
> immutable SEP = ", ";
> 
> ["a","b"].join(SEP);
> // -- nope! Guess what? That is because your array is not a range! Why?
> Easy! It cannot be changed! See?
> // -- I don't care, but this should work... I'll implement the function
> myself!
> 
> immutable SEP = ", ";
> string sep=SEP;
> ["a","b"].join(sep); //sure thing
> 
> > There are plenty of things that are essentially
> > useless if they're immutable. Take a stream for instance. Like a range,
> > it's altered as you use. So, you can't have an immutable stream - or if
> > you did, it would be useless.
> 
> You can. You can even have immutable IO if you choose the right abstractions ;).
> 
> > Immutability can be very useful, but not being able to
> > mutate something can really get in the way of doing anything with it. The
> > same goes for const.
> 
> You can always use it's value to create others.
> 
> > There has been some discussion in the past of trying to find a way to
> > convert an immutable or const range to a tail-immutable or tail-const
> > range (similar to how you can pass immutable(char[]) to a function which
> > takes
> > immutable(char)[] and have it work),
> > but the language doesn't currently
> > provide any way to do that, and it wouldn't necessarily make sense for
> > all range types anyway. It's highly dependent on how their implemented
> > and what they're actually iterating over.
> 
> I completely agree. It would be unnecessarily complicated (impossible?) to make all ranges work as immutable that would meaningfully be able to. Also it would give almost no benefit.
> 
> But immutable(T[]) should clearly be accepted where immutable(T)[] would
> be. I've run across that problem myself
> It is just annoying, even more so if you do not actually need the
> generosity provided by those functions at all.
> 
> > It's definitely true that the template situation could use some improvement to better deal with immutable arrays, and an improvement to the type system to better handle tail-const for ranges where applicable would be desirable, but the fact remains that not everything can be fully const or immutable and be useful. It's just a fact of how const and immutable work. There's stuff that needs to be mutable, and when you make it so that you can't mutate them, they're useless.
> > 
> > - Jonathan M Davis
> 
> Nothing needs to be mutable. It is possible to work with immutable values
> only, but it is no always efficient.
> Having an immutable range interface would be quite easy. popFront would
> just have to return the next range instead of changing the current one.

The solution is to fix template instantiation so that it's smarter when dealing with static arrays and const or immutable arrays: http://d.puremagic.com/issues/show_bug.cgi?id=6148

It's a language issue, not a design issue. Making ranges function like slists (with head and tail or car and cdr) would be almost certainly be too inefficient (particularly for ranges where save is not super cheap, though it's at least supposed to be fairly cheap). Immutability might be nice, but it does have its costs, and in this case, D's templates aren't currenly smart enough to use immutable(E)[] instead of immutable(E[]). And it's not like immutable ranges are going to work with non-array ranges anyway, so it's arguably a good idea to just expect immutable and const ranges to not work anyway.

- Jonathan M Davis

On Tue, 12 Jul 2011 00:01:55 +0300, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> The solution is to fix template instantiation so that it's smarter when
> dealing with static arrays and const or immutable arrays:
> http://d.puremagic.com/issues/show_bug.cgi?id=6148
>
> It's a language issue, not a design issue. Making ranges function like slists
> (with head and tail or car and cdr) would be almost certainly be too
> inefficient (particularly for ranges where save is not super cheap, though
> it's at least supposed to be fairly cheap). Immutability might be nice, but it
> does have its costs, and in this case, D's templates aren't currenly smart
> enough to use immutable(E)[] instead of immutable(E[]). And it's not like
> immutable ranges are going to work with non-array ranges anyway, so it's
> arguably a good idea to just expect immutable and const ranges to not work
> anyway.

There is a simple workaround for this type of ranges that are like iterators, which we know the beginning and the end.
We can improve isForwardRange!R by adding a line hasForwardRange!R. If it does have, we return an adaptor which gives us a mutable range.
Good thing is because the original range is mutable we don't need to worry about anything else.

On Tue, 12 Jul 2011 00:37:04 +0300, so <so@so.so> wrote:

> Good thing is because the original range is mutable we don't need to worry about anything else.

Typo, "original range is immutable"

Jonathan M Davis wrote:
> The solution is to fix template instantiation so that it's smarter when dealing with static arrays and const or immutable arrays: http://d.puremagic.com/issues/show_bug.cgi?id=6148
>

Unless I am missing something essential, supporting this would turn the complexity
of failing template function instantiations to Omega((time for matching)*2^(number
of const/immutable arrays in the input)). I don't think this is too great.

> It's a language issue, not a design issue. Making ranges function like slists (with head and tail or car and cdr) would be almost certainly be too inefficient (particularly for ranges where save is not super cheap, though it's at least supposed to be fairly cheap).

That Eg. join cannot take an immutable(char[]) is certainly a design issue.
I think what you find to be a language issue are limitations inherent to templates
that are very hard (NP hard in the general case) to overcome.

> Immutability might be nice, but it
> does have its costs, and in this case, D's templates aren't currenly smart
> enough to use immutable(E)[] instead of immutable(E[]).

I claim they cannot get that smart. If I'm right, this turns it into a complete design issue.

> And it's not like
> immutable ranges are going to work with non-array ranges anyway, so it's
> arguably a good idea to just expect immutable and const ranges to not work
> anyway.
>
> - Jonathan M Davis

immutable ranges don't exist in D. immutable arrays do. The issue is that many Phobos functions can only take ranges, what excludes immutable arrays. That is bad as immutable arrays share many properties with ranges and could be used as input to similar algorithms.

This does not have top priority (at least for me), as it is just a mild annoyance that can be worked around. But it needs some thinking.

Cheers,
-Timon