View mode: basic / threaded / horizontal-split · Log in · Help
December 10, 2011
Fixing const arrays
Walter and I discussed today and decided to fix this long-standing issue:

import std.algorithm;
void main() {
  const arr = [1, 2, 3];
  reduce!"a*b"(arr);
}

The problem here is that D's built-in arrays are at the same time 
containers and their own ranges. When the array is constant the matter 
becomes confusing because the range itself must be non-constant such 
that it can iterate the constant array.

Put simply, the type of the array in this case should be const(int[]) 
and the type of its range should be const(int)[].

This problem does not commonly occur with ranges because people do 
expect that a constant range can't be used for iteration, and an 
elaborate container will emit the proper type of range even when the 
container itself is constant.

We decided to fix this issue by automatically shedding the top-level 
const when passing an array or a pointer by value into a function.

The rule is simple and does not complicate lookup rules, type deduction, 
or template specialization. It is a semantic rewrite as follows.

Consider x an array of type qualifier(T[]). In any function call in 
which it is established that x is passed by value (i.e. no "ref" with 
the parameter), the call:

fun(..., x, ...)

is lowered into:

fun(..., cast(qualifier(T)[]) x, ...)

after which the usual language rules apply. Similarly, if x has type 
qualifier(T*), AND if x is passed by value into a function, the call:

fun(..., x, ...)

is lowered into:

fun(..., cast(qualifier(T)*) x, ...)

after which, again, the usual rules apply. Note that fun does not need 
to be a template and generally must meet no special conditions aside 
from taking x by value. If fun takes specifically a const(T[]), the call 
will go through no problem because const(T)[] is implicitly convertible 
to const(T[]).

This allows template functions to accept arrays by value yet modify 
their own private copy of the array's bounds, which is reasonable and 
expected.

This rule solves a host of issues related to applying range algorithms 
to arrays. I hope we will have this rule in action in dmd 2.057.


Andrei
December 10, 2011
Re: Fixing const arrays
Andrei Alexandrescu Wrote:
> We decided to fix this issue by automatically shedding the top-level 
> const when passing an array or a pointer by value into a function.

Thank you. Finally.
December 10, 2011
Re: Fixing const arrays
On Saturday, December 10, 2011 15:47:13 Andrei Alexandrescu wrote:
> Walter and I discussed today and decided to fix this long-standing issue:
[snip]

This looks like a solid decision, and I was actually about to suggest it when 
thinking about the issues with const ranges today. It's the only sensible 
solution that I can think of. I was wondering if there might be any negative 
side effects to it, but I haven't been able to think of any.

It would be nice to do the same thing with static arrays, but that would make 
it harder to pass static arrays to templated functions, and it actually 
becomes quite dangerous, since it then becomes very easy to mistakenly return 
references to that static array when the scope is exited. So, I think that the 
issue of having to slice static arrays should probably remain, but unlike 
dynamic arrays, static arrays _do_ own their memory, so it's quite reasonably 
to have to slice them IMHO.

The other remaining issue is how to handle tail-const in the general case. 
Ideally, there would be a way to return a tail-const range from a const range. 
The only way I can think of enabling that at the moment is to make it so that 
if opSlice is defined for a type, and it returns the same type (save for how 
const or immutable it is), then it could be treated the same way as arrays 
would be as far as template instantiations go (that is, the newly suggested 
way). But I don't know quite what side-effects that has. Still, I do think that 
we should find a way to define tail-constness for ranges which are user-defined 
types. It's not as critical as with arrays, but given the transivity of const, 
it's still important, since it can be very easy to end up in a situation where 
it becomes somewhat problematic to get a tail-const range when you're using 
const much.

In any case, I applaud this decision. It should definitely help dealing with 
arrays as ranges easier. The fact that const and immutable don't work very 
well at all with ranges at this point is problematic given how const and 
immutable are supposed to be major features of the language, and this is 
definitely a step in the right direction.

- Jonathan M Davis
December 10, 2011
Re: Fixing const arrays
Le 10/12/2011 22:47, Andrei Alexandrescu a écrit :
> Walter and I discussed today and decided to fix this long-standing issue:
>
> import std.algorithm;
> void main() {
> const arr = [1, 2, 3];
> reduce!"a*b"(arr);
> }
>
> The problem here is that D's built-in arrays are at the same time
> containers and their own ranges. When the array is constant the matter
> becomes confusing because the range itself must be non-constant such
> that it can iterate the constant array.
>
> Put simply, the type of the array in this case should be const(int[])
> and the type of its range should be const(int)[].
>
> This problem does not commonly occur with ranges because people do
> expect that a constant range can't be used for iteration, and an
> elaborate container will emit the proper type of range even when the
> container itself is constant.
>
> We decided to fix this issue by automatically shedding the top-level
> const when passing an array or a pointer by value into a function.
>
> The rule is simple and does not complicate lookup rules, type deduction,
> or template specialization. It is a semantic rewrite as follows.
>
> Consider x an array of type qualifier(T[]). In any function call in
> which it is established that x is passed by value (i.e. no "ref" with
> the parameter), the call:
>
> fun(..., x, ...)
>
> is lowered into:
>
> fun(..., cast(qualifier(T)[]) x, ...)
>
> after which the usual language rules apply. Similarly, if x has type
> qualifier(T*), AND if x is passed by value into a function, the call:
>
> fun(..., x, ...)
>
> is lowered into:
>
> fun(..., cast(qualifier(T)*) x, ...)
>
> after which, again, the usual rules apply. Note that fun does not need
> to be a template and generally must meet no special conditions aside
> from taking x by value. If fun takes specifically a const(T[]), the call
> will go through no problem because const(T)[] is implicitly convertible
> to const(T[]).
>
> This allows template functions to accept arrays by value yet modify
> their own private copy of the array's bounds, which is reasonable and
> expected.
>
> This rule solves a host of issues related to applying range algorithms
> to arrays. I hope we will have this rule in action in dmd 2.057.
>
>
> Andrei

I love to see that. This is going the right way.

You deserve your wikipedia page (semi private joke inside).
December 10, 2011
Re: Fixing const arrays
Treating whole constant arrays as ranges by automatically shedding the
top-level const is good.
But realizing it by language semantic change is definitely bad.It
breaks IFTI rule, and adding special case will make difficult to learn
language.

Instead of language change, we can add specializations that receive
non-ranges and convert them to ranges by removing top-level const.
I believe that it is Phobos issue and is never the issue of language.

Kenji

2011/12/11 Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:
> Walter and I discussed today and decided to fix this long-standing issue:
>
> import std.algorithm;
> void main() {
>  const arr = [1, 2, 3];
>  reduce!"a*b"(arr);
> }
>
> The problem here is that D's built-in arrays are at the same time containers
> and their own ranges. When the array is constant the matter becomes
> confusing because the range itself must be non-constant such that it can
> iterate the constant array.
>
> Put simply, the type of the array in this case should be const(int[]) and
> the type of its range should be const(int)[].
>
> This problem does not commonly occur with ranges because people do expect
> that a constant range can't be used for iteration, and an elaborate
> container will emit the proper type of range even when the container itself
> is constant.
>
> We decided to fix this issue by automatically shedding the top-level const
> when passing an array or a pointer by value into a function.
>
> The rule is simple and does not complicate lookup rules, type deduction, or
> template specialization. It is a semantic rewrite as follows.
>
> Consider x an array of type qualifier(T[]). In any function call in which it
> is established that x is passed by value (i.e. no "ref" with the parameter),
> the call:
>
> fun(..., x, ...)
>
> is lowered into:
>
> fun(..., cast(qualifier(T)[]) x, ...)
>
> after which the usual language rules apply. Similarly, if x has type
> qualifier(T*), AND if x is passed by value into a function, the call:
>
> fun(..., x, ...)
>
> is lowered into:
>
> fun(..., cast(qualifier(T)*) x, ...)
>
> after which, again, the usual rules apply. Note that fun does not need to be
> a template and generally must meet no special conditions aside from taking x
> by value. If fun takes specifically a const(T[]), the call will go through
> no problem because const(T)[] is implicitly convertible to const(T[]).
>
> This allows template functions to accept arrays by value yet modify their
> own private copy of the array's bounds, which is reasonable and expected.
>
> This rule solves a host of issues related to applying range algorithms to
> arrays. I hope we will have this rule in action in dmd 2.057.
>
>
> Andrei
December 10, 2011
Re: Fixing const arrays
This is just about removing the constness for whatever the part of a 
type that is passed by value. This is not really a special rule and make 
sense.

Le 10/12/2011 23:31, kenji hara a écrit :
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
>
> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.
>
> Kenji
>
> 2011/12/11 Andrei Alexandrescu<SeeWebsiteForEmail@erdani.org>:
>> Walter and I discussed today and decided to fix this long-standing issue:
>>
>> import std.algorithm;
>> void main() {
>>   const arr = [1, 2, 3];
>>   reduce!"a*b"(arr);
>> }
>>
>> The problem here is that D's built-in arrays are at the same time containers
>> and their own ranges. When the array is constant the matter becomes
>> confusing because the range itself must be non-constant such that it can
>> iterate the constant array.
>>
>> Put simply, the type of the array in this case should be const(int[]) and
>> the type of its range should be const(int)[].
>>
>> This problem does not commonly occur with ranges because people do expect
>> that a constant range can't be used for iteration, and an elaborate
>> container will emit the proper type of range even when the container itself
>> is constant.
>>
>> We decided to fix this issue by automatically shedding the top-level const
>> when passing an array or a pointer by value into a function.
>>
>> The rule is simple and does not complicate lookup rules, type deduction, or
>> template specialization. It is a semantic rewrite as follows.
>>
>> Consider x an array of type qualifier(T[]). In any function call in which it
>> is established that x is passed by value (i.e. no "ref" with the parameter),
>> the call:
>>
>> fun(..., x, ...)
>>
>> is lowered into:
>>
>> fun(..., cast(qualifier(T)[]) x, ...)
>>
>> after which the usual language rules apply. Similarly, if x has type
>> qualifier(T*), AND if x is passed by value into a function, the call:
>>
>> fun(..., x, ...)
>>
>> is lowered into:
>>
>> fun(..., cast(qualifier(T)*) x, ...)
>>
>> after which, again, the usual rules apply. Note that fun does not need to be
>> a template and generally must meet no special conditions aside from taking x
>> by value. If fun takes specifically a const(T[]), the call will go through
>> no problem because const(T)[] is implicitly convertible to const(T[]).
>>
>> This allows template functions to accept arrays by value yet modify their
>> own private copy of the array's bounds, which is reasonable and expected.
>>
>> This rule solves a host of issues related to applying range algorithms to
>> arrays. I hope we will have this rule in action in dmd 2.057.
>>
>>
>> Andrei
December 10, 2011
Re: Fixing const arrays
On Sunday, December 11, 2011 07:31:28 kenji hara wrote:
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
> 
> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.

Making the fix in Phobos is a pain in that you have to duplicate functions all 
over the place (though they'd just be wrappers, so you wouldn't have to 
duplicate their implementations). And everyone who defines their own range-
based functions have to do the same thing.

The question is whether we actually lose anything by making it so that so that 
IFTI won't give you a fully immutable or const array. And I don't see anything 
that you'd lose or how it could break any code. Sure, it then becomes possible 
to alter the array within the function where it wasn't before, but if it 
couldn't before, making it possible now won't break anything, since the 
function would have to be changed to even try. And for new functions, if they 
_really_ want the array to be const or immutable, they can assign it to a new 
array locally. I don't see how any expressiveness is really lost here or that 
anything would be broken.

So, while this _does_ change the language's semantics, it seems to me that it 
just changes them for the better, not worse.

Making this sort of change for static arrays _would_ be a problem, but I don't 
see how it's an issue for dynamic arrays.

- Jonathan M Davis
December 10, 2011
Re: Fixing const arrays
On 12/10/11 4:31 PM, kenji hara wrote:
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.

There is no breakage of IFTI, just a reduction of what IFTI sees.

> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.

The problem here is scale. We're looking at an absolutely massive code 
duplication for every single function. I don't think this will ever 
work; if we do it in Phobos it will make Phobos and by extension the 
language more difficult to understand AND more bloated for the sake of 
needless consistency.

The language change is legitimate when you think of it this way - it 
gives template functions back the natural right to change their local 
state. This right was already there for non-template functions.


Andrei
December 10, 2011
Re: Fixing const arrays
On 12/10/11 4:31 PM, kenji hara wrote:
> Treating whole constant arrays as ranges by automatically shedding the
> top-level const is good.
> But realizing it by language semantic change is definitely bad.It
> breaks IFTI rule, and adding special case will make difficult to learn
> language.
>
> Instead of language change, we can add specializations that receive
> non-ranges and convert them to ranges by removing top-level const.
> I believe that it is Phobos issue and is never the issue of language.

I should add there is precedent. C++ also removes top-level const when 
passing objects by value to templates. Deducing top-level const with 
pass-by-value is inherently nonsensical.

Andrei
December 10, 2011
Re: Fixing const arrays
kenji hara:

> It breaks IFTI rule,

What do you mean?


> and adding special case will make difficult to learn language.

I think that change proposed by Andrei A. (that I like) doesn't add a special case to D2. Currently (2.057beta) this works:


void main() {
   immutable(int[]) a = [1, 2];
   immutable(int)[] b = a;
}

Calling a function that accepts a immutable(int)[] with a immutable(int[]) means creating a local slice, it's similar. So I think Andrei A. removes a special case.

Bye,
bearophile
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home