On Mon, 14 Nov 2011 16:28:52 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
>> Look at the code generated for enum a = [1, 2, 3]. using a is replaced
>> with a call to _d_arrayliteral. There is no CTFE going on.
>>
>
> There is some ctfe going on, but the compiler has to allocate the result anew every time it is used. So there is also some runtime overhead.
>
> To make my point clearer:
>
> int foo(){return 100;}
> enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100, 100];
>
> void main(){
>      auto x = a; // this does *not* call foo. But it allocates a new array literal
> }

Yes, you are right.  The issue is that the resulting array is initialized at runtime, not that CTFE is being avoided.  After doing some of these tests, I have a better understanding of the issues.

>> The compiler has no choice. It must develop the array at runtime, or
>> else the type allows one to modify the source value (just like in D1 how
>> you could modify string literals). In essence, the compiler is creating
>> a new copy for every usage (and building it from scratch).
>>
>
> That is a quality of implementation issue. The language semantics do not require that.

The language semantics require that if an enum type points at mutable data, a runtime allocation *must* occur to avoid corruption of literals.  I think a rule requiring an enum to be immutable or implicitly cast to immutable puts the burden of runtime allocation on the coder, making it clear what's going on.

In C++, novice coders typically pass classes by value not knowing what a horrible thing this is doing.  Then they are puzzled why the code is so slow, the syntax is so short!  This is another case of a hidden allocation which can be either avoided or made visible.

>>> enum a = [2,1,4];
>>> enum b = sort(a); // should be fine.
>>
>> I was actually surprised that this compiles. But this should not be a
>> problem even if a was immutable(int)[]. sort should be able to create a
>> copy of an immutable array in order to sort it. It doesn't matter the
>> performance hit, because this should all be done at compile time.
>>
>
> It does not, but explicitly calling .dup works
> immutable x = [3,2,1];
> immutable y = sort(x.dup);

I'm saying sort (or another symbol, ctfesort?) can be made to do the dup automatically for you so you don't have to have it when using ctfe.  Extra allocations during CTFE cost nothing (well, with a properly GC'd compiler, that is).

Update: I have a better idea, see below.

>> When I see an enum, I think "evaluated at compile time". No matter how
>> complex it is to build that value, it should be built at compile-time
>> and *used* at runtime. No complex function calls should be done at
>> runtime, an enum is a value.
>
> Exactly. Therefore you assign from it by copying it.
>
> Compare to static array.
>
> int[10] x = [1,2,3,4,5,6,7,8,9,0];
>
> x still needs to be initialized at runtime.

Yes, but this is spelled out because copying a static array requires moving data.  However, this does *not* require a hidden allocation (even though it does do a hidden allocation currently).

I'm not worried about copying data as much as I am about hidden allocations.  Hidden allocations are a huge drag on performance.  Every time you allocate, you need to take a global GC lock, and it's an unbounded operation (doing one allocation could run a collection cycle).

>> I did an interesting little test:

[snip]

>
> That just tells us that DMD sucks at generating code for array literals.
>
> This generates identical code:
>
> import std.stdio;
>
> void main() {
>      writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3]);
> }
>
> You don't need enums for that.
>
>
> What it actually should for both our examples is more like the following:
>
> import std.stdio;
>
> immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];
>
> void main() {
>      writeln(_somewhereinrom.dup);
> }
>
> push   %ebp
> mov    %esp,%ebp
> pushl  0x8097184
> pushl  0x8097180
> mov    $0x80975c8,%eax
> push   %eax
> call   8079470 <_adDupT>
> add    $0xc,%esp
> push   %edx
> push   %eax
> call   807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
> xor    %eax,%eax
> pop    %ebp
> ret
>
>
> If writeln would actually be const correct, the compiler could even get rid of the allocation.

That is the idea.  Get rid of the hidden allocation.  Writeln *is* const correct, it can certainly print immutable(int)[].  The issue is not writeln, it's what the type of the array  literal/enum is.

Technically, an array literal is equivalent to an enum, and should follow the same rules.

> This is not about enums that much, it is about array literals.
>
> The fact that stack static array initialization allocates is one of DMDs bigger warts.
>
> Look at the ridiculous code generated for the following example:
>
> void main() {
>      int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];
>      writeln(x);
> }

Yes, these are all cases of the same issue.

>>> enum a = [2,1,4];
>>> enum b = sort(a.dup); // what exactly is that 'a.dup' thing?
>>
>> I don't think .dup should be necessary at compile time. Creating a
>> sorted copy of an immutable array should be quite doable.
>>
>
> I agree, phobos won't currently do it though.

This is easily fixed.  But maybe there is a better way (see below).

>>> enum d = sort(c); // does not work?
>>>
>>> enum e = foo(a.dup, b.dup, c.dup, d.dup);
>>
>> Again, I don't think .dup would be used for dependent enums, I was
>> rather thinking dup would be used where you need a mutable copy of an
>> array during enum usage in normal code.
>>
>
> But if the type of a,b,c,d is immutable(int)[] and foo is a function that takes 4 int[]s then the .dup's are necessary to pass type checking.

What about this idea:

At a global level, expressions that result in CTFE being triggered, can be implicitly cast from mutable to immutable and vice versa via a deep-dup.  This allows you to use enums as parameters to functions accepting mutable references.  Then enums that are derived from other enums do not need to follow the same rules as runtime code that uses the enums.

This of course, only happens at the global-expressions level, as function internals must compile at runtime as well.

What I thought of as a solution was to create CTFE only functions that wrap other functions to do a dup.  But you wouldn't want to do this during runtime, because dup is expensive.  During compile time, dup costs nothing.  This idea essentially takes the place of that boilerplate code.

-Steve

On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:
> On Mon, 14 Nov 2011 16:28:52 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 11/14/2011 09:39 PM, Steven Schveighoffer wrote:
>>> Look at the code generated for enum a = [1, 2, 3]. using a is replaced
>>> with a call to _d_arrayliteral. There is no CTFE going on.
>>>
>>
>> There is some ctfe going on, but the compiler has to allocate the
>> result anew every time it is used. So there is also some runtime
>> overhead.
>>
>> To make my point clearer:
>>
>> int foo(){return 100;}
>> enum a = [foo(), foo(), foo()]; // a is the array literal [100, 100,
>> 100];
>>
>> void main(){
>> auto x = a; // this does *not* call foo. But it allocates a new array
>> literal
>> }
>
> Yes, you are right. The issue is that the resulting array is initialized
> at runtime, not that CTFE is being avoided. After doing some of these
> tests, I have a better understanding of the issues.
>
>>> The compiler has no choice. It must develop the array at runtime, or
>>> else the type allows one to modify the source value (just like in D1 how
>>> you could modify string literals). In essence, the compiler is creating
>>> a new copy for every usage (and building it from scratch).
>>>
>>
>> That is a quality of implementation issue. The language semantics do
>> not require that.
>
> The language semantics require that if an enum type points at mutable
> data, a runtime allocation *must* occur to avoid corruption of literals.
> I think a rule requiring an enum to be immutable or implicitly cast to
> immutable puts the burden of runtime allocation on the coder, making it
> clear what's going on.
>
> In C++, novice coders typically pass classes by value not knowing what a
> horrible thing this is doing. Then they are puzzled why the code is so
> slow, the syntax is so short! This is another case of a hidden
> allocation which can be either avoided or made visible.
>
>>>> enum a = [2,1,4];
>>>> enum b = sort(a); // should be fine.
>>>
>>> I was actually surprised that this compiles. But this should not be a
>>> problem even if a was immutable(int)[]. sort should be able to create a
>>> copy of an immutable array in order to sort it. It doesn't matter the
>>> performance hit, because this should all be done at compile time.
>>>
>>
>> It does not, but explicitly calling .dup works
>> immutable x = [3,2,1];
>> immutable y = sort(x.dup);
>
> I'm saying sort (or another symbol, ctfesort?) can be made to do the dup
> automatically for you so you don't have to have it when using ctfe.
> Extra allocations during CTFE cost nothing (well, with a properly GC'd
> compiler, that is).
>
> Update: I have a better idea, see below.
>
>>> When I see an enum, I think "evaluated at compile time". No matter how
>>> complex it is to build that value, it should be built at compile-time
>>> and *used* at runtime. No complex function calls should be done at
>>> runtime, an enum is a value.
>>
>> Exactly. Therefore you assign from it by copying it.
>>
>> Compare to static array.
>>
>> int[10] x = [1,2,3,4,5,6,7,8,9,0];
>>
>> x still needs to be initialized at runtime.
>
> Yes, but this is spelled out because copying a static array requires
> moving data. However, this does *not* require a hidden allocation (even
> though it does do a hidden allocation currently).
>
> I'm not worried about copying data as much as I am about hidden
> allocations. Hidden allocations are a huge drag on performance. Every
> time you allocate, you need to take a global GC lock, and it's an
> unbounded operation (doing one allocation could run a collection cycle).

You don't actually _need_ a global GC lock. It is just how it is implemented in this case. Note that this is an explicit allocation:

int[] a = [1,2,3]; // just as explicit as a NewExpression

Only the enums "hide" it sometimes, but actually you should know the involved types.

>
>>> I did an interesting little test:
>
> [snip]
>
>>
>> That just tells us that DMD sucks at generating code for array literals.
>>
>> This generates identical code:
>>
>> import std.stdio;
>>
>> void main() {
>> writeln([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3,
>> 3, 3, 3, 3]);
>> }
>>
>> You don't need enums for that.
>>
>>
>> What it actually should for both our examples is more like the following:
>>
>> import std.stdio;
>>
>> immutable _somewhereinrom = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,
>> 2, 2, 3, 3, 3, 3, 3, 3, 3, 3];
>>
>> void main() {
>> writeln(_somewhereinrom.dup);
>> }
>>
>> push %ebp
>> mov %esp,%ebp
>> pushl 0x8097184
>> pushl 0x8097180
>> mov $0x80975c8,%eax
>> push %eax
>> call 8079470 <_adDupT>
>> add $0xc,%esp
>> push %edx
>> push %eax
>> call 807041c <_D3std5stdio15__T7writelnTAiZ7writelnFAiZv>
>> xor %eax,%eax
>> pop %ebp
>> ret
>>
>>
>> If writeln would actually be const correct, the compiler could even
>> get rid of the allocation.
>
> That is the idea. Get rid of the hidden allocation. Writeln *is* const
> correct, it can certainly print immutable(int)[].

Well, there is a function called writeln that can do that. That is a different function. But the one that gets actually called is not const correct as well.


This is writeln:

// Most general instance
void writeln(T...)(T args)
if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) : const(char)[]))
{
    stdout.write(args, '\n');
}


=>
writeln([1,2,3]);
// modulo IFTY:
writeln!(int[])([1,2,3]);
// const correct?
writeln!(int[])([1,2,3].idup); // nope!

Error: cannot implicitly convert expression (_adDupT(& _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]


Now if there was a const there, the compiler could infer that writeln will not change the .duped array. Ergo it could pass the reference to immutable data directly. It would maybe help against template code bloat a bit, but not that much because const(immutable(int)[]) and the like are distinct types to const(int[]).




> The issue is not
> writeln, it's what the type of the array literal/enum is.
>

> Technically, an array literal is equivalent to an enum, and should
> follow the same rules.
>

Remember that immutable is transitive. That can really get in your way in this case.



>> This is not about enums that much, it is about array literals.
>>
>> The fact that stack static array initialization allocates is one of
>> DMDs bigger warts.
>>
>> Look at the ridiculous code generated for the following example:
>>
>> void main() {
>> int[24] x = [1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3,
>> 3, 3, 3, 3, 3];
>> writeln(x);
>> }
>
> Yes, these are all cases of the same issue.
>
>>>> enum a = [2,1,4];
>>>> enum b = sort(a.dup); // what exactly is that 'a.dup' thing?
>>>
>>> I don't think .dup should be necessary at compile time. Creating a
>>> sorted copy of an immutable array should be quite doable.
>>>
>>
>> I agree, phobos won't currently do it though.
>
> This is easily fixed. But maybe there is a better way (see below).
>
>>>> enum d = sort(c); // does not work?
>>>>
>>>> enum e = foo(a.dup, b.dup, c.dup, d.dup);
>>>
>>> Again, I don't think .dup would be used for dependent enums, I was
>>> rather thinking dup would be used where you need a mutable copy of an
>>> array during enum usage in normal code.
>>>
>>
>> But if the type of a,b,c,d is immutable(int)[] and foo is a function
>> that takes 4 int[]s then the .dup's are necessary to pass type checking.
>
> What about this idea:
>
> At a global level, expressions that result in CTFE being triggered, can
> be implicitly cast from mutable to immutable and vice versa via a
> deep-dup. This allows you to use enums as parameters to functions
> accepting mutable references. Then enums that are derived from other
> enums do not need to follow the same rules as runtime code that uses the
> enums.
>
> This of course, only happens at the global-expressions level, as
> function internals must compile at runtime as well.
>
> What I thought of as a solution was to create CTFE only functions that
> wrap other functions to do a dup. But you wouldn't want to do this
> during runtime, because dup is expensive. During compile time, dup costs
> nothing. This idea essentially takes the place of that boilerplate code.
>
> -Steve


That could work, but I think this is cluttering up the semantics a bit.

On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:

>> Yes, but this is spelled out because copying a static array requires
>> moving data. However, this does *not* require a hidden allocation (even
>> though it does do a hidden allocation currently).
>>
>> I'm not worried about copying data as much as I am about hidden
>> allocations. Hidden allocations are a huge drag on performance. Every
>> time you allocate, you need to take a global GC lock, and it's an
>> unbounded operation (doing one allocation could run a collection cycle).
>
> You don't actually _need_ a global GC lock. It is just how it is implemented in this case.

This is all fine in theory.  I haven't seen any implementations.  But memory allocations are not cheap, even without a GC lock.

> Note that this is an explicit allocation:
>
> int[] a = [1,2,3]; // just as explicit as a NewExpression
>
> Only the enums "hide" it sometimes, but actually you should know the involved types.

As I've said, there are already ways to explicitly allocate memory.  A suggested replacement for this is:

int[] a = array(1, 2, 3);

And you could always do:

int[] a = [1, 2, 3].dup;

Nobody complains about having to do:

char[] a = "hello".dup;

I don't see why we couldn't do the same for all array literals.

>> That is the idea. Get rid of the hidden allocation. Writeln *is* const
>> correct, it can certainly print immutable(int)[].
>
> Well, there is a function called writeln that can do that. That is a different function. But the one that gets actually called is not const correct as well.
>
>
> This is writeln:
>
> // Most general instance
> void writeln(T...)(T args)
> if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) : const(char)[]))
> {
>      stdout.write(args, '\n');
> }
>
>
> =>
> writeln([1,2,3]);
> // modulo IFTY:
> writeln!(int[])([1,2,3]);
> // const correct?
> writeln!(int[])([1,2,3].idup); // nope!
>
> Error: cannot implicitly convert expression (_adDupT(& _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]

I'm not sure what this means.  If [1, 2, 3] is typed as immtuable(int)[], it will compile.

Simple test:

immutable(int)[] a = [1, 2, 3];
writeln(a);

works with 2.056.

>> The issue is not
>> writeln, it's what the type of the array literal/enum is.
>>
>
>> Technically, an array literal is equivalent to an enum, and should
>> follow the same rules.
>>
>
> Remember that immutable is transitive. That can really get in your way in this case.

If the data is stored in ROM, it should be typed as immutable.  If it's not, then it could be modified.  The only other option is heap allocation and construction on use, which I've already said is undesirable.

If we start from "all array literals and enums that contain references store the referenced data in ROM," then we will find ways to deal with any inconveniences.  It's all a matter of what's more important in a system language, performance or convenience.

>> What about this idea:
>>
>> At a global level, expressions that result in CTFE being triggered, can
>> be implicitly cast from mutable to immutable and vice versa via a
>> deep-dup. This allows you to use enums as parameters to functions
>> accepting mutable references. Then enums that are derived from other
>> enums do not need to follow the same rules as runtime code that uses the
>> enums.
>>
>> This of course, only happens at the global-expressions level, as
>> function internals must compile at runtime as well.
>>
>> What I thought of as a solution was to create CTFE only functions that
>> wrap other functions to do a dup. But you wouldn't want to do this
>> during runtime, because dup is expensive. During compile time, dup costs
>> nothing. This idea essentially takes the place of that boilerplate code.
>
>
> That could work, but I think this is cluttering up the semantics a bit.

It's just a shortcut.  In essence, during compile time execution, .dups are added for you because they are free.  During runtime, they are not free, so you must add them.

-Steve

On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
> On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 11/15/2011 04:53 PM, Steven Schveighoffer wrote:
>
>>> Yes, but this is spelled out because copying a static array requires
>>> moving data. However, this does *not* require a hidden allocation (even
>>> though it does do a hidden allocation currently).
>>>
>>> I'm not worried about copying data as much as I am about hidden
>>> allocations. Hidden allocations are a huge drag on performance. Every
>>> time you allocate, you need to take a global GC lock, and it's an
>>> unbounded operation (doing one allocation could run a collection cycle).
>>
>> You don't actually _need_ a global GC lock. It is just how it is
>> implemented in this case.
>
> This is all fine in theory. I haven't seen any implementations. But
> memory allocations are not cheap, even without a GC lock.
>
>> Note that this is an explicit allocation:
>>
>> int[] a = [1,2,3]; // just as explicit as a NewExpression
>>
>> Only the enums "hide" it sometimes, but actually you should know the
>> involved types.
>
> As I've said, there are already ways to explicitly allocate memory. A
> suggested replacement for this is:
>
> int[] a = array(1, 2, 3);
>
> And you could always do:
>
> int[] a = [1, 2, 3].dup;
>
> Nobody complains about having to do:
>
> char[] a = "hello".dup;
>
> I don't see why we couldn't do the same for all array literals.
>

Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.


>>> That is the idea. Get rid of the hidden allocation. Writeln *is* const
>>> correct, it can certainly print immutable(int)[].
>>
>> Well, there is a function called writeln that can do that. That is a
>> different function. But the one that gets actually called is not const
>> correct as well.
>>
>>
>> This is writeln:
>>
>> // Most general instance
>> void writeln(T...)(T args)
>> if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
>> const(char)[]))
>> {
>> stdout.write(args, '\n');
>> }
>>
>>
>> =>
>> writeln([1,2,3]);
>> // modulo IFTY:
>> writeln!(int[])([1,2,3]);
>> // const correct?
>> writeln!(int[])([1,2,3].idup); // nope!
>>
>> Error: cannot implicitly convert expression (_adDupT(&
>> _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]
>
> I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[],
> it will compile.
>
> Simple test:
>
> immutable(int)[] a = [1, 2, 3];
> writeln(a);
>
> works with 2.056.

That is like saying

void foo(int[] a){...}             // prints a
void bar(immutable(int)[] a){...}  // prints a

int[] a = [1,2,3];
immutable(int)[] b = [1, 2, 3];
foo(a);
bar(b);

"=> Oh, bar is const correct because I can call foo with mutable data and bar with immutable data."

foo is writeln!(int[])
bar is writeln!(immutable(int)[])

in effect, if you do:

writeln("hello".dup);

The compiler cannot assume that the IFTI'd writeln!(char[]) will not change the argument, ergo it cannot optimize away the .dup, even though it would be a valid transformation as long as the programmer does not make the program semantics depend on the identity relation on immutable references (doing so defeats the purpose of immutability).


>
>>> The issue is not
>>> writeln, it's what the type of the array literal/enum is.
>>>
>>
>>> Technically, an array literal is equivalent to an enum, and should
>>> follow the same rules.
>>>
>>
>> Remember that immutable is transitive. That can really get in your way
>> in this case.
>
> If the data is stored in ROM, it should be typed as immutable. If it's
> not, then it could be modified. The only other option is heap allocation
> and construction on use, which I've already said is undesirable.
>
> If we start from "all array literals and enums that contain references
> store the referenced data in ROM," then we will find ways to deal with
> any inconveniences. It's all a matter of what's more important in a
> system language, performance or convenience.

Both are more important. It is D.
Everything you need to do is make the enum static immutable instead.
D is also a scripting language and an application programming language.

auto a = [new Foo, new Bar, new Qux]; // I want that to work.

On 11/16/2011 08:26 PM, Timon Gehr wrote:
> auto a = [new Foo, new Bar, new Qux]; // I want that to work.
>

(It currently does, if that was unclear)

On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
>> On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>
>>> Note that this is an explicit allocation:
>>>
>>> int[] a = [1,2,3]; // just as explicit as a NewExpression
>>>
>>> Only the enums "hide" it sometimes, but actually you should know the
>>> involved types.
>>
>> As I've said, there are already ways to explicitly allocate memory. A
>> suggested replacement for this is:
>>
>> int[] a = array(1, 2, 3);
>>
>> And you could always do:
>>
>> int[] a = [1, 2, 3].dup;
>>
>> Nobody complains about having to do:
>>
>> char[] a = "hello".dup;
>>
>> I don't see why we couldn't do the same for all array literals.
>>
>
> Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.

string is a reference type.  We hear no complaints about strings being stored in ROM and the type of literals being immutable.

Make no mistake, this doesn't cover every current instance array literals, such as ones which contain necessarily-heap-allocated entities.  Those would be covered by a library function (e.g. array(...)).  But those are not array literals anyways, they are constructors.

>>> Well, there is a function called writeln that can do that. That is a
>>> different function. But the one that gets actually called is not const
>>> correct as well.
>>>
>>>
>>> This is writeln:
>>>
>>> // Most general instance
>>> void writeln(T...)(T args)
>>> if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
>>> const(char)[]))
>>> {
>>> stdout.write(args, '\n');
>>> }
>>>
>>>
>>> =>
>>> writeln([1,2,3]);
>>> // modulo IFTY:
>>> writeln!(int[])([1,2,3]);
>>> // const correct?
>>> writeln!(int[])([1,2,3].idup); // nope!
>>>
>>> Error: cannot implicitly convert expression (_adDupT(&
>>> _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]
>>
>> I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[],
>> it will compile.
>>
>> Simple test:
>>
>> immutable(int)[] a = [1, 2, 3];
>> writeln(a);
>>
>> works with 2.056.
>
> That is like saying
>
> void foo(int[] a){...}             // prints a
> void bar(immutable(int)[] a){...}  // prints a
>
> int[] a = [1,2,3];
> immutable(int)[] b = [1, 2, 3];
> foo(a);
> bar(b);
>
> "=> Oh, bar is const correct because I can call foo with mutable data and bar with immutable data."
>
> foo is writeln!(int[])
> bar is writeln!(immutable(int)[])
>
> in effect, if you do:
>
> writeln("hello".dup);
>
> The compiler cannot assume that the IFTI'd writeln!(char[]) will not change the argument, ergo it cannot optimize away the .dup, even though it would be a valid transformation as long as the programmer does not make the program semantics depend on the identity relation on immutable references (doing so defeats the purpose of immutability).

writeln is not contractually const correct, but that's only because most of D is not const correct either.  For example, Object.toString is not const, so if you const-ified all writeln args, you couldn't print objects.  But writeln itself does not change any data (a rogue toString might change data, but that is not common practice).

Once D becomes const correct, writeln *can* apply const to all it's parameters.

>> If the data is stored in ROM, it should be typed as immutable. If it's
>> not, then it could be modified. The only other option is heap allocation
>> and construction on use, which I've already said is undesirable.
>>
>> If we start from "all array literals and enums that contain references
>> store the referenced data in ROM," then we will find ways to deal with
>> any inconveniences. It's all a matter of what's more important in a
>> system language, performance or convenience.
>
> Both are more important. It is D.
> Everything you need to do is make the enum static immutable instead.
> D is also a scripting language and an application programming language.
>
> auto a = [new Foo, new Bar, new Qux]; // I want that to work.

auto a = array(new Foo, new Bar, new Qux);

The one case which is difficult to do is initializing a fixed-size array with a literal that uses runtime data.  I suppose we'd need a function that returns a fixed-sized array made of its arguments, and doing the init builds it in place.  i.e.:

Object[3] objs = array_fixed(new Foo, new Bar, new Qux);

would not do any moving of references, it would construct the fixed sized array in-place.  Initializing fixed sized arrays with array literals already needs attention anyway.

-Steve

On Wed, 16 Nov 2011 15:00:16 -0500, Steven Schveighoffer <schveiguy@yahoo.com> wrote:

> The one case which is difficult to do is initializing a fixed-size array with a literal that uses runtime data.  I suppose we'd need a function that returns a fixed-sized array made of its arguments, and doing the init builds it in place.  i.e.:
>
> Object[3] objs = array_fixed(new Foo, new Bar, new Qux);
>
> would not do any moving of references, it would construct the fixed sized array in-place.  Initializing fixed sized arrays with array literals already needs attention anyway.

one benefit here, we could use auto:

auto objs = array_fixed(new Foo, new Bar, new Qux);

-Steve

On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
> On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
>>> On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr@gmx.ch>
>>> wrote:
>>>
>>>> Note that this is an explicit allocation:
>>>>
>>>> int[] a = [1,2,3]; // just as explicit as a NewExpression
>>>>
>>>> Only the enums "hide" it sometimes, but actually you should know the
>>>> involved types.
>>>
>>> As I've said, there are already ways to explicitly allocate memory. A
>>> suggested replacement for this is:
>>>
>>> int[] a = array(1, 2, 3);
>>>
>>> And you could always do:
>>>
>>> int[] a = [1, 2, 3].dup;
>>>
>>> Nobody complains about having to do:
>>>
>>> char[] a = "hello".dup;
>>>
>>> I don't see why we couldn't do the same for all array literals.
>>>
>>
>> Because 'immutable' behaves nicely on built-in value types, but not on
>> arbitrary reference types.
>
> string is a reference type. We hear no complaints about strings being
> stored in ROM and the type of literals being immutable.
>

string is not an immutable type. It is immutable(char)[] and char is a built-in value type.

static assert(isMutable!string);

> Make no mistake, this doesn't cover every current instance array
> literals, such as ones which contain necessarily-heap-allocated
> entities. Those would be covered by a library function (e.g.
> array(...)). But those are not array literals anyways, they are
> constructors.
>

It is true that they are constructors, but they are currently also called array literals.

>>>> Well, there is a function called writeln that can do that. That is a
>>>> different function. But the one that gets actually called is not const
>>>> correct as well.
>>>>
>>>>
>>>> This is writeln:
>>>>
>>>> // Most general instance
>>>> void writeln(T...)(T args)
>>>> if (T.length > 1 || T.length == 1 && !is(typeof(args[0]) :
>>>> const(char)[]))
>>>> {
>>>> stdout.write(args, '\n');
>>>> }
>>>>
>>>>
>>>> =>
>>>> writeln([1,2,3]);
>>>> // modulo IFTY:
>>>> writeln!(int[])([1,2,3]);
>>>> // const correct?
>>>> writeln!(int[])([1,2,3].idup); // nope!
>>>>
>>>> Error: cannot implicitly convert expression (_adDupT(&
>>>> _D11TypeInfo_Ai6__initZ,[1,2,3])) of type immutable(int)[] to int[]
>>>
>>> I'm not sure what this means. If [1, 2, 3] is typed as immtuable(int)[],
>>> it will compile.
>>>
>>> Simple test:
>>>
>>> immutable(int)[] a = [1, 2, 3];
>>> writeln(a);
>>>
>>> works with 2.056.
>>
>> That is like saying
>>
>> void foo(int[] a){...} // prints a
>> void bar(immutable(int)[] a){...} // prints a
>>
>> int[] a = [1,2,3];
>> immutable(int)[] b = [1, 2, 3];
>> foo(a);
>> bar(b);
>>
>> "=> Oh, bar is const correct because I can call foo with mutable data
>> and bar with immutable data."
>>
>> foo is writeln!(int[])
>> bar is writeln!(immutable(int)[])
>>
>> in effect, if you do:
>>
>> writeln("hello".dup);
>>
>> The compiler cannot assume that the IFTI'd writeln!(char[]) will not
>> change the argument, ergo it cannot optimize away the .dup, even
>> though it would be a valid transformation as long as the programmer
>> does not make the program semantics depend on the identity relation on
>> immutable references (doing so defeats the purpose of immutability).
>
> writeln is not contractually const correct, but that's only because most
> of D is not const correct either. For example, Object.toString is not
> const, so if you const-ified all writeln args, you couldn't print
> objects. But writeln itself does not change any data (a rogue toString
> might change data, but that is not common practice).
>
> Once D becomes const correct, writeln *can* apply const to all it's
> parameters.

Ok, I see. writeln still could be const-correct in principle. It would need to apply const to the parameters that have a const toString method. But the language is not powerful enough to do that. There is no way to intercept IFTI... I am commonly missing that feature.

>
>>> If the data is stored in ROM, it should be typed as immutable. If it's
>>> not, then it could be modified. The only other option is heap allocation
>>> and construction on use, which I've already said is undesirable.
>>>
>>> If we start from "all array literals and enums that contain references
>>> store the referenced data in ROM," then we will find ways to deal with
>>> any inconveniences. It's all a matter of what's more important in a
>>> system language, performance or convenience.
>>
>> Both are more important. It is D.
>> Everything you need to do is make the enum static immutable instead.
>> D is also a scripting language and an application programming language.
>>
>> auto a = [new Foo, new Bar, new Qux]; // I want that to work.
>
> auto a = array(new Foo, new Bar, new Qux);
>

Requiring that works modulo template bloat and breakage of existing code. Also, it does not really buy us anything imho.

> The one case which is difficult to do is initializing a fixed-size array
> with a literal that uses runtime data. I suppose we'd need a function
> that returns a fixed-sized array made of its arguments, and doing the
> init builds it in place. i.e.:
>
> Object[3] objs = array_fixed(new Foo, new Bar, new Qux);
>
> would not do any moving of references, it would construct the fixed
> sized array in-place. Initializing fixed sized arrays with array
> literals already needs attention anyway.
>

From the compiler side. Not necessarily from the std lib side.

On Wed, 16 Nov 2011 21:00:16 +0100, Steven Schveighoffer <schveiguy@yahoo.com> wrote:

> On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
>>> On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>>
>>>> Note that this is an explicit allocation:
>>>>
>>>> int[] a = [1,2,3]; // just as explicit as a NewExpression
>>>>
>>>> Only the enums "hide" it sometimes, but actually you should know the
>>>> involved types.
>>>
>>> As I've said, there are already ways to explicitly allocate memory. A
>>> suggested replacement for this is:
>>>
>>> int[] a = array(1, 2, 3);
>>>
>>> And you could always do:
>>>
>>> int[] a = [1, 2, 3].dup;
>>>
>>> Nobody complains about having to do:
>>>
>>> char[] a = "hello".dup;
>>>
>>> I don't see why we couldn't do the same for all array literals.
>>>
>>
>> Because 'immutable' behaves nicely on built-in value types, but not on arbitrary reference types.
>
> string is a reference type.  We hear no complaints about strings being stored in ROM and the type of literals being immutable.

immutable(int[][]) a = [[1]];
int[][] b = a.dup; // Fails.

The problem is with more than a single layer of indirection.

On Wed, 16 Nov 2011 16:16:48 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 11/16/2011 09:00 PM, Steven Schveighoffer wrote:
>> On Wed, 16 Nov 2011 14:26:57 -0500, Timon Gehr <timon.gehr@gmx.ch> wrote:
>>
>>> On 11/16/2011 02:22 PM, Steven Schveighoffer wrote:
>>>> On Tue, 15 Nov 2011 13:45:02 -0500, Timon Gehr <timon.gehr@gmx.ch>
>>>> wrote:
>>>>
>>>>> Note that this is an explicit allocation:
>>>>>
>>>>> int[] a = [1,2,3]; // just as explicit as a NewExpression
>>>>>
>>>>> Only the enums "hide" it sometimes, but actually you should know the
>>>>> involved types.
>>>>
>>>> As I've said, there are already ways to explicitly allocate memory. A
>>>> suggested replacement for this is:
>>>>
>>>> int[] a = array(1, 2, 3);
>>>>
>>>> And you could always do:
>>>>
>>>> int[] a = [1, 2, 3].dup;
>>>>
>>>> Nobody complains about having to do:
>>>>
>>>> char[] a = "hello".dup;
>>>>
>>>> I don't see why we couldn't do the same for all array literals.
>>>>
>>>
>>> Because 'immutable' behaves nicely on built-in value types, but not on
>>> arbitrary reference types.
>>
>> string is a reference type. We hear no complaints about strings being
>> stored in ROM and the type of literals being immutable.
>>
>
> string is not an immutable type. It is immutable(char)[] and char is a built-in value type.
>
> static assert(isMutable!string);

It fits my definition of a valid enum reference type (immutable or implicitly castable to immutable).  The point is that the data referenced is stored in ROM and therefore a) immutable and b) fully defined at compile-time.

>> Make no mistake, this doesn't cover every current instance array
>> literals, such as ones which contain necessarily-heap-allocated
>> entities. Those would be covered by a library function (e.g.
>> array(...)). But those are not array literals anyways, they are
>> constructors.
>>
>
> It is true that they are constructors, but they are currently also called array literals.

I'm looking to redefine what a literal means in D.

>>
>> writeln is not contractually const correct, but that's only because most
>> of D is not const correct either. For example, Object.toString is not
>> const, so if you const-ified all writeln args, you couldn't print
>> objects. But writeln itself does not change any data (a rogue toString
>> might change data, but that is not common practice).
>>
>> Once D becomes const correct, writeln *can* apply const to all it's
>> parameters.
>
> Ok, I see. writeln still could be const-correct in principle. It would need to apply const to the parameters that have a const toString method. But the language is not powerful enough to do that. There is no way to intercept IFTI... I am commonly missing that feature.

I have an enhancement request in for intercepting IFTI (not sure if it applies here): http://d.puremagic.com/issues/show_bug.cgi?id=4998

>>
>>>> If the data is stored in ROM, it should be typed as immutable. If it's
>>>> not, then it could be modified. The only other option is heap allocation
>>>> and construction on use, which I've already said is undesirable.
>>>>
>>>> If we start from "all array literals and enums that contain references
>>>> store the referenced data in ROM," then we will find ways to deal with
>>>> any inconveniences. It's all a matter of what's more important in a
>>>> system language, performance or convenience.
>>>
>>> Both are more important. It is D.
>>> Everything you need to do is make the enum static immutable instead.
>>> D is also a scripting language and an application programming language.
>>>
>>> auto a = [new Foo, new Bar, new Qux]; // I want that to work.
>>
>> auto a = array(new Foo, new Bar, new Qux);
>>
>
> Requiring that works modulo template bloat and breakage of existing code. Also, it does not really buy us anything imho.

I think the bloat is a wash.  Every time I use an array literal, there is a complete instantiation of what would be in an array template inline in the calling function.

Yes, it breaks code.  It's worth it.  To avoid a heap allocation for [1, 2, 3] is worth breaking code that uses runtime data in an array literal.  The compiler can be helpful in the error message:

Error somefile.d(345): array literals cannot contain runtime data.  Maybe you meant:
 array(new Foo, new Bar, new Qux);

I want to point out that currently there is *NO* way to make an immutable array literal that lives in ROM.  This is unacceptable.

-Steve