December 20, 2006
== Quote from Andrei Alexandrescu (See Website For Email)
...
> That remains to be seen, but I think the buck stops at functions. The problem of duplicating templates for inout (lvalues) and rvalues stays, but I have an idea about that, that I might tell about tomorrow.
>
> Andrei

Did you ever work out how to do this lvalue/rvalue idea?

Kevin
December 20, 2006
Kevin Bealer wrote:
> == Quote from Andrei Alexandrescu (See Website For Email)
> ...
>> That remains to be seen, but I think the buck stops at functions. The
>> problem of duplicating templates for inout (lvalues) and rvalues
>> stays, but I have an idea about that, that I might tell about
>> tomorrow.
>>
>> Andrei
> 
> Did you ever work out how to do this lvalue/rvalue idea?

I had to leave to Romania before having the time to post thoughts on the lvalue/rvalue discussion I've had with Walter on Saturday. We both agree that it's a serious problem with D's type system that needs fixing (and that he needs to do all the work :o)). I've continued to think of the issue, at least enough to figure that the solution we initially thought of is not sound.

Walter is reluctant to offering the ability to overload on lvalue/rvalue, while it turns out that that can't be avoided. Let's return to my litmus test - the identity function ident(e), which can snuggle any expression e and leave its semantics unchanged. My thesis is that this function is an important test of a language's power. The starting point would be:

template ident(T) {
  T ident(T e) { return e; }
  inout T ident(inout T e) { return e; }
}

The problem with this approach is that it doesn't scale. Right now "inout" is about the only interesting storage class of a function parameter, but "lazy" comes to mind (which I hope to get rid of soon via a much better solution) and later on we'll have "const", and each of these combinations will mean one more duplication of the ident body (and, by extension, of any function that wants to just pass the storage class outside). Walter had an idea along the line:

template ident(T) {
  return T ident(return T e) { return e; }
}

which allows you to reuse the return keyword as a symbolic placeholder for passing out the storage class. This solution is severely shortsighted in that it fixes ident and only ident, whereas the purpose of ident is to serve as a simplified case for functions with multiple parameters. So this fell as well.

We then discussed another solution that I won't bore you with, as it's so wrong it hurts. My current thoughts navigate around two possible solutions. One is to make the storage part of the template parameters:

template ident(S T) {
  S T ident(S T e) { return e; }
}

When two adjacent symbols appear in a template parameter list, they unambiguously denote a storage class followed by a type. So "S" can bind to things like "in", "inout" etc., while "T" can bind to types. In the example above, the compiler will deduce both S and T from the argument type. It already does that, so that's no extra difficulty. The key point that makes this scale is that you can bind S and T multiple times in a variadic template. Another interesting detail is that it clarifies that you can't solve the problem without somehow compiling two versions of the ident function. So in the end overloading on "inout" is a must.

Another solution that works is to commit to the idea of associating the storage class with the parameter (and divorce it from the type entirely). In that case, the following syntax describes what happens:

template ident(T) {
  storageof(e) T ident(storageof(e) T e) { return e; }
}

The storageof(symbol) meta-operator yields the storage of that symbol. The problem with this notation is that it uses a symbol without having seen it. That's not too bad (it already happens due to the way symbols at global scope are looked up) but in this case it does have a fishy smell. Another thing that I don't like it that the code obscures what's going on - namely that one ident will be generated for each storage class, even though that's not reflected in the parameter type list.

Finally, one related but slightly different topic is the necessity of deduced return types for functions, e.g. by using "auto" to denote the return type. Automatic deduction of return types is very useful in that it allows compact template function definition - no more need for a template that defines a homonym function. With deduced argument types, ident can be written as:

auto ident(S T)(S T e) {
  return e;
}

which is, I think, the Platonic ideal of ident as far as expressing it in D goes.


Andrei
December 20, 2006
Andrei Alexandrescu (See Website for Email) wrote:
> Kevin Bealer wrote:
>> == Quote from Andrei Alexandrescu (See Website For Email)
>> ...
>>> That remains to be seen, but I think the buck stops at functions. The
>>> problem of duplicating templates for inout (lvalues) and rvalues
>>> stays, but I have an idea about that, that I might tell about
>>> tomorrow.
>>>
>>> Andrei
>>
>> Did you ever work out how to do this lvalue/rvalue idea?

<snip>

I'd like "auto" to appear on more places as it will make templates more accessible than ever before. For example, this notation has been suggested a couple of times:

void func(auto x) { }


I feel confident that having you and Walter on the issue, we should have a nice solution in no-time :)

Sarbatori fericite!

L.

I'm actually leaving Romania to go to the Netherlands :)
December 20, 2006
Lionello Lunesu wrote:
> Andrei Alexandrescu (See Website for Email) wrote:
>> Kevin Bealer wrote:
>>> == Quote from Andrei Alexandrescu (See Website For Email)
>>> ...
>>>> That remains to be seen, but I think the buck stops at functions. The
>>>> problem of duplicating templates for inout (lvalues) and rvalues
>>>> stays, but I have an idea about that, that I might tell about
>>>> tomorrow.
>>>>
>>>> Andrei
>>>
>>> Did you ever work out how to do this lvalue/rvalue idea?
> 
> <snip>
> 
> I'd like "auto" to appear on more places as it will make templates more accessible than ever before. For example, this notation has been suggested a couple of times:
> 
> void func(auto x) { }

Thanks for the kind wishes. One problem I see with the shortcut above is that the notation does not clarify whether "auto" catches both the type and the storage class. Even if it does, more linguistic machinery is needed to extract the type and storage class of x and to properly transport them, if needed, to the return type.

So in the end the "auto" suggestion is a mere shortcut for:

void func(S T)(S T x) { }

with the difference that the storage and type entities are clearly bound to symbols, style that's more in the spirit of D.


Andrei
December 20, 2006
Andrei Alexandrescu (See Website for Email) wrote:
> Lionello Lunesu wrote:
>> Andrei Alexandrescu (See Website for Email) wrote:
>>> Kevin Bealer wrote:
>>>> == Quote from Andrei Alexandrescu (See Website For Email)
>>>> ...
>>>>> That remains to be seen, but I think the buck stops at functions. The
>>>>> problem of duplicating templates for inout (lvalues) and rvalues
>>>>> stays, but I have an idea about that, that I might tell about
>>>>> tomorrow.
>>>>>
>>>>> Andrei
>>>>
>>>> Did you ever work out how to do this lvalue/rvalue idea?
>>
>> <snip>
>>
>> I'd like "auto" to appear on more places as it will make templates more accessible than ever before. For example, this notation has been suggested a couple of times:
>>
>> void func(auto x) { }

> So in the end the "auto" suggestion is a mere shortcut for:
> 
> void func(S T)(S T x) { }
> 
> with the difference that the storage and type entities are clearly bound to symbols, style that's more in the spirit of D.

The motivation I see for such a shortcut is that this template pattern is used so often that it is worth simplifying. Furthermore, the variables S and T are often just dummy variables to satisfy the requirements of the template; you don't even need these symbols, and they fill the namespace.

Cheers,

Reiner
December 20, 2006
Reiner Pope wrote:
> Andrei Alexandrescu (See Website for Email) wrote:
>> Lionello Lunesu wrote:
>>> I'd like "auto" to appear on more places as it will make templates more accessible than ever before. For example, this notation has been suggested a couple of times:
>>>
>>> void func(auto x) { }
> 
>> So in the end the "auto" suggestion is a mere shortcut for:
>>
>> void func(S T)(S T x) { }
>>
>> with the difference that the storage and type entities are clearly bound to symbols, style that's more in the spirit of D.
> 
> The motivation I see for such a shortcut is that this template pattern is used so often that it is worth simplifying. Furthermore, the variables S and T are often just dummy variables to satisfy the requirements of the template; you don't even need these symbols, and they fill the namespace.

I'm definitely one for shortcuts, but let's not forget that at the moment we lack the feature that the shortcut is supposed to simplify. At this moment, "auto" helps solving the storage class parameterization issue as much as a great color for the seat covers helps designing an economic automobile.

Let me illustrate further why ident is important and what solution we should have for it. Consider C's response to ident:

#define IDENT(e) (e)

Now, you can use IDENT with many C expressions (except those that have top-level commas), so we should be glad. Or should we? A true solution is to intercept the type of the expression and intelligently pass it as the return type of the function. In the general case, that means the language offers total control to the programmer of the types tossed around by a program. Instead, the C solution cheats its way around in that it has absolutely no grip on the expression type; it just gives the illusion that a function call is going on.

Similarly, let's say that a group of revolutionaries convinces Walter (as I understand happened in case of using "length" and "$" inside slice expressions, which is a shame and an absolute disaster that must be undone at all costs) to implement "auto", therefore leading to the following implementation of ident:

auto ident(auto x) {
  return x;
}

In the euphoria of the celebration, it will be initially forgotten that we have the shortcut without the feature, for the feature means having access to the exact type and storage of x, not finding a way to comfortably specify a deduction process.


Andrei
December 20, 2006
Andrei Alexandrescu (See Website For Email) wrote:
> Similarly, let's say that a group of revolutionaries convinces Walter (as I understand happened in case of using "length" and "$" inside slice expressions, which is a shame and an absolute disaster that must be undone at all costs) to implement "auto"

This off-hand remark worries me. I presume that you mean being able to reference the length of a string, from inside the slice? (rather than simply the notation).
And the problem being that it requires a sliceable entity to know its length? Or is the problem more serious than that?
It's worrying because any change would break an enormous amount of code.

These issues you're raising seem to be far too fundamental to be fixed in the next few days, casting grave doubts on whether a D1.0 release on Jan 1 is a good idea.
December 20, 2006
Andrei Alexandrescu (See Website for Email) schrieb am 2006-12-20:
> Kevin Bealer wrote:
>> == Quote from Andrei Alexandrescu (See Website For Email)
>> ...
>>> That remains to be seen, but I think the buck stops at functions. The problem of duplicating templates for inout (lvalues) and rvalues stays, but I have an idea about that, that I might tell about tomorrow.

<snip>

> We then discussed another solution that I won't bore you with, as it's so wrong it hurts. My current thoughts navigate around two possible solutions. One is to make the storage part of the template parameters:
>
> template ident(S T) {
>    S T ident(S T e) { return e; }
> }
>
> When two adjacent symbols appear in a template parameter list, they unambiguously denote a storage class followed by a type. So "S" can bind to things like "in", "inout" etc., while "T" can bind to types.

Unambiguously?

template Templ_1(int i) {
}

Is "int" now a type or a storage class?

> Another solution that works is to commit to the idea of associating the storage class with the parameter (and divorce it from the type entirely). In that case, the following syntax describes what happens:
>
> template ident(T) {
>    storageof(e) T ident(storageof(e) T e) { return e; }
> }
>
> The storageof(symbol) meta-operator yields the storage of that symbol.

This seems to be much cleaner. How would storageof handle the following:

mixin ident!(extern(C) inout float function(int)) foo;

Thomas


December 20, 2006
Don Clugston wrote:
> Andrei Alexandrescu (See Website For Email) wrote:
>> Similarly, let's say that a group of revolutionaries convinces Walter (as I understand happened in case of using "length" and "$" inside slice expressions, which is a shame and an absolute disaster that must be undone at all costs) to implement "auto"
> 
> This off-hand remark worries me. I presume that you mean being able to reference the length of a string, from inside the slice? (rather than simply the notation).
> And the problem being that it requires a sliceable entity to know its length? Or is the problem more serious than that?
> It's worrying because any change would break an enormous amount of code.

It would indeed break an enormous amount of code, but "all costs" includes "enormous costs". :o) A reasonable migration path is to deprecate them soon and make them illegal over the course of one year.

A small book could be written on just how bad language design is using "length" and "$" to capture slice size inside a slice expression. I managed to write two lengthy emails to Walter about them, and just barely got started. Long story short, "length" introduces a keyword through the back door, effectively making any use of "length" anywhere unrecommended and highly fragile. Using "$" is a waste of symbolic real estate to serve a narrow purpose; the semantics isn't naturally generalized to its logical conclusion; and the choice of symbol itself as a reminiscent of Perl's regexp is at best dubious ("#" would have been vastly better as it has count connotation in natural language, and making it into an operator would have fixed the generalization issue). As things stand now, the rules governing the popping up of "length" and "$" constitute a sudden boo-boo on an otherwise carefully designed expression landscape.

> These issues you're raising seem to be far too fundamental to be fixed in the next few days, casting grave doubts on whether a D1.0 release on Jan 1 is a good idea.

The lvalue/rvalue issue is fundamental. I'm not in the position to assess whether it's a maker or breaker of D 1.0.

The "length"/"$" issue is not fundamental the same way that C's declaration syntax, Java's throw specifications, C++'s use of "<" and ">" for templates, and Mao Zedong's refusal to use a toothbrush are not fundamental. It will "just" go down in history as a huge embarrassment and a good resource for cheap shooters and naysayers. If I understand its genesis, it will also be a canonical example of why design by committee is bad.


Andrei
December 20, 2006
Thomas Kuehne wrote:
>> We then discussed another solution that I won't bore you with, as it's so wrong it hurts. My current thoughts navigate around two possible solutions. One is to make the storage part of the template parameters:
>>
>> template ident(S T) {
>>    S T ident(S T e) { return e; }
>> }
>>
>> When two adjacent symbols appear in a template parameter list, they unambiguously denote a storage class followed by a type. So "S" can bind to things like "in", "inout" etc., while "T" can bind to types.
> 
> Unambiguously?
>  template Templ_1(int i) {
> }
> 
> Is "int" now a type or a storage class?

It's a type because the symbol "int" is already bound as a keyword. There's no way (to the best of my knowledge) to specify two adjacent non-keyword symbols in a template parameter list.

>> Another solution that works is to commit to the idea of associating the storage class with the parameter (and divorce it from the type entirely). In that case, the following syntax describes what happens:
>>
>> template ident(T) {
>>    storageof(e) T ident(storageof(e) T e) { return e; }
>> }
>>
>> The storageof(symbol) meta-operator yields the storage of that symbol.
> 
> This seems to be much cleaner.

The more I think of it, the more I think it sucks :o).

> How would storageof handle the following:
> 
> mixin ident!(extern(C) inout float function(int)) foo;

C does not have lvalue return types.


Andrei