Hi there,

I've just recently come upon the D language and was captured right away by it. Of all those programming languages I've found over the years, this definitely is one of the most promising!

Anyway, the template mechanism has a number of deficiencies that makes it drop far behind that of C++. To those, who have never worked with stuff like the "expression template" method, this will look like minor details, but after implementing my own matrix library in C++ using that method, I definitely have come to realize the real power that lies behind the template concept in C++.

Two examples of what is possible in C++, which I could not find in D:

-------------------------
* Templated functions - allowing stuff like:

        template <typename T>
        T special_addition(T a,T b) { ... }

i.e. a function that takes two arguments of an arbitrary -- but identical -- type. Of course, the concept goes far beyond that and is absolutely indispensible in many cases.

-------------------------
* Integers as template arguments:

        template <int N>
        class Vector {
                double data[N];
        };

making Vector<2> a different type than Vector<3>, allowing stuff like direct vector or matrix manipulations via operators with complete type checking. (Going far beyond the simple possibilities of direct array manipulations in D.)

-------------------------
The one thing that I dislike most about C++ templates, and which has been
carried over to D as well, is, that template parameters are not typed. I.e.
if I have:
        template <typename T>
        class Test {
                static T add(T a,T b) { return a + b; } /* X */
        };

        bool foo() {
                return Test<bool>::add(true,false); /* Y */
        };
the compiler actually complains about line /*X*/ which is absolutely
correct, instead of telling the user that Test simply should not be
instatiated with bool. Now if Test is somewhere deep within a library, the
application programmer may get a huge bunch of error messages looking like
a bug somewhere in the library, even though, the actual error was simply
the wrong instatiation of a template.

Solution to this would be something like interfaces in D, specifying a set of capabilities of a type (be it a class or a predefined type), and the possibility to restrict parameters in a template definition to such an interface.
-------------------------

Now, this was just a rough sketch of what I found to be missing in D making it seriously fall back behind C++. It is nothing about philosophy or far-fetched theory, but just plain experience after many hours of digging into the depths of C++ templates.

Really bringing anything of this power into D would probably mean a redesign of large parts of the template and interface mechanism. It would also mean quite a bit of research and genious in language design. The mechanisms of expression templates are still quite new and much work is yet to be done to really understand the basic principles behind it.

Anyhow: unless this power is somehow carried over from C++, the language D can not seriously aim at measuring up with its ancestor.

Anyone interested in that whole topic, best start reading at
        http://www.oonumerics.org
perhaps starting out on blitz++ for the moment.
(I've done my own, smaller and simpler library, but it's hardly documented
at all, so it is not yet a good starting point...)

Ciao,
Nobbi

I guess, my remark directly collides with the message "C and/or C++ in D" by Ben Woodhead from yesterday, therefore an additional remark:

Everything I mentioned, is, of course somehow possible to be done in D. The one crucial point simply is PERFORMANCE. What makes templates in C++ such a powerful tool in C++ is not, that they allow stuff impossible before. They do not even make life any more comfortable (quite in the contrary, they make life awfully complicated!!) Anyhow:

        They allow much of the work to be done at compile time!

I you don't want to worry about performance, simply ignore this whole topic.

Anyhow, in numerical computation, where performance often is one of the most crucial goals, expression templates seem to be the only way so far to allow the full expressiveness of object orientated languages without a huge performance penalty. Thousands of programmers in the numerical field still use Fortran 77 simply because of its unbeaten performance and it was not until the discovery of expression templates, that C++ could close up with that speed.

Anyhow: the suggestion of allowing to restrict template parameter types is probably independant of the other extensions, and I would seriously advise the language designers to consider this unless they want to make using template libraries a pain to anyone.

Ciao,
Nobbi

On Wed, 15 Jan 2003 11:34:14 +0100
Norbert Nemec <nobbi_at_theorie3.physik.uni-erlangen.de@NOSPAM.COM>
wrote:

> * Templated functions - allowing stuff like:
> 
>         template <typename T>
>         T special_addition(T a,T b) { ... }
> 
> i.e. a function that takes two arguments of an arbitrary -- but identical -- type. Of course, the concept goes far beyond that and is absolutely indispensible in many cases.

I thought this was possible:

template special (T) {
    T addition(T a, T b) { ... }
}

From what I under stand, there isn't anything you can't templatize, although your int example is interesting. I didn't even know C++ templates could do that.

-- 
Theodore Reed (rizen/bancus)       -==-       http://www.surreality.us/ ~OpenPGP Signed/Encrypted Mail Preferred; Finger me for my public key!~

"Those who hammer their guns into plowshares will plow for those who do not." -- Thomas Jefferson

Comments embedded...

"Norbert Nemec" <nobbi_at_theorie3.physik.uni-erlangen.de@NOSPAM.COM> wrote in message news:b03d7n$2m9r$1@digitaldaemon.com...
> Hi there,
>
> I've just recently come upon the D language and was captured right away by it. Of all those programming languages I've found over the years, this definitely is one of the most promising!
>
> Anyway, the template mechanism has a number of deficiencies that makes it drop far behind that of C++. To those, who have never worked with stuff like the "expression template" method, this will look like minor details, but after implementing my own matrix library in C++ using that method, I definitely have come to realize the real power that lies behind the template concept in C++.
>
> Two examples of what is possible in C++, which I could not find in D:
>
> -------------------------
> * Templated functions - allowing stuff like:
>
>         template <typename T>
>         T special_addition(T a,T b) { ... }
>
> i.e. a function that takes two arguments of an arbitrary -- but
identical --
> type. Of course, the concept goes far beyond that and is absolutely indispensible in many cases.

This does work in D, as Theodore's example shows.


> -------------------------
> * Integers as template arguments:
>
>         template <int N>
>         class Vector {
>                 double data[N];
>         };
>
> making Vector<2> a different type than Vector<3>, allowing stuff like
direct
> vector or matrix manipulations via operators with complete type checking. (Going far beyond the simple possibilities of direct array manipulations
in
> D.)

This is probably going to have to be added to D. Nobody seems to like the alternative mechanism I proposed <g>.

> -------------------------
> The one thing that I dislike most about C++ templates, and which has been carried over to D as well, is, that template parameters are not typed.
I.e.
> if I have:
>         template <typename T>
>         class Test {
>                 static T add(T a,T b) { return a + b; } /* X */
>         };
>
>         bool foo() {
>                 return Test<bool>::add(true,false); /* Y */
>         };
> the compiler actually complains about line /*X*/ which is absolutely
> correct, instead of telling the user that Test simply should not be
> instatiated with bool. Now if Test is somewhere deep within a library, the
> application programmer may get a huge bunch of error messages looking like
> a bug somewhere in the library, even though, the actual error was simply
> the wrong instatiation of a template.
>
> Solution to this would be something like interfaces in D, specifying a set of capabilities of a type (be it a class or a predefined type), and the possibility to restrict parameters in a template definition to such an interface.
> -------------------------

A good point. Error messages are always a pain with templates.

>
> Now, this was just a rough sketch of what I found to be missing in D
making
> it seriously fall back behind C++. It is nothing about philosophy or far-fetched theory, but just plain experience after many hours of digging into the depths of C++ templates.
>
> Really bringing anything of this power into D would probably mean a
redesign
> of large parts of the template and interface mechanism. It would also mean quite a bit of research and genious in language design. The mechanisms of expression templates are still quite new and much work is yet to be done
to
> really understand the basic principles behind it.
>
> Anyhow: unless this power is somehow carried over from C++, the language D can not seriously aim at measuring up with its ancestor.
>
> Anyone interested in that whole topic, best start reading at
>         http://www.oonumerics.org
> perhaps starting out on blitz++ for the moment.
> (I've done my own, smaller and simpler library, but it's hardly documented
> at all, so it is not yet a good starting point...)
>
> Ciao,
> Nobbi
>

> 
> This is probably going to have to be added to D. Nobody seems to like the
> alternative mechanism I proposed <g>.
> 
I'm sorry, i've searched through a bit but was unable to find it.

Hi,

Answers embedded.

In article <b03d7n$2m9r$1@digitaldaemon.com>, Norbert Nemec says...
>
>Hi there,
>
>I've just recently come upon the D language and was captured right away by it. Of all those programming languages I've found over the years, this definitely is one of the most promising!
>
>Anyway, the template mechanism has a number of deficiencies that makes it drop far behind that of C++. To those, who have never worked with stuff like the "expression template" method, this will look like minor details, but after implementing my own matrix library in C++ using that method, I definitely have come to realize the real power that lies behind the template concept in C++.
>
>Two examples of what is possible in C++, which I could not find in D:
>
>-------------------------
>* Templated functions - allowing stuff like:
>
>        template <typename T>
>        T special_addition(T a,T b) { ... }
>
>i.e. a function that takes two arguments of an arbitrary -- but identical -- type. Of course, the concept goes far beyond that and is absolutely indispensible in many cases.


As others pointed out this is possible in D:

template TAddition(T) {
T special_addition(T a,T b) { ... }
}

But we need to name the template.So usually D templates pack a set of related functions.


>
>-------------------------
>* Integers as template arguments:
>
>        template <int N>
>        class Vector {
>                double data[N];
>        };
>
>making Vector<2> a different type than Vector<3>, allowing stuff like direct vector or matrix manipulations via operators with complete type checking. (Going far beyond the simple possibilities of direct array manipulations in D.)
>


As Walter pointed out this will get in D. That is a Good Thing :-)


>-------------------------
>The one thing that I dislike most about C++ templates, and which has been carried over to D as well, is, that template parameters are not typed. I.e. if I have:
>        template <typename T>
>        class Test {
>                static T add(T a,T b) { return a + b; } /* X */
>        };
>
>        bool foo() {
>                return Test<bool>::add(true,false); /* Y */
>        };
>the compiler actually complains about line /*X*/ which is absolutely correct, instead of telling the user that Test simply should not be instatiated with bool. Now if Test is somewhere deep within a library, the application programmer may get a huge bunch of error messages looking like a bug somewhere in the library, even though, the actual error was simply the wrong instatiation of a template.
>
>Solution to this would be something like interfaces in D, specifying a set of capabilities of a type (be it a class or a predefined type), and the possibility to restrict parameters in a template definition to such an interface.
>-------------------------


If you restrict yourself to classes, you can restrict the type parameters:

template TSort(T : Comparable) {
}

where Comparable is a type or class. AFAIK it's valid D code, if not is should
;-) . There's some info at http://www.digitalmars.com/d/template.html under
Argument Deduction section.
I don't think there's any reason for the compiler to always complain about line
/*X*/, instead of saying: Template instantiation at line /*Y*/ is invalid
because of /*X*/ requirement in template body. Systems with type inference
algorithms usually provide some intelligent error messages. There's also some
papers about giving intelligent error messages in type inference algorithms.


>
>Now, this was just a rough sketch of what I found to be missing in D making it seriously fall back behind C++. It is nothing about philosophy or far-fetched theory, but just plain experience after many hours of digging into the depths of C++ templates.
>
>Really bringing anything of this power into D would probably mean a redesign of large parts of the template and interface mechanism. It would also mean quite a bit of research and genious in language design. The mechanisms of expression templates are still quite new and much work is yet to be done to really understand the basic principles behind it.
>
>Anyhow: unless this power is somehow carried over from C++, the language D can not seriously aim at measuring up with its ancestor.


IIRC D's ancestor is C. C++ is just an older cousin ;-)


>
>Anyone interested in that whole topic, best start reading at
>        http://www.oonumerics.org
>perhaps starting out on blitz++ for the moment.
>(I've done my own, smaller and simpler library, but it's hardly documented
>at all, so it is not yet a good starting point...)
>
>Ciao,
>Nobbi
>


C++ template mechanism is very powerful, but almost impossible to get right. Walter writes C++ compiler writes since the beginning of C++, so he knows about the pitfalls of template mechanism. All these issues about template are causes of several threads in this newsgroup (check the archives where lots of people, including me, argue for a better template mechanism). Blitz++ does a very good usageof templates, I like particularly the tensor notation. But it carries a 750 page language standard behind it, defining the mechanism it uses. Walter wants something better, we just have to keep throwing him good ideas and critics.

Best regards,
Daniel Yokomiso.

In article <b03ehf$2muu$1@digitaldaemon.com>, Norbert Nemec says...
>
>I guess, my remark directly collides with the message "C and/or C++ in D" by Ben Woodhead from yesterday, therefore an additional remark:
>
>Everything I mentioned, is, of course somehow possible to be done in D. The one crucial point simply is PERFORMANCE. What makes templates in C++ such a powerful tool in C++ is not, that they allow stuff impossible before. They do not even make life any more comfortable (quite in the contrary, they make life awfully complicated!!) Anyhow:
>
>        They allow much of the work to be done at compile time!
>
>I you don't want to worry about performance, simply ignore this whole topic.
>
>Anyhow, in numerical computation, where performance often is one of the most crucial goals, expression templates seem to be the only way so far to allow the full expressiveness of object orientated languages without a huge performance penalty. Thousands of programmers in the numerical field still use Fortran 77 simply because of its unbeaten performance and it was not until the discovery of expression templates, that C++ could close up with that speed.
>
>Anyhow: the suggestion of allowing to restrict template parameter types is probably independant of the other extensions, and I would seriously advise the language designers to consider this unless they want to make using template libraries a pain to anyone.
>
>Ciao,
>Nobbi

Hi,

Expression templates aren't the only way of achieving performance in "correct" OO systems. In a good type system we can do more agressive inlining of operations, reduce the need for dynamic dispatch, do strictness analysis to ignore partial unnecessary results, do loop unrolling, etc.. All of these are good in every piece of code, not just in some particular mechanism. The problem with expression templates is that they require someone (the library writer) to know how to write code with optimization hints. In C++ template syntax is almost a language of its own. In Common Lisp macro expansion (which can do anything expression templates do and more) gives you similar performance, but the syntax is much more similar to Common Lisp. Scheme also provides a expressive and simple to use macro system.

Best regards,
Daniel Yokomiso.

In article <b04fq8$8t8$1@digitaldaemon.com>, Walter says...
>

[snip]

>> -------------------------
>> * Integers as template arguments:
>>
>>         template <int N>
>>         class Vector {
>>                 double data[N];
>>         };
>>
>> making Vector<2> a different type than Vector<3>, allowing stuff like
>direct
>> vector or matrix manipulations via operators with complete type checking. (Going far beyond the simple possibilities of direct array manipulations
>in
>> D.)
>
>This is probably going to have to be added to D. Nobody seems to like the alternative mechanism I proposed <g>.
>

[snip]


Hi,

I'm glad you're considering adding this to the language. Which kind of value
parameters will be possible: just integers or any kind of value?
Also there's another problem with current template syntax. We can't define
templates which define templatized methods. E.g. a List(T) class with a method
List(U) map(U (*operation)(T)). This is a very good thing to have. C# generics
proposal allows definition of generic classes with generic methods, and you have
to instantiate the generic method when you use it, not when you instantiate the
generic class. This, together with integer template parameters will allow D to
have type-safe unit classes for scientific programming:

// Using CGS unit system
public template class Unit(int C, int G, int S) {
private double _value;
public this(double value) {
this._value = value;
}
public double value() {
return this._value;
}
public Unit(C, G, S) mul(int other) {
return new Unit(C, G, S)(value() * other);
}
public template Unit(C - X, G - Y, S - Z) div(Unit(X, Y, Z) other) {
return new Unit(C - X, G - Y, S - Z)(value() / other.value());
}
}

alias Unit(1, 0, 0) Length;
alias Unit(0, 1, 0) Mass;
alias Unit(0, 0, 1) Time;
alias Unit(1, 0, -1) Speed;
alias Unit(2, 1, 2) Energy;

const Length metre = new Length(100);
const Length second = new Time(1);
const Length kilogram = new Mass(1000);
const Speed c = 299792458 * metre / second;

Mass m = 100 * kilogram;
Energy e;

e = m * c * c;

Nice isn't it? Some issues about invalid instantiation can happen if the code uses variables to define dimensions, but I think limiting dimensions to be constant isn't a big problem. I don't think the compiler complexity is a burden when we compare it to the benefits it brings.

Best regards,
Daniel Yokomiso.

Theodore Reed wrote:

> On Wed, 15 Jan 2003 11:34:14 +0100
> Norbert Nemec <nobbi_at_theorie3.physik.uni-erlangen.de@NOSPAM.COM>
> wrote:
> 
>> * Templated functions - allowing stuff like:
>> 
>>         template <typename T>
>>         T special_addition(T a,T b) { ... }
>> 
>> i.e. a function that takes two arguments of an arbitrary -- but identical -- type. Of course, the concept goes far beyond that and is absolutely indispensible in many cases.
> 
> I thought this was possible:
> 
> template special (T) {
>     T addition(T a, T b) { ... }
> }
> 
> From what I under stand, there isn't anything you can't templatize, although your int example is interesting. I didn't even know C++ templates could do that.

OK, but as I understand it, this needs to be called like:
        x = special(int).addition(3,5);
whereas in C++, a call to addition(3,5) does implicit instatiation.

Don't know, though, whether that would be a fundamental problem.

Daniel Yokomiso wrote:

> In article <b03d7n$2m9r$1@digitaldaemon.com>, Norbert Nemec says...
>>-------------------------
>>The one thing that I dislike most about C++ templates, and which has been carried over to D as well, is, that template parameters are not typed. I.e. if I have:
>>        template <typename T>
>>        class Test {
>>                static T add(T a,T b) { return a + b; } /* X */
>>        };
>>
>>        bool foo() {
>>                return Test<bool>::add(true,false); /* Y */
>>        };
>>the compiler actually complains about line /*X*/ which is absolutely correct, instead of telling the user that Test simply should not be instatiated with bool. Now if Test is somewhere deep within a library, the application programmer may get a huge bunch of error messages looking like a bug somewhere in the library, even though, the actual error was simply the wrong instatiation of a template.
>>
>>Solution to this would be something like interfaces in D, specifying a set of capabilities of a type (be it a class or a predefined type), and the possibility to restrict parameters in a template definition to such an interface.
> 
> If you restrict yourself to classes, you can restrict the type parameters:
> 
> template TSort(T : Comparable) {
> }
> 
> where Comparable is a type or class. AFAIK it's valid D code, if not is should ;-) .

I'm not sure whether that should really be the correct way to say it.
Otherwise
        template X(T:T[]) {}
would mean that T has to be a subtype of T[], which is complete nonsense. It
is hard to name the actual meaning of the ":", but it definitely is not
"subtype of". Actually, I don't have the slightest idea what would be
better. Perhaps:
        template X(T:T < Comparable) {}
as in Sather?

> I don't think there's any reason for the compiler to always
> complain about line /*X*/, instead of saying: Template instantiation at
> line /*Y*/ is invalid because of /*X*/ requirement in template body.
> Systems with type inference algorithms usually provide some intelligent
> error messages. There's also some papers about giving intelligent error
> messages in type inference algorithms.

Problem is, that in C++ code (and in D code as well), one cannot even say where the error lies. Did the library author do something illegal with the argument, or did the application programmer do a wrong instatiation? As long as there is no way to specify what capabilities the parameter types are supposed to have, even the most intelligent compiler cannot say more than: /*Y*/ seems to be wrong because of /*X*/, or even more nested error messages. In a case of 10 nested instantiations, the actual error might be located at any level.

And what it worse: there is no way to completely test the compilability of a template without doing all possible instantiations.

> C++ template mechanism is very powerful, but almost impossible to get right. Walter writes C++ compiler writes since the beginning of C++, so he knows about the pitfalls of template mechanism. All these issues about template are causes of several threads in this newsgroup (check the archives where lots of people, including me, argue for a better template mechanism). Blitz++ does a very good usageof templates, I like particularly the tensor notation. But it carries a 750 page language standard behind it, defining the mechanism it uses. Walter wants something better, we just have to keep throwing him good ideas and critics.

Nothing to say against that! Probably, the whole power of the C++ template mechanism was never actually planned. Actually, the fact that it hides a complete language on its own was even found by accident. And up to now, I guess nobody has really pinned down what the details are that make it so powerful. Guess it will take a good portion of genious to extract that part without carrying over all the drawbacks...

Ciao,
Nobbi