On 12/30/2010 07:08 PM, Steven Schveighoffer wrote:

>
> auto opAdd(Foo other)
>
> vs.
>
> auto opBinary(string op)(Foo other) if (op == "+")

For the latter not to look so intimidating, it can be shortened to:

auto opBinary(string op : "+")(Foo other)

Andrei Alexandrescu wrote:
> Many good examples do prove a ton though. Just off the top of my head:
> 
> - complex numbers
	Multiplication and division are different from each other and from
addition and subtraction.

> - checked integers
> - checked floating point numbers
> - ranged/constrained numbers
	More or less the same case, so I'm not sure that they make three.
Other than that agreed.

> - big int
> - big float
> - matrices and vectors
> - dimensional analysis (SI units)
> - rational numbers
> - fixed-point numbers
	For all of those, multiplication and division are different from
each other and from addition and subtraction.

	So what your examples do is actually prove *Steven's* point: most
of the time, the code is not shared between operators.

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr

On Thu, 30 Dec 2010 12:52:32 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 12/30/10 11:08 AM, Steven Schveighoffer wrote:
>> I'd have to see how it works. I also thought the new operator
>> overloading scheme was reasonable -- until I tried to use it.
>
> You mean until you tried to use it /once/.
>
>> Note this is even more bloated because you generate one function per
>> pair of types used in concatenation, vs. one function per class defined.
>
> That function is inlined and vanishes out of existence. I wish one day we'd characterize this bloating issue more precisely. Right now anything generic has the "bloated!!" alarm stuck to it indiscriminately.

Functions inline out of existence during runtime, but the function itself remains resident in the compiled binary.

I don't know that it's an important aspect to keep it in there or not, I just know it's kept.  There are a whole slew of improvements we can make in this regard, but I'm not sure they are possible, because I'm not a compiler writer.

One such nuisance in particular is the proliferation of types when you use something like isInputRange.  That invariably is *only* used at compile time, yet the type and its typeinfo are injected into the binary.

>> I mean bloated because you are generating template functions that just
>> forward to other functions. Those functions are compiled in and take up
>> space, even if they are inlined out.
>
> I think we can safely leave this matter to compiler technology.

I hope that can be done.  D already suffers from the 'hey what gives, how come hello world is 1MB?!!' syndrome.

>> Let's also realize that the mixin is going to be required *per
>> interface* and *per class*, meaning even more bloat.
>
> The bloating argument is a complete red herring in this case. I do agree that generally it could be a concern and I also agree that the compiler needs to be improved in that regard. But by and large I think we can calmly and safely think that a simple short function is not a source of worry.

Short template functions still have template mangled names.  I have found that template names with lots of parameters can slow down compilation, but I think Walter is working on fixing that.

If we can get to a point where language constructs such as this can be truly inlined out of existence, then I think we will be on another level from other languages.  It's not an unimportant nuisance to be dealt with later.

And at that point, I can agree that the template solution is not bloated ;)

>> So I'd say, while my example is not proof that this is a disaster, I
>> think it shows the change in operator overloading cannot yet be declared
>> a success. One good example does not prove anything just like one bad
>> example does not prove anything.
>
> Many good examples do prove a ton though. Just off the top of my head:
>
> - complex numbers
>
> - checked integers
>
> - checked floating point numbers
>
> - ranged/constrained numbers
>
> - big int
>
> - big float
>
> - matrices and vectors
>
> - dimensional analysis (SI units)
>
> - rational numbers
>
> - fixed-point numbers
>
> If I agree with something is that opCat is an oddity here as it doesn't usually group with others. Probably it would have helped if opCat would have been left named (just like opEquals or opCmp) but then uniformity has its advantages too. I don't think it's a disaster one way or another, but I do understand how opCat in particular is annoying to your case.

Probably the most common operator overload in D is opEquals, luckily that is not a template (even though it sadly does not work with interfaces yet).

It seems that operator overloads are in categories.  There are the numeric overloads, which I agree are generally overloaded in groups.  When I defined cursors to be more like C++ iterators in dcollections instead of small ranges, I used the ++ and -- overloads, which you typically define together.

When designing the mixin that allows you to define various operator overloads, I think it would be hugely beneficial to take into account these groupings and make the mixins modular.

>> I haven't had that experience. This is just me talking. Maybe others
>> believe it is good.
>>
>> I agree that the flexibility is good, I really think it should have that
>> kind of flexibility. Especially when we start talking about the whole
>> opAddAssign mess that was in D1. It also allows making wrapper types
>> easier.
>>
>> The problem with flexibility is that it comes with complexity. Most
>> programmers looking to understand how to overload operators in D are
>> going to be daunted by having to use both templates and template
>> constraints, and possibly mixins.
>
> Most programmers looking to understand how to overload operators in D will need to bundle them (see the common case argument above) and will go with the TDPL examples, which are clear, short, simple, and useful.

The code itself is simple, it's the "how does x + y match up with this template thingy" which is the problem I think.  We've already had several posts on d.learn ask how operator overloads work even after reading TDPL.

>> There once was a discussion on how to improve operators on the phobos
>> mailing list (don't have the history, because i think it was on
>> erdani.com). Essentially, the two things were:
>>
>> 1) let's make it possible to easily specify template constraints for
>> typed parameters (such as string) like this:
>>
>> auto opBinary("+")(Foo other)
>>
>> which would look far less complex and verbose than the current
>> incarnation. And simple to define when all you need is one or two
>> operators.
>
> I don't see this slight syntactic special case a net improvement over what we have.

It's less intimidating.  Max pointed out opBinary(string op : "+"), which is close, but still has some seemingly superfluous syntax (why do I need string op? and what is that : for?)

Compare that to C++ operators:

operator+(Foo rhs)

I'd call that very simple to understand in the context of operator overloading.

Also, the proposal is a specialization of templates in general, not just for operator overloading.  It translates to the same thing as if you wrote:

opBinary(string $)(Foo other) if($ == "+")

where $ is an inaccessible symbol.  It basically optimizes out the parts you don't care about if you don't care about them.

>
>> 2) make template instantiations that provably evaluate to a single
>> instance virtual. Or have a way to designate they should be virtual.
>> e.g. the above operator syntax can only have one instantiation.
>
> This may be worth exploring, but since template constraints are arbitrary expressions I fear it will become a mess of special cases designed to avoid the Turing tarpit.

That's why I conditioned it as "provably" evaluate to single instance.  I meant provable by the compiler, so even something that may look obvious to a user as only instantiating to one instance may not be provable by the compiler.  It probably requires you to use specific constructs (like the one mentioned above) to help the compiler out.

>>> Using operator overloading in conjunction with class inheritance is rare.
>>
>> I don't use operator overloads and class inheritance, but I do use
>> operator overloads with interfaces. I think rare is not the right term,
>> it's somewhat infrequent, but chances are if you do a lot of interfaces,
>> you will encounter it at least once. It certainly doesn't dominate the
>> API being defined.
>
> Maybe a more appropriate characterization is that you use catenation with interfaces.

Concatenation, equality comparison, indexing, and assignment.  Out of those, only concatenation gives me headaches because it requires templates.

If you propose we remove concatenation from opBinary and give it its own form, then I thing that would solve the problem too.

>> Actually, the functionality almost exists in template this parameters.
>> At least, the reevaluation part is working. However, you still must
>> incur a performance penalty to cast to the derived type, plus the
>> template nature of it adds unnecessary bloat.
>
> Saw that. I have a suspicion that we'll see a solid solution from you soon!

Alas, no solution is possible without templates being allowed in interfaces :(  But yes, I plan to use this technique as soon as it's possible.

-Steve

On Thu, 30 Dec 2010 21:15:30 +0200, Jérôme M. Berger <jeberger@free.fr> wrote:

> Andrei Alexandrescu wrote:
>> Many good examples do prove a ton though. Just off the top of my head:
>>
>> - complex numbers
> 	Multiplication and division are different from each other and from
> addition and subtraction.
>
>> - checked integers
>> - checked floating point numbers
>> - ranged/constrained numbers
> 	More or less the same case, so I'm not sure that they make three.
> Other than that agreed.
>
>> - big int
>> - big float
>> - matrices and vectors
>> - dimensional analysis (SI units)
>> - rational numbers
>> - fixed-point numbers
> 	For all of those, multiplication and division are different from
> each other and from addition and subtraction.
>
> 	So what your examples do is actually prove *Steven's* point: most
> of the time, the code is not shared between operators.
>
> 		Jerome

First, most of these don't even have a division operator defined in math.
Second, you prove Andrei's point, not the other way around, since it makes the generic case easier, particular case harder.

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

s/most/some

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On 12/30/10 1:15 PM, "Jérôme M. Berger" wrote:
> Andrei Alexandrescu wrote:
>> Many good examples do prove a ton though. Just off the top of my head:
>>
>> - complex numbers
> 	Multiplication and division are different from each other and from
> addition and subtraction.
>
>> - checked integers
>> - checked floating point numbers
>> - ranged/constrained numbers
> 	More or less the same case, so I'm not sure that they make three.
> Other than that agreed.
>
>> - big int
>> - big float
>> - matrices and vectors
>> - dimensional analysis (SI units)
>> - rational numbers
>> - fixed-point numbers
> 	For all of those, multiplication and division are different from
> each other and from addition and subtraction.

That's where the flexibility of grouping really helps:  Let's also not forget about things such as unary + and -.

> 	So what your examples do is actually prove *Steven's* point: most
> of the time, the code is not shared between operators.

I thought these examples effectively settle the matter.


Andrei

On 12/30/10 1:17 PM, Steven Schveighoffer wrote:
> On Thu, 30 Dec 2010 12:52:32 -0500, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 12/30/10 11:08 AM, Steven Schveighoffer wrote:
>>> I'd have to see how it works. I also thought the new operator
>>> overloading scheme was reasonable -- until I tried to use it.
>>
>> You mean until you tried to use it /once/.
>>
>>> Note this is even more bloated because you generate one function per
>>> pair of types used in concatenation, vs. one function per class defined.
>>
>> That function is inlined and vanishes out of existence. I wish one day
>> we'd characterize this bloating issue more precisely. Right now
>> anything generic has the "bloated!!" alarm stuck to it indiscriminately.
>
> Functions inline out of existence during runtime, but the function
> itself remains resident in the compiled binary.

Doesn't have to!

> I don't know that it's an important aspect to keep it in there or not, I
> just know it's kept. There are a whole slew of improvements we can make
> in this regard, but I'm not sure they are possible, because I'm not a
> compiler writer.

Please take my word for it: this is a solved problem.

> One such nuisance in particular is the proliferation of types when you
> use something like isInputRange. That invariably is *only* used at
> compile time, yet the type and its typeinfo are injected into the binary.
>
>>> I mean bloated because you are generating template functions that just
>>> forward to other functions. Those functions are compiled in and take up
>>> space, even if they are inlined out.
>>
>> I think we can safely leave this matter to compiler technology.
>
> I hope that can be done. D already suffers from the 'hey what gives, how
> come hello world is 1MB?!!' syndrome.

More like 600KB, but yah :o). Note that the size of the executable is caused by other issues, compared to which the concerns at hand are puny.

[snip]
>>> So I'd say, while my example is not proof that this is a disaster, I
>>> think it shows the change in operator overloading cannot yet be declared
>>> a success. One good example does not prove anything just like one bad
>>> example does not prove anything.
>>
>> Many good examples do prove a ton though. Just off the top of my head:
>>
>> - complex numbers
>>
>> - checked integers
>>
>> - checked floating point numbers
>>
>> - ranged/constrained numbers
>>
>> - big int
>>
>> - big float
>>
>> - matrices and vectors
>>
>> - dimensional analysis (SI units)
>>
>> - rational numbers
>>
>> - fixed-point numbers

One more:

- Variant types

>> If I agree with something is that opCat is an oddity here as it
>> doesn't usually group with others. Probably it would have helped if
>> opCat would have been left named (just like opEquals or opCmp) but
>> then uniformity has its advantages too. I don't think it's a disaster
>> one way or another, but I do understand how opCat in particular is
>> annoying to your case.
>
> Probably the most common operator overload in D is opEquals, luckily
> that is not a template (even though it sadly does not work with
> interfaces yet).
>
> It seems that operator overloads are in categories. There are the
> numeric overloads, which I agree are generally overloaded in groups.
> When I defined cursors to be more like C++ iterators in dcollections
> instead of small ranges, I used the ++ and -- overloads, which you
> typically define together.
>
> When designing the mixin that allows you to define various operator
> overloads, I think it would be hugely beneficial to take into account
> these groupings and make the mixins modular.

That's a valuable insight! Introspection can help a lot, e.g. you can synthesize opAdd from opAddAssign (or vice versa) etc.

>>> I haven't had that experience. This is just me talking. Maybe others
>>> believe it is good.
>>>
>>> I agree that the flexibility is good, I really think it should have that
>>> kind of flexibility. Especially when we start talking about the whole
>>> opAddAssign mess that was in D1. It also allows making wrapper types
>>> easier.
>>>
>>> The problem with flexibility is that it comes with complexity. Most
>>> programmers looking to understand how to overload operators in D are
>>> going to be daunted by having to use both templates and template
>>> constraints, and possibly mixins.
>>
>> Most programmers looking to understand how to overload operators in D
>> will need to bundle them (see the common case argument above) and will
>> go with the TDPL examples, which are clear, short, simple, and useful.
>
> The code itself is simple, it's the "how does x + y match up with this
> template thingy" which is the problem I think. We've already had several
> posts on d.learn ask how operator overloads work even after reading TDPL.

I'm not worried about this most at all as I think (unintentionally) things have fallen in the right place: operator overloading is an advanced, specialized topic. I believe the set of users who are sophisticated enough to sit down and start overloading operators at large, yet at the same time are beginners enough to not grasp the notion of a generic function in D (which is much simpler than in other languages) may as well be not empty, but is small enough to not cater for.

>> Saw that. I have a suspicion that we'll see a solid solution from you
>> soon!
>
> Alas, no solution is possible without templates being allowed in
> interfaces :( But yes, I plan to use this technique as soon as it's
> possible.

I voted for it now, too. As I always use all of my 10 votes, I had asked on the Phobos list to increase that limit, but was thoroughly shredded into little pieces.


Andrei

so wrote:
> On Thu, 30 Dec 2010 21:15:30 +0200, Jérôme M. Berger <jeberger@free.fr> wrote:
> 
>> Andrei Alexandrescu wrote:
>>> Many good examples do prove a ton though. Just off the top of my head:
>>>
>>> - complex numbers
>>     Multiplication and division are different from each other and from
>> addition and subtraction.
>>
>>> - checked integers
>>> - checked floating point numbers
>>> - ranged/constrained numbers
>>     More or less the same case, so I'm not sure that they make three.
>> Other than that agreed.
>>
>>> - big int
>>> - big float
>>> - matrices and vectors
>>> - dimensional analysis (SI units)
>>> - rational numbers
>>> - fixed-point numbers
>>     For all of those, multiplication and division are different from
>> each other and from addition and subtraction.
>>
>>     So what your examples do is actually prove *Steven's* point: most
>> of the time, the code is not shared between operators.
>>
>>         Jerome
> 
> First, most of these don't even have a division operator defined in math.

	?? The only type in this list without a division operator is vector
all the others have it.

> Second, you prove Andrei's point, not the other way around, since it makes the generic case easier, particular case harder.
> 
	I'm sorry? What do you call the "generic case" here? All this list
shows is that each operator needs to be implemented individually
anyway. Andrei's point was exactly the reverse: he claims that most
operators can be implemented in groups which clearly isn't the case
here.

		Jerome
-- 
mailto:jeberger@free.fr
http://jeberger.free.fr
Jabber: jeberger@jabber.fr

On 12/30/10 3:36 PM, "Jérôme M. Berger" wrote:
> so wrote:
>> On Thu, 30 Dec 2010 21:15:30 +0200, Jérôme M. Berger<jeberger@free.fr>
>> wrote:
>>
>>> Andrei Alexandrescu wrote:
>>>> Many good examples do prove a ton though. Just off the top of my head:
>>>>
>>>> - complex numbers
>>>      Multiplication and division are different from each other and from
>>> addition and subtraction.
>>>
>>>> - checked integers
>>>> - checked floating point numbers
>>>> - ranged/constrained numbers
>>>      More or less the same case, so I'm not sure that they make three.
>>> Other than that agreed.
>>>
>>>> - big int
>>>> - big float
>>>> - matrices and vectors
>>>> - dimensional analysis (SI units)
>>>> - rational numbers
>>>> - fixed-point numbers
>>>      For all of those, multiplication and division are different from
>>> each other and from addition and subtraction.
>>>
>>>      So what your examples do is actually prove *Steven's* point: most
>>> of the time, the code is not shared between operators.
>>>
>>>          Jerome
>>
>> First, most of these don't even have a division operator defined in math.
>
> 	?? The only type in this list without a division operator is vector
> all the others have it.
>
>> Second, you prove Andrei's point, not the other way around, since it
>> makes the generic case easier, particular case harder.
>>
> 	I'm sorry? What do you call the "generic case" here? All this list
> shows is that each operator needs to be implemented individually
> anyway. Andrei's point was exactly the reverse: he claims that most
> operators can be implemented in groups which clearly isn't the case
> here.

And I stand by that claim. One aspect that seems to have been forgotten is that types usually implement either op= in terms of op or vice versa. That savings alone is large.

Andrei

On 12/30/10 3:36 PM, "Jérôme M. Berger" wrote:
> so wrote:
>> On Thu, 30 Dec 2010 21:15:30 +0200, Jérôme M. Berger<jeberger@free.fr>
>> wrote:
>>
>>> Andrei Alexandrescu wrote:
>>>> Many good examples do prove a ton though. Just off the top of my head:
>>>>
>>>> - complex numbers
>>>      Multiplication and division are different from each other and from
>>> addition and subtraction.
>>>
>>>> - checked integers
>>>> - checked floating point numbers
>>>> - ranged/constrained numbers
>>>      More or less the same case, so I'm not sure that they make three.
>>> Other than that agreed.
>>>
>>>> - big int
>>>> - big float
>>>> - matrices and vectors
>>>> - dimensional analysis (SI units)
>>>> - rational numbers
>>>> - fixed-point numbers
>>>      For all of those, multiplication and division are different from
>>> each other and from addition and subtraction.
>>>
>>>      So what your examples do is actually prove *Steven's* point: most
>>> of the time, the code is not shared between operators.
>>>
>>>          Jerome
>>
>> First, most of these don't even have a division operator defined in math.
>
> 	?? The only type in this list without a division operator is vector
> all the others have it.
>
>> Second, you prove Andrei's point, not the other way around, since it
>> makes the generic case easier, particular case harder.
>>
> 	I'm sorry? What do you call the "generic case" here? All this list
> shows is that each operator needs to be implemented individually
> anyway. Andrei's point was exactly the reverse: he claims that most
> operators can be implemented in groups which clearly isn't the case
> here.

Oh, not to mention opIndexXxx. In fact I remember that the breaking point for Walter (where he agreed to implement my design pronto) was the spectrum of having to define opIndexAddAssign etc. which effectively doubled the number of named operators AND is virtually ALWAYS implemented in a uniform manner.

So let's not forget that the new design not only does what the D1 design does (and better), it also does things that the old design didn't, and in a scalable manner.

Does this settle the argument?


Andrei