On Thu, Dec 17, 2009 at 9:28 AM, dsimcha <dsimcha@yahoo.com> wrote:
> == Quote from Nick Sabalausky (a@a.a)'s article
>> Pardon my ignorance, but why is it that templated functions can't be virtual?
>
> This is an unfortunate consequence of compilation model leaking out into language design.  You're supposed to be able to subclass a base class even if you don't have the full source code to it.  Let's say we have:
>
> class A {
>    void doStuff(T)(T arg) {}
> }
>
> class B : A {
>    void doStuff(T)(T arg) {
>        writeln("In B.");
>    }
> }
>
> And then we do:
>
> B b = new B;
> b.doStuff(1);
>
> The instantiation of B.doStuff() with an int would require that the vtable for A be updated to include doStuff!(int).  This is not possible if we also allow base classes to be compiled separately from derived classes and the code that uses the derived class.
>
> The only solution I see would be to completely get rid of separate compilation. For my personal use, since most of my projects are under 10k lines and take a negligible amount of time to compile anyhow, I'd be in favor of this.  However, for larger projects or projects that require things to work based only on interface, without access to the full source code, this is probably impractical.

This 'auto ref' stuff and the multi-flavor 'vconst' functions are
basically templates with a known list of instantiations (ref /no-ref,
and const/immutable/plain)
In theory there's no reason you couldn't allow templates to also
create virtual functions, if you limit yourself to listing the
possible instantiations up front.

It feels like we may be missing out on a more general construct here that could embrace all these cases via some extension to template syntax.

I'm reminded of how Go interfaces do the job of both specifying that a function is a template and checking constraints on use.

In D-ish syntax, Go uses something like this:
interface Foo { ... }
void doSomething(Foo f, int x) { ... }

To mean something like

template CheckIfIsFoo!(T) { ... }
void doSomething(T)(T f, int x) if (CheckIfIsFoo!(T)) { ... }

The advantage is that the compiler can offer better error messages about what part of the check fails, and the syntax is cleaned up significantly (less repetition, less proliferation of parentheses).

Maybe a similar approach could be used to separate the specification of these alternatives into pseudo-types that can then be used as parameters.

Like
static interface VConst(T) {
   constnessOf(T) is in [const, immutable, !const];
}
VConst!(Bar) doSomething(VConst!(Bar) b, int y) {...}

static interface MaybeRef(T) {
   refnessOf(T) is in [ref, !ref];
}
MaybeRef!(Bar) doSomethingElse(MaybeRef!(Bar) b, int y) { ... }


Anyway, the more I look at standard C++ and D template syntax the more I think that Go is onto something.  Yeh, there are some generic container-like templated types where T really can be any type (which Go inexcusably ignores), but very often there are some restrictions on T that are important enough and common enough that they deserve some support with a terse syntax.

--bb

Bill Baxter wrote:
> On Thu, Dec 17, 2009 at 9:28 AM, dsimcha <dsimcha@yahoo.com> wrote:
>> == Quote from Nick Sabalausky (a@a.a)'s article
>>> Pardon my ignorance, but why is it that templated functions can't be
>>> virtual?
>> This is an unfortunate consequence of compilation model leaking out into language
>> design.  You're supposed to be able to subclass a base class even if you don't
>> have the full source code to it.  Let's say we have:
>>
>> class A {
>>    void doStuff(T)(T arg) {}
>> }
>>
>> class B : A {
>>    void doStuff(T)(T arg) {
>>        writeln("In B.");
>>    }
>> }
>>
>> And then we do:
>>
>> B b = new B;
>> b.doStuff(1);
>>
>> The instantiation of B.doStuff() with an int would require that the vtable for A
>> be updated to include doStuff!(int).  This is not possible if we also allow base
>> classes to be compiled separately from derived classes and the code that uses the
>> derived class.
>>
>> The only solution I see would be to completely get rid of separate compilation.
>> For my personal use, since most of my projects are under 10k lines and take a
>> negligible amount of time to compile anyhow, I'd be in favor of this.  However,
>> for larger projects or projects that require things to work based only on
>> interface, without access to the full source code, this is probably impractical.
> 
> This 'auto ref' stuff and the multi-flavor 'vconst' functions are
> basically templates with a known list of instantiations (ref /no-ref,
> and const/immutable/plain)
> In theory there's no reason you couldn't allow templates to also
> create virtual functions, if you limit yourself to listing the
> possible instantiations up front.

This notion of templates "restricted enough" to be virtual has been intensively discussed a couple of years ago between Walter, Bartosz and myself. Back then it was much less clear where to draw the line, but right now the idea is well worth revisiting.

Andrei

On Thu, 17 Dec 2009 13:03:51 -0500, Bill Baxter <wbaxter@gmail.com> wrote:

> This 'auto ref' stuff and the multi-flavor 'vconst' functions are
> basically templates with a known list of instantiations (ref /no-ref,
> and const/immutable/plain)
> In theory there's no reason you couldn't allow templates to also
> create virtual functions, if you limit yourself to listing the
> possible instantiations up front.

Technically, the vconst functions provide one other feature -- implicit casting back to the correct type.  You can't support that with templates, and still have the compiler ensure const is obeyed during the function all with a function signature.  Also, there is only ever a single vconst function, not 3^n overloaded ones.

But I agree with allowing a restricted template to be able to be virtual.  One recent discussion that applies here is templatized operator overloading -- wouldn't it be nice if:

opBinary(op : "+")(T rhs) {...}

could be a virtual function?  That gives us immediate support for virtual functions without the hoaky:

opBinary(op : "+")(T rhs) {return opAdd(rhs);}

mixin boilerplate.

(BTW, this was Denis Koroshin's idea, not mine)

-Steve

dsimcha:

>The only solution I see would be to completely get rid of separate compilation.<

"All problems in computer science can be solved by another level of indirection;"
-- David Wheeler
But that also slows code down a little :-)

--------------------

Bill Baxter:

Static interfaces are an easy idea, it was discussed some weeks ago, and probably it's not too much hard to implement in the language. I like them enough, but they don't add that much to the template constraints already present, so it's mostly a duplication of syntax and semantics. So I am not sure they are a good idea.


>static interface VConst(T) {
   constnessOf(T) is in [const, immutable, !const];
}
VConst!(Bar) doSomething(VConst!(Bar) b, int y) {...}

static interface MaybeRef(T) {
   refnessOf(T) is in [ref, !ref];
}
MaybeRef!(Bar) doSomethingElse(MaybeRef!(Bar) b, int y) { ... }<

Walter has just added traits to perform this, so I think this is already doable, with __trait/meta. and template constraints.
The opIn_r defined for arrays is something that D2 must eventually have, there's no doubt about this.

But "is in", followed by an array of those modifiers is currently impossible (you may create a tuple of templates, where each template tests for constness, etc). Maybe in future you can create an array of annotations:
[@const, @immutable, @notConst]

Bye,
bearophile

On Wed, 16 Dec 2009 19:05:09 -0500, Michel Fortin <michel.fortin@michelf.com> wrote:

> On 2009-12-16 16:46:14 -0500, Jason House <jason.james.house@gmail.com> said:
>
>> Walter Bright Wrote:
>>
>>> Jason House wrote:
>>>> KennyTM~ Wrote:
>>>>
>>>>> auto const?
>>>>  I was wondering the same thing.
>>>  The const transport thing is, unfortunately, a very different problem.
>>  Of course, but it may still go through bikeshed issues. This morning I read about inout, return, vconst, aconst, sameconst, autoconst, auto const, and bikeshed. At least one of those was in jest :) auto const isn't that bad, and you obviously liked auto ref...
>
> Since this is just a special kind of const, it could be called "const^" (const or a derived constness):
>
> 	const?(Object) func(const?(Object) o) {
> 		return o;
> 	}
>
> The interesting thing about it, beside not taking a keyword, is that it can scale in the future if we need to add many distinct constness to the same function signature:
>
> 	const?(Object) func(const?(Object) o, const?2(Object) o2, out const?2(Object) o3) {
> 		o3 = o2;
> 		return o;
> 	}

This can never work.  a const?(Object) is const during the function execution, and cannot be assigned to.

> Not that you'd need that often, but if it does becomes necessary in the future we'll still have some options.
>
> Furthermore, the concept could be extended to any type. This could be useful with class hierarchies:
>
> 	Object? func(Object? o) {
> 		writeln(o.toString());
> 		return o;
> 	}
>
> 	MyObject o = func(new MyObject);
>
> Here, "Object?" means Object or a derived type.

This doesn't have the same utility as vconst, since you can't apply Object to other types like you can constancy.

Plus you can already do this with a template and have a virtual version of the func:

T func(T)(T o) if(T : Object) {func_virt(o); return o; }
protected void func_virt(Object o) {writeln(o.toString());}

-Steve

On Thu, 17 Dec 2009 17:26:52 +0100, Leandro Lucarella <llucax@gmail.com> wrote:

> Simen kjaeraas, el 17 de diciembre a las 02:16 me escribiste:
>> On Wed, 16 Dec 2009 12:00:46 +0100, KennyTM~ <kennytm@gmail.com> wrote:
>>
>> >On Dec 16, 09 15:18, Walter Bright wrote:
>> >>There's a need in generic code to have a function take a parameter by
>> >>ref if it is an lvalue, and by value if it is an rvalue. This can be
>> >>addressed by making it a template using auto ref:
>> >>
>> >>T foo(T)(auto ref T x) { ... }
>> >>
>> >>foo(3) // call by value
>> >>int y;
>> >>foo(y) // call by reference
>> >>
>> >>There is also a need to 'transmit' the ref'ness to the return value.
>> >>This can be done with auto ref:
>> >>
>> >>auto ref foo(T)(auto ref T x) { return x; }
>> >>
>> >>foo(3) => int foo(int x)
>> >>foo(y) => ref int foo(ref int x)
>> >>
>> >>This means that the generic forwarding function would look like:
>> >>
>> >>auto ref foo(alias F, T...)(auto ref T args) { return F(args); }
>> >
>> >auto const?
>>
>> auto const auto ref Foo bar( auto const auto ref Foo arg ) {
>>     return arg;
>> }
>>
>> Am I the only one who finds this confusing?
>
> Just call "auto const auto ref" "auto auto":
>
> auto auto Foo bar( auto auto Foo arg ) {
>     return arg;
> }
>
> =P
>

Even better, with return type inference:

auto auto auto bar( T )( auto auto T arg ) {
    return arg;
}

:p
-- 
Simen

On 2009-12-17 14:52:40 -0500, "Steven Schveighoffer" <schveiguy@yahoo.com> said:

>> The interesting thing about it, beside not taking a keyword, is that it  can scale in the future if we need to add many distinct constness to the  same function signature:
>> 
>> 	const?(Object) func(const?(Object) o, const?2(Object) o2, out  const?2(Object) o3) {
>> 		o3 = o2;
>> 		return o;
>> 	}
> 
> This can never work.  a const?(Object) is const during the function  execution, and cannot be assigned to.

I'm not sure why, but I always forget that const(Object) is not rebindable. My mistake, here is the corrected example:

	const?(MyStruct)* func(const?(MyStruct)* s, const?2(MyStruct)* s2, const?2(MyStruct)* s3) {
		o2 = o3;
		return s;
	}


>> Furthermore, the concept could be extended to any type. This could be  useful with class hierarchies:
>> 
>> 	Object? func(Object? o) {
>> 		writeln(o.toString());
>> 		return o;
>> 	}
>> 
>> 	MyObject o = func(new MyObject);
>> 
>> Here, "Object?" means Object or a derived type.
> 
> This doesn't have the same utility as vconst, since you can't apply Object  to other types like you can constancy.
> 
> Plus you can already do this with a template and have a virtual version of  the func:
> 
> T func(T)(T o) if(T : Object) {func_virt(o); return o; }
> protected void func_virt(Object o) {writeln(o.toString());}

Indeed. I was mostly trying to show that the "const?" notation can easily be extended to all sort of things, which makes it a better choice than other notations.

We could do the same with an old idea that didn't get in the language: scope arguments.

	// *a and *b are in the same scope, so you can swap a and b
	void swap(scope?(int)* a, scope?(int)* b) {
		int tmp = a;
		a = b;
		b = tmp;
	}

But the scope problem would require more thought.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Thu, 17 Dec 2009 15:17:22 -0500, Michel Fortin <michel.fortin@michelf.com> wrote:

> On 2009-12-17 14:52:40 -0500, "Steven Schveighoffer" <schveiguy@yahoo.com> said:
>
>>> The interesting thing about it, beside not taking a keyword, is that it  can scale in the future if we need to add many distinct constness to the  same function signature:
>>>  	const?(Object) func(const?(Object) o, const?2(Object) o2, out  const?2(Object) o3) {
>>> 		o3 = o2;
>>> 		return o;
>>> 	}
>>  This can never work.  a const?(Object) is const during the function  execution, and cannot be assigned to.
>
> I'm not sure why, but I always forget that const(Object) is not rebindable. My mistake, here is the corrected example:
>
> 	const?(MyStruct)* func(const?(MyStruct)* s, const?2(MyStruct)* s2, const?2(MyStruct)* s3) {
> 		o2 = o3;
> 		return s;
> 	}

That doesn't do anything :)  It may as well be written:

const?(MyStruct)* func(const?(MyStruct)* s) { return s; }

If you want something like this:


 	const?(MyStruct)* func(const?(MyStruct)* s, const?2(MyStruct)** s2,
 const?2(MyStruct)** s3) {
 		*s2 = *s3;
 		return s;
 	}

This will not accept any args except for const?(MyStruct)** for s2 and s3.  That is, you cannot pass a MyStruct** or a const(MyStruct)** or an immutable(MyStruct)** because it is 2 levels of indirection (this is the test we ran earlier).

>
>
>>> Furthermore, the concept could be extended to any type. This could be  useful with class hierarchies:
>>>  	Object? func(Object? o) {
>>> 		writeln(o.toString());
>>> 		return o;
>>> 	}
>>>  	MyObject o = func(new MyObject);
>>>  Here, "Object?" means Object or a derived type.
>>  This doesn't have the same utility as vconst, since you can't apply Object  to other types like you can constancy.
>>  Plus you can already do this with a template and have a virtual version of  the func:
>>  T func(T)(T o) if(T : Object) {func_virt(o); return o; }
>> protected void func_virt(Object o) {writeln(o.toString());}
>
> Indeed. I was mostly trying to show that the "const?" notation can easily be extended to all sort of things, which makes it a better choice than other notations.
>
> We could do the same with an old idea that didn't get in the language: scope arguments.
>
> 	// *a and *b are in the same scope, so you can swap a and b
> 	void swap(scope?(int)* a, scope?(int)* b) {
> 		int tmp = a;
> 		a = b;
> 		b = tmp;
> 	}
>
> But the scope problem would require more thought.

OK, that makes more sense.  Except scope is not a type constructor (yet) :)

-Steve

"dsimcha" <dsimcha@yahoo.com> wrote in message news:hgdpn2$t1c$1@digitalmars.com...
> == Quote from Nick Sabalausky (a@a.a)'s article
>> Pardon my ignorance, but why is it that templated functions can't be virtual?
>
> This is an unfortunate consequence of compilation model leaking out into
> language
> design.  You're supposed to be able to subclass a base class even if you
> don't
> have the full source code to it.  Let's say we have:
>
> class A {
>    void doStuff(T)(T arg) {}
> }
>
> class B : A {
>    void doStuff(T)(T arg) {
>        writeln("In B.");
>    }
> }
>
> And then we do:
>
> B b = new B;
> b.doStuff(1);
>
> The instantiation of B.doStuff() with an int would require that the vtable
> for A
> be updated to include doStuff!(int).  This is not possible if we also
> allow base
> classes to be compiled separately from derived classes and the code that
> uses the
> derived class.
>
> The only solution I see would be to completely get rid of separate
> compilation.
> For my personal use, since most of my projects are under 10k lines and
> take a
> negligible amount of time to compile anyhow, I'd be in favor of this.
> However,
> for larger projects or projects that require things to work based only on
> interface, without access to the full source code, this is probably
> impractical.

But we already can't instantiate templates without the source anyway, so I don't see how doing that vtable update for A would create any additional requirements on source availability that we don't already have, regardless of whether or not the function is virtual. Either way, the issue of "Do I need to provide the source?" still boils down to just "Am I defining a public template?" and is not at all dependent on "Is this going to be externally subclassed?".

== Quote from Nick Sabalausky (a@a.a)'s article
> "dsimcha" <dsimcha@yahoo.com> wrote in message news:hgdpn2$t1c$1@digitalmars.com...
> > == Quote from Nick Sabalausky (a@a.a)'s article
> >> Pardon my ignorance, but why is it that templated functions can't be virtual?
> >
> > This is an unfortunate consequence of compilation model leaking out into
> > language
> > design.  You're supposed to be able to subclass a base class even if you
> > don't
> > have the full source code to it.  Let's say we have:
> >
> > class A {
> >    void doStuff(T)(T arg) {}
> > }
> >
> > class B : A {
> >    void doStuff(T)(T arg) {
> >        writeln("In B.");
> >    }
> > }
> >
> > And then we do:
> >
> > B b = new B;
> > b.doStuff(1);
> >
> > The instantiation of B.doStuff() with an int would require that the vtable
> > for A
> > be updated to include doStuff!(int).  This is not possible if we also
> > allow base
> > classes to be compiled separately from derived classes and the code that
> > uses the
> > derived class.
> >
> > The only solution I see would be to completely get rid of separate
> > compilation.
> > For my personal use, since most of my projects are under 10k lines and
> > take a
> > negligible amount of time to compile anyhow, I'd be in favor of this.
> > However,
> > for larger projects or projects that require things to work based only on
> > interface, without access to the full source code, this is probably
> > impractical.
> But we already can't instantiate templates without the source anyway, so I don't see how doing that vtable update for A would create any additional requirements on source availability that we don't already have, regardless of whether or not the function is virtual. Either way, the issue of "Do I need to provide the source?" still boils down to just "Am I defining a public template?" and is not at all dependent on "Is this going to be externally subclassed?".

To clarify, the requirement is somewhat stronger than having the source.  A, B, and all code that instantiates template methods of A or B would probably all have to be compiled as one compilation unit to get the vtable layout right.