On 10/1/20 2:26 PM, H. S. Teoh wrote:
> On Thu, Oct 01, 2020 at 02:08:11PM -0400, Steven Schveighoffer via Digitalmars-d wrote:
>> On 10/1/20 1:35 PM, Andrei Alexandrescu wrote:
> [...]
>>> How do you implement Variant.get(T) without reifying is(T : U)?
>>
>> How do you implement Variant.get(T) *with* reifying is(U : T)?
>>
>> step 1: determine using reified constructs that U is convertible to T
>> step 2: ?????
>> step 3: Return a T from a U!
>>
>> BTW, Variant already does a *limited* form of this. reification
>> doesn't change what needs to happen.
>>
>> I don't see why Variant's needs have to dictate how you need to reason
>> about types at compile time in CTFE.
> [...]
> 
> I think what Andrei is getting at is, we want to be able to transfer the
> compiler's knowledge and implementation of implicit type conversions to
> runtime.  I.e., we'd like to do this:
> 
> 	Variant v;
> 	v = 100;	// v now stores an int
> 	...
> 	long l = v.get!long; // currently doesn't work

Again, this does work, but I don't think this completely destroys your point.

> Or maybe, a more motivating example:
> 
> 	T calculate(T)(Variant number) {
> 		T val = number.get!T;
> 		return val*2 + 1;
> 	}
> 
> 	Variant v;
> 	v = 100;
> 	assert(calculate!long(v) == 201L);
> 	assert(calculate!float(v) == 201.0f);

This currently works as well.

There *are* cases that don't work. But that's a matter of fixing ImplicitlyConversionTargets.

> 
> I.e., you don't know what type you'd like to get from the Variant until
> the caller calls you, but you also don't know what type is stored in
> Variant until runtime.  Currently, there's a hack in Variant to handle
> some of these implicit conversions, but as Andrei says, it's incomplete
> and may have some wrong cases.  This knowledge is already in the
> compiler; there should be a way to transfer this to Variant at runtime
> without having to re-implement implicit conversions in library code.

That's all fine. The problem I see is:

1. Variant (a true variant that can hold ANY type) is really the only use case for runtime type information. Like, literally the only. Algebraic types don't have to use reification *at all*. Any reification system is still not going to encompass all you can do with a type. Sure, you can probably 100% cover type conversion. It will never 100% cover, e.g. comparison or type coersion.
2. Compile time is the main focus here. Variant works, and can be made better, and we don't have to introduce a new concept to make it work. But in CTFE, pulling back the Oz curtain a bit can make things an insane amount more pleasant. We don't need to reimplement what the compiler does at runtime for this.

-Steve

On Thursday, 1 October 2020 at 18:30:31 UTC, Andrei Alexandrescu wrote:
> std.variant. The latter can be considered a problem of having the right primitives in std.traits.

I don't think that std.variant can ever work cleanly.
Either you are a dynamically typed language, or you aren't.
We can approach, dynamism by emulating it in templates, but reaching it ... quite probably impossible.
While is an interesting consideration, it's not at all what type functions solve.

An accurate definition of what type functions allow you to do, and what they are designed to do is:
  - Replacing templates which could be functions with functions.
  - i.e. If the only reason you are using a template instead of a function is that functions can't take types or a list of types. Then that's in the problem domain of type functions.

On 10/1/20 2:32 PM, Andrei Alexandrescu wrote:
> On 10/1/20 2:26 PM, Steven Schveighoffer wrote:
>> On 10/1/20 2:10 PM, Andrei Alexandrescu wrote:
>>> On 10/1/20 1:55 PM, Stefan Koch wrote:
>>>> Please show me in the code where is(T : U) is used.
>>>> I don't find it in std variant.
>>>
>>> It's in the use of ImplicitConversionTargets in std.traits.
>>
>> The only use of is(T:U) in that trait is:
>>
>> is(T : typeof(null))
> 
> Of course. That's the point - the template is not using "is", because it can't: it only has one type available. It "implements" a subset of it. See what I'm saying? ImplicitConversionTargets helps Variant implement a kinda sorta "is" at runtime.
> 

Yes, I see. This subthread confused me...

-Steve

On Thu, Oct 01, 2020 at 02:28:37PM -0400, Steven Schveighoffer via Digitalmars-d wrote:
> On 10/1/20 2:19 PM, H. S. Teoh wrote:
[...]
> > We would like to be able to do this:
> > 
> > 	Variant v;
> > 	v = 100; // v now stores an int
> > 	long l = v.get!long; // currently does not work
> 
> I'm not going to respond to anything else here, but just want to point out, this does work.
> 
> https://run.dlang.io/is/S3sqDF

OK, I didn't check before I posted, :-/ but the point is that the implicit conversion is manually hacked with a library template that does not cover all the cases.  We should not have to reimplement implicit conversions in library code when the compiler is already perfectly capable of doing it.

It's not just implicit conversions, though.  There's a host of other things that we cannot do currently.  Like conversion to string: in theory, since std.conv.to is able to convert just about any type to string, we ought to be able to support v.get!string for *any* stored type.  Yet we cannot because at runtime all we have is typeid(T), so we cannot dispatch to the correct overload of std.conv.to to create the string.


T

-- 
Being forced to write comments actually improves code, because it is easier to fix a crock than to explain it. -- G. Steele

On 10/1/20 2:40 PM, Steven Schveighoffer wrote:
> 1. Variant (a true variant that can hold ANY type) is really the only use case for runtime type information.Like, literally the only.

I agree. But it's also the keystone: Any (let's call it Any...) is the only way to do things like creating objects and invoking functions from dynamically-loaded library in a sane manner. This has become clearer to me when I was wrestling typeid() - the right compile-time introspection is the kind that allows you to create runtime layouts safely.

> Algebraic types don't have to use reification *at all*.

Correct. SumType is awesome btw :o).

> Any reification system is still not going to encompass all you can do with a type. Sure, you can probably 100% cover type conversion. It will never 100% cover, e.g. comparison or type coersion.

Coercion seems to be on the same level of difficulty as implicit conversions, so probably having one done right means having the other as well. As to what the capabilities are, it comes down to what context things can be dereified in (i.e. your opening post).

> 2. Compile time is the main focus here. Variant works, and can be made better, and we don't have to introduce a new concept to make it work. But in CTFE, pulling back the Oz curtain a bit can make things an insane amount more pleasant. We don't need to reimplement what the compiler does at runtime for this.

At best of course we'd have everything, and py nothing for it. One good question is how the quite baroque introspection facilities in std.traits can be helped. Things like, give me the function parameters with types and modifiers and attributes and all that. A struct that can be manipulated during compilation containing all that information would be very valuable.

On 10/1/20 2:51 PM, H. S. Teoh wrote:
> Like conversion to string: in
> theory, since std.conv.to is able to convert just about any type to
> string, we ought to be able to support v.get!string for*any*  stored
> type.

Yah, that's quite problematic indeed.

On Thursday, 1 October 2020 at 18:55:17 UTC, Andrei Alexandrescu wrote:
>
> A struct that can be manipulated during compilation containing all that information would be very valuable.

Indeed, I considered and implemented that a few months ago.
Type functions can store types in structs. Right now.
I think I did demonstrate that somewhere ....
Structs holding types can never be used outside of CTFE of course.
Because a type is not given an ABI.

On 10/1/20 2:51 PM, H. S. Teoh wrote:
> On Thu, Oct 01, 2020 at 02:28:37PM -0400, Steven Schveighoffer via Digitalmars-d wrote:
>> On 10/1/20 2:19 PM, H. S. Teoh wrote:
> [...]
>>> We would like to be able to do this:
>>>
>>> 	Variant v;
>>> 	v = 100; // v now stores an int
>>> 	long l = v.get!long; // currently does not work
>>
>> I'm not going to respond to anything else here, but just want to point
>> out, this does work.
>>
>> https://run.dlang.io/is/S3sqDF
> 
> OK, I didn't check before I posted, :-/ but the point is that the
> implicit conversion is manually hacked with a library template that does
> not cover all the cases.  We should not have to reimplement implicit
> conversions in library code when the compiler is already perfectly
> capable of doing it.
> 
> It's not just implicit conversions, though.  There's a host of other
> things that we cannot do currently.  Like conversion to string: in
> theory, since std.conv.to is able to convert just about any type to
> string, we ought to be able to support v.get!string for *any* stored
> type.  Yet we cannot because at runtime all we have is typeid(T), so we
> cannot dispatch to the correct overload of std.conv.to to create the
> string.

It's not implicit conversions that we are talking about. It's implicit conversion *checks*. Actually doing the conversion requires calling a function, which so far has not been proposed (and I'm a little scared to see how big this beast is going to end up being).

And by the way, Variant.toString works... and calls the correct overload of to!string ;) https://github.com/dlang/phobos/blob/b0b64c3f41014addc6f5284f9eb20f8e90116ec2/std/variant.d#L438

"solving the Variant problem" needs a problem definition first.

-Steve

On 10/1/20 2:30 PM, Andrei Alexandrescu wrote:

> The mechanics of carrying the conversion are also needed, especially if a change in format is necessary (e.g. int to long or float to double). It looks like a pointer to function needs to be part of the solution.

Yes, Variant already does this. And it does this by a static unrolled loop over the actual types. Not the reified types.

So you'd have to build that mechanism into reification to "replace" Variant's usage of ImplicitConversionTargets. In other words, you can't just hoist this one piece into reification. You need to hoist ALL OF IT.

At some point, you are going to end up with a complete runtime reflection system.

> 
>> BTW, Variant already does a *limited* form of this. reification doesn't change what needs to happen.
>>
>> I don't see why Variant's needs have to dictate how you need to reason about types at compile time in CTFE.
> 
> The more applicability to difficult problems a mechanism has, the more useful it is. Looking good on cherry-picked examples is not enough. Difficult related problems that we have are introspection, std.meta, std.traits, and std.variant. The latter can be considered a problem of having the right primitives in std.traits.

Variant already has a solution. And it's tailored to Variant. I do not consider *at all* the problems Variant has to be related to a nicer compile time usage of types inside CTFE.

FWIW, type functions can replace ImplicitConversionTargets quite easily.

-Steve

On 10/1/20 3:01 PM, Steven Schveighoffer wrote:
> "solving the Variant problem" needs a problem definition first.

Problem definition: Implement Variant.get correctly. :o)

E.g. this produces a type error somewhere in the innards of std.variant: https://run.dlang.io/is/joIbeV. I'm not sure how many other cases are out there.

BTW thanks Steve for choosing the right angle. This is not a contest, and not a search for the proverbial nails for the use of a given hammer. The converse approach is best - find what the difficult related problems are, and figure how to solve them well.