Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...).

Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain?

On 08/19/2018 08:27 PM, Nick Sabalausky (Abscissa) wrote:
> Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...).
> 
> Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain?

Maybe something involving using Variant.coerce to convert the payload to a single common type? Not sure how I would do that though.

On Sunday, August 19, 2018 6:33:06 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote:
> On 08/19/2018 08:27 PM, Nick Sabalausky (Abscissa) wrote:
> > Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...).
> >
> > Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain?
>
> Maybe something involving using Variant.coerce to convert the payload to a single common type? Not sure how I would do that though.

You could always create a wrapper type. Whatever code you're dealing with is going to need to statically know what type it's using even that's a base type in a class hierarchy. So, you need to present a type which actually has the appropriate API even if internally, it can dynamically handle several types which have that API, and it's not known ahead of time which type that is.

- Jonathan M Davis

On 08/19/2018 10:23 PM, Jonathan M Davis wrote:
> On Sunday, August 19, 2018 6:33:06 PM MDT Nick Sabalausky (Abscissa) via
> Digitalmars-d-learn wrote:
>>
>> Maybe something involving using Variant.coerce to convert the payload to
>> a single common type? Not sure how I would do that though.
> 
> You could always create a wrapper type. Whatever code you're dealing with is
> going to need to statically know what type it's using even that's a base
> type in a class hierarchy. So, you need to present a type which actually has
> the appropriate API even if internally, it can dynamically handle several
> types which have that API, and it's not known ahead of time which type that
> is.
> 

I guess the parts I'm unclear on are:

1. What mechanism does Variant.coerce!SomeType use to attempt conversion to SomeType?

2. How (if possible) can I provide a way to convert something to type SomeType which Variant.coerce!SomeType will then recognize and use?

On Monday, 20 August 2018 at 00:27:04 UTC, Nick Sabalausky (Abscissa) wrote:
> Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...).
>
> Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain?

You are basically reinventing OOP here.

Instead of Variants, use objects that implement an interface (e.g., `std.range.interfaces.InputRange`). Then you can call methods that belong to that interface and rely on virtual method dispatch to choose the correct implementation at runtime.

On Sunday, August 19, 2018 9:08:39 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote:
> On 08/19/2018 10:23 PM, Jonathan M Davis wrote:
> > On Sunday, August 19, 2018 6:33:06 PM MDT Nick Sabalausky (Abscissa) via
> >
> > Digitalmars-d-learn wrote:
> >> Maybe something involving using Variant.coerce to convert the payload
> >> to
> >> a single common type? Not sure how I would do that though.
> >
> > You could always create a wrapper type. Whatever code you're dealing with is going to need to statically know what type it's using even that's a base type in a class hierarchy. So, you need to present a type which actually has the appropriate API even if internally, it can dynamically handle several types which have that API, and it's not known ahead of time which type that is.
>
> I guess the parts I'm unclear on are:
>
> 1. What mechanism does Variant.coerce!SomeType use to attempt conversion to SomeType?
>
> 2. How (if possible) can I provide a way to convert something to type SomeType which Variant.coerce!SomeType will then recognize and use?

Glancing at coerce's implementation, it just uses std.conv.to, but it's fairly restricted on which types it will convert to. The target type has to be numeric, bool, convertible to Object, or an array of characters. It looks like it will convert the object to string to do the conversion, so it's doing slightly more than just using std.conv.to and restricting the conversions, but really, it's basically just calling std.conv.to in a restricted manner.

So, looking at coerce, I don't really understand why it exists. It just seems like it would make more sense to tell folks to get the object out with get and then convert it themselves using std.conv.to or whatever makes the most sense for their use case. Presenting a function that uses std.conv.to but restricts which conversions work really doesn't make sense to me.

Either way, if you're doing something like using a Variant to hold multiple range types, I very much doubt that coerce is going to do you much good.

- Jonathan M Davis

On 08/19/2018 11:31 PM, Paul Backus wrote:
> 
> You are basically reinventing OOP here.
> 

Yes, I am. Deliberately, in fact. Class inheritance is my backup plan, though.

My use-case is actually very, very similar to std.digest:

There are various benefits to using a template-and-constraint based interface. But one notable downside is the inability to handle the situation where the actual type needed is only known at runtime. In std.digest, this dilemma is dealt with by duplicating the entire interface in BOTH template/constraint AND class-inheritance varieties.

Additionally, an important thing to note in the case of std.digest is that `isDigest!T` returns FALSE for types using the OOP version of the interface. I think there is (arguably) a certain merit in that:

The OOP interface obviously uses classes, whereas (in order to obtain the full potential benefits of a template-and-constraint approach) the template interface uses (and even requires) structs. Because of this, instances should be constructed differently ("T instance;" vs "T instance = new T();"). Thus, any generic code that wants to handle ANY digest type must be very careful to mind this distinction whenever creating new instances. Presumably, this may be why std.digest chose to NOT let OO digests satisfy the template-oriented isDigest.

And then std.digest also needs WrapperDigest, needed to convert a template-style digest to OO-style.

So...while std.digest succeeds at offering the best-of-both-worlds between template and OO approaches, it also creates a whole new mess via completely duplicated interfaces that aren't 100% compatible.

I want to see if I can do better.

So back to the root of the problem:

We have a bunch of types, including user-defined types we don't have advance knowledge of. And we need a type which can hold any of them at runtime.

AFAIK, there are two classic ways to do that: One is OOP, the other is an unbounded discriminated union (a variant). Unlike class-based inheritance, a variant CAN be implemented as a struct. So, at least in theory, a variant-based type could potentially be made which implements the *true* template-based interface, eliminating the need for duplicate APIs and the mess that entails.

There are a bunch of discriminated union types available for D, but the only one I'm aware of that *doesn't* require a finite-sized list of types known ahead-of-time is Phobos's Variant. The only problem is calling a function on a type not already known ahead-of-time. Maybe that's unsolvable? If so, then I'll fallback to the std.digest approach. But I'd prefer to avoid that, if possible.

On Monday, August 20, 2018 1:38:13 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote:
> There are a bunch of discriminated union types available for D, but the only one I'm aware of that *doesn't* require a finite-sized list of types known ahead-of-time is Phobos's Variant. The only problem is calling a function on a type not already known ahead-of-time. Maybe that's unsolvable? If so, then I'll fallback to the std.digest approach. But I'd prefer to avoid that, if possible.

It can be done if you know the list of types ahead of time, but you basically have to check which type it is at runtime and then pick which code to run based on that type, which means that you're essentially duplicating a portion of the code for every type - though depending, it would be possible to do something like templatize the code so that you don't have to explicitly dulpicate it. But in the end, aside from using classes, if you're using a variant type, I think that you're stuck doing something like

// exact API made up here rather than bothering to take the type to look up
// Variant's API.
void popFront()
{
    foreach(T; TypesThatVariantHolds)
    {
        if(_variant.contains!T)
        {
            auto v = _variant.get!T;
            v.popFront();
            return;
        }
    }
}

On the other hand, if you don't know the exact list of types ahead of time or require that the code calling this code tell you which type to use, then you're pretty much stuck using something like classes or delegates. You can't just call functions on completely unknown types, because the compiler wouldn't know what code to generate or what to link against.

- Jonathan M Davis

On 08/20/2018 04:34 PM, Jonathan M Davis wrote:
> 
>      foreach(T; TypesThatVariantHolds)

Yea, that's what I would've just done, but I wanted to support user-created types not already known to my library.

> You
> can't just call functions on completely unknown types, because the compiler
> wouldn't know what code to generate or what to link against.
> 

Right. D's templates have really spoiled me. Because of them, I've grown accustomed to invoking functions on unknown types ;)

But you're right, that's not going to work for runtime dispatch without enumerating each possible type. Even if I try (like I was thinking) to provide a templated function to convert any of the concrete types to an internal-only supertype, I'd still need something to runtime dispatch to the right template instance. Darn.

A lot of times, I really wish I could introspect across *all* linked modules, not just specific ones. Ex: "Hey compiler, give me a compile-time list of ALL types in this program with UDA @xyz". Then things like this could be built out of that. No need to worry about unknown types/symbols because they would all be known and a finite list could always be constructed.

Although, what would REALLY be nice is if, for every type, the compiler built a list of every member and how to access it (which...really it has to do anyway), but then uses that info to build a master runtime dispatch system. Types such as Variant already have runtime knowledge of what type is currently being held, so that info could be passed to the compiler-generated "master runtime dispatch" system along with what member to invoke/access.

Come to think of it...aren't I just describing standard run-of-the-mill runtime reflection? I mean, hasn't Java already been able to do that for ages? But then, in Java everything is already part of the class hierarchy anyway so maybe that's why it's possible there?

On Monday, August 20, 2018 8:29:30 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote:
> On 08/20/2018 04:34 PM, Jonathan M Davis wrote:
> >      foreach(T; TypesThatVariantHolds)
>
> Yea, that's what I would've just done, but I wanted to support user-created types not already known to my library.
>
> > You
> > can't just call functions on completely unknown types, because the
> > compiler wouldn't know what code to generate or what to link against.
>
> Right. D's templates have really spoiled me. Because of them, I've grown accustomed to invoking functions on unknown types ;)
>
> But you're right, that's not going to work for runtime dispatch without enumerating each possible type. Even if I try (like I was thinking) to provide a templated function to convert any of the concrete types to an internal-only supertype, I'd still need something to runtime dispatch to the right template instance. Darn.
>
> A lot of times, I really wish I could introspect across *all* linked modules, not just specific ones. Ex: "Hey compiler, give me a compile-time list of ALL types in this program with UDA @xyz". Then things like this could be built out of that. No need to worry about unknown types/symbols because they would all be known and a finite list could always be constructed.
>
> Although, what would REALLY be nice is if, for every type, the compiler built a list of every member and how to access it (which...really it has to do anyway), but then uses that info to build a master runtime dispatch system. Types such as Variant already have runtime knowledge of what type is currently being held, so that info could be passed to the compiler-generated "master runtime dispatch" system along with what member to invoke/access.
>
> Come to think of it...aren't I just describing standard run-of-the-mill runtime reflection? I mean, hasn't Java already been able to do that for ages? But then, in Java everything is already part of the class hierarchy anyway so maybe that's why it's possible there?

Runtime reflection is theoretically possible in D, but it requires generating the appropriate stuff for every type involved so that the runtime stuff has something to work with. Java built all of that into the language and has the JVM to boot, which fundamentally changes some of what can be done. With D, we're basically in exactly the same boat as C or C++ except that we have better compile-time type introspection. In principle, a runtime reflection facility could be built using that, but even if it were part of Phobos, it would still have to be opt-in. So, I don't know how useful such a solution would ever be outside of very specific use cases. Regardless, given that D's object files, linking, etc. are using the C tools, it was never going to be the case that Java-style runtime reflection would be built in to D like it is with Java.

- Jonathan M Davis