Hi all,

I finally found some time to work on TypeInfoEx. It's usable, in that you can view information about a type; but you can't do anything with that information. One large barrier is the lack of a usable Variant type.

To invoke a function on an object or struct, you need to pass it as either Object or void* or Variant.

If you invoke a function, you want to pass in an array of Variant for the arguments. You want to get the return type as a Variant.

If you get a field, you want to get it as a Variant.

You could use templates instead, but that's not usable in many situations, so I won't have that as the only option.

Andrei's std.variant has an interesting design, but it has two flaws that render it unusable:
 - It cannot be used with structs of arbitrary size. It's parameterized on size, but no size will always be sufficient; and due to its allocation scheme, choosing to support (for instance) 1KB structs will force a bool to take 1KB.
 - It cannot be created with RTTI; it requires compile-time type information.

So, my options are:
 - Write my own Variant supporting the necessary operations. This makes things annoying for anyone wanting to use my code with Phobos, since now there are two Variant structs running around.
 - Use std.boxer, which is deprecated.
 - Pass around an opaque Pair!(TypeInfo, void*).

Any suggestions? I've written such a Variant for D1 and Tango, but I relish neither the idea of supporting both with the same class, nor code duplication.

Christopher Wright wrote:
> Hi all,
> 
> I finally found some time to work on TypeInfoEx. It's usable, in that you can view information about a type; but you can't do anything with that information. One large barrier is the lack of a usable Variant type.
> 
> To invoke a function on an object or struct, you need to pass it as either Object or void* or Variant.
> 
> If you invoke a function, you want to pass in an array of Variant for the arguments. You want to get the return type as a Variant.
> 
> If you get a field, you want to get it as a Variant.
> 
> You could use templates instead, but that's not usable in many situations, so I won't have that as the only option.
> 
> Andrei's std.variant has an interesting design, but it has two flaws
> that render it unusable:
>  - It cannot be used with structs of arbitrary size. It's parameterized
> on size, but no size will always be sufficient; and due to its
> allocation scheme, choosing to support (for instance) 1KB structs will
> force a bool to take 1KB.
>  - It cannot be created with RTTI; it requires compile-time type
> information.
> 
> So, my options are:
>  - Write my own Variant supporting the necessary operations. This makes
> things annoying for anyone wanting to use my code with Phobos, since now
> there are two Variant structs running around.
>  - Use std.boxer, which is deprecated.
>  - Pass around an opaque Pair!(TypeInfo, void*).
> 
> Any suggestions? I've written such a Variant for D1 and Tango, but I relish neither the idea of supporting both with the same class, nor code duplication.

Use the Tango Variant code.  It supports values of any size and doesn't require knowledge of the compile-time type.  It's in tango.core so the dependencies are relatively small, so porting should be fairly simple (the code originally compiled under both Phobos and Tango, but that version's out of date now.)

If you look at http://dsource.org/projects/tango/ticket/821 there's a patch to the current Variant that adds support for varargs as well.

  -- Daniel

== Quote from Christopher Wright (dhasenan@gmail.com)'s article
> Hi all,
> I finally found some time to work on TypeInfoEx. It's usable, in that
> you can view information about a type; but you can't do anything with
> that information. One large barrier is the lack of a usable Variant type.
> To invoke a function on an object or struct, you need to pass it as
> either Object or void* or Variant.
> If you invoke a function, you want to pass in an array of Variant for
> the arguments. You want to get the return type as a Variant.
> If you get a field, you want to get it as a Variant.
> You could use templates instead, but that's not usable in many
> situations, so I won't have that as the only option.
> Andrei's std.variant has an interesting design, but it has two flaws
> that render it unusable:
>   - It cannot be used with structs of arbitrary size. It's parameterized
> on size, but no size will always be sufficient; and due to its
> allocation scheme, choosing to support (for instance) 1KB structs will
> force a bool to take 1KB.
>   - It cannot be created with RTTI; it requires compile-time type
> information.
> So, my options are:
>   - Write my own Variant supporting the necessary operations. This makes
> things annoying for anyone wanting to use my code with Phobos, since now
> there are two Variant structs running around.
>   - Use std.boxer, which is deprecated.
>   - Pass around an opaque Pair!(TypeInfo, void*).
> Any suggestions? I've written such a Variant for D1 and Tango, but I
> relish neither the idea of supporting both with the same class, nor code
> duplication.

Andrei has said that he would like to fix std.variant to support arbitrary sized stuff, but just doesn't have time right now.  If you are able and willing, submit a patch.  If it's decent, it will probably be accepted.  If you don't have time or don't want to, someone who does should submit one since this seems to be something a lot of people want.  I've looked at doing this before, and I'd be willing to step up and submit a patch after some of my coursework dies down and I have more free time (fairly soon), if noone else gets around to it before then.

Daniel Keep wrote:
> Use the Tango Variant code.  It supports values of any size and doesn't
> require knowledge of the compile-time type.

Not so. That was one of three reasons that I wrote my own Variant previously (the others being toString and sameness comparison).

Christopher Wright wrote:
> Daniel Keep wrote:
>> Use the Tango Variant code.  It supports values of any size and doesn't require knowledge of the compile-time type.
> 
> Not so. That was one of three reasons that I wrote my own Variant previously (the others being toString and sameness comparison).

What not so?  I know for a fact it supports values of any size, so I assume you're referring to compile-time.  With the new patch you can pack and unpack Variants from TypeInfo and a void* so you DON'T need to know the type ahead of time.  There isn't a way to construct from a single TypeInfo,void* pair since I wasn't sure if that would actually be needed.

toString is a design issue, not a technical one; they didn't want stuff in tango.core depending on stuff outside of that.  If you look at the history of Variant, there's an old version with "proper" toString code that should port forward painlessly.

On a side note, I'm working on getting Tango's Layout to understand Variants and do the "proper" stringification.  Still trying to work out how to get the changes to compile without having to rebuild Tango... *grumble*

As for comparisons, it supports opEqual and opCmp between Variants, it just doesn't attempt to do any automatic conversion between non-identical but comparable types (like comparing ints to longs.)

Is there something specific you think Tango's Variant should do that it doesn't?

  -- Daniel

dsimcha wrote:
> == Quote from Christopher Wright (dhasenan@gmail.com)'s article
>> Hi all,
>> I finally found some time to work on TypeInfoEx. It's usable, in that
>> you can view information about a type; but you can't do anything with
>> that information. One large barrier is the lack of a usable Variant type.
>> To invoke a function on an object or struct, you need to pass it as
>> either Object or void* or Variant.
>> If you invoke a function, you want to pass in an array of Variant for
>> the arguments. You want to get the return type as a Variant.
>> If you get a field, you want to get it as a Variant.
>> You could use templates instead, but that's not usable in many
>> situations, so I won't have that as the only option.
>> Andrei's std.variant has an interesting design, but it has two flaws
>> that render it unusable:
>>   - It cannot be used with structs of arbitrary size. It's parameterized
>> on size, but no size will always be sufficient; and due to its
>> allocation scheme, choosing to support (for instance) 1KB structs will
>> force a bool to take 1KB.
>>   - It cannot be created with RTTI; it requires compile-time type
>> information.
>> So, my options are:
>>   - Write my own Variant supporting the necessary operations. This makes
>> things annoying for anyone wanting to use my code with Phobos, since now
>> there are two Variant structs running around.
>>   - Use std.boxer, which is deprecated.
>>   - Pass around an opaque Pair!(TypeInfo, void*).
>> Any suggestions? I've written such a Variant for D1 and Tango, but I
>> relish neither the idea of supporting both with the same class, nor code
>> duplication.
> 
> Andrei has said that he would like to fix std.variant to support arbitrary sized
> stuff, but just doesn't have time right now.  If you are able and willing, submit
> a patch.  If it's decent, it will probably be accepted.  If you don't have time or
> don't want to, someone who does should submit one since this seems to be something
> a lot of people want.  I've looked at doing this before, and I'd be willing to
> step up and submit a patch after some of my coursework dies down and I have more
> free time (fairly soon), if noone else gets around to it before then.

It would be great if somebody could find the time to implement the pesky arbitrarily-sized structs for Variant.

I don't have much time to add to this interesting discussion, but let me make a few quick points. First off, I'm very happy that discriminated unions are receiving increased interest from the community. I think they are an interesting abstraction mechanism with cool applications in dynamic code.

Second, I have hacked discriminated unions for years and wrote a fairly influential paper on them as far back as 2001 (which has inspired many others, including Boost's), and some more articles later on. I can say that std.variant is the best implementation of a discriminated I've seen, by a mile. std.variant is truly general (at no point in its design or implementation appears a list of supported types), well positioned for working with full introspection when that will become available, tight in memory usage, and extremely fast because it uses only one indirect call for dispatching. The latter point is very important in heavy-duty usage; hash-based dispatch sounds and looks good on paper, until you need to invoke it in a tight loop. Then you'd start trying to avoid it, which undermines motivation for using variants.

Lack of construction from RTTI is not a major limitation and can be obviated. RTTI objects cannot be forged, so they were created from some objects that did have static type information available. This suggests that code can be shuffled such that Variant construction is done at that point, instead of getting the RTTI alone (and losing static type information) and attempting to construct the Variant later.


Andrei

On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep <daniel.keep.lists@gmail.com> wrote:

> On a side note, I'm working on getting Tango's Layout to understand
> Variants and do the "proper" stringification.  Still trying to work out
> how to get the changes to compile without having to rebuild Tango...
> *grumble*


Have you tried building your test app with your modified Tango file specifically?  That generally works for me:

dmd variantTest.d ~/d/include/d/tango/core/Variant.d

-Steve

Steven Schveighoffer wrote:
> On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep <daniel.keep.lists@gmail.com> wrote:
> 
>> On a side note, I'm working on getting Tango's Layout to understand Variants and do the "proper" stringification.  Still trying to work out how to get the changes to compile without having to rebuild Tango... *grumble*
> 
> 
> Have you tried building your test app with your modified Tango file specifically?  That generally works for me:
> 
> dmd variantTest.d ~/d/include/d/tango/core/Variant.d
> 
> -Steve

That works for Variant, but it doesn't seem to work with Layout.  It compiles and runs, but it doesn't seem to actually use the code I give it.

  -- Daniel

On Fri, 17 Apr 2009 07:11:33 -0400, Daniel Keep <daniel.keep.lists@gmail.com> wrote:

>
>
> Steven Schveighoffer wrote:
>> On Thu, 16 Apr 2009 21:37:29 -0400, Daniel Keep
>> <daniel.keep.lists@gmail.com> wrote:
>>
>>> On a side note, I'm working on getting Tango's Layout to understand
>>> Variants and do the "proper" stringification.  Still trying to work out
>>> how to get the changes to compile without having to rebuild Tango...
>>> *grumble*
>>
>>
>> Have you tried building your test app with your modified Tango file
>> specifically?  That generally works for me:
>>
>> dmd variantTest.d ~/d/include/d/tango/core/Variant.d
>>
>> -Steve
>
> That works for Variant, but it doesn't seem to work with Layout.  It
> compiles and runs, but it doesn't seem to actually use the code I give it.

I think I ran into this before.  I think it's because Layout is a template, and is compiled when it is used, not when the module is compiled.  You have to recompile the user of Layout as well (i.e. Stdout.d, and probably Print.d).

-Steve

Daniel Keep wrote:
> 
> Christopher Wright wrote:
>> Daniel Keep wrote:
>>> Use the Tango Variant code.  It supports values of any size and doesn't
>>> require knowledge of the compile-time type.
>> Not so. That was one of three reasons that I wrote my own Variant
>> previously (the others being toString and sameness comparison).
> 
> What not so?  I know for a fact it supports values of any size, so I
> assume you're referring to compile-time.  With the new patch you can
> pack and unpack Variants from TypeInfo and a void* so you DON'T need to
> know the type ahead of time.  There isn't a way to construct from a
> single TypeInfo,void* pair since I wasn't sure if that would actually be
> needed.

Okay, you must have done that very recently, since I can't find it in my working copy.

> toString is a design issue, not a technical one; they didn't want stuff
> in tango.core depending on stuff outside of that.  If you look at the
> history of Variant, there's an old version with "proper" toString code
> that should port forward painlessly.

Yes, I'm familiar with that issue. I'm not faulting Tango for lacking a proper toString, but it was a problem, one that I couldn't work around with Tango's Variant, so I did what I had to do.

> On a side note, I'm working on getting Tango's Layout to understand
> Variants and do the "proper" stringification.  Still trying to work out
> how to get the changes to compile without having to rebuild Tango...
> *grumble*
> 
> As for comparisons, it supports opEqual and opCmp between Variants, it
> just doesn't attempt to do any automatic conversion between
> non-identical but comparable types (like comparing ints to longs.)

Ah. Mine does that -- it stores all floating point types as reals, for instance, and all integer types as longs (except for ulongs).

> Is there something specific you think Tango's Variant should do that it
> doesn't?

Apparently, nothing that won't be in the next release, aside from bitwise comparisons.

>   -- Daniel