Hi

Straight off, I should ask if Variant[Variant] is a feasable idea. I am writing a serialization module which is trying to make use of this.

This is the function

  Variant unserialize(V:Variant)(ref const(char)[] str)
  {
    switch (str[0])
    {
      case 'b':
        return Variant(unserialize!(bool)(str));
      case 'i':
        return Variant(unserialize!(long)(str));
      case 'f':
        return Variant(unserialize!(double)(str));
      case 's':
        return Variant(unserialize!(string)(str));
      case 'a':
        auto len = checkLengthTypeStart(str, 'a');

        Variant[] va;
        foreach (i; 0..len)
        {
          va ~= unserialize!(Variant)(str);
        }
        return Variant(va);
      case 'o':
        auto leno = checkLengthTypeStart(str, 'o');

        Variant[Variant] vaa;
        foreach (j; 0..leno)
        {
          auto v = unserialize!(Variant)(str);
          vaa[v] = unserialize!(Variant)(str);
        }
        return Variant(vaa);
      default:
        throw new Exception(format("Unknown serialize type '%c'",str[0]));
    }
  }

It seems to happily unserialize what I give into a Variant[Variant], but I can't get access to it. Here is the unittest code.

  const(char)[] tcd_ss = "a2o3s1:ai1s1:bi2s1:ci3o3s1:di4s1:ei5s1:fi6";
  auto tcd_v = unserialize!(Variant)(tcd_ss);
  assert(tcd_v.type() == typeid(Variant[]));
  assert(tcd_v.length == 2);
  auto tcd1_v = tcd_v[0];
  assert(tcd1_v.length == 3);
  assert(tcd1_v.type() == typeid(Variant[Variant]));
  auto va = Variant("a");
  auto tcd1a_v = tcd1_v[va];  // <======= Blows up here.

By my reckoning, the key Variant("a") exists in tcd1_v. If not, shouldn't it give a "Range Violation" error? Instead it gives:

std.variant.VariantException@std/variant.d(1231): Variant: attempting to use incompatible types immutable(char)[] and std.variant.VariantN!(24u).VariantN
----------------
./serialize(@trusted int std.variant.VariantN!(24u).VariantN.handler!(std.variant.VariantN!(24u).VariantN[std.variant.VariantN!(24u).VariantN]).handler(std.variant.VariantN!(24u).VariantN.OpID, ubyte[24]*, void*)+0x286) [0x811cd2a]
./serialize(@trusted std.variant.VariantN!(24u).VariantN std.variant.VariantN!(24u).VariantN.opIndex!(std.variant.VariantN!(24u).VariantN).opIndex(std.variant.VariantN!(24u).VariantN)+0x68) [0x811e654]
./serialize(void serialize.__unittestL257_1()+0x1839) [0x80d296d]
./serialize(void serialize.__modtest()+0x8) [0x8120b84]
./serialize(extern (C) bool core.runtime.runModuleUnitTests().int __foreachbody352(ref object.ModuleInfo*)+0x24) [0x812faac]
./serialize(int rt.minfo.moduleinfos_apply(scope int delegate(ref object.ModuleInfo*)).int __foreachbody541(ref rt.sections_linux.DSO)+0x37) [0x8128057]
./serialize(int rt.sections_linux.DSO.opApply(scope int delegate(ref rt.sections_linux.DSO))+0x2c) [0x81282ac]
./serialize(int rt.minfo.moduleinfos_apply(scope int delegate(ref object.ModuleInfo*))+0x14) [0x8128004]
./serialize(runModuleUnitTests+0x87) [0x812f9b7]
./serialize(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void runAll()+0x25) [0x81258e5]
./serialize(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void tryExec(scope void delegate())+0x18) [0x8125550]
./serialize(_d_run_main+0x121) [0x8125521]
./serialize(main+0x14) [0x81253f4]
/lib/libc.so.6(__libc_start_main+0xf3) [0x429a5413]

So I ask again, is Variant[Variant] too ambitious? If not ... help!

Thanks in advance
Regards
Jason

P.S. The full code can be found at http://jaypha.com.au/serialize.d if that helps.

On 08/24/2013 04:12 AM, Jason den Dulk wrote:
> Hi
>
> Straight off, I should ask if Variant[Variant] is a feasable idea.

It looks like it:

import std.variant;

void main()
{
    Variant[Variant] aa;
    aa[Variant(42)] = Variant("forty two");

    assert(Variant(42) in aa);
    assert(aa == [ Variant(42) : Variant("forty two") ]);
}

> This is the function

I spent 15 minutes trying to complete the code but failed. Could you please provide a minimal program that almost compiles.

Ali

Hi Ali, thanks for helping me out.

On Tuesday, 27 August 2013 at 01:38:54 UTC, Ali Çehreli wrote:
> I spent 15 minutes trying to complete the code but failed. Could you please provide a minimal program that almost compiles.

I uploaded a new version with a main routine. So you know, I am using DMD 2.063.2 on Fedora 15.

The code should compile with "dmd serialize.d".

I since realized that I was not dealing with a Variant[Variant] but a Variant(Variant[Variant]). It didn't seem to like having "[Variant]" called upon it.

After extracting it to a V[V] (called tcd1_vn), I noticed something else.

Doing a foreach on it,

  foreach (_a_, _b_;tcd1_vn)
  {
    writeln(_a_.type(),",",_b_.type());
    writeln(_a_.get!(string),",",_b_.get!(long));
  }

gives this:

  immutable(char)[],long
  a,1
  immutable(char)[],long
  b,2
  immutable(char)[],long
  c,3

So it appears that Variant("a") is a key, but

  assert(Variant("a") in tcd1_vn);

fails. Any ideas?

Thanks again for helping. I have been reading your online book. I have found it quite helpful.

Regards
Jason

On 08/27/2013 03:53 AM, Jason den Dulk wrote:

> I uploaded a new version with a main routine. So you know, I am using
> DMD 2.063.2 on Fedora 15.
>
> The code should compile with "dmd serialize.d".

And it's here:

  http://jaypha.com.au/serialize.d

> I since realized that I was not dealing with a Variant[Variant] but a
> Variant(Variant[Variant]). It didn't seem to like having "[Variant]"
> called upon it.
>
> After extracting it to a V[V] (called tcd1_vn), I noticed something else.
>
> Doing a foreach on it,
>
>    foreach (_a_, _b_;tcd1_vn)
>    {
>      writeln(_a_.type(),",",_b_.type());
>      writeln(_a_.get!(string),",",_b_.get!(long));
>    }
>
> gives this:
>
>    immutable(char)[],long
>    a,1
>    immutable(char)[],long
>    b,2
>    immutable(char)[],long
>    c,3
>
> So it appears that Variant("a") is a key, but
>
>    assert(Variant("a") in tcd1_vn);
>
> fails. Any ideas?

It is definitely a bug. I spent a lot of time on this and figured it out just before giving up. :) (Actually I figured it out *after* giving up. :p)

The signature of VariantN.toHash() does not match D's expectation so it is not considered at all. As a result, the default toHash for structs gets called, which happens to hash by the members of the struct. The problem is, being a variant type, the members of VariantN are a function pointer and the storage:

struct VariantN(size_t maxDataSize, AllowedTypesX...)
{
// ...
    ptrdiff_t function(OpID selector, ubyte[size]* store, void* data) fptr
        = &handler!(void);
    union
    {
        ubyte[size] store;
        // conservatively mark the region as pointers
        static if (size >= (void*).sizeof)
            void* p[size / (void*).sizeof];
    }
// ...
}

Since 'store' is just a ubyte[] array, the default toHash for structs cannot hash it as strings.

VariantN.toHash's signature must be changed accordingly and Phobos must be recompiled:

    // WARNING: Unnecessarily slow because type.getHash() is not nothrow.
    size_t toHash() const nothrow @safe
    {
        try {
            return type.getHash(&store);

        } catch (Exception) {
            return 0;
        }
    }

The original toHash was this:

    size_t toHash()
    {
        return type.getHash(&store);
    }

The other issue is, the compiler did not warn me until I added a const to the signature. Try to compile this:

    size_t toHash() const
    {
        return type.getHash(&store);
    }

./phobos/std/variant.d(822): Warning: toHash() must be declared as extern (D) size_t toHash() const nothrow @safe, not const @trusted ulong()

The same warning should be given to the naked signature 'size_t toHash()' as well. (I remember bearophile warning about such problems.)

So, VariantN is broken. It cannot be used as an AA key correctly, at least when it stores a string. To make matters worse, testing with Variants that are initialized by literal strings works because the compiler optimizes the same literal strings.

> Thanks again for helping. I have been reading your online book. I have
> found it quite helpful.

You are very kind. :) However, as this issue proves, the book is outdated in some parts. At least there is a note for me to update the toHash signatures in the book: :)


https://code.google.com/p/ddili/source/browse/trunk/src/ders/d.en/object.d#681

So, I will get to this issue eventually. :-/

May I ask you or somebody else to create a bug about VariantN.toHash not being considered at all. Thank you! :)

Ali

On Tuesday, 27 August 2013 at 18:32:46 UTC, Ali Çehreli wrote:

> May I ask you or somebody else to create a bug about VariantN.toHash not being considered at all. Thank you! :)

I have filed a bug report for this, and I have placed a work around in my code, so that's it for now, I suppose. Thanks again.