Over the weekend, I attacked opDispatch again and found some old Variant bugs were killed. I talked about that in the Who uses D thread.

Today, I couldn't resist revisiting a dynamic kind of object, and made some decent progress on it.

http://arsdnet.net/dcode/dynamic.d

(You can compile that; there's a main() at the bottom of that file)

It isn't quite done - still needs op overloading, and probably better errors, but it basically works.

It works sort of like a Javascript object.

Features:

opDispatch and assignment functions:
Dynamic obj;

// assign from various types
obj = 10;
obj = "string";

obj.a = 10; // assign properties from simple types naturally

// can set complex types with one compromise: the () after the
// property tells it you want opAssign instead of property opDispatch
obj.a() = { writefln("hello, world"); }

// part two of the compromise - to call it with zero args, use call:
obj.a.call();

// delegte with arguments works too
obj.a() = delegate void(string a) { writeln(a); };

// Calling with arguments works normally
obj.a("some arguments", 30);


Those are just the basics. What about calling a D function? You need to convert them back to regular types:

string mystring = obj.a.as!string; // as forwards to Variant.coerce
    // to emulate weak typing

Basic types are great, but what about more advanced types? So far, I've implemented interfaces:

interface Cool {
    void a();
}

void takesACool(Cool thing) { thing.a(); }



takesACool(obj.as!Cool); // it creates a temporary class implementing // the interface by forwarding all its methods to the dynamic obj



I can make it work with structs too but haven't written that yet.
I want to add some kind of Javascript like prototype inheritance too.



I just thought it was getting kinda cool so I'd share it :)

On Mon, 3 Jan 2011 22:23:29 +0000 (UTC)
Adam Ruppe <destructionator@gmail.com> wrote:

> Over the weekend, I attacked opDispatch again and found some old Variant bugs were killed. I talked about that in the Who uses D thread.
> 
> Today, I couldn't resist revisiting a dynamic kind of object, and made some decent progress on it.
> 
> http://arsdnet.net/dcode/dynamic.d
> 
> (You can compile that; there's a main() at the bottom of that file)
> 
> It isn't quite done - still needs op overloading, and probably better errors, but it basically works.
> 
> It works sort of like a Javascript object.
> 
> Features:
> 
> opDispatch and assignment functions:
> Dynamic obj;
> 
> // assign from various types
> obj = 10;
> obj = "string";
> 
> obj.a = 10; // assign properties from simple types naturally
> 
> // can set complex types with one compromise: the () after the
> // property tells it you want opAssign instead of property opDispatch
> obj.a() = { writefln("hello, world"); }
> 
> // part two of the compromise - to call it with zero args, use call:
> obj.a.call();
> 
> // delegte with arguments works too
> obj.a() = delegate void(string a) { writeln(a); };
> 
> // Calling with arguments works normally
> obj.a("some arguments", 30);
> 
> 
> Those are just the basics. What about calling a D function? You need to convert them back to regular types:
> 
> string mystring = obj.a.as!string; // as forwards to Variant.coerce
>     // to emulate weak typing
> 
> Basic types are great, but what about more advanced types? So far, I've implemented interfaces:
> 
> interface Cool {
>     void a();
> }
> 
> void takesACool(Cool thing) { thing.a(); }
> 
> 
> 
> takesACool(obj.as!Cool); // it creates a temporary class implementing // the interface by forwarding all its methods to the dynamic obj
> 
> 
> 
> I can make it work with structs too but haven't written that yet.
> I want to add some kind of Javascript like prototype inheritance too.
> 
> 
> 
> I just thought it was getting kinda cool so I'd share it :)


Waow, quite cool, indeed! I'll have a look at your code as soon as I can. Esp
	obj.a() = { writefln("hello, world"); }
is a real mystery for me ;-)

I have a toy language project in mind (for years, already) that looks somewhat similar to your dynamic D. Main difference is that (unlike prototype-based ones like JS, Io, etc) inheritance is not done via delegating but by plain copy of (references to) methods. This better maps my views and also greatly simplifies the model --in particuliar, no method lookup chain issue: a method is either on the object or on its type. Also, there would be some kind of type, again unlike true prototype-based, but they would really be true objects, like when writing a simple OO typing system in Lua.

Denis
-- -- -- -- -- -- --
vit esse estrany ☣

spir.wikidot.com

On Mon, 03 Jan 2011 22:19:49 -0500, spir <denis.spir@gmail.com> wrote:

> On Mon, 3 Jan 2011 22:23:29 +0000 (UTC)
> Adam Ruppe <destructionator@gmail.com> wrote:
>
>> Over the weekend, I attacked opDispatch again and found some old
>> Variant bugs were killed. I talked about that in the Who uses D
>> thread.
>>
>> Today, I couldn't resist revisiting a dynamic kind of object, and
>> made some decent progress on it.
>>
>> http://arsdnet.net/dcode/dynamic.d
>>
>> (You can compile that; there's a main() at the bottom of that file)
>>
>> It isn't quite done - still needs op overloading, and probably better
>> errors, but it basically works.
>>
>> It works sort of like a Javascript object.
>>
>> Features:
>>
>> opDispatch and assignment functions:
>> Dynamic obj;
>>
>> // assign from various types
>> obj = 10;
>> obj = "string";
>>
>> obj.a = 10; // assign properties from simple types naturally
>>
>> // can set complex types with one compromise: the () after the
>> // property tells it you want opAssign instead of property opDispatch
>> obj.a() = { writefln("hello, world"); }
>>
>> // part two of the compromise - to call it with zero args, use call:
>> obj.a.call();
>>
>> // delegte with arguments works too
>> obj.a() = delegate void(string a) { writeln(a); };
>>
>> // Calling with arguments works normally
>> obj.a("some arguments", 30);
>>
>>
>> Those are just the basics. What about calling a D function? You
>> need to convert them back to regular types:
>>
>> string mystring = obj.a.as!string; // as forwards to Variant.coerce
>>     // to emulate weak typing
>>
>> Basic types are great, but what about more advanced types? So far,
>> I've implemented interfaces:
>>
>> interface Cool {
>>     void a();
>> }
>>
>> void takesACool(Cool thing) { thing.a(); }
>>
>>
>>
>> takesACool(obj.as!Cool); // it creates a temporary class implementing
>> // the interface by forwarding all its methods to the dynamic obj
>>
>>
>>
>> I can make it work with structs too but haven't written that yet.
>> I want to add some kind of Javascript like prototype inheritance too.
>>
>>
>>
>> I just thought it was getting kinda cool so I'd share it :)
>
>
> Waow, quite cool, indeed! I'll have a look at your code as soon as I can. Esp
> 	obj.a() = { writefln("hello, world"); }
> is a real mystery for me ;-)

Well, obj.a() returns by ref, which is how that works. The expression itself is a zero arg delegate literal.

spir wrote:
> obj.a() = { writefln("hello, world"); }
> is a real mystery for me ;-)

Yea, like Robert said, what happens is the opDispatch method (which turns the part after the string into a template parameter instead of a compile error, see: http://digitalmars.com/d/2.0/operatoroverloading.html#Dispatch for more info) just returns a reference to the property.

Each property has the same type as everything else - struct Dynamic.
It's opAssign takes arbitrary types, wraps them if necessary,
and stores a reference to the wrapper function.


You'll notice that every time I create one of these internally, I go through the extra step to new it on the heap. I got some random segfaults before, because a stack allocated object gets wiped out when the function returns (possibly a bug in dmd's closure detection algorithm, but maybe not, since this behavior is expected coming from C, C++, D1, etc). Anyway, I want to keep the intermediates around in case they are used later by a ref return somewhere and the new keyword ensures exactly that.

Actually I think I want Dynamic to be a class instead of a struct so it is always by reference, but that might be weird too. I don't know for sure yet.

The wrapper function is created by the method makeDynamic. It
returns a delegate that makes the static parameter list out
of the dynamic arguments. I used this same technique in my web.d
to automatically call D functions from cgi parameters - it generates
a wrapper function that takes string[string] and converts it to
the static type, by looping through and calling to!type on each
string.

(This looked a bit weird the first time I did it too - it doesn't loop the dynamic args, but instead the static args. The reason is the static arguments, via std.traits.ParameterTypeTuple, can be worked with as static types and passed to other templates, like std.conv.to. A dynamic type can't.

Basically if you know what to expect, you can work with the compiler, but if you don't, it is guess and check. This comes up again later.)


Anyway, the reference to that wrapper function is what's stored.
This is advantageous because then all methods you assign have
a uniform signature - suitable for a runtime array.


> Main difference is that (unlike prototype-ba sed ones like JS,
> Io, etc) inheritance is not done via delegating but by plain
> copy of (references to) methods.

That could be done from inside D too! In fact, it might make
sense here. But maybe not, since the this for the wrapper functions
(which you'd actually be passing around) would get weird.

I'd have to play with it, but I'm pretty confident D could do it itself - no need to make a new toy language!

> Also, there would be some kind of type, again unlike true prototype-based, but they would really be true objects, like when writing a simple OO typing system in Lua.

I've never actually used Lua, but the concept sounds simple enough. I think it'd be a fairly small modification of where I'm taking my code.

On 1/4/2011 12:23 AM, Adam Ruppe wrote:
> Over the weekend, I attacked opDispatch again and found some old
> Variant bugs were killed. I talked about that in the Who uses D
> thread.
>
> Today, I couldn't resist revisiting a dynamic kind of object, and
> made some decent progress on it.
>
> http://arsdnet.net/dcode/dynamic.d
>
> (You can compile that; there's a main() at the bottom of that file)
>
> It isn't quite done - still needs op overloading, and probably better
> errors, but it basically works.
>
> It works sort of like a Javascript object.
>
> Features:
>
> opDispatch and assignment functions:
> Dynamic obj;
>
> // assign from various types
> obj = 10;
> obj = "string";
>
> obj.a = 10; // assign properties from simple types naturally
>
> // can set complex types with one compromise: the () after the
> // property tells it you want opAssign instead of property opDispatch
> obj.a() = { writefln("hello, world"); }
>
> // part two of the compromise - to call it with zero args, use call:
> obj.a.call();
>
> // delegte with arguments works too
> obj.a() = delegate void(string a) { writeln(a); };
>
> // Calling with arguments works normally
> obj.a("some arguments", 30);
>
>
> Those are just the basics. What about calling a D function? You
> need to convert them back to regular types:
>
> string mystring = obj.a.as!string; // as forwards to Variant.coerce
>      // to emulate weak typing
>
> Basic types are great, but what about more advanced types? So far,
> I've implemented interfaces:
>
> interface Cool {
>      void a();
> }
>
> void takesACool(Cool thing) { thing.a(); }
>
>
>
> takesACool(obj.as!Cool); // it creates a temporary class implementing
> // the interface by forwarding all its methods to the dynamic obj
>
>
>
> I can make it work with structs too but haven't written that yet.
> I want to add some kind of Javascript like prototype inheritance too.
>
>
>
> I just thought it was getting kinda cool so I'd share it :)

To cool!
I remember trying to do the same for D1 for my Hessian lib, but you hit the spot here.

This is a great little start for easy RPC proxy creation and for interoperability with COM and others.

Great work!

Very cool!

The restriction with calling zero-args functions is unfortunate, could this be solved by turning it into a class and dynamically checking whether the type is a function and then invoke it if it is?

I would also change implicit declarations into an error, I see you have an throw statement for that but commented out.

Lutger Blijdestijn wrote:
> The restriction with calling zero-args functions is unfortunate, could this be solved by turning it into a class and dynamically checking whether the type is a function and then invoke it if it is?

Maybe, but I think it'd still break the property get, which breaks setting complex types. This, and your other concern about implicit declarations are actually caused by the same root problem: the imperfect compiler implementation of properties combined with variadic opDispatch templates.

(Like I said in that other thread, each of those features works quite well on its own, but the combination reveals some minor bugs.)


To set a complex type, you do the:

dyn.property() = some fancy thing;

Which returns a property by reference, then does its opAssign template, which is much more capable of fanciness than the runtime variadic function you get without the parenthesis.

Why do that? Because overloading on property and non-property
doesn't work right now. Through trial and error, I found
the compiler doesn't add a property assignment's type to the
template argument list, but does pass it as a func argument.
(Quite possibly a (minor) compiler bug itself - I may be
depending on one bug to work around another!)

So, if you try to do a template, it will complain about wrong number of arguments to the function. A variadic function gets that error to go away and I can handle it at runtime. This comes with trade-offs though: the big one being that it doesn't work with complex types. Like the code's comment says, I really need the compiler's help to make them work, and it isn't around by the time any of that code is actually run.


The best situation would be:

@property Dynamic opDispatch(string fieldName)() // getter
Dynamic opDispatch(string fieldName, T...)(T t) // function call,
                                              // any num of args
@property Dynamic opDispatch(string fieldName, T)(T t) // setter

Then everything would Just Work, without the limitations of the runtime variadic and associated workarounds. But the first two are broken by @property not being recognized in overloads currently*, and the last is broke by the right-hand side of the property assignment not being passed as a template argument.


* My preferred fix here would be to give @property one more point in overload resolution so it is slightly preferred over non-property in these situations, but otherwise do nothing. It's the most conservative solution here. I looked at dmd's code, but I don't know the compiler well enough to make it happen.

I guess if @property was strengthened, I could work with that too, just by adding an opCall and leaving opDispatch to do nothing but get and set. The property assign not being passed to the template would have to be solved somehow there too though.



Anyway, none of the fixes are in place today, so I had to make a choice - either setting requires a named function call or zero arg calling requires a named function. I went with the latter since I figured it is a bit prettier in usage.

> I would also change implicit declarations into an error, I see you have an throw statement for that but commented out.

Yes, the main reason is so the ref returns don't do range violation when you are trying to do an assignment, due to the above situation.

It would work with simple assignment:

dynobj.name; // throws
dynobj.name = 10; // works
dynobj.name; // still works

But complex assignment won't work there:

dynobj.name = someDelegate; // this won't work right, probably
// will segfault down the line if I don't throw at runtime over it

So my compromise was:

dynobj.name() = someDelegate; // uses the opAssign template
                              // which works well

But... if dynobj.name didn't already exist, the left hand side of that would throw a range violation, getting a property that doesn't exist.

That's why the exception is commented. If I required a .set()
method or something like that:

dynobj.set("name", whatever you want);

That would work, but it no longer looks like a dynamic language...



When the compiler progresses a little more (or maybe someone will think of a nicer workaround), we can have the best of all worlds, but for now this is the best I can do while keeping a mostly dynamic language like appearance.


(Now, a related question here is weak typing in general. I'm writing this to be very weakly typed - coercing all over the place. Dynamic languages can throw a runtime exception on type mismatch, but I want to basically mimic javascript here, which does not. Besides, D itself is strongly typed - if you want strong types, just use standard D types! Possibly including std.variant.)

On Mon, 03 Jan 2011 17:23:29 -0500, Adam Ruppe <destructionator@gmail.com> wrote:
> Over the weekend, I attacked opDispatch again and found some old
> Variant bugs were killed. I talked about that in the Who uses D
> thread.
>
> Today, I couldn't resist revisiting a dynamic kind of object, and
> made some decent progress on it.
>
> http://arsdnet.net/dcode/dynamic.d
>
> (You can compile that; there's a main() at the bottom of that file)
>
> It isn't quite done - still needs op overloading, and probably better
> errors, but it basically works.
>
> It works sort of like a Javascript object.
>
[snip]

I've been working on an update to both std.json and std.variant. Previews of both are available here: https://jshare.johnshopkins.edu/rjacque2/public_html/
though they are still works in progress. Two of the big enhancements that you might be interested in are call support and opDispatch + reflection + prototype structs. To paraphrase your example:

    Variant v;
    v.a( 10 );
    assert(v.a == 10);
    v.a( { writefln("hello, world"); } );
    v.a.call;  //To be replaced by opCall, once struct opCall is fixed (Bug 4053)
    v.a( delegate void(string a, int x) { foreach(i;0..x) writeln(i+1," ",a); } );
    v.a("potatoes", 3);

I've also stubbed out a prototype style object, but I haven't really tested it yet. Thoughts, comments and use/test cases are always welcomed.

Robert Jacques wrote:
> I've been working on an update to both std.json and std.variant.

Yes, I remember looking at it before. Both look outstanding, but I haven't actually used them yet so the improvements haven't quite sunk into my brain.

I think the differentiating point here is mainly that I'm just playing around, so the general quality will be lower, and I want to try to make a weakly typed object work with the rest of D, just to see how far that can be pushed.

On 1/6/11 1:22 AM, Robert Jacques wrote:
> On Mon, 03 Jan 2011 17:23:29 -0500, Adam Ruppe
> <destructionator@gmail.com> wrote:
>> Over the weekend, I attacked opDispatch again and found some old
>> Variant bugs were killed. I talked about that in the Who uses D
>> thread.
>>
>> Today, I couldn't resist revisiting a dynamic kind of object, and
>> made some decent progress on it.
>>
>> http://arsdnet.net/dcode/dynamic.d
>>
>> (You can compile that; there's a main() at the bottom of that file)
>>
>> It isn't quite done - still needs op overloading, and probably better
>> errors, but it basically works.
>>
>> It works sort of like a Javascript object.
>>
> [snip]
>
> I've been working on an update to both std.json and std.variant.
> Previews of both are available here:
> https://jshare.johnshopkins.edu/rjacque2/public_html/
> though they are still works in progress. Two of the big enhancements
> that you might be interested in are call support and opDispatch +
> reflection + prototype structs. To paraphrase your example:
>
> Variant v;
> v.a( 10 );
> assert(v.a == 10);
> v.a( { writefln("hello, world"); } );
> v.a.call; //To be replaced by opCall, once struct opCall is fixed (Bug
> 4053)
> v.a( delegate void(string a, int x) { foreach(i;0..x) writeln(i+1,"
> ",a); } );
> v.a("potatoes", 3);
>
> I've also stubbed out a prototype style object, but I haven't really
> tested it yet. Thoughts, comments and use/test cases are always welcomed.

I think this transgresses the charter of Variant. Variant is meant to hold an object of some _preexisting_ type, not to morph into anything. We should have three abstractions:

* Algebraic holds any of a closed set of types. It should define method calls like a.fun(args) if and only if all of its possible types support the call with compatible arguments and return types.

* Variant holds any of an unbounded set of types. Reflection may allow us to define v.fun(args) to look up the method name dynamically and issue a runtime error if it doesn't exist (sort of what happens now with operators).

* Dynamic is a malleable type that you get to add state and methods to, just like in Javascript.


Andrei