On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote:
> On Thursday, 7 September 2017 at 16:55:02 UTC, EntangledQuanta wrote:
>> Sorry, I think you missed the point completely... or I didn't explain things very well.
>
> I don't think I did - your new explanation didn't change my understanding at least. This indicates I'm the one who's bad at explaining. Ah well.
>
> The point of my post was mostly to rewrite the code you'd posted in a form that I (and, I hope, others) found easier to understand.
>
>> I see no where in your code where you have a variant like type.
>
> True. I've now rewritten it to use std.variant.Algebraic with these semantics:
>
> auto foo(T1, T2)(T1 a, T2 b, int n) {
>     import std.conv;
>     return T1.stringof~": "~to!string(a)~" - "~T2.stringof~": "~to!string(b);
> }
>
> unittest {
>     import std.variant;
>     Algebraic!(float, int) a = 4f;
>     Algebraic!(double, byte) b = 1.23;
>
>     auto res = varCall!foo(a, b, 3);
>     assert(res == "float: 4 - double: 1.23");
> }
>
> template varCall(alias fn) {
>     import std.variant;
>     auto varCall(int n = 0, Args...)(Args args) {
>         static if (n == Args.length) {
>             return fn(args);
>         } else {
>             auto arg = args[n];
>             static if (is(typeof(arg) == VariantN!U, U...)) {
>                 foreach (T; arg.AllowedTypes) {
>                     if (arg.type == typeid(T))
>                         return varCall!(n+1)(args[0..n], arg.get!T, args[n+1..$]);
>                 }
>                 assert(false);
>             } else {
>                 return varCall!(n+1)(args);
>             }
>         }
>     }
> }
>
> Sadly, by using std.variant, I've given up on the elegant switch/case in exchange for a linear search by typeid. This can be fixed, but requires changes in std.variant.
>
> Of course, it would be possible to hide all this behind compiler magic. Is that desirable? I frankly do not think so. We should be wary of adding too much magic to the compiler - it complicates the language and its implementation. This is little more than an optimization, and while a compiler solution would be less intrusive and perhaps more elegant, I do not feel it provides enough added value to warrant its inclusion.
>
> Next, I'm curious about this code:
>
>> void bar(var t)
>> {
>>     writeln("\tbar: Type = ", t.type, ", Value = ", t);
>> }
>> 
>> void main()
>> {
>>    bar(3); // calls bar as if bar was `void bar(int)`
>>    bar(3.4f); // calls bar as if bar was `void bar(float)`
>>    bar("sad"); // calls bar as if bar was `void bar(string)`
>> }
>
> What does 'var' add here, that regular templates do not? (serious question, I'm not trying to shoot down your idea, only to better understand it) One possible problem with var here (if I understand it correctly) would be separate compilation - a generated switch would need to know about types in other source files that may not be available at the time it is compiled.
>
> Next:
>
>> var foo(var x)
>> {
>>    if (x == 3)
>>        return x;
>>    return "error!";
>> }
>
> This looks like a sort of reverse alias this, which I've argued for on many occasions. Currently, it is impossible to implement a type var as in that function - the conversion from string to var would fail. A means of implementing this has been discussed since at least 2007, and I wrote a DIP[1] about it way back in 2013. It would make working with variants and many other types much more pleasant.
>
> [1]: https://wiki.dlang.org/DIP52

I use something similar where I use structs behaving like enums. Each field in the struct is an "enum value" which an attribute, this is because I have not had luck with using attributes on enum values directly and that structs allow enums with a bit more power.

When a runtime value depends on these structs one can build a mapping between the values and functional aspects of program. Since D has a nice type system, one can provide one templated function that represents code for all the enum values.


E.g.,

enum TypeID // actually done with a struct
{
   @("int") i, @("float") f
}


struct someType
{
   TypeID id;
}

someType.id is runtime dependent. But we want to map behavior for each type.

if (s.id == TypeID.i) fooInt();
if (s.id == TypeID.f) fooFloat();

For lots of values this is tedius and requires N functions. Turning foo in to a template and autogenerating the mapping using mixins we can get something like

mixin(MapEnum!(TypeID, "foo")(s.id))

which generates the following code:

switch(s.id)
{
   case TypeID.i: foo!int(); break;
   case TypeID.f: foo!float(); break;
}


and of course we must create foo:

void foo(T)()
{

}


but rather than one for each enum member, we just have to have one. For certain types of code, this works wonders. We can treat runtime dependent values as if they were compile time values without too much work. MapEnum maps runtime to compile time behavior allowing us to use use templates to handle runtime variables. T in foo is actually acting in a runtime fashion depending on the value of id.

My code is not elegant as it suits my specific needs but if a general purpose framework was created, variant types could easily be treated as compile time template parameters. It has been a pretty powerful concept in the code I write which has to handle many of the primitive types and combinations of them. I can create functions like

Add!(A,B,C)(A a,B b,C c)

and depending on what the runtime values of some object are, have the mapping code call the appropriate Add function, but only have to create one since there is a canonical form such as

Add!(A,B,C)(A a,B b,C c)
{
    return a + b + c;
}

the type system even verifies the code is correct! Template instantiates that are illegal will create errors.

Of course, If one has to specialize for each combination then this method is not much more convenient than doing it all by hand. It still lets one think about runtime types that enumerate behavior as compile time templates though and leverage all the functionality they have rather than using using runtime code.

This, in fact, is why I use the technique. Instead of having runtime checks I can move them in to compile time increasing performance(the cost is the switch statement). What makes it performant generally is because of how code can be organized in a more streamlined manner rather than getting in to a rats nest of handling all the possible combinations at runtime.


For example:

Suppose we must process a void[]. We do not know the underlying type at runtime. Our processing does not depend on the primitive underlying type.

We just need to specify

void Process(T)(T[] buf)
{
   T t;
   foreach(b; buf)
     t = max(b, t);
   if (max > 0) assert("error!");
}

and now we have T that corresponds to the buffer type. The MapEnum hooks up the runtime type variable's value to the compile type template. Because process does not depend on the specifics of T except that they are primitive, we only have to have one general function that handles them all.

Of course, return types are more difficult. I do not deal with return types in my code but I suppose one could use the same type of technique where we store the return type in a variant along with its type and then use the same techniques to deal with their values.

Not all difficult and would make coding runtime very nearly like compile time. Unfortunately what is really going on here is all combinations that a runtime variable can take are mapped in compile time code and this could lead to exponential increases in code size. OTOH, it probably would be nearly as performant as possible and easier to code with the proper tools rather than having to do it by hand.

A compiler feature set regarding this type of coding scheme would definition be nice it could manage keeping everything consistent.

On Saturday, 2 June 2018 at 23:12:46 UTC, DigitalDesigns wrote:
> On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote:
>> [...]
>
> I use something similar where I use structs behaving like enums. Each field in the struct is an "enum value" which an attribute, this is because I have not had luck with using attributes on enum values directly and that structs allow enums with a bit more power.
>
> [...]

You might want to have a look at https://wiki.dlang.org/Dynamic_typing
This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea.

On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote:
> On Saturday, 2 June 2018 at 23:12:46 UTC, DigitalDesigns wrote:
>> On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote:
>>> [...]
>>
>> I use something similar where I use structs behaving like enums. Each field in the struct is an "enum value" which an attribute, this is because I have not had luck with using attributes on enum values directly and that structs allow enums with a bit more power.
>>
>> [...]
>
> You might want to have a look at https://wiki.dlang.org/Dynamic_typing
> This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea.

No, this is not what I'm talking about, although maybe it could be related in some way.

What I am talking about is hooking up a runtime variable that can take a few values, such as from an enum and have those be mapped to a compile time template value.

This way you get full static time checking of runtime code. Seems impossible? It's not!

What it does is leverage D's meta programming engine to deal with all the routine possiblities.

A large switch statement is what makes the runtime to compile time magic happen

int x;

switch(x)
{
    default: foo!void();
    case 0: foo!int();
    case 1: foo!double();
    etc...
}

See how the switch maps a runtime value x to a templated function foo?

we then can handle the x values with foo

void foo(T)()
{
   // if x = 0 then T = int
   // if x = 1 then T = double


}

But inside foo we have T, the template variable that is the compile time representation of the dynamic variable x. Remember, x's value is unknown at compile time... the switch is what maps the runtime to the compile time.

But in foo, because we have T, the type system all works fine.

What makes this very useful that we can call templated functions using T and the meta engine will pick the right template function specialization.

e.g.,

void bar(S)()
{

}


can be used inside foo by calling bar!T(). It doesn't seem like much here if you had to use x it would be a pain. Either you would have to manually create switches or create a rats nest of if statements. But we never have to worry about that stuff when using the above method because it is exactly like programming at compile time as if x a compile time value(like say, just int).

It works great when you have several template variables and just want everything to work together without having to go in to much trouble:


foo(A,B)()
{
   B b = 4;
   bar!(A)(b)
}


suppose A can come from int, double, and B from float and long

That is 4 different combinations one would normally have to represent. Not a big deal until you have to handle every combination.


Suppose you are working with void arrays. They contain types but you don't know the type except after compile time.

Without using this technique you have to use casts and tricks and you'll find out if you screwed up some typing stuff at runtime. Using this technique you will not have a void array but a T[] with T being any of the possible types that you specify using UDA's.

if you could

if (x == 0)
{
    foo(cast(int[])a);
} else

if (x == 1)
{
    foo(cast(double[])a);
} else


but I can do that with one line which simply generates the switch for me. Really all I'm doing is hiding the switch so it all looks like some magic is happening in one line. But the fact that it becomes really simple to do seems to open up the use of it and conceptually on can then think of "x" as a compile time variable that can take on several possibilities.

On Monday, 4 September 2017 at 03:26:23 UTC, EntangledQuanta wrote:
> Take a variant type. It contains the "type" and the data. To simplify, we will treat look at it like
>
> (pseudo-code, use your brain)
>
> enum Type { int, float }
>
> foo(void* Data, Type type);
>
> The normal way to deal with this is a switch:
>
> switch(type)
> {
>     case int: auto val = *(cast(int*)Data);
>     case float: auto val = *(cast(float*)Data);
> }
>
>
> But what if the switch could be generated for us?
>
> [...]
>
> But, in fact, since we can specialize on the type we don't have to use switch and in some cases do not even need to specialize:
>
> for example:
>
> foo(T)(T* Data) { writeln(*Data); }
>
> is a compile time template that is called with the correct type value at run-time due to the "magic" which I have yet to introduce.
>
> Note that if we just use a standard runtime variant, writeln would see a variant, not the correct type that Data really is. This is the key difference and what makes this "technique" valuable. We can treat our dynamic variables as compile time types(use the compile time system) without much hassle. They fit naturally in it and we do not clutter our code switches. We can have a true auto/var like C# without the overhead of the IR. The cost, of course, is that switches are still used, they are generated behind the scenes though and the runtime cost is a few instructions that all switches have and that we cannot avoid.
>
> To get a feel for what this new way of dealing with dynamic types might look like:
>
> void foo(var y) { writeln(y); }
>
> var x = "3"; // or possibly var!(string, int) for the explicit types used
> foo(x);
> x = 3;
> foo(x);

It sounds like what you are describing is a sum type. There is an implementation of one in the standard library, std.variant.Algebraic, as well as several alternative implementations on code.dlang.org, including my own, "sumtype" [1].

Using sumtype, your example would look like this:

alias Var = SumType!(string, int);

void foo(Var y) {
    var.match!(
        (value) { writeln(value); } // template lambda
    );
}

Var x = "3";
foo(x);
x = 3;
foo(x);

The match method takes a list of functions as template arguments, and generates a switch statement that maps each possible type of Var to one of those functions. All type checking is done at compile time.

[1] https://code.dlang.org/packages/sumtype

On Sunday, 3 June 2018 at 14:57:37 UTC, DigitalDesigns wrote:
> On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote:
>> You might want to have a look at https://wiki.dlang.org/Dynamic_typing
>> This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea.
>
> No, this is not what I'm talking about, although maybe it could be related in some way.

Actually, it sort of is. Your mapEnum is essentially the same as std.variant.visit (https://dlang.org/phobos/std_variant#.visit), and std.variant.Algebraic is the type that encapsulates both the runtime tag and the void[] containing the data of unknown type.

Now, there may be many important differences - Algebraic encapsulates the data and tag, which may or may not be what you want, visit only takes one algebraic argument, mapEnum may be faster, more or less generic, etc. The idea of converting a run-time value to a compile-time value is the same, though.

--
  Simen

On Sunday, 3 June 2018 at 16:36:52 UTC, Simen Kjærås wrote:
> On Sunday, 3 June 2018 at 14:57:37 UTC, DigitalDesigns wrote:
>> On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote:
>>> You might want to have a look at https://wiki.dlang.org/Dynamic_typing
>>> This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea.
>>
>> No, this is not what I'm talking about, although maybe it could be related in some way.
>
> Actually, it sort of is. Your mapEnum is essentially the same as std.variant.visit (https://dlang.org/phobos/std_variant#.visit), and std.variant.Algebraic is the type that encapsulates both the runtime tag and the void[] containing the data of unknown type.
>
> Now, there may be many important differences - Algebraic encapsulates the data and tag, which may or may not be what you want, visit only takes one algebraic argument, mapEnum may be faster, more or less generic, etc. The idea of converting a run-time value to a compile-time value is the same, though.
>
> --
>   Simen

I didn't know that variants had those functions! pretty nice. Yes, it is similar to what I'm doing. Same principles but just a little different perspective. I use enums, UDA's, and templates rather than a Algebraic and delegates.

The difference is that the enum stores only the type information rather than the variable and the type info that Algebraic stores.

If I were to have know about this before I might have used it instead and everything would have probably been fine.

The only thing is that the enum version lets me store the type info separately than with the data. When several variables depend on the type id I think it will make it a little easier than having to manage several Algebraic type info's across several variables to sync them.

For example

dataType type;
void[] in, out;

rather than

Algebraic!(type1,..., typen) in, out;

and then having to make sure the types are synced between in and out. At least in my case it might be a little easier. Also my way uses a templated function directly rather than an array of lambads, although they are equivalent:

Algebraic!(string, int) variant;

variant.visit!((string s) => cast(int) s.length, (int i)    => i)();

which could be written as

variant.visit!((string s) => foo(s), (int i)    => foo(i))();

auto foo(T)(T t) { }


would become

enum variant
{
    @("int") _int,
    @("string") _string,
}

mixin(variant.MapEnum!("foo")());

auto foo(T)(T t) { }


So, they definitely are very similar and actually might be identical. I haven't used Algebraic and visit any to know.

What I do know is that for several Algebraics you would have to do something like

variant.visit!((string s) => variant2.visit!((double d) => { foo(s,d); })), (int i)    => foo(i))();

etc. Which is creating the nested switch structure and can become complicated while my method still remains one line but foo just takes more than one template parameter. My feeling is mine is a little less robust since it's more for specific types of code while visit is a little more general. Mainly because of the hard coding of the mixin structure.