Thread overview
std.traits.ParameterDefaults implementation
Apr 08, 2019
Adam D. Ruppe
Apr 09, 2019
Meta
Apr 09, 2019
Adam D. Ruppe
Apr 09, 2019
Alex
April 08, 2019
I'm looking further into some pains with parameter default reflection and I feel Phobos' overcomplicated implementation is to blame for the troubles and am trying to figure why it is that way.

First, let me paste some code:

----
class C {
	override @safe string toString() const { return "C"; }
}

class A {
	void foo(lazy scope inout C a = new inout(C)) {}
}
@safe void main() {
	import std.stdio;
	foreach(overload; __traits(getOverloads, A, "foo")) {
		static if(is(typeof(overload) Params == __parameters))
			static foreach(idx, _; Params) {{
				alias param = Params[idx .. idx + 1];
				writeln("\t", __traits(identifier, param),
					" = ",
					function(param p) { return p[0]; }().toString
				);
			}}
	}
}
---


This is my testbed for default argument without Phobos. And this line is all there really is to it:


function(param p) { return p[0]; }()


I'm building with dmd -dip25 -dip1000 in an attempt to break it with attributes. (the explicit toString on the outside is because writeln didn't like the const class being passed to it)


On the other hand, this is Phobos' implementation:

---
        template Get(size_t i)
        {
            // `PT[i .. i+1]` declares a parameter with an arbitrary name.
            // To avoid a name clash, generate local names that are distinct
            // from the parameter name, and mix them in.
            enum name = param_names[i];
            enum args = "args" ~ (name == "args" ? "_" : "");
            enum val = "val" ~ (name == "val" ? "_" : "");
            enum ptr = "ptr" ~ (name == "ptr" ? "_" : "");
            mixin("
                // workaround scope escape check, see
                // https://issues.dlang.org/show_bug.cgi?id=16582
                // should use return scope once available
                enum get = (PT[i .. i+1] " ~ args ~ ") @trusted
                {
                    // If the parameter is lazy, we force it to be evaluated
                    // like this.
                    auto " ~ val ~ " = " ~ args ~ "[0];
                    auto " ~ ptr ~ " = &" ~ val ~ ";
                        // workaround Bugzilla 16582
                    return *" ~ ptr ~ ";
                };
            ");
            static if (is(typeof(get())))
                enum Get = get();
---


It handles a missing default too, but that's trivial, just the is(typeof()) check.

What gets me here is the function body. In mine, I just defined an inline function and returned the parameter.

Phobos uses what appears to be a gratuitous mixin and defines two local variables inside the function. These cause trouble with inout

        writeln(ParameterDefaults!(A.foo)[0].toString);

/home/me/d/dmd2/linux/bin32/../../src/phobos/std/traits.d(1492): Error: variable `std.traits.ParameterDefaults!(foo).Get!0u.Get` only parameters or stack based variables can be inout
/home/me/d/dmd2/linux/bin32/../../src/phobos/std/traits.d(1512): Error: template instance `std.traits.ParameterDefaults!(foo).Get!0u` error instantiating


I've seen people blame inout for this before... but it seems to me to be that Phobos has an overcomplicated implementation.

Looking at bug 16582 which introduced the new code

https://issues.dlang.org/show_bug.cgi?id=16582

the test case is similar to what I just wrote, and my simple code works. The other stuff in there talks about lazy. Mine worked with that too.


Is the Phobos implementation just working around compiler bugs that have since been fixed? Or am I missing something else in there?
April 09, 2019
On Monday, 8 April 2019 at 18:57:59 UTC, Adam D. Ruppe wrote:
> Is the Phobos implementation just working around compiler bugs that have since been fixed? Or am I missing something else in there?

If I had to guess, I would say it's a combination of this, as well as the fact that this template is 7 years old:

https://github.com/dlang/phobos/commit/c8daf366230356c338f1dc3631bceca8f575e3fa

I was only starting to look into D in winter of 2012, so I don't know if compile-time metaprogramming was very widely-used at the time, or if the necessary idioms and best practices had developed yet.
April 09, 2019
On Tuesday, 9 April 2019 at 01:16:18 UTC, Meta wrote:
> If I had to guess, I would say it's a combination of this, as well as the fact that this template is 7 years old:

Yeah, though the original version is actually closer to what I'd expect it to be; the other stuff I find really questionable was added later in response to bugs.

But as far as I can tell, those bugs have since been fixed.

> I was only starting to look into D in winter of 2012, so I don't know if compile-time metaprogramming was very widely-used at the time, or if the necessary idioms and best practices had developed yet.

I was using it pretty heavily at the time; my web.d was actively in production on a few sites as early as 2011, and web.d uses the reflection capabilities to generate html for D functions.

But yeah, it took a lot of gnarly workarounds to make it work back then.
April 09, 2019
On Monday, 8 April 2019 at 18:57:59 UTC, Adam D. Ruppe wrote:
> I'm looking further into some pains with parameter default reflection and I feel Phobos' overcomplicated implementation is to blame for the troubles and am trying to figure why it is that way.
>
> First, let me paste some code:
>
> ----
> class C {
> 	override @safe string toString() const { return "C"; }
> }
>
> class A {
> 	void foo(lazy scope inout C a = new inout(C)) {}
> }
> @safe void main() {
> 	import std.stdio;
> 	foreach(overload; __traits(getOverloads, A, "foo")) {
> 		static if(is(typeof(overload) Params == __parameters))
> 			static foreach(idx, _; Params) {{
> 				alias param = Params[idx .. idx + 1];
> 				writeln("\t", __traits(identifier, param),
> 					" = ",
> 					function(param p) { return p[0]; }().toString
> 				);
> 			}}
> 	}
> }
> ---
>
>
> This is my testbed for default argument without Phobos. And this line is all there really is to it:
>
>
> function(param p) { return p[0]; }()
>
>
> I'm building with dmd -dip25 -dip1000 in an attempt to break it with attributes. (the explicit toString on the outside is because writeln didn't like the const class being passed to it)
>
>
> On the other hand, this is Phobos' implementation:
>
> ---
>         template Get(size_t i)
>         {
>             // `PT[i .. i+1]` declares a parameter with an arbitrary name.
>             // To avoid a name clash, generate local names that are distinct
>             // from the parameter name, and mix them in.
>             enum name = param_names[i];
>             enum args = "args" ~ (name == "args" ? "_" : "");
>             enum val = "val" ~ (name == "val" ? "_" : "");
>             enum ptr = "ptr" ~ (name == "ptr" ? "_" : "");
>             mixin("
>                 // workaround scope escape check, see
>                 // https://issues.dlang.org/show_bug.cgi?id=16582
>                 // should use return scope once available
>                 enum get = (PT[i .. i+1] " ~ args ~ ") @trusted
>                 {
>                     // If the parameter is lazy, we force it to be evaluated
>                     // like this.
>                     auto " ~ val ~ " = " ~ args ~ "[0];
>                     auto " ~ ptr ~ " = &" ~ val ~ ";
>                         // workaround Bugzilla 16582
>                     return *" ~ ptr ~ ";
>                 };
>             ");
>             static if (is(typeof(get())))
>                 enum Get = get();
> ---
>
>
> It handles a missing default too, but that's trivial, just the is(typeof()) check.
>
> What gets me here is the function body. In mine, I just defined an inline function and returned the parameter.
>
> Phobos uses what appears to be a gratuitous mixin and defines two local variables inside the function. These cause trouble with inout
>
>         writeln(ParameterDefaults!(A.foo)[0].toString);
>
> /home/me/d/dmd2/linux/bin32/../../src/phobos/std/traits.d(1492): Error: variable `std.traits.ParameterDefaults!(foo).Get!0u.Get` only parameters or stack based variables can be inout
> /home/me/d/dmd2/linux/bin32/../../src/phobos/std/traits.d(1512): Error: template instance `std.traits.ParameterDefaults!(foo).Get!0u` error instantiating
>
>
> I've seen people blame inout for this before... but it seems to me to be that Phobos has an overcomplicated implementation.
>
> Looking at bug 16582 which introduced the new code
>
> https://issues.dlang.org/show_bug.cgi?id=16582
>
> the test case is similar to what I just wrote, and my simple code works. The other stuff in there talks about lazy. Mine worked with that too.
>
>
> Is the Phobos implementation just working around compiler bugs that have since been fixed? Or am I missing something else in there?


I appreciate you looking in to it instead of writing it off. It basically proves my point that there are issues with D. The problem with these types of issues is that because D's meta programming is so complex(mainly traits) that one doesn't know if things are bugs or ones code. So when I code something and something is not working out right, which sometimes I might not find out until later due to the way D silently gives wrong information or it might only happen in certain situations(such as the inout parameter here, which initially gave me no error and I noticed later on it wasn't giving the default value and then changed something and got the error(I think I was using compiles to bypass stuff)). This leads one down wrong paths. I spend hours trying to figure out what is going on and why my code isn't working and usually change it drastically which then either creates or obscures other problems.

This is precisely why I want a better reflection library. One that is uniform in design and bypasses these issues(although it will still suffer from the cases where D returns the wrong information but I suppose unittests could get most of that).

There are too many obscure issues with the type system and doing certain things. It doesn't feel cohesive but patched together. If I need to do something that I can't remember I have to check both __traits and std.traits. Sometimes neither has basic meta programming functionality and one has to build it, but it is very common. This dirties the code and makes it less readable.(such as having two nested static foreach's to filter out stuff)

It really doesn't have to be complicated. The library I wrote shows this. One has an CT object that has all the info in it. As I've said before, the D compiler can easily build this object very fast and make it available. My design, which is rather straight forward and simple is very slow. It takes about 10 seconds to get all the info on a class(well, not that long but close) that is very bare bones. It does get everything though, but the compiler also has all this info already so a lot of work is duplicated.

We could have something like this

static foreach(m; SomeType.__reflect.AllMembers.OnlyPublic.Types)

where __reflect returns whatever information. It consumes __traits and std.traits in to a simple to use interface. Some other syntax could be used.


Basically one could have range like semantics for reflecting(I probably could add this to my library solution).