On Tuesday, 9 April 2019 at 18:33:21 UTC, Seb wrote:
> On Tuesday, 9 April 2019 at 16:30:53 UTC, Alex wrote:
>> On Tuesday, 9 April 2019 at 14:59:03 UTC, Adam D. Ruppe wrote:
>>>[...]
>> I didn't say the language. The point with the language is that it could have built in semantics to do reflection in a inform way(__traits is somewhat uniform but messy and then one has std.traits which then makes it not uniform).
>>
>> [...]
>
> Have you considered writing a DIP?

No, because I expect it won't even be considered. The typical responses I have seen to this sort of thing is that "we already have it". Hence why I decided to write my own lib, and then had a few issues.

Andre did look in to some of the ideas and said he would pass it off to some GSOC students to look in to making it better, so maybe something will come out of that.

I believe that basically the compiler simply needs to present a better interface in to reflection. This should have been done from the get go but I guess __traits has been patched together over time as as things were needed.

Essentially the compiler would need to do what I have done and wrap __traits and std.traits in to a more uniform interface. It would add an extra layer of indirection though since the wrapper needs then to stay consistent with any traits changes.


My library works well but is incomplete and has some bugs to due bugs in traits. It is, though, rather slow and I haven't worked on fixing it. (actually it doesn't seem to add too much overhead after the first call but I imagine that is due to the template being instantiated already in the module, might multiply for each new module it is used in)


What I'm thinking is that maybe it is better to have some range like semantics used for reflection. I have thought about it much but I got the idea earlier.

Type.Reflect.Accessor.Accessor.Selector.Selector.Accessor...

The idea is that one can access stuff from the type and then use certain keywords to select and filter and then access more.

I say it is range like but I'm not sure if ranges would work or not or if it would just be dispatching.

E.g.,

MyClass.Reflect.TemplateParameters.Select!int.Names

Would get the template parameters of MyClass, then select only those with int and then get the names of those parameters.

(this could work in general for runtime too)

MyClass.SomeArray.Select("x.value == 4").name;

which is why I say it is sorta like ranges(which I'm thinking more of piping).


I just haven't thought about it much and there doesn't seem to be much interest in going this route... I think it would be a great addition to D but others seem to not care or are more interested in arguing to relatively insignificant issues.

On Tuesday, 9 April 2019 at 18:56:58 UTC, H. S. Teoh wrote:
> On Tue, Apr 09, 2019 at 06:33:21PM +0000, Seb via Digitalmars-d-learn wrote:
>> On Tuesday, 9 April 2019 at 16:30:53 UTC, Alex wrote:
>> > On Tuesday, 9 April 2019 at 14:59:03 UTC, Adam D. Ruppe wrote:
>> > > [...]
>> > I didn't say the language. The point with the language is that it could have built in semantics to do reflection in a inform way(__traits is somewhat uniform but messy and then one has std.traits which then makes it not uniform).
>> > 
>> > [...]
>> 
>> Have you considered writing a DIP?
>
> I think the original intent was that __traits was supposed to be low-level, implementation-specific calls into the compiler, while std.traits is supposed to be the user-friendly syntactic sugar built on top that user code is supposed to use.
>
> Unfortunately, it turned out that people prefer using __traits directly instead, and std.traits hasn't quite been keeping up with new __traits (e.g., AFAIK __traits(compiles) does not have a std.traits equivalent), and is also less familiar to many people, so now we have a mishmash of code sometimes using __traits and sometimes std.traits.
>

That may be the case but I recently saw some one here say "All the roads to hell are paved with good intentions", which isn't hyperbole. The better the initial plan the better things will turn out. Planning is the key to success, without it there is only luck.

My point has been and is that there is a better way that is more natural.  I make no claims about anything else. It may be a cop out to say something that is a tautology but I make the claim as an emphasis that I believe that it is more true in this case than others. i.e., D's traits are PITA. Ok, they are not as bad as some things but I know they can be better since I have used other languages that have far more logical reflection(such as C#) than D.

Because D has such an extensive meta programming language(the best I've seen for procedural) it needs a well thought out and natural reflection scheme. traits works but the more I use it the more I believe it could have been done better. It's one thing to use it here and there, but when you use it everywhere the issues multiply.

I think the library I through together at least demonstrates a much more natural way(and it was just a first attempt without any planning). I imagine something much better could be created with some proper planning and teamwork. My impression though is few people here actually care about such things.

On Tue, Apr 09, 2019 at 08:49:17PM +0000, Alex via Digitalmars-d-learn wrote:
> On Tuesday, 9 April 2019 at 18:56:58 UTC, H. S. Teoh wrote:
[...]
> My point has been and is that there is a better way that is more natural.  I make no claims about anything else. It may be a cop out to say something that is a tautology but I make the claim as an emphasis that I believe that it is more true in this case than others. i.e., D's traits are PITA. Ok, they are not as bad as some things but I know they can be better since I have used other languages that have far more logical reflection(such as C#) than D.

As I said, __traits was never designed to be user-friendly, or even "logical".  It was supposed to be a quick-and-easy way to tap into compiler internals, and std.traits was supposed to be the proper API to introspection.  But std.traits, as it stands, is far, far from what it ought to be, and so we have today's sorry situation.

(And on a side note: don't even get me started on is(...) expressions.)

Anyway, since you seem to be passionate about this, this could be your chance of making std.traits what it ought to have been all along, or making a sensible reflection library that can for all practical purposes replace std.traits.

I, for one thing, would appreciate a nicer API to introspection than what we currently have.  One of the main things that drew me to D was its metaprogramming capabilities, and while overall it has been a positive experience, there *are* flies in the ointment like __traits, is(...) expressions, etc..  If you or somebody else could come up with a saner way to do introspection in D, it'd be very much appreciated.


T

-- 
Life would be easier if I had the source code. -- YHL

On Tuesday, 9 April 2019 at 20:45:18 UTC, Alex wrote:
> On Tuesday, 9 April 2019 at 18:33:21 UTC, Seb wrote:
>> Have you considered writing a DIP?
>
> No, because I expect it won't even be considered.

You won't pass review if you don't show knowledge of the existing language, but there's a lot of people who are interested in a few changes.

A few years ago, there was a proposal for a magical `meta` namespace that would call into the compiler for syntax sugar on CT reflection. The old idea was to replace `__traits(X, args)` with `meta.X(args)` - pure syntax change - but you could perhaps revive and expand that as the entry point to your new idea and get some people on board.

I'm skeptical of any reflection library, but compiler changes might be able to change that. The status quo for D's reflection libraries are:

1) weird bugs and/or omissions since the language does not let you express all the function parameter details as return values or local variables.

2) slow compile times (you said your thing was 10 seconds! that's utterly unacceptable)

3) not actually significantly easier to use than language built-ins (if they are actually easier to use at all!)


So any library without compiler changes is, as of today, I believe *impossible* to get right, and getting as close as you can is horribly slow. So, your DIP would necessarily include language+compiler changes.

And the compiler is *already* (almost) fully capable, so changing that needs to show that the change is worth it. Maybe you can do that, even little cosmetic things can indeed be a win, but to write a convincing case here, you'll need to compare and contrast various cases.

On Tuesday, 9 April 2019 at 21:00:55 UTC, H. S. Teoh wrote:
> (And on a side note: don't even get me started on is(...) expressions.)

is expressions rock. And I betcha if we did do libraries for them, it would eventually go full circle, with someone doing a pattern-matching DSL that reinvents the original syntax, just much slower to compile :P

Another couple ideas you might get some traction on are making typeid() values be convertible back into typeof types iff CTFE.

That would let you use types as OOP class values in intermediate structures while still turning them back into template args later.

Not sure just how realistic that is, but might be fun to explore.


Related would be to use RTInfo in druntime to extend the typeid info to runtime data, based on a -version switch. This should be possible in today's language fairly easily, though it would not be customizable by the end user, you could perhaps just make it include it all.

On Wed, Apr 10, 2019 at 02:22:35AM +0000, Adam D. Ruppe via Digitalmars-d-learn wrote:
> On Tuesday, 9 April 2019 at 21:00:55 UTC, H. S. Teoh wrote:
> > (And on a side note: don't even get me started on is(...)
> > expressions.)
> 
> is expressions rock. And I betcha if we did do libraries for them, it would eventually go full circle, with someone doing a pattern-matching DSL that reinvents the original syntax, just much slower to compile :P

The functionality rocks, but the syntax is a horrendous hairball of inconsistencies and design-by-chance.


T

-- 
It's bad luck to be superstitious. -- YHL

On Wednesday, 10 April 2019 at 10:18:35 UTC, H. S. Teoh wrote:
> The functionality rocks, but the syntax is a horrendous hairball of inconsistencies and design-by-chance.

There's a little bit weird about it, but it makes sense.

I used to find it hard to even remember how it works, but then I had to describe it for the book and it all came together. Consider this.

Declaring a normal local variable:

---
int foo = 0;
---

We can generalize that to
---
Type_Name Local_Name Operator Initial_Value
---

Well, now, compare to the is expression:

---
is(typeof(func) Params == __parameters)
---


It is a bit of a stretch, but the general shape is the same. We can define the is expression generally to be:

is(Type_Name Local_Name Deconstruction_Pattern)


And then both Local_Name and Deconstruction_Pattern are optional.

This gives us all the forms from the website https://dlang.org/spec/expression.html#IsExpression (from point #5 there)

1: is(Type). This has the type, but left out the local name and deconstruction pattern. All it cares about is if the type exists.

2: is(Type : something). This deconstruction pattern, using :, means "can implicitly convert to something"

The deconstruction pattern mimics common ways of writing such types; class A : B {} defines a type A that can implicitly cast to B, so is(A : B) would pass and return true.

3: is(Type == something). This deconstruction pattern, using ==, just needs an exact match. It is probably the easiest one for people to remember.

4: is(Type Identifier). This skips the deconstruction pattern, meaning it is the most basic "this type must exist" check, but includes the local name.

Like with `int a;`, the name comes after the type.

5: is(Type Identifier : something). This is just #2 again, but including the optional local name.

6: is(Type Identifier == something). Just #3 including the optional local name.

At this point, the documentation also includes other deconstruction patterns, like the `class` keyword, etc. These are mostly very simple, but it is slightly magical in places because it can initialize Identifier to other thing.. but is that really weird?

int a = 10;

The value of a is dependent on the right side, and so is the Identifier here.

It looks totally wrong because it is using == here rather than =, so we intuitively think

is(A B == C)

is comparing B and C... but just rewrite in your brain that this is actually declaring a variable with a funky deconstruction initializer and it will make sense again.

So the whole `== C` part is the deconstruction pattern, and B is the variable being set to it. And being a deconstruction pattern, it is trying to pull layers off, so stuff like

is(MyClass Parents == super),

the ==super pattern is deconstructing its super keyword; its base classes.

Think of it not as comparison, but as matching a deconstruction pattern and it will make more sense.

I know its weird, but we can work with it. Let's move on:

7: This one has a lot of examples, but it really just expands on the deconstruction pattern. For example:

static if(is(int[10] == T[N], T, size_t N))

What's the pattern here?

== means use exact comparison, not implicitly converting comparison.

T[N] writes out a model of what the declaration is expected to look like.

Then commas separate the definitions of each placeholder variable, just as if they  were template argument definitions. Same syntax, different location.

The one tricky thing is the compiler will not tell you when you malformed the pattern (for the most part), just nothing will happen to match it. So match it on some known quantity to test via static assert or something.

To use a complex pattern with the optional name:


static if(is(int[10] OriginalType == T[N], T, size_t N))

that's all the pieces together. Not so bad when you know how it breaks down.

On Wednesday, 10 April 2019 at 14:06:53 UTC, Adam D. Ruppe wrote:
> On Wednesday, 10 April 2019 at 10:18:35 UTC, H. S. Teoh wrote:
>> [...]
>
> There's a little bit weird about it, but it makes sense.
>
> I used to find it hard to even remember how it works, but then I had to describe it for the book and it all came together. Consider this.
>
> [...]

I wonder if there are some interesting patterns of nesting is's?

is(...is(...is(...)...)...)

On Wednesday, 10 April 2019 at 02:19:42 UTC, Adam D. Ruppe wrote:
> On Tuesday, 9 April 2019 at 20:45:18 UTC, Alex wrote:
>> On Tuesday, 9 April 2019 at 18:33:21 UTC, Seb wrote:
>>> Have you considered writing a DIP?
>>
>> No, because I expect it won't even be considered.
>
> You won't pass review if you don't show knowledge of the existing language, but there's a lot of people who are interested in a few changes.
>
> A few years ago, there was a proposal for a magical `meta` namespace that would call into the compiler for syntax sugar on CT reflection. The old idea was to replace `__traits(X, args)` with `meta.X(args)` - pure syntax change - but you could perhaps revive and expand that as the entry point to your new idea and get some people on board.

Yeah, but that isn't really effective, virtually identical. I am talking about one level higher abstraction. I mean, it's all the same but I'd like things to be sorta based in oop and ranges(UFCS like stuff). I haven't though about it enough to know how it would work out but I imagine things could be done very nicely. E.g., some accessors get information and some process that information and one can combine and match as needed. X.Type.name, X.Members.Select!"foo".Memebers.Select!"bar"(assuming foo is an aggregate), etc.

Range functionality could even be used to process arrays(I guess the return of the traits would need to be ranges).

> I'm skeptical of any reflection library, but compiler changes might be able to change that. The status quo for D's reflection libraries are:
>
> 1) weird bugs and/or omissions since the language does not let you express all the function parameter details as return values or local variables.
>
> 2) slow compile times (you said your thing was 10 seconds! that's utterly unacceptable)
>

It actually is not to bad. I haven't done much testing but at least repeatedly calling the Reflect on the same base type added little overhead(it wasn't scaling linearly with the number of calls). So maybe it has a chance. Still will never be as efficient as doing it internally though.

> 3) not actually significantly easier to use than language built-ins (if they are actually easier to use at all!)
>
>
> So any library without compiler changes is, as of today, I believe *impossible* to get right, and getting as close as you can is horribly slow. So, your DIP would necessarily include language+compiler changes.
>
> And the compiler is *already* (almost) fully capable, so changing that needs to show that the change is worth it. Maybe you can do that, even little cosmetic things can indeed be a win, but to write a convincing case here, you'll need to compare and contrast various cases.

I'd definitely rather see this in the compiler. I just no zero about Dmd's design and the specifics. It's not something I'm capable of doing without a significant investment that I'm not willing to do(only if I could add the design on top without having to "learn" everything about the compiler).


My goal was to sort of bridge the gap of the ideal solution and what we have. Something I could use personally to solve most of the frustrations I have but I wasn't thinking of trying to make it in to something ideal.