Progress is moving along nicely with __typeid. Currently all types aside from `struct`s, `class`es and `interface`s are supported:

https://github.com/dlang/druntime/pull/3174

I don't see major issues ahead, so __typeid!(T) should supplant the functionality provided by typeid(T) with no or little magic needed on the compiler side. It would allow deleting a gnarly part of the compiler - so convoluted, in fact, that Walter himself gave up on modularizing it and decided to wait for __typeid to be finished so he throws all that away in one fell swoop. That's going to be fun.

Reading a bit more about TypeInfo and related stuff, I figure (and correct me if I'm wrong) that the entire machinery is currently used for the following purposes (only):

* Built-in associative arrays (which ought to be phased out)

* Inform the GC of what it needs to scan (provided in an awkward manner though, this is subject to a later discussion)

* Dynamic cast information for class objects

* Object creation for Object.factory() (which also should be phased out)

The API provided by TypeInfo has evolved with these uses in mind, so the bulk of it is concerned with primitives for associative arrays such as comparison, hashing, and swapping.

The more exciting part is not just replacing the existing functionality, but taking it forward. In the future:

* Built-in associative arrays should be templated and make no use of typeid at all

* The GC interface should be done through functions, not data structures. __typeid!(T) would provide a function scan(T*) that scans the given object and all pointers it embeds, transitively. That is easy to implement in an efficient manner because the static information about T (and therefore its .tupleof) is available when the function is generated. Templates, wheee!

* Dynamic cast information for class objects should be kept, perhaps made more efficiently than a linear search.

* Object creation primitives should be only provided in an opt-in manner. If a class hierarchy wants to expose a factory function, all it needs to do is inherit a tag interface:

interface DynamicallyConstructible {}

During the generation of __typeid, the inheritance of this tag is detected and the appropriate construction code is generated.

* An important use of __typeid will be to implement Variant "the right way". Using the __typeid instead of the unsightly pointer to handler function in Variant would go a long way toward simplifying it. the __typeid API would be therefore geared toward the needs of that type. I've come to the realization that that type is essential for the use of D in dynamic contexts. I'd name that type Box, put it in druntime, and make it available to Das Besser C.

* Once Box is present, much more is possible. It's not difficult to do introspection during compilation on a class object and expose its methods at runtime. We'd do that only if another tag interface is inherited, again in an opt-in manner:

interface DynamicallyInvokable {}

So it becomes possible to expose objects at runtime, making uses like this possible:

// Inside the library
class Widget : DynamicallyInvokable
{
    int frobnicate(double a, int b) { ... }
}

// Application that does NOT import the widget module
// Load the typeid(Widget) from a dynamic library
TypeInfo ti = loadType("./mylibrary.so", "widget.Widget");
// Make a widget. Note that we do NOT have the Widget class definition!
Object w = ti.make();
// Call Widget.frobnicate an get its result
Box result = ti.invoke(w, "frobnicate", 3.14, 42);
// The Box contains an int
int x = result.get!int;

All of this is well within reach, but it's a fair amount of work.

Oh good so we are finally tackling runtime reflection-esque capabilities!

Obvious concern of mine: how it handles shared library support including when unloading.

On Saturday, 1 August 2020 at 16:12:58 UTC, Andrei Alexandrescu wrote:
> Progress is moving along nicely with __typeid. Currently all types aside from `struct`s, `class`es and `interface`s are supported:
>
> [...]

I'm also interested in the runtime implications but I'm even happier with the compile-time ramifications.  Knocking back complexity while boosting capability?  You betcha!

Judging from the issues forum Walter and the other front-enders have been fighting a rising tide of ICEs.  Sounds like this will help a bunch.  Thanks.

On 2020-08-01 18:12, Andrei Alexandrescu wrote:
> Progress is moving along nicely with __typeid.

I recommend __typeid to be private and have the compiler lowering to __traits(getMember, object, "__typeid") to bypass private. This is to make sure this symbol is not accessed directly.

-- 
/Jacob Carlborg

On 8/1/20 12:23 PM, rikki cattermole wrote:
> Oh good so we are finally tackling runtime reflection-esque capabilities!
> 
> Obvious concern of mine: how it handles shared library support including when unloading.

In an ideal world the GC would automatically close libraries of which code is no longer used. I don't know if that's possible.

On 8/1/20 1:29 PM, Jacob Carlborg wrote:
> On 2020-08-01 18:12, Andrei Alexandrescu wrote:
>> Progress is moving along nicely with __typeid.
> 
> I recommend __typeid to be private and have the compiler lowering to __traits(getMember, object, "__typeid") to bypass private. This is to make sure this symbol is not accessed directly.

That's related to the matter of immutability. Currently all __typeid objects are immutable and live in static read-only storage. That's nice and would make it impossible to mess with them. However, that makes them very unpleasant to use, e.g. you can't use them as out parameters, can't assign them etc. All that old story with tail const.

On 02/08/2020 6:23 AM, Andrei Alexandrescu wrote:
> On 8/1/20 12:23 PM, rikki cattermole wrote:
>> Oh good so we are finally tackling runtime reflection-esque capabilities!
>>
>> Obvious concern of mine: how it handles shared library support including when unloading.
> 
> In an ideal world the GC would automatically close libraries of which code is no longer used. I don't know if that's possible.

globalVar = new SharedLibThingy;

unload(sharedLib);

// ugh oh globalVar still exists!

Something I was thinking about this, would be a kind of 'audit pass' that the GC could offer. Maybe even nullifying references that it finds.

On Saturday, 1 August 2020 at 18:26:54 UTC, Andrei Alexandrescu wrote:
> However, that makes them very unpleasant to use, e.g. you can't use them as out parameters, can't assign them etc.

It seems to me that immutable out params SHOULD work - it would follow the same rules as initializing an immutable class member in a constructor.

An out param is only actually written once, so the implementation just needs to realize it is construction, not assignment, and then perhaps it can be made to work.

I suppose the rule would be an out param may be declared before use (indeed, it must be), but then it could not actually be initialized or assigned outside the function.


void foo(out immutable T t) {
   t = new immutable T; // OK, initial construction
}

immutable T a;

foo(a); // if a was already initialized, this would be an error.
// but if not it should allow it



We do something similar for classes so it seems doable in theory.


 * * *

Of course we should just have const(Object) ref.

On Saturday, 1 August 2020 at 18:26:54 UTC, Andrei Alexandrescu wrote:
> On 8/1/20 1:29 PM, Jacob Carlborg wrote:
>> On 2020-08-01 18:12, Andrei Alexandrescu wrote:
>>> Progress is moving along nicely with __typeid.
>> 
>> I recommend __typeid to be private and have the compiler lowering to __traits(getMember, object, "__typeid") to bypass private. This is to make sure this symbol is not accessed directly.
>
> That's related to the matter of immutability. Currently all __typeid objects are immutable and live in static read-only storage. That's nice and would make it impossible to mess with them. However, that makes them very unpleasant to use, e.g. you can't use them as out parameters, can't assign them etc. All that old story with tail const.

That unpleasantness, and the attendant complexities of the current workarounds, motivates Stefan's type function work.  I agree with beerconf Atila, IIUC, that a good solution to this will probably involve a generalization to some form of restricted type variables.  Perhaps mutable forms that are canonized to unique? immutables.

On Saturday, 1 August 2020 at 16:12:58 UTC, Andrei Alexandrescu wrote:
> * An important use of __typeid will be to implement Variant "the right way". Using the __typeid instead of the unsightly pointer to handler function in Variant would go a long way toward simplifying it. the __typeid API would be therefore geared toward the needs of that type. I've come to the realization that that type is essential for the use of D in dynamic contexts. I'd name that type Box, put it in druntime, and make it available to Das Besser C.

In other languages, a Box is a wrapper that turns a value type into a reference type. [1][2][3] It is not a container for a single dynamically-typed value--in fact, its value is typically statically-typed.

Most languages do not have a type like the one D currently calls Variant, but in those that do, the most widely-accepted name seems to be Any. [4][5]

[1] https://docs.oracle.com/javase/tutorial/java/data/autoboxing.html
[2] https://doc.rust-lang.org/std/boxed/struct.Box.html
[3] https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/types/boxing-and-unboxing
[4] https://en.cppreference.com/w/cpp/utility/any
[5] https://www.scala-lang.org/api/current/scala/Any.html