On Monday, 21 June 2021 at 14:15:27 UTC, SealabJaster wrote:

SELECT * FROM people WHERE age >= $1;

You can now get the Variable declaration as reflection parameter.
also you can now pass scopes ;)
Look at the following code:

import core.reflect.type;
import core.reflect.node;
import core.reflect.decl;

int[2] myArr;

static immutable arr_node = nodeFromName("myArr");
static immutable arr_type = cast(immutable TypeArray) (cast(immutable VariableDeclaration)arr_node).type;
static assert(
    arr_type.kind == "sarray" // this name is subject to change.
    // TypeKinds are supposed to be an enum not a string but I didn't feel like implementing it yet :)
    && arr_type.identifier == "int[2]"
    && arr_type.nextOf.identifier == "int"
    && arr_type.dim == 2
);

pragma(msg, arr_node);

this will output:
VariableDeclaration("myArr", [], 0, "", TypeArray("sarray", 4u, 8u, "int[2]", Type("int", 4u, 4u, "int"), 2LU))
where VariableDeclaration is defined as

class Declaration : Node
{
    string name;
    Node[] attributes;
    Linkage linkage;
    string comment;

    abstract immutable DeclarationKind kind() pure;
}

class VariableDeclaration : Declaration
{
    Type type;
    override final immutable DeclarationKind kind() pure { return DeclarationKind.VariableDeclaration; }
}

type is defined as:

class Type : Node
{
    string kind;
    uint alignSize;
    ulong size;
    string identifier = null; /// optional Types may be anonymous
}

class TypeNext : Type
{
    Type nextOf;
}
class TypeArray : TypeNext
{
    int dim;
}

so this reads:

name: "myArr",
attributes: [],
Linkage: 0, //(Linkage.init) we don't capture it yet
comment: "",
type: {
  kind: "sarray",
  alignSize: 4
  size: 8,
  identifier: "int[2]",

  nextOf: { // element type
    kind: "int",
    alignSize: 4,
    size: 4,
    identifier: "int"
  },
dim: 2
}

I am rather happy with the structure of the output actually.
The implementation is such much cleaner than type functions are as well ;)

Let me know what you think.

P.S. I do apologize for this very long post, I don't really know how to present this work in progress stuff properly.

On Wednesday, 23 June 2021 at 21:06:23 UTC, Stefan Koch wrote:

I very much like where this is headed: "Meta programming is just programming. Here are the very-nearly-self-documenting objects that you'll be using".

On Wednesday, 23 June 2021 at 21:06:23 UTC, Stefan Koch wrote:

I like it so far. I can already foresee the arguments about whether capturing comments is something it should do >:D.

My main frame of reference will always be from C#, simply because it's the only runtime reflection I'm used to:

see Expression trees: https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/expression-trees/#creating-expression-trees-from-lambda-expressions

see Type reflection: https://docs.microsoft.com/en-us/dotnet/framework/reflection-and-codedom/reflection

Obviously D's potential ability for reflection will be different and have different capabilities, but I'm sadly not able to really think up anything unique or different.

Other people of course will have more of a better eye for specific details and pain points, something I'm unable to provide at the moment.

Keep up the good work though!

On Thursday, 24 June 2021 at 06:44:49 UTC, SealabJaster wrote:

You missed out some toys,

F# code quotations,

https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/code-quotations

C++/CLI

https://docs.microsoft.com/en-us/cpp/dotnet/dotnet-programming-with-cpp-cli-visual-cpp?view=msvc-160

Code Generators,

https://github.com/dotnet/roslyn/blob/main/docs/features/source-generators.md

C++/CLI and F# might feel like cheating, but that is the beauty of having a polyglot runtime sharing library code.

On Sunday, 20 June 2021 at 22:51:06 UTC, Stefan Koch wrote:

No more assumptions now I have proof :P

check the following example:

import core.reflect.type;
import core.reflect.node;
import core.reflect.decl;

// reflection functions may ne annotated with @(core.reflect)
// to prevent code generation for them
// in this case the function itself does not call reflection functions
// such as nodeFromName. but if it did we would have to prevent codegen
// as nodeFromName does not have a function body
@(core.reflect) immutable(Type) getType(immutable Node node)
{
  if (immutable vd = cast(immutable VariableDeclaration)node)
  {
     return vd.type;
  }

  return null;
}

int[2] myArr;
static immutable arr_node = nodeFromName("myArr");

static immutable arr_type =
  cast(immutable TypeArray)
    (cast(immutable VariableDeclaration)arr_node).type;

static assert(arr_type.identifier == "int[2]"
  && arr_type.nextOf.identifier == "int"
  && arr_type.dim == 2
);

pragma(msg, "typeof(x).stringof before decl: '", nodeFromName("x").getType().identifier, "'");
typeof(myArr)* x;
pragma(msg, "typeof(x).stringof after decl: '", nodeFromName("x").getType().identifier, "'");

The output of this will be:

typeof(x).stringof before decl: 'typeof(myArr)*'
typeof(x).stringof after decl: 'int[2]*'

as you can see the implicitly forward declared symbol x "changes" or rather "refines" and resolves it's type after it has been actually declared.
I hope I can find a way to compute a schedule that will leave a symbol in a stable state as reflection is not supposed to be able to modify anything.

On Thursday, 24 June 2021 at 11:53:00 UTC, Stefan Koch wrote:

more news on this.
I am now forcing the resolution of types as part of the reflection.
That does get rid of the uncomplicated forward refs.

Also I checked that the data can be used at runtime without issues.

If you run to following program:

    extern (C) void*** fnXXfn(void* p, void** pp);
    static immutable fn_x = cast(immutable FunctionDeclaration) nodeFromName("fnXXfn");
    void main()
    {
      import core.stdc.stdio;
      printf(StructToString(fn_x).ptr);
    }


    string StructToString(S)(S _struct, uint indent_level = 1)
    {
      import std.conv : to;

      char[] indent_m;
      indent_m.length = indent_level * 4;
      indent_m[] = ' ';
      string indent = cast(string) indent_m;

      string result = S.stringof ~ " = {\n";

      //TODO: if it's a class cast to base_type;

      foreach(i, e;_struct.tupleof)
      {
        auto name = __traits(identifier, _struct.tupleof[i]);
        result ~= indent ~ name ~ ": ";
        enum isAggregate(T) = is(T == class) || is(T == struct);
        if (e)
        {
            static if (is(typeof(e) : const(char)[]))
            {
               result ~= "\"" ~ e ~ "\"" ~ "\n";
            }
            else static if (is(typeof(e) : E[], E))
            {
                if (!e.length)
                {
                  result ~= "[]\n";
                }
                else
                {
                    result ~= "[\n";
                    indent ~= "    ";
                    foreach(elem;e)
                    {
                        pragma(msg, "isArray: " , typeof(e));
                        static if (isAggregate!(typeof(elem)))
                        {
                            result ~= indent ~ StructToString(elem, indent_level + 2) ~ ",\n";
                        }
                        else
                        {
                           result ~= indent ~ to!string(elem) ~ "\n";
                        }
                    }
                    indent = indent[0 .. $ - 4];
                    result ~= indent ~ "]\n";
                }
            }
            else static if (isAggregate!(typeof(e)))
            {
                result ~= StructToString(e, indent_level + 1) ~ "\n";
            }
            else
            {
               result ~= to!string(e) ~ "\n";
            }
        }
        else
        {
            result ~= "null\n";
        }
      }
      result ~= indent[4 .. $] ~ "}";
      return result;
    }

it will give you the output:

    immutable(FunctionDeclaration) = {
        type: immutable(FunctionType) = {
            returnType: immutable(Type) = {
                kind: "pointer"
                alignSize: 8
                size: 8
                identifier: "void***"
            }
            parameterTypes: [
                immutable(FunctionParameter) = {
                    type: immutable(Type) = {
                        kind: "pointer"
                        alignSize: 8
                        size: 8
                        identifier: "void*"
                    }
                    identifier: "p"
                },
                immutable(FunctionParameter) = {
                    type: immutable(Type) = {
                        kind: "pointer"
                        alignSize: 8
                        size: 8
                        identifier: "void**"
                    }
                    identifier: "pp"
                },
            ]
        }
        parameters: null
        fbody: null
    }

If you look closely you will see that it matches type of the function fnXXfn
Declared in the code up above.
If my printout code would recurse into super-classes you would also be able to see name of the declaration.
However I didn't do that yet ;)
This is just an example to show that these reflection classes can be used at runtime as well as at compile time.

Lastly you can play with this yourself if you do:
git clone https://github.com/UplinkCoder/dmd -b core_reflect
and
git clone https://github.com/UplinkCoder/druntime -b core_reflect
and
git clone https://github.com/dlang/phobos -b v2.094.2
then you go into build it
then you build phobos
you might have to remove '-transition=complex' from the makefiles in both druntime and phobos.

and now you should be able to run the exmaple posted above.

I am not quite sure why; but although this is based on ~master as of roughly 5 days ago
phobos only seems to compile if you checkout v2.094.2 ...

On Thursday, 24 June 2021 at 09:17:15 UTC, Paulo Pinto wrote:

I really need to get around to learning F#, it always looks like such an interesting language.

e.g. https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/units-of-measure

On Friday, 25 June 2021 at 15:07:54 UTC, Stefan Koch wrote:

Hi there, I have just been able to do the first thing I wanted to use core reflect for.
dynamically loading function pointers for debugging functions.
In case the function cannot be loaded because it's not running in the debug environment I want the function pointers to be initialized to dummy functions which do nothing.

Let's have a look at the code:

import core.reflect.reflect;
static immutable decls = declarationsFromTokenString(
q{
//  extern (C) @nogc pure nothrow __gshared {
     void __itt_sync_create (void *addr, const char *objtype, const char *objname, int attribute);
     void  __itt_sync_cancel (void* addr);
     void __itt_sync_acquired (void* addr);
     void  __itt_sync_releasing (void* addr);
     void __itt_sync_prepare (void* addr);
//  }
});
import core.reflect.nodeToSource;

pragma(msg, () {
    string result;

    foreach(d;decls)
    {
        if (auto fd = cast(FunctionDeclaration) d)
        {
            result ~= fd
                .makeFunctionVariableWithDummyInit()
                .nodeToSource()
            ;
        }
    }
    return result;
} ());

VariableDeclaration makeFunctionVariableWithDummyInit(const FunctionDeclaration fd) {
  auto type = new TypePointer();
  type.nextOf = cast()fd.type;
  auto name = fd.name;

  auto result = new VariableDeclaration();
  result.type = type;
  result.name = name;
  auto func = new FunctionLiteral();
  func.parameters = cast()fd.parameters;
  func.fbody = makeDummyBody(fd.type.returnType);
  result._init = func;
  return result;
}

BlockStatement makeDummyBody(const Type returnType)
{
    BlockStatement stmt = new BlockStatement();

    if (cast(TypeBasic) returnType && returnType.identifier == "void")
    {
        stmt.statements ~= new ReturnStatement();
    }
    else
        assert(0, "non void return not supported as of now");

    return stmt;
}

and the output of this will be

void function (void* addr, char* objtype, char* objname, int attribute) __itt_sync_create = () {
    return ;
};
void function (void* addr) __itt_sync_cancel = () {
    return ;
};
void function (void* addr) __itt_sync_acquired = () {
    return ;
};
void function (void* addr) __itt_sync_releasing = () {
    return ;
};
void function (void* addr) __itt_sync_prepare = () {
    return ;
};

which does exactly what I want it declares function pointers with the same name as the functions extracted from the intel header.
and initializes them to dummy functions be default.
yay.

Creating a string is not how it's supposed to be spliced in though!
rather I will special case mixin to splice in core.reflect directly nodes when it is used on them.
Or maybe something more clever :)

On Friday, 25 June 2021 at 15:07:54 UTC, Stefan Koch wrote:

Hi there, I have just been able to do the first thing I wanted to use core reflect for.
dynamically loading function pointers for debugging functions.
In case the function cannot be loaded because it's not running in the debug environment I want the function pointers to be initialized to dummy functions which do nothing.

Let's have a look at the code:

import core.reflect.reflect;
static immutable decls = declarationsFromTokenString(
q{
//  extern (C) @nogc pure nothrow __gshared {
     void __itt_sync_create (void *addr, const char *objtype, const char *objname, int attribute);
     void  __itt_sync_cancel (void* addr);
     void __itt_sync_acquired (void* addr);
     void  __itt_sync_releasing (void* addr);
     void __itt_sync_prepare (void* addr);
//  }
});
import core.reflect.nodeToSource;

pragma(msg, () {
    string result;

    foreach(d;decls)
    {
        if (auto fd = cast(FunctionDeclaration) d)
        {
            result ~= fd
                .makeFunctionVariableWithDummyInit()
                .nodeToSource()
            ;
        }
    }
    return result;
} ());

VariableDeclaration makeFunctionVariableWithDummyInit(const FunctionDeclaration fd) {
  auto type = new TypePointer();
  type.nextOf = cast()fd.type;
  auto name = fd.name;

  auto result = new VariableDeclaration();
  result.type = type;
  result.name = name;
  auto func = new FunctionLiteral();
  func.parameters = cast()fd.parameters;
  func.fbody = makeDummyBody(fd.type.returnType);
  result._init = func;
  return result;
}

BlockStatement makeDummyBody(const Type returnType)
{
    BlockStatement stmt = new BlockStatement();

    if (cast(TypeBasic) returnType && returnType.identifier == "void")
    {
        stmt.statements ~= new ReturnStatement();
    }
    else
        assert(0, "non void return not supported as of now");

    return stmt;
}

and the output of this will be

void function (void* addr, char* objtype, char* objname, int attribute) __itt_sync_create = () {
    return ;
};
void function (void* addr) __itt_sync_cancel = () {
    return ;
};
void function (void* addr) __itt_sync_acquired = () {
    return ;
};
void function (void* addr) __itt_sync_releasing = () {
    return ;
};
void function (void* addr) __itt_sync_prepare = () {
    return ;
};

which does exactly what I want it declares function pointers with the same name as the functions extracted from the intel header.
and initializes them to dummy functions be default.
yay.

Creating a string is not how it's supposed to be spliced in though!
rather I will special case mixin to splice in core.reflect directly nodes when it is used on them.
Or maybe something more clever :)

On Tuesday, 29 June 2021 at 14:28:36 UTC, Stefan Koch wrote:

void function (void* addr, char* objtype, char* objname, int attribute) __itt_sync_create = () {
    return ;
};
void function (void* addr) __itt_sync_cancel = () {
    return ;
};
void function (void* addr) __itt_sync_acquired = () {
    return ;
};
void function (void* addr) __itt_sync_releasing = () {
    return ;
};
void function (void* addr) __itt_sync_prepare = () {
    return ;
};

Oops as you can see the output the result is not as intended.
The parameters are missing.
That's because we haven't run semantic yet and therefore we have to get the parameters from the type.

if you replace the makeFunctionVariableWithDummyInit function from the previous post with this one:

VariableDeclaration makeFunctionVariableWithDummyInit(const FunctionDeclaration fd) {
  auto type = new TypePointer();
  type.nextOf = cast()fd.type;
  auto name = fd.name;

  auto result = new VariableDeclaration();
  result.type = type;
  result.name = name;
  auto func = new FunctionLiteral();

  typeof(func.parameters) parameters = [];
  parameters.length = fd.type.parameterTypes.length;

  foreach(i, p;fd.type.parameterTypes)
  {
   parameters[i] = new typeof(parameters[0])();
   parameters[i].name = p.identifier;
   parameters[i].type = cast()p.type;
  }
  func.parameters = parameters;
  func.fbody = makeDummyBody(fd.type.returnType);
  result._init = func;
  return result;
}

The output will be correct:

void function (void* addr, char* objtype, char* objname, int attribute) __itt_sync_create = (void* addr, char* objtype, char* objname, int attribute) {
    return ;
};
void function (void* addr) __itt_sync_cancel = (void* addr) {
    return ;
};
void function (void* addr) __itt_sync_acquired = (void* addr) {
    return ;
};
void function (void* addr) __itt_sync_releasing = (void* addr) {
    return ;
};
void function (void* addr) __itt_sync_prepare = (void* addr) {
    return ;
};