Hello,

I recently started using D, and I am exploring some of the functionality in CTFE. I ran into a case that I can't get past. The goal is to take a JSON API Schema and convert it into D types using CTFE. The problem is, the types can be self-referential.

A simplified form of the language would look like:

{ "A" : { "type": struct, "fields" : { "b": { "type": "string" }, "a" : { "type": "A" }}}

This would yield a type "A" that has two fields, "a" and "b". Where "a" is a string and "b" is of type "A". Or in D:

struct A {
  string b;
  A* a;
}

This is a simplified example, but imagine this is highly nested to generate arbitrarily complex types that could reference on one another.

I got this working in the simple case if none of the fields are self-referential, but it breaks as soon as there is a loop.

Here is a gist which demonstrates the issue: https://gist.github.com/rbscott/ee0f3ba94296f9c8224a8c4c13c2f026.

I think I can workaround this by with string mixins everywhere, but the structure generation logic is complex enough that I would prefer not to do that. This approach could be totally off, so any suggestions would be greatly appreciated!

thanks,
rbscott

On Sunday, 17 November 2019 at 07:13:52 UTC, rbscott wrote:
> Hello,
>
> I recently started using D, and I am exploring some of the functionality in CTFE. I ran into a case that I can't get past. The goal is to take a JSON API Schema and convert it into D types using CTFE. The problem is, the types can be self-referential.
>
> A simplified form of the language would look like:
>
> { "A" : { "type": struct, "fields" : { "b": { "type": "string" }, "a" : { "type": "A" }}}
>
> This would yield a type "A" that has two fields, "a" and "b". Where "a" is a string and "b" is of type "A". Or in D:
>
> struct A {
>   string b;
>   A* a;
> }
>
> This is a simplified example, but imagine this is highly nested to generate arbitrarily complex types that could reference on one another.
>
> I got this working in the simple case if none of the fields are self-referential, but it breaks as soon as there is a loop.
>
> Here is a gist which demonstrates the issue: https://gist.github.com/rbscott/ee0f3ba94296f9c8224a8c4c13c2f026.
>
> I think I can workaround this by with string mixins everywhere, but the structure generation logic is complex enough that I would prefer not to do that. This approach could be totally off, so any suggestions would be greatly appreciated!
>
> thanks,
> rbscott

But "a" is not strictly of type A, but rather A*, which makes a big difference.

Anyway, upon the mixin template instantiation, you can try to 'declare' all your struct's on top and then properly 'define' them afterwards:

struct A;
struct B;

struct A {
  B* b;
}

struct B {
  A* a;
}


Have you tried that?

On Sunday, 17 November 2019 at 07:13:52 UTC, rbscott wrote:
> I got this working in the simple case if none of the fields are self-referential, but it breaks as soon as there is a loop.
>
> Here is a gist which demonstrates the issue: https://gist.github.com/rbscott/ee0f3ba94296f9c8224a8c4c13c2f026.

This is a DMD bug, you should open an issue.

It's part of a bigger family of issues caused by how DMD's semantic analysis is architectured. I tried to tackle it some time ago but didn't see it through the end :

https://github.com/dlang/dmd/pull/7018
https://forum.dlang.org/post/auvbfyzmqjxtfhkvmkxu@forum.dlang.org

As I see it the proper fix requires rewritting a large part of the compiler frontend, but this is definitely something worth doing at some point to make DMD less brittle esp. regarding advanced usage of mixins/CTFE (I don't have time anymore, and IMHO someone should be paid 2 months to work full-time on re-architecturing DMD for this)

On Sunday, 17 November 2019 at 09:16:19 UTC, Claude wrote:
> Anyway, upon the mixin template instantiation, you can try to 'declare' all your struct's on top and then properly 'define' them afterwards:
>
> struct A;
> struct B;
>
> struct A {
>   B* b;
> }
>
> struct B {
>   A* a;
> }
>
>
> Have you tried that?

Forward declaring structs isn't allowed in D.

On Sunday, 17 November 2019 at 17:35:07 UTC, Elie Morisse wrote:
>
> It's part of a bigger family of issues caused by how DMD's semantic analysis is architectured. I tried to tackle it some time ago but didn't see it through the end :
>
> https://github.com/dlang/dmd/pull/7018
> https://forum.dlang.org/post/auvbfyzmqjxtfhkvmkxu@forum.dlang.org
>
> As I see it the proper fix requires rewritting a large part of the compiler frontend, but this is definitely something worth doing at some point to make DMD less brittle esp. regarding advanced usage of mixins/CTFE (I don't have time anymore, and IMHO someone should be paid 2 months to work full-time on re-architecturing DMD for this)
>
>

Thanks for pointing that out. Given what you understand about the issue, do you have any recommendations for working around this? I am pretty sure a string mixin will work, but hopefully there is an alternative. Maybe a different compiler will work?

> Forward declaring structs isn't allowed in D.

I did try forward declarations. I thought because of "Opaque Pointers", this would work, but it generates a compiler error.

On Sunday, 17 November 2019 at 19:32:58 UTC, rbscott wrote:
> Thanks for pointing that out. Given what you understand about the issue, do you have any recommendations for working around this? I am pretty sure a string mixin will work, but hopefully there is an alternative. Maybe a different compiler will work?

DMD, GDC, LDC all share the same frontend so you'd get the same errors.

As a workaround a single string mixin to generate the full body of Structures would work yes. I can't think of other alternatives..