Thread overview
How can I have those "template instance recursive expansion" errors under control?
Nov 30
realhet
Nov 30
realhet
Dec 01
realhet
Dec 01
realhet
Dec 01
realhet
Dec 01
monkyyy
Dec 03
realhet
November 30

I have a big example code (compared to me) for this particular case, I can't include example as narrowing it down would take several hours.

I just ask you to give me general tips to avoid it.

How can I detect them earlier for example?
Is it a way analyze the --vtemplates compiler output and detect weak spots?

I already know a few effective workarounds (I learned from the other template recursion topics here):

  • Don't use dynamic returns. Put all the cases into static if else chains.
  • If the "template instance recursive expansion" pops up, narrow the import clause to individual names.

I have a vector/matrix/2d-image math module. Most of the times the error pops there because there is a lot of template work there. But I know it can pop anywhere, this time it is inside my traits.isNumeric() inside my vector.opBinary(). Is there a way to access the full instation path? Maybe it could show me a bug.

I just put there some sources near the error message, maybe someone can spot a terrible mistake.
Thank you in advance!

(LDC 1.40 Win64)

private template OperationResultType(string op, A, B)
{
	static if(op.among("<<", ">>", ">>>"))	alias OperationResultType = A;
	else	alias OperationResultType = CommonType!(A, B);
}

auto generateVector(CT, alias fun, T...)(in T args)
{
	static if(anyVector!T)
	{
		Vector!(CT, CommonVectorLength!T) res;
		static foreach(i; 0..res.length)
		res[i] = cast(CT) mixin("fun(", T.length.iota.map!(j => "args["~j.text~"].vectorAccess!i").join(','), ")");
		return res;
	}
	else
	{ return cast(CT) fun(args); }
}

struct Vector(CT, int N)
if(N.inRange(2, 4))
{

   ... lots of code ...

   private static binaryVectorOp(string op, A, B)(in A a, in B b)
   {
	alias CT = OperationResultType!(op, ScalarType!A, ScalarType!B);
	return generateVector!(CT, (a, b) => mixin("a", op, "b") )(a, b);
   }

   auto opBinary(string op, T)(in T other) const
   {
	static if(isNumeric!T || isVector!T)
                //^^^^^^^^^^^  Here the compiler decided that enough is enough :S
                //Error: template instance /+Code: std.traits.isNumeric!(Vector!(float, 3))+/ recursive expansion
	{
		//vector * (scalar or vector)
		return binaryVectorOp!op(this, other);
	}
	else static if(op=="*" && isMatrix!T && T.height==length)
	{
		//vector * matrix
		return other.transpose * this;
		//Opt: this is slow if it is not unrolled.
	}
	else static if(op=="in" && isBounds!T && T.VectorLength == length)
	{ return other.contains(this); }
	else static if(op=="in" && isImage!T && T.Dimension == length)
	{ return other.contains(this); }
	else static if(op.among("+", "*") && isBounds!T && T.VectorLength == length)
	{ return other.opBinary!op(this); }
	else
	{ static assert(false, "Unhandled operation: "~op~" "~T.stringof); }
   }

   ... lots of code ...
}
November 30

On Saturday, 30 November 2024 at 18:11:56 UTC, realhet wrote:

>

narrowing it down would take

I was lucky, I narrowed down to only 2 files.

The recursion error occurs when I call this 'abomination' of a template function:
https://github.com/realhet/hetlib/blob/fe028689791d011cd98bc63042ee76e28fcffad2/het/Math.d#L2271

With a vector color:

image2D(1, 2, RGB(0,0,0))

This 'statically' calls itself: image2D(ivec2(1, 2), RGB(0,0,0))

And generates a 1 step deep recursion:

hetmath.image2D!("", int, int, Vector!(ubyte, 3)).image2D(int __param_0, int __param_1, Vector!(ubyte, 3) __param_2)
hetmath.image2D!("", Vector!(int, 2), Vector!(ubyte, 3)).image2D(Vector!(int, 2) __param_0, Vector!(ubyte, 3) __param_1)
hetmath.image2D!("", Vector!(int, 2), MapResult!(__lambda3, MapResult!(__lambda4, Result))).image2D(Vector!(int, 2) __param_0, MapResult!(__lambda3, MapResult!(__lambda4, Result)) __param_1)
hetmath.d(609): Error: template instance `std.traits.isNumeric!(Vector!(float, 3))` recursive expansion

It doesn't seems too much...

image2D(ivec2(1, 2), 123) and image2D(1, 2, 123)

work well, but then I'm restricted to one color channels. :/

For example this could be a recursion violation:

return image2D!fun(ivec2(args[0], args[1]), args[2..$])

I 'call' itself but with a single 2d vector parameter instead of 2 integer width/height parameters.
I do a lot of these unifications to keep the amnount of code low.

And I learned long ago that better to use a single declaration that supports everything,

auto image2D(alias fun="", A...)(A args) {}

A big overload set with all the possible parameter specializations will be not usable. At least this is my feeling based on my early experiences.

What annoys me the most is that I don't see the mechanics of what things are going towards the template recursion error. :S It's just seems like try and error and sometimes I find the voodoo magic that will solve it for a while :D
But now it's kinda broken for me. (I had a big change since it worked: LDC 1.28->1.40)

December 01
The trick for such a function, is you have the public wrappers that sanitize the input into something that you want to work with, then internally have a function that accepts only that input.
December 01
On Sunday, 1 December 2024 at 00:08:02 UTC, Richard (Rikki) Andrew Cattermole wrote:
> The trick for such a function, is you have the public wrappers that sanitize the input into something that you want to work with, then internally have a function that accepts only that input.

I'm putting things out into smaller functions.
Now I have az image2D_impl() too. And 3 more. But the recursive expansion error is still there.

Now I tracked down wich specific error is that:
"%s `%s` recursive expansion"
It has 2 occurences in dmd/dsymbolsem.d

I thint it's the 2nd one.
```d
if (doSemantic3)
            tempinst.trySemantic3(sc2);

        TemplateInstance ti = tempinst.tinst;
        int nest = 0;
        while (ti && !ti.deferred && ti.tinst)
        {
            ti = ti.tinst;
            if (++nest > global.recursionLimit)
            {
                global.gag = 0; // ensure error message gets printed
                .error(tempinst.loc, "%s `%s` recursive expansion", tempinst.kind, tempinst.toPrettyChars);
                fatal();
            }
        }
```

But how can a standard template like CommonType or isNumeric go nuts recursively? o.O

I will check them, maybe they don't like my Vector/Image structs...
December 01

On Sunday, 1 December 2024 at 00:08:02 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

The trick...

I've managed to 'fix' it. Sadly it's not a rational fix, just voodoo magic :D

When I mixin() my aliases for my Vector types, I have to 'mention' all types first, because if I use the simple alias name from a different module, inside the complicated image2D template, the compiler reaches a limit and gives up. (I'm sure it's not an infinite limit, although I have no data on this, it just works with the magic trick.)

static foreach(T; vectorElementTypes)
static foreach(N; vectorElementCounts)
{
	/+voodoo magic ->+/pragma(msg, Vector!(T, N).stringof[0..0]);
	/+
		Note: BugFix: 241201: Without 'mentioning' the vector type first,
		/+Code: image2D(1, 2, RGB(1, 2, 3))+/ drops a template instance recursion rerror
		at random places when it tries to resolve the RGB alias.
		❗ Must be placed in front of the alias declaration!
		❗ Can't use __traits(compiles, ...) or static assert(...) or mixin(dummy function with variable declaration)
			because those fail with various errors like: "forward reference" error.
	+/
	mixin(iq{alias $(Vector!(T, N).VectorTypeName) = $(Vector!(T, N).stringof); }.text);
}

So this feels like a bug, but it has no simple case to reproduce, it needs a big complexity to fail.

Recap:

This fails in a separate module:

import het.math; void main(){ auto img = image2D(1, 2, RGB(1, 2, 3)); }

This is one ugly fix:

import het.math; void main(){ auto img = image2D(1, 2, Vector!(ubyte, 3)(1, 2, 3)); }
//not using the nice alias name: RGB

This is another ugly fix:

import het.math; void main(){ auto col = RGB(1, 2, 3); auto img = image2D(1, 2, col); }

This is the voodoo magic fix:

import het.math; void main(){ pragma(msg, Vector!(ubyte, 3));auto img = image2D(1, 2, col); }

And if I write the pragma(msg,) into the het.math module, I'm able to express my thoughts with minimal redundancy:

import het.math; void main(){ auto img = image2D(1, 2, RGB(1, 2, 3)); }

This is where I started hobby programming with a positive mood yesterday... I tried a simple thing and it failed... But anyways, a matrix of template types are not simple stuff, not for me, neither for the compiler :D

December 01

On Sunday, 1 December 2024 at 19:55:59 UTC, realhet wrote:

>

On Sunday, 1 December 2024 at 00:08:02 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

The trick...

Update: This not works!

>

This is another ugly fix:

import het.math; void main(){ auto col = RGB(1, 2, 3); auto img = image2D(1, 2, col); }

The trigger of the error is accessing Vector!(ubyte, 3) by the alias WITHOUT mentioning Vector!(ubyte, 3) previously!

A harmless way to 'mention' is -> pragma(msg, (Vector!(ubyte, 3)).stringof[0..0]);

The alias declaration must be in a different module (minimum 2 modules needed to reproduce.).

December 01

On Sunday, 1 December 2024 at 20:29:30 UTC, realhet wrote:

>

Update: This not works!

You have more code in that one file then I try to have in a project, so idk but if your still looking at the file, may as well say my thing

>

But how can a standard template like CommonType or isNumeric go nuts recursively?

I believe phoboes.is* are all terribly implemented, the "contacts" add more code that can fail for less functionality. WHile Im unsure about the isNumberic falls for the same pitfalls as say isRange, it wouldnt surprize me in the least

>

Error: template instance std.traits.isNumeric!(Vector!(float, 3))

I honestly hate what I can tell about your code from this one error message but as a bandaid add

enum isNumberic(T:Vector!(S,N),S,size_T N)=true;

if you have any more recursive types or use other type deduction you will likely need to fix those as well; then ... dont do this type of code again, you are likely just making a recursive type in your opOverloads and your keep finding these sorts of issues

Avoid "contracts", make every function in structs templates even if its a void void function (void foo()()=>static assert(0); compiles); template wont be correct, so make things simple as possible with the ugly syntax and small.


syntax test for posterity

struct data(T,size_t N){}
enum isdata(T:data!(S,N),S,size_t N)=true;
enum isdata(T)=false;

unittest{
	import std;
	isdata!(int).writeln;
	isdata!(data!(int,100)).writeln;
}
December 03

On Sunday, 1 December 2024 at 21:35:43 UTC, monkyyy wrote:

>

On Sunday, 1 December 2024 at 20:29:30 UTC, realhet wrote:

>

Update: This not works!

You have more code in that one file then I try to have in a project, so idk but if your still looking at the file, may as well say my thing

The reason for my big modules is this:
I build them in parallel, one module = one LDC instance. So I just can't have a module for every sneeze :D (Until I make possible to group them...)

To compensate it, I have a lot of region blocks: version(/+$DIDE_REGION region caption+/all){}
I use those to make a hierarchical order inside large modules. These are files, they can be nested.

>

enum isNumberic(T:Vector!(S,N),S,size_T N)=true;

Thanks for the example code, I was surprised that it runs. This crazy enum format with the separate =true and =false was new to me. I've checked isNumeric(), and the same stuff is inside that.

I use this format:

enum isVector(T) = is(T==Vector!(CT, N), CT, int N);
enum isScalar(T) = is(T==bool) || isNumeric!T;

I tried yours, but nothing changed, the current solution to this inter-module template-type-alias problem is:

  1. In the target module: 'mentioning' the vector type auto dummy = Vector!(ubyte, 3)(0); before using the alias RGB(1, 2, 3)
  2. In the math module: 'mentioning' it with a pragma(msg, Vector!(ubyte, 3).stringof[0..0]); (Only this works, not a dummy declaration.)

Both solutions are lame :D
But the one inside the math module is hidden, so the production modules can contain nice non-redundant code.