I just got such an error while trying to instantiate a rather longish template. This is pretty much a blocker for more complex template code. Couldn't symbol names be truncated at some predefined length ?

It also seems like long symbols are killing DMD performance-wise as they can eat lots of RAM while compiling. Then mem allocations + cache misses + GC stalls make it crawl. I've tried instantiating a struct template (the Tuple from my 'bind' lib) with a nesting level of 50 and before I managed to kill DMD, it ate 700MB of virtual mem out of my 512MB RAM system.

Can anything be done with it ? It seems like compile times and memory use explode when dealing with more complex templates.


-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/

In article <e8pf88$22ki$1@digitaldaemon.com>, Tom S says...
>
>I just got such an error while trying to instantiate a rather longish template. This is pretty much a blocker for more complex template code. Couldn't symbol names be truncated at some predefined length ?
>
>It also seems like long symbols are killing DMD performance-wise as they can eat lots of RAM while compiling. Then mem allocations + cache misses + GC stalls make it crawl. I've tried instantiating a struct template (the Tuple from my 'bind' lib) with a nesting level of 50 and before I managed to kill DMD, it ate 700MB of virtual mem out of my 512MB RAM system.
>
>Can anything be done with it ? It seems like compile times and memory use explode when dealing with more complex templates.
>
>
>-- 
>Tomasz Stachowiak  /+ a.k.a. h3r3tic +/


Tsk, tsk, tsk... what mad template experiments are you up to now, Tom?  Your computer must be smoking at this point. lol!

-JJR

John Reimer wrote:
> Tsk, tsk, tsk... what mad template experiments are you up to now, Tom?  Your
> computer must be smoking at this point. lol!

You know... the usual stuff ;D
I'm just working with a *bit* more complex mixins, like this one:

	mixin Input!(`in`)
		.field!(DataStream!(vec3), `positions`)
		.field!(DataStream!(vec3), `normals`)
	.end input;

When I add too many fields, DMD eats many many rams and gives me that error.


-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/

Tom S wrote:
> ...DMD eats many many rams...

*Gasp!*  Those poor sheep/goats!

I would be interested in seeing exactly what your 'Input' mixin does.  The use of self-referancing returns to initialize it is pretty nifty though.  Might be a way to get rid of the '.end' at the end, though?

The error baffles me, though.

-- Chris Nicholson-Sauls

Chris Nicholson-Sauls wrote:
> Tom S wrote:
>> ...DMD eats many many rams...
> 
> *Gasp!*  Those poor sheep/goats!
> 
> I would be interested in seeing exactly what your 'Input' mixin does.  The use of self-referancing returns to initialize it is pretty nifty though.  Might be a way to get rid of the '.end' at the end, though?
> 
> The error baffles me, though.

Symbols are limited to ~255 chars on Win32.  I think it's a limitation of the object file format.  "symbol too long" errors are painfully common in C++, though they're typically truncated to the max length to allow for debugging.


Sean

Sean Kelly wrote:
> Chris Nicholson-Sauls wrote:
>> Tom S wrote:
>>> ...DMD eats many many rams...
>>
>> *Gasp!*  Those poor sheep/goats!
>>
>> I would be interested in seeing exactly what your 'Input' mixin does.  The use of self-referancing returns to initialize it is pretty nifty though.  Might be a way to get rid of the '.end' at the end, though?
>>
>> The error baffles me, though.
> 
> Symbols are limited to ~255 chars on Win32.  I think it's a limitation of the object file format.  "symbol too long" errors are painfully common in C++, though they're typically truncated to the max length to allow for debugging.

I thought the OMF lib-imposed limit was about 4K. I've just made a template with a symbol name of 480 characters, and that was OK.
If Tuple!() works with a nesting level of 50, the limit would have to be pretty big.
Maybe in the future, we'll be able to add some kind of 'final' declaration to templates, which would indicate that they don't need to be stored in the object file or library, and can be discarded from the symbol table as soon as they are instantiated. This would be particularly useful for recursive templates, which can easily spew reams of garbage into the object file.

Chris Nicholson-Sauls wrote:
> Tom S wrote:
>> ...DMD eats many many rams...
> 
> *Gasp!*  Those poor sheep/goats!
> 
> I would be interested in seeing exactly what your 'Input' mixin does.

There you go: http://158.75.59.9/~h3/code/processing.rar
It's in processing/input.d


> The use of self-referancing returns to initialize it is pretty nifty though.  Might be a way to get rid of the '.end' at the end, though?

Hmm... check the code and decide for yourself... I thought using the 'end' was the most straightforward approach. Otherwise the implicit name promotion wouldn't work and I'd have to use field.field all over the place. 'field.add' could work, but I considered forcing the user to add one 'end' to be simpler.


-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/

Don Clugston wrote:
> If Tuple!() works with a nesting level of 50, the limit would have to be pretty big.

Yeah, and it would cause DMD to eat RAMs like a medium-sized dragon.


> Maybe in the future, we'll be able to add some kind of 'final' declaration to templates, which would indicate that they don't need to be stored in the object file or library, and can be discarded from the symbol table as soon as they are instantiated. This would be particularly useful for recursive templates, which can easily spew reams of garbage into the object file.

My thoughts exactly :) Right now I'd be happy with anything that worked though...


And while we're at recursive templates, don't you think that the limitation for recursive templates, which causes the error 'template instance recursive template expansion for template argument X' is pretty arbitary ? It can be hacked around, after all:


// tmp.d
import std.stdio;


struct Foo(T/*, int dummy=0*/) {
	Foo!(Foo!(T/*, dummy+1*/)) rec()() {    // line 5
		Foo!(Foo!(T/*, dummy+1*/)) res;
		return res;
	}
}


void main() {
	Foo!(int) a;
	writefln(typeid(typeof(a)));
	auto b = a.rec!()();
	writefln(typeid(typeof(b)));
}
// ----

tmp.d(5): template instance recursive template expansion for template argument Foo

But when you uncomment the 'dummies', it compiles and runs without any problems...

Yeah, I know, the error is an artifact of how the instantiation works, that it's not 'lazy' like in C++. But since it can be fooled into working, couldn't DMD apply the hack automagically ?


-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/

Don Clugston wrote:
> Sean Kelly wrote:
>> Chris Nicholson-Sauls wrote:
>>> Tom S wrote:
>>>> ...DMD eats many many rams...
>>>
>>> *Gasp!*  Those poor sheep/goats!
>>>
>>> I would be interested in seeing exactly what your 'Input' mixin does.  The use of self-referancing returns to initialize it is pretty nifty though.  Might be a way to get rid of the '.end' at the end, though?
>>>
>>> The error baffles me, though.
>>
>> Symbols are limited to ~255 chars on Win32.  I think it's a limitation of the object file format.  "symbol too long" errors are painfully common in C++, though they're typically truncated to the max length to allow for debugging.
> 
> I thought the OMF lib-imposed limit was about 4K. I've just made a template with a symbol name of 480 characters, and that was OK.

Perhaps it is--it's been a while since I developed on Windows.  I only remember seeing the errors quite often.

> Maybe in the future, we'll be able to add some kind of 'final' declaration to templates, which would indicate that they don't need to be stored in the object file or library, and can be discarded from the symbol table as soon as they are instantiated. This would be particularly useful for recursive templates, which can easily spew reams of garbage into the object file.

This would be nice.  I generally expect this to occur anyway, but it would be nice to have some sort of insurance.


Sean

Tom S wrote:
> Don Clugston wrote:
>> If Tuple!() works with a nesting level of 50, the limit would have to be pretty big.
> 
> Yeah, and it would cause DMD to eat RAMs like a medium-sized dragon.
> 
> 
>> Maybe in the future, we'll be able to add some kind of 'final' declaration to templates, which would indicate that they don't need to be stored in the object file or library, and can be discarded from the symbol table as soon as they are instantiated. This would be particularly useful for recursive templates, which can easily spew reams of garbage into the object file.
> 
> My thoughts exactly :) Right now I'd be happy with anything that worked though...
> 
> 
> And while we're at recursive templates, don't you think that the limitation for recursive templates, which causes the error 'template instance recursive template expansion for template argument X' is pretty arbitary ? It can be hacked around, after all:
> 
> 
> // tmp.d
> import std.stdio;
> 
> 
> struct Foo(T/*, int dummy=0*/) {
>     Foo!(Foo!(T/*, dummy+1*/)) rec()() {    // line 5
>         Foo!(Foo!(T/*, dummy+1*/)) res;
>         return res;
>     }
> }
> 
> 
> void main() {
>     Foo!(int) a;
>     writefln(typeid(typeof(a)));
>     auto b = a.rec!()();
>     writefln(typeid(typeof(b)));
> }
> // ----
> 
> tmp.d(5): template instance recursive template expansion for template argument Foo
> 
> But when you uncomment the 'dummies', it compiles and runs without any problems...
> 
> Yeah, I know, the error is an artifact of how the instantiation works, that it's not 'lazy' like in C++. But since it can be fooled into working, couldn't DMD apply the hack automagically ?

I don't know.  To do so would be like allowing:

struct S
{
    S val;
}

I'm not sure I'd want the compiler to try and "fix" a mistake I'd made involving this.  Adding an integer is pretty easy anyway.


Sean