Thread overview
Re: staticIndexOf is incredibly slow and memory intensive
Aug 26, 2012
Andrej Mitrovic
Aug 26, 2012
Artur Skawina
Aug 26, 2012
Philippe Sigaud
Aug 26, 2012
Artur Skawina
Aug 26, 2012
Andrej Mitrovic
Aug 27, 2012
Philippe Sigaud
Aug 27, 2012
Timon Gehr
Aug 27, 2012
Andrej Mitrovic
August 26, 2012
On 8/26/12, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:
> If I replace it
> with my own hardcoded function below it takes only 5 seconds and uses
> 150 MB RAM. This is what the function looks like:

An even faster version, twice as fast:

template myStaticIndexOf(T, TList...)
{
    static if (!__traits(compiles, TList[0]))
        enum myStaticIndexOf = -1;
    else
    static if (is(T == TList[0]))
        enum myStaticIndexOf = 0;
    else
    // repeat..
}
August 26, 2012
On 08/26/12 23:03, Andrej Mitrovic wrote:
> Maybe this is CTFE to blame more than the function itself.
> 
> I have a project where I have a TypeTuple that holds a class tree of a wrapped C++ library. The tuple is iterated from in several places where an index has to be retrieved. Compiling this project takes 46 seconds when using staticIndexOf and uses 700 MB RAM. If I replace it with my own hardcoded function below it takes only 5 seconds and uses 150 MB RAM. This is what the function looks like:
> 
> template myStaticIndexOf(T, TList...)
> {
>     static if (is(typeof(T == TList[0])) && is(T == TList[0]))
>         enum myStaticIndexOf = 0;
>     else
>     static if (is(typeof(T == TList[1])) && is(T == TList[1]))
>         enum myStaticIndexOf = 1;
>     else
>     // ... and so on ...
> }
> 
> The body is pregenerated of course, using mixin() would slow down
> compilation here as well.
> 
> When wrapping larger libraries (and hence having a larger TypeTuple) and using staticIndexOf the memory usage becomes so high that the compiler runs out of memory and crashes.
> 
> I really think it sucks that I have to resort to manually pre-generating a function body externally as if I were using a lame (CTFE-wise) language like C++03.
> 
> D *has* to be better than this..

   template staticIndexOf(T, TS...) {
      enum staticIndexOf = {
         foreach(size_t i, I; TS)
            static if (is(typeof(T == I)) && is(T == I))
               return i;
         return -1;
      }();
   }

But i have no idea how it will do - it appears to be faster than the std one, but i couldn't really test it, because the std one keeps failing when i try it with a large enough tuple...

artur
August 26, 2012
On Sun, Aug 26, 2012 at 11:58 PM, Artur Skawina <art.08.09@gmail.com> wrote:

>    template staticIndexOf(T, TS...) {
>       enum staticIndexOf = {
>          foreach(size_t i, I; TS)
>             static if (is(typeof(T == I)) && is(T == I))
>                return i;
>          return -1;
>       }();
>    }

Is the is(typeof(T == I)) really necessary? If T == I is an error,
then is() returns false, or am I mistaken?
August 26, 2012
On 08/27/12 00:31, Philippe Sigaud wrote:
> On Sun, Aug 26, 2012 at 11:58 PM, Artur Skawina <art.08.09@gmail.com> wrote:
> 
>>    template staticIndexOf(T, TS...) {
>>       enum staticIndexOf = {
>>          foreach(size_t i, I; TS)
>>             static if (is(typeof(T == I)) && is(T == I))
>>                return i;
>>          return -1;
>>       }();
>>    }
> 
> Is the is(typeof(T == I)) really necessary? If T == I is an error,
> then is() returns false, or am I mistaken?
> 

It didn't look necessary to me either; i took the condition verbatim from Andrej's example. Removing the check made no difference to performance, so i kept it to minimize the diff between both versions.

artur
August 26, 2012
On 8/27/12, Philippe Sigaud <philippe.sigaud@gmail.com> wrote:
> Is the is(typeof(T == I)) really necessary? If T == I is an error,
> then is() returns false, or am I mistaken?
>

In my OP example I've had to use a typeof() test because otherwise I
could get an out of bounds compile-time error. The is() expression
doesn't seem to gag these errors, e.g.:

void main()
{
    alias TypeTuple!(int) x;
    static if (is(int == x[4]))
    {
    }
}

test.d(17): Error: tuple index 4 exceeds 1

I just noticed something though, if the order is swapped there's no error:
void main()
{
    alias TypeTuple!(int) x;
    static if (is(x[4] == int))
    {
    }
}

Bug maybe?
August 27, 2012
On Mon, Aug 27, 2012 at 1:00 AM, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:

>
> test.d(17): Error: tuple index 4 exceeds 1
>
> I just noticed something though, if the order is swapped there's no error:
> void main()
> {
>     alias TypeTuple!(int) x;
>     static if (is(x[4] == int))
>     {
>     }
> }
>
> Bug maybe?

Certainly.
August 27, 2012
On 08/27/2012 07:12 AM, Philippe Sigaud wrote:
> On Mon, Aug 27, 2012 at 1:00 AM, Andrej Mitrovic
> <andrej.mitrovich@gmail.com> wrote:
>
>>
>> test.d(17): Error: tuple index 4 exceeds 1
>>
>> I just noticed something though, if the order is swapped there's no error:
>> void main()
>> {
>>      alias TypeTuple!(int) x;
>>      static if (is(x[4] == int))
>>      {
>>      }
>> }
>>
>> Bug maybe?
>
> Certainly.
>

Actually it is according to the specification:

"3. is ( Type == TypeSpecialization )
The condition is satisfied if Type is semantically correct and is the same type as TypeSpecialization."

=> TypeSpecialization is compiled without suppressing errors.
August 27, 2012
On 8/27/12, Philippe Sigaud <philippe.sigaud@gmail.com> wrote:
> Certainly.

http://d.puremagic.com/issues/show_bug.cgi?id=8593