Kagamin Wrote:

> Jason Spencer Wrote:
> 
> > If I want to avoid the copy, am I relegated back
> > to pointers?
> 
or something like this

struct ReferenceArray!(ArrayType)
{
  ArrayType* back;
  this(byte[] data)
  {
    assert(data.length==ArrayType.sizeof);
    back = cast(ArrayType*)data.ptr;
  }
  //will it be inlined?
  auto opIndex(int column, int row)
  {
    return (*back)[row][column];
  }
}

byte[] data = getData();
ReferenceArray!(float[100][100]) matrix = data;
writeln(matrix[1][2]);

On Wed, 28 Jul 2010 14:52:11 -0400, Jason Spencer <spencer8@sbcglobal.net> wrote:

> I'm working on a program to do statistics on matrices of different sizes, and
> I've run into a handful of situations where I just can't seem to find the
> trick I need.  In general, I'm trying to make my functions work on static
> arrays of the proper size, and template that up for the different sizes I need
> to support.  I appeal to your experience and mercy on the various questions
> below.  Any help on any point appreciated.
>
> 1.  How can I cast a single dimension dynamic array to a multi-dimension
> static array?  I'm trying to do roughly the following:
>
>    auto data = cast(float[100][100])std.file.read(fname, fsize);
>
> which fails on the cast.  Is there any way to treat they memory returned
> from read as a static array?  If I want to avoid the copy, am I relegated back
> to pointers?  Is there a cast path through a pointer that would work?  I think
> I'm essentially asking if I can make a reference to a static array and assign
> that to the memory contained in a dynamic array.  I have a suspicion about the
> answer....
>
> 2.  To work around 1., I've left one dimension of the cast floating.  This
> means to bind both dimensions in my template args, I can no longer use
> type parameter inference, so I have to explicitly instantiate the template
> with the static size, but pass in a type that's dynamic in one dimension.

That is the correct way to do this.  std.file.read is reading data into a heap array.  If you want to cast this into a static array that lives on the stack, you will end up copying the data from the heap to the stack.  Essentially, you are asking the type system to ignore the fact that std.file.read may *not* read 100x100 floats, which is quite unsafe.

Another thing you can do:

float[100][100] data;
File f = File(fname);
enforce(std.file.rawRead(data[]).length == data.length);

Now, data should be filled in, and you haven't used the heap.

> If I could solve that, then next I'd need to make this work:
>
>    U foo(T: U[C][R], U, size_t C, size_t R)(U[C][] data, size_t c, size_t r)
>    {
>       return data[r][c];
>    }
>
>    int[4][] data = [ [...] ];
>    int bar = foo(data, 2, 3);
>
> Extra points for that one!

That one will never work, because data's type does not include the main dimension, so it can't be decided at compile time.

-Steve

Ok, I've gone over this, adapted it, and mostly understand it.  I just have one question left:

== Quote from bearophile (bearophileHUGS@lycos.com)'s article
> template Iota(int stop) {
> ...
>         alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
> }
> ...
>     foreach (t; Iota!(str_types.length))

What happens at compile-time with this foreach loop?

I nievely went and replaced "foreach (t; Iota!(str_types.length))" with "foreach (t; str_types.length)", since the length of that array is known at compile-time.  That of course bombed, but I don't quite get why.  Is the compiler actually evaluating the foreach loop at compile time?  How could it, when the body makes run-time checks?  If it's not, why doesn't my change work?

Jason

Jason Spencer wrote:

> Ok, I've gone over this, adapted it, and mostly understand it.  I just have one question left:
> 
> == Quote from bearophile (bearophileHUGS@lycos.com)'s article
>> template Iota(int stop) {
>> ...
>>         alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
>> }
>> ...
>>     foreach (t; Iota!(str_types.length))
> 
> What happens at compile-time with this foreach loop?
> 
> I nievely went and replaced "foreach (t; Iota!(str_types.length))" with "foreach (t; str_types.length)", since the length of that array is known at compile-time.  That of course bombed, but I don't quite get why.  Is the compiler actually evaluating the foreach loop at compile time?  How could it, when the body makes run-time checks?  If it's not, why doesn't my change work?
> 
> Jason

your replacement tries to loop over an uint called str_types.length. Never gonna happen.

Iota!(str_types.length) seems to generate str_types.length(a number of) integer indexes. Can't use 0 .. str_types.length in the foreach because compiler is expecting Integer constants so it can make the template "foo" into actual code.

== Quote from Rory Mcguire (rjmcguire@gm_no_ail.com)'s article
> Jason Spencer wrote:

> > I nievely went and replaced "foreach (t; Iota!(str_types.length))"
> > with "foreach (t; str_types.length)", since the length of that
> > array is known at compile-time.

> your replacement tries to loop over an uint called str_types.length.
> Never gonna happen.
> Iota!(str_types.length) seems to generate str_types.length(a number
> of)integer indexes. Can't use 0 .. str_types.length in the foreach
> because compiler is expecting Integer constants so it can make the
> template "foo" into actual code.

Not quite sure I follow.  I think you're saying the range in foreach has to be actual literals, and not just an expression that can be evaluated at compile time to generate the same range...close?

If that's the case, then why does it work to instantiate Iota! with str_types.length?  It can obviously get the value behind it at compile time.  I'm still missing something.

Jason

Jason Spencer wrote:

> == Quote from Rory Mcguire (rjmcguire@gm_no_ail.com)'s article
>> Jason Spencer wrote:
> 
>> > I nievely went and replaced "foreach (t; Iota!(str_types.length))"
>> > with "foreach (t; str_types.length)", since the length of that
>> > array is known at compile-time.
> 
>> your replacement tries to loop over an uint called str_types.length.
>> Never gonna happen.
>> Iota!(str_types.length) seems to generate str_types.length(a number
>> of)integer indexes. Can't use 0 .. str_types.length in the foreach
>> because compiler is expecting Integer constants so it can make the
>> template "foo" into actual code.
> 
> Not quite sure I follow.  I think you're saying the range in foreach has to be actual literals, and not just an expression that can be evaluated at compile time to generate the same range...close?
> 
> If that's the case, then why does it work to instantiate Iota! with str_types.length?  It can obviously get the value behind it at compile time.  I'm still missing something.
> 
> Jason

I convert str_types.length to its actual value below:
foreach (t; 3) {
...
}

You can't do that (dmd : t.d(6): Error: int is not an aggregate type)

Jason Spencer:
> I nievely went and replaced "foreach (t; Iota!(str_types.length))" with "foreach (t; str_types.length)", since the length of that array is known at compile-time.  That of course bombed, but I don't quite get why.  Is the compiler actually evaluating the foreach loop at compile time?  How could it, when the body makes run-time checks?  If it's not, why doesn't my change work?

foreach (t; str_types.length) doesn't work because integral values are not iterable. You need a range, collection or tuple.

foreach (t; 0 .. str_types.length) doesn't work in that code because it's a run-time foreach.

The  Iota!(n) creates a tuple of the values 0, 1, 2 ... n-1.

The foreach done on a tuple is a static foreach, it's done at compile-time, so my code is a way to perform a compile-time unrolling of the if-goto body. You can see it with this example (normally DMD is not able to perform loop unrolling):


import std.typetuple: TypeTuple;
import std.c.stdio: printf;

template Iota(int stop) {
    static if (stop <= 0)
        alias TypeTuple!() Iota;
    else
        alias TypeTuple!(Iota!(stop-1), stop-1) Iota;
}
void main() {
    foreach (i; Iota!(5)) {
        enum int n = i;
        printf("%d\n", n);
    }
}


The resulting asm:

__Dmain comdat
        push    EBX
        push    0
        mov EAX,offset FLAT:_DATA
        push    EAX
        call    near ptr _printf
        add ESP,8
        push    1
        mov ECX,offset FLAT:_DATA
        push    ECX
        call    near ptr _printf
        add ESP,8
        push    2
        mov EDX,offset FLAT:_DATA
        push    EDX
        call    near ptr _printf
        add ESP,8
        push    3
        mov EBX,offset FLAT:_DATA
        push    EBX
        call    near ptr _printf
        add ESP,8
        push    4
        push    EBX
        call    near ptr _printf
        add ESP,8
        xor EAX,EAX
        pop EBX
        ret


I suggest you to use a variant of my last version of the code because it contains some asserts and static asserts that improve code safety a bit.

See also this enhancement request of mine: http://d.puremagic.com/issues/show_bug.cgi?id=4085

Bye,
bearophile

== Quote from Rory Mcguire (rjmcguire@gm_no_ail.com)'s article
> Jason Spencer wrote:
> > == Quote from Rory Mcguire (rjmcguire@gm_no_ail.com)'s article
> >> Jason Spencer wrote:
> >
> >> > I nievely went and replaced "foreach (t;
Iota!(str_types.length))"
> >> > with "foreach (t; str_types.length)", since the length of that
> >> > array is known at compile-time.

> >> Can't use 0 .. str_types.length in the foreach
> >> because compiler is expecting Integer constants so it can make
> >> the template "foo" into actual code.

This is the part I'm still not getting.  Why shouldn't

   foreach (t; 0..3)

work?  Those are integer constants.

Actually, I think I'm getting it.  str_types.length is actually (or close to) an integer literal, but t is not.  t over a range is an int variable.  So at best, the compiler will infer the type of t and try to get TypeTuple![int] from the mixin, which doesn't help.  But it works in Iota because it only needs a value, and the length property is not a variable, but a compile-time constant.

I'm not sure what magic gets worked when t is bound to a TypeTuple that has int literals, but I'm guessing t in that case is not an int variable, but a compile-time type variable, and it iterates over int literals.  Those work with templ.  What I really want to know is "does that foreach run at compile-time or run-time?"  I suspect compile-time because it iterates over type variables.  But documentation is shakey :)



> I convert str_types.length to its actual value below:
> foreach (t; 3) {
> ...
> }
> You can't do that (dmd : t.d(6): Error: int is not an aggregate
type)

Yeah, I mis-typed orginally.  It was "foreach (t;
0..str_types.length)" as a range.

@bearophile:

Got it.  Had it mostly worked out before reading, but this helps clarify the static foreach part.

Thanks, gang!

Jason

Jason Spencer wrote:

> == Quote from Rory Mcguire (rjmcguire@gm_no_ail.com)'s article
>> Jason Spencer wrote:
>> > == Quote from Rory Mcguire (rjmcguire@gm_no_ail.com)'s article
>> >> Jason Spencer wrote:
>> >
>> >> > I nievely went and replaced "foreach (t;
> Iota!(str_types.length))"
>> >> > with "foreach (t; str_types.length)", since the length of that
>> >> > array is known at compile-time.
> 
>> >> Can't use 0 .. str_types.length in the foreach
>> >> because compiler is expecting Integer constants so it can make
>> >> the template "foo" into actual code.
> 
> This is the part I'm still not getting.  Why shouldn't
> 
>    foreach (t; 0..3)
> 
> work?  Those are integer constants.
> 
> Actually, I think I'm getting it.  str_types.length is actually (or close to) an integer literal, but t is not.  t over a range is an int variable.  So at best, the compiler will infer the type of t and try to get TypeTuple![int] from the mixin, which doesn't help.  But it works in Iota because it only needs a value, and the length property is not a variable, but a compile-time constant.
> 
> I'm not sure what magic gets worked when t is bound to a TypeTuple that has int literals, but I'm guessing t in that case is not an int variable, but a compile-time type variable, and it iterates over int literals.  Those work with templ.  What I really want to know is "does that foreach run at compile-time or run-time?"  I suspect compile-time because it iterates over type variables.  But documentation is shakey :)
> 
> 
> 
>> I convert str_types.length to its actual value below:
>> foreach (t; 3) {
>> ...
>> }
>> You can't do that (dmd : t.d(6): Error: int is not an aggregate
> type)
> 
> Yeah, I mis-typed orginally.  It was "foreach (t;
> 0..str_types.length)" as a range.

The foreach using Iota is unrolled at compile time, bearofile mentioned this somewhere as well.