On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
> [ ... ]

Hi Guys, I managed to hack around the issue of multi-dimensional static array parameters.

making the following code work in newCTFE:

int[] fold (int[4][3] a)
{
  int[] result;
  result.length = 4 * 3;
  int pos;
  foreach (i; 0 .. 3)
  {
    foreach (j; 0 .. 4)
    {
      result[pos++] = a[i][j];
    }
  }
  return result;
}



static assert (fold ([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]) ==
              [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]);

Enjoy!

On Monday, 24 July 2017 at 11:17:23 UTC, Stefan Koch wrote:
>
> static assert (fold ([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]) ==
>               [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]);
>
> Enjoy!

I barely have any idea what any of this means, but it looks really cool. Keep up the good work!

On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
> [ ... ]
Hi Guys I just fixed another bug that had me puzzled for a while.

The following function

ulong swap(ulong val)
{
    ulong result;

        result |= (val & 0xFF) << 56;
        result |= (val & 0xFF00) << 40;
        result |= (val & 0xFF00_00) << 24;
        result |= (val & 0xFF00_0000) << 8;

        result |= (val & 0xFF00_0000_00) >> 8;
        result |= (val & 0xFF00_0000_0000) >> 24;
        result |= (val & 0xFF00_0000_0000_00) >> 40;
        result |= (val & 0xFF00_0000_0000_0000) >> 56;

    return result;
}

would return strange values.
pragma(msg, swap(0xABCD_EF01_2345_6789)); // returns  9900958322423496704
                                          // expected 9900958322455989675
On further inspection the actual return value has only zeros in the lower 32 bits.

The generated IR however looks completey correct.

    Initialize();
        Line(1);

        auto val_1 = genParameter(BCType(BCTypeEnum.i64));//SP[4]
        beginFunction(0);//swap
            Line(2);
            Line(3);
            auto result_1 = genLocal(BCType(BCTypeEnum.i64), "result");//SP[8]
            Set(result_1, BCValue(Imm64(0)));
            Line(5);
            auto tmp1 = genTemporary(BCType(BCTypeEnum.i64));//SP[16]
            auto tmp2 = genTemporary(BCType(BCTypeEnum.i64));//SP[24]
            And3(tmp2, val_1, BCValue(Imm64(255)));
            auto tmp3 = genTemporary(BCType(BCTypeEnum.i32));//SP[32]
            Le3(tmp3, BCValue(Imm32(56)), BCValue(Imm32(63)));
            auto tmp4 = genTemporary(BCType(BCTypeEnum.i32));//SP[36]
            Set(tmp4, BCValue(Imm32(56)));
            auto tmp5 = genTemporary(BCType(BCTypeEnum.i32));//SP[40]
            Set(tmp5, BCValue(Imm32(63)));
            Assert(tmp3, Imm32(1) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(56)), BCValue(Imm32(63))*/;);
            Lsh3(tmp1, tmp2, BCValue(Imm32(56)));
            Or3(result_1, result_1, tmp1);
            Line(6);
            auto tmp6 = genTemporary(BCType(BCTypeEnum.i64));//SP[44]
            auto tmp7 = genTemporary(BCType(BCTypeEnum.i64));//SP[52]
            And3(tmp7, val_1, BCValue(Imm64(65280)));
            auto tmp8 = genTemporary(BCType(BCTypeEnum.i32));//SP[60]
            Le3(tmp8, BCValue(Imm32(40)), BCValue(Imm32(63)));
            auto tmp9 = genTemporary(BCType(BCTypeEnum.i32));//SP[64]
            Set(tmp9, BCValue(Imm32(40)));
            auto tmp10 = genTemporary(BCType(BCTypeEnum.i32));//SP[68]
            Set(tmp10, BCValue(Imm32(63)));
            Assert(tmp8, Imm32(2) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(40)), BCValue(Imm32(63))*/;);
            Lsh3(tmp6, tmp7, BCValue(Imm32(40)));
            Or3(result_1, result_1, tmp6);
            Line(7);
            auto tmp11 = genTemporary(BCType(BCTypeEnum.i64));//SP[72]
            auto tmp12 = genTemporary(BCType(BCTypeEnum.i64));//SP[80]
            And3(tmp12, val_1, BCValue(Imm64(16711680)));
            auto tmp13 = genTemporary(BCType(BCTypeEnum.i32));//SP[88]
            Le3(tmp13, BCValue(Imm32(24)), BCValue(Imm32(63)));
            auto tmp14 = genTemporary(BCType(BCTypeEnum.i32));//SP[92]
            Set(tmp14, BCValue(Imm32(24)));
            auto tmp15 = genTemporary(BCType(BCTypeEnum.i32));//SP[96]
            Set(tmp15, BCValue(Imm32(63)));
            Assert(tmp13, Imm32(3) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(24)), BCValue(Imm32(63))*/;);
            Lsh3(tmp11, tmp12, BCValue(Imm32(24)));
            Or3(result_1, result_1, tmp11);
            Line(8);
            auto tmp16 = genTemporary(BCType(BCTypeEnum.i64));//SP[100]
            auto tmp17 = genTemporary(BCType(BCTypeEnum.i64));//SP[108]
            And3(tmp17, val_1, BCValue(Imm64(4278190080)));
            auto tmp18 = genTemporary(BCType(BCTypeEnum.i32));//SP[116]
            Le3(tmp18, BCValue(Imm32(8)), BCValue(Imm32(63)));
            auto tmp19 = genTemporary(BCType(BCTypeEnum.i32));//SP[120]
            Set(tmp19, BCValue(Imm32(8)));
            auto tmp20 = genTemporary(BCType(BCTypeEnum.i32));//SP[124]
            Set(tmp20, BCValue(Imm32(63)));
            Assert(tmp18, Imm32(4) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(8)), BCValue(Imm32(63))*/;);
            Lsh3(tmp16, tmp17, BCValue(Imm32(8)));
            Or3(result_1, result_1, tmp16);
            Line(10);
            auto tmp21 = genTemporary(BCType(BCTypeEnum.i64));//SP[128]
            auto tmp22 = genTemporary(BCType(BCTypeEnum.i64));//SP[136]
            And3(tmp22, val_1, BCValue(Imm64(1095216660480)));
            auto tmp23 = genTemporary(BCType(BCTypeEnum.i32));//SP[144]
            Le3(tmp23, BCValue(Imm32(8)), BCValue(Imm32(63)));
            auto tmp24 = genTemporary(BCType(BCTypeEnum.i32));//SP[148]
            Set(tmp24, BCValue(Imm32(8)));
            auto tmp25 = genTemporary(BCType(BCTypeEnum.i32));//SP[152]
            Set(tmp25, BCValue(Imm32(63)));
            Assert(tmp23, Imm32(5) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(8)), BCValue(Imm32(63))*/;);
            Rsh3(tmp21, tmp22, BCValue(Imm32(8)));
            Or3(result_1, result_1, tmp21);
            Line(11);
            auto tmp26 = genTemporary(BCType(BCTypeEnum.i64));//SP[156]
            auto tmp27 = genTemporary(BCType(BCTypeEnum.i64));//SP[164]
            And3(tmp27, val_1, BCValue(Imm64(280375465082880)));
            auto tmp28 = genTemporary(BCType(BCTypeEnum.i32));//SP[172]
            Le3(tmp28, BCValue(Imm32(24)), BCValue(Imm32(63)));
            auto tmp29 = genTemporary(BCType(BCTypeEnum.i32));//SP[176]
            Set(tmp29, BCValue(Imm32(24)));
            auto tmp30 = genTemporary(BCType(BCTypeEnum.i32));//SP[180]
            Set(tmp30, BCValue(Imm32(63)));
            Assert(tmp28, Imm32(6) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(24)), BCValue(Imm32(63))*/;);
            Rsh3(tmp26, tmp27, BCValue(Imm32(24)));
            Or3(result_1, result_1, tmp26);
            Line(12);
            auto tmp31 = genTemporary(BCType(BCTypeEnum.i64));//SP[184]
            auto tmp32 = genTemporary(BCType(BCTypeEnum.i64));//SP[192]
            And3(tmp32, val_1, BCValue(Imm64(71776119061217280)));
            auto tmp33 = genTemporary(BCType(BCTypeEnum.i32));//SP[200]
            Le3(tmp33, BCValue(Imm32(40)), BCValue(Imm32(63)));
            auto tmp34 = genTemporary(BCType(BCTypeEnum.i32));//SP[204]
            Set(tmp34, BCValue(Imm32(40)));
            auto tmp35 = genTemporary(BCType(BCTypeEnum.i32));//SP[208]
            Set(tmp35, BCValue(Imm32(63)));
            Assert(tmp33, Imm32(7) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(40)), BCValue(Imm32(63))*/;);
            Rsh3(tmp31, tmp32, BCValue(Imm32(40)));
            Or3(result_1, result_1, tmp31);
            Line(13);
            auto tmp36 = genTemporary(BCType(BCTypeEnum.i64));//SP[212]
            auto tmp37 = genTemporary(BCType(BCTypeEnum.i64));//SP[220]
            And3(tmp37, val_1, BCValue(Imm64(18374686479671623680)));
            auto tmp38 = genTemporary(BCType(BCTypeEnum.i32));//SP[228]
            Le3(tmp38, BCValue(Imm32(56)), BCValue(Imm32(63)));
            auto tmp39 = genTemporary(BCType(BCTypeEnum.i32));//SP[232]
            Set(tmp39, BCValue(Imm32(56)));
            auto tmp40 = genTemporary(BCType(BCTypeEnum.i32));//SP[236]
            Set(tmp40, BCValue(Imm32(63)));
            Assert(tmp38, Imm32(8) /*"shift by %d is outside the range 0..%d", BCValue(Imm32(56)), BCValue(Imm32(63))*/;);
            Rsh3(tmp36, tmp37, BCValue(Imm32(56)));
            Or3(result_1, result_1, tmp36);
            Line(26);
            Ret(result_1);
            Line(27);
        endFunction();

        Line(28);
    Finalize();

So what is going wrong here ?

The 4 lines responsible for the lower 32bit of the result
have immediate and-instructions with constants larger then 32 bits.
like : And3(tmp37, val_1, BCValue(Imm64(18374686479671623680)));
which will cause the bytecode-interpreter to the immediate-bytecode instruction And tmp37, (immvalue & 0xFFFF_FFFF)
Which in this case will become And tmp37, #0 (since the lower bits of 18374686479671623680 are zero)

This is now fixed by pushing 64bit value into a register and issuing a reg-reg instruction.

So, you see, there is never a boring day in newCTFE-land :)

Cheers, Stefan

On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
> [ ... ]

Hey Guys!

working on the ctfe brainfuck compiler I finally figured out what was going wrong.

consider this:

uint fn ()
{
  uint[1] pointlessArray = [0];
  foreach(0 .. 42)
      pointlessArray[0]++;
  return pointlessArray[0];
}

static assert(fn() == 42);

until a few minutes ago this would have failed and the output would have been 0;
can you guess why ?

Well, while the ++array[0] would lower to BinAssignExp (array[0] += 1) which does correctly deal with references
++ is actually not lowerd but it's own special expression.
which is handled with the following code:


                auto expr = genExpr(e.e1); // in x++ expr is the x
                if (!canWorkWithType(expr.type) || !canWorkWithType(retval.type))
                {
                    bailout("++ only i32 is supported not " ~ to!string(expr.type.type) ~ " -- " ~ e.toString);
                    return;
                }
                assert(expr.vType != BCValueType.Immediate,
                    "++ does not make sense as on an Immediate Value");

                Set(retval, expr); // return a copy of the old value
                // the following code adds one the the original value
                if (expr.type.type == BCTypeEnum.f23)
                {
                    Add3(expr, expr, BCValue(Imm23f(1.0f)));
                }
                else if (expr.type.type == BCTypeEnum.f52)
                {
                    Add3(expr, expr, BCValue(Imm52f(1.0)));
                }
                else
                {
                    Add3(expr, expr, imm32(1));
                }

 of course arr[x]++ will load the value into a temporary and add one to that temporary
 never modifying the array.
Luckily I introduced a a way for rmw (read-modify-write) operations to be done on structs on arrays a while back.

if the expr is not normal local i.e. a stack-variable it will have heapRef which tells you to which pointer you have to write to modify the actual value rather then just modifying the temporary in which it was loaded.

so this was fixed by adding the following 3 lines.

                if (expr.heapRef)
                {
                    StoreToHeapRef(expr);
                }

Which will work for array's slices and structs alike :)

On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
> [ ... ]

Hi Guys,

After getting the brainfuck to D transcompiler to work.
I now made it's output compatible with newCTFE.

See it here:
https://gist.github.com/UplinkCoder/002b31572073798897552af4e8de2024

Unfortunately the above code does seem to get mis-compiled,
As it does not output Hello World, but rather:




On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
> [ ... ]

Hello Guys,

The bug preventing newCTFE from executing bf_ctfe[1] correctly  (a peculiarity in which for for and if statement-conditions  other then 32bit integers where ignored) is now fixed.

newCTFE is about 5.7 times faster compiling bf_ctfe.
(compiling bf_ctfe includes a test where a brainfuck interpreter written in brainfuck executes the brainfuck hello-world programm)

Here are the numbers:

uplink@uplink-desktop:~/d/dmd$ time src/dmd ../bf-ctfe/source/*.d -c -o- > x
Hello World!

Hello World!

real	0m0.113s
user	0m0.104s
sys	0m0.008s
uplink@uplink-desktop:~/d/dmd$ time dmd ../bf-ctfe/source/*.d -c > x
Hello World!

Hello World!

real	0m0.633s
user	0m0.600s
sys	0m0.028s

[1] https://github.com/UplinkCoder/bf_ctfe

Am Sun, 30 Jul 2017 14:44:07 +0000
schrieb Stefan Koch <uplink.coder@googlemail.com>:

> On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
> > [ ... ]
> 
> Hi Guys,
> 
> After getting the brainfuck to D transcompiler to work.
> I now made it's output compatible with newCTFE.
> 
> See it here: https://gist.github.com/UplinkCoder/002b31572073798897552af4e8de2024
> 
> Unfortunately the above code does seem to get mis-compiled,
> As it does not output Hello World, but rather:
> 

> 

Funny, it is working and mis-compiling at the same time.
I figure with such complex code, it is working if it ends up
*printing anything* at all and not segfaulting. :)

-- 
Marco

On Monday, 31 July 2017 at 17:58:56 UTC, Marco Leise wrote:
> Am Sun, 30 Jul 2017 14:44:07 +0000
> schrieb Stefan Koch <uplink.coder@googlemail.com>:
>
>> On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
>> > [ ... ]
>> 
>> Hi Guys,
>> 
>> After getting the brainfuck to D transcompiler to work.
>> I now made it's output compatible with newCTFE.
>> 
>> See it here: https://gist.github.com/UplinkCoder/002b31572073798897552af4e8de2024
>> 
>> Unfortunately the above code does seem to get mis-compiled,
>> As it does not output Hello World, but rather:
>> 

>> 
>
> Funny, it is working and mis-compiling at the same time.
> I figure with such complex code, it is working if it ends up
> *printing anything* at all and not segfaulting. :)

I fixed the bug which cause this to miscompile it works now at ctfe.
This code is not really that complex, it only looks confusing.
complex code uses slices  of struct-types and pointer-slicing and that stuff.

On Sunday, 30 July 2017 at 20:40:24 UTC, Stefan Koch wrote:
> On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
>> [...]
>
> Hello Guys,
>
> The bug preventing newCTFE from executing bf_ctfe[1] correctly  (a peculiarity in which for for and if statement-conditions  other then 32bit integers where ignored) is now fixed.
>
> [...]

Aren't you disabling codegen by passing a -o- to your engine, so it starts to compile faster?

On Monday, 31 July 2017 at 23:03:21 UTC, Temtaime wrote:
> On Sunday, 30 July 2017 at 20:40:24 UTC, Stefan Koch wrote:
>> On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
>>> [...]
>>
>> Hello Guys,
>>
>> The bug preventing newCTFE from executing bf_ctfe[1] correctly
>>  (a peculiarity in which for for and if statement-conditions  other then 32bit integers where ignored) is now fixed.
>>
>> [...]
>
> Aren't you disabling codegen by passing a -o- to your engine, so it starts to compile faster?

Ah yes.
An oversight while posting the results. it does not affect the meassurements in any real way though the difference it 3-5 milliseconds.

In the testcase the ctfe workload is totally dominant.