Hello,

Brief context description:
I'm trying to replace calls to druntime functions such as _d_arraycopy [0] with calls to templates. These calls are generated by the compiler here [1] for array assignments like below:

int[2] a = [1, 2];
int[2] b;

b[] = a[]; // _d_array_copy call

There are several such druntime functions and the purpose of this is to avoid pulling druntime as a dependency unless necessary. The templates don't "exist" if not used and many of them can be implemented without pulling any dependency.

--------------------------------------------------------------

I implemented a lowering in AssignExp:semantic to a template call for the use case described above. However compilation fails for this code (I extracted this bit from druntime code):

  1
  2 char[] mangle(T)(const(char)[] fqn, char[] dst = null) @safe pure nothrow
  3 {
  4     import core.internal.string : numDigits, unsignedToTempString;
  5
  6     static struct DotSplitter
  7     {
  8         @safe pure nothrow:
  9             const(char)[] s;
 10
 11         @property bool empty() const { return !s.length; }
 12
 13         @property const(char)[] front() const
 14         {
 15             immutable i = indexOfDot();
 16             return i == -1 ? s[0 .. $] : s[0 .. i];
 17         }
 18
 19         void popFront()
 20         {
 21         }
 22
 23         private ptrdiff_t indexOfDot() const
 24         {
 25             foreach (i, c; s) if (c == '.') return i;
 26             return -1;
 27         }
 28     }
 29
 30     size_t len = "_D".length;
 31     foreach (comp; DotSplitter(fqn))
 32         len += numDigits(comp.length) + comp.length;
 33
 34     return ['a', 'b', 'c'];
 35 }
 36
 37 char[] mangleFunc(T:FT*, FT)(const(char)[] fqn, char[] dst = null) @safe pure nothrow if (is(FT == function))
 38 {
 39         return mangle!FT(fqn, dst);
 40 }
 41
 42 pragma(mangle, mangleFunc!(int function(int))("a.b"));
 43
 44 int main() {
 45     return 0;
 46 }

The error is:
ctfe.d(28): Error: 2 variable __r57 cannot be read at compile time
ctfe.d(28):        called from here: _d_arraycopyT(this.s[], __r57, 1u)
ctfe.d(18):        called from here: this.indexOfDot()
ctfe.d(34):        called from here: __r58.front()
ctfe.d(42):        called from here: mangle(fqn, dst)
ctfe.d(45):        called from here: mangleFunc("a.b", null)

This is probably because of an issue with CTFE and the solution may be to avoid generating calls to the template in a CTFE context. The code in AssignExp:semantic [2] would look like this:

if (__ctfe)
{
  // add template call to AST
}
else
{
  // old code
}
However I was not able to detect this situation from AssignExp:semantic.
1. __ctfe doesn't seem to be set
2. performing similar checks with the ones in dinterpret which throw the error [3] don't distinguish this situation
3. checking the scope for ctfe doesn't work either: sc.flags & SCOPEctfe

Is there a way to achieve this? Maybe I am looking at this problem from a wrong angle.

Thank you,
Lucia

[0]https://github.com/dlang/druntime/blob/2db828bd4f21807254b770b3ec304f14596a9805/src/rt/arraycat.d#L22

[1] https://github.com/dlang/dmd/blob/master/src/e2ir.d#L2702

[2] https://github.com/dlang/dmd/blob/master/src/expression.d#L13545

[3] https://github.com/dlang/dmd/blob/master/src/dinterpret.d#L2381

The way to attack these sorts of problems is to not attempt the complete solution as the first step.

Start with a massive simplification - for example, just having mangle() return "hello". If that fails in a confusing manner, go even simpler.

Once the very simple solution works, then start adding in the complex solution bit by bit, ensuring each piece works before adding the next.

On 01/10/2017 07:13 AM, Lucia Cojocaru wrote:
> Hello,
>
> Brief context description:
> I'm trying to replace calls to druntime functions such as _d_arraycopy
> [0] with calls to templates.
[snip]

You may also want to push your WIP into your dmd branch on github so people can take a look. Thanks! -- Andrei

On Tuesday, 10 January 2017 at 12:13:18 UTC, Lucia Cojocaru wrote:
> Hello,
>
> Brief context description:
> I'm trying to replace calls to druntime functions such as _d_arraycopy [0] with calls to templates. These calls are generated by the compiler here [1] for array assignments like below:
>
> [...]

I cannot say much from this snippet alone. Please discribe what you want to archive concretely.
As previously mentioned it would also help to push your code to a visible branch.
I will take a look shortly.

The dmd code (e2ir.d and expression.d are of interest):
https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090

The druntime template:
https://github.com/somzzz/druntime/commit/6cf9cbc6650697d8a038be7076e588601aefe954

The example which doesn't compile is a standalone code snippet which reproduces the error encountered. I started from the code in druntime and simplified it to that point. As of your suggestions, I will simplify it further and come back with another example.

Thanks!

On Wednesday, 11 January 2017 at 09:05:44 UTC, Lucia Cojocaru wrote:
> The dmd code (e2ir.d and expression.d are of interest):
> https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090

Calls for the old dmd<->C-API are very different from template functions calls, e.g. take a look at how _xOpEquals is called.
https://github.com/dlang/dmd/blob/538a895157acdbbfc5869791f9504f7e86b4fdd0/src/clone.d#L496

> The druntime template:

https://github.com/somzzz/druntime/commit/6cf9cbc6650697d8a038be7076e588601aefe954

> The example which doesn't compile is a standalone code snippet which reproduces the error encountered. I started from the code in druntime and simplified it to that point. As of your suggestions, I will simplify it further and come back with another example.

You cannot distinguish between ctfe/non-ctfe during semantic. Only the backend/glue layer differs between CTFE (interpret.d) and IR/codegen (e2ir).
If you want to convert a C-API intrinsic to a template function call, you'd usually deal with __ctfe in druntime not in the compiler.
DMD will always call the druntime function and if that happens during CTFE, it'll get interpreted.

On Wednesday, 11 January 2017 at 09:05:44 UTC, Lucia Cojocaru wrote:
> The dmd code (e2ir.d and expression.d are of interest):
> https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090
>
> The druntime template:
> https://github.com/somzzz/druntime/commit/6cf9cbc6650697d8a038be7076e588601aefe954
>
> The example which doesn't compile is a standalone code snippet which reproduces the error encountered. I started from the code in druntime and simplified it to that point. As of your suggestions, I will simplify it further and come back with another example.
>
> Thanks!

You should not need to special case ctfe inside the compiler for this.
Rather the template should have if (__ctfe) inside it if those are needed.
However I would advise against splitting code-paths, if it is not _strictly_ necessary.

I will be able to provide more help should you need it.
Just contact me if you encounter further problems.

On 1/11/17 3:16 PM, Martin Nowak wrote:
> On Wednesday, 11 January 2017 at 09:05:44 UTC, Lucia Cojocaru wrote:
>> The dmd code (e2ir.d and expression.d are of interest):
>> https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090
>>
>
> Calls for the old dmd<->C-API are very different from template functions
> calls, e.g. take a look at how _xOpEquals is called.
> https://github.com/dlang/dmd/blob/538a895157acdbbfc5869791f9504f7e86b4fdd0/src/clone.d#L496

Cool. That looks different from https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090, is the latter appropriate as well? Or perhaps that's why the error with reading the variable during compilation?

>> The druntime template:
>
> https://github.com/somzzz/druntime/commit/6cf9cbc6650697d8a038be7076e588601aefe954
>
>
>> The example which doesn't compile is a standalone code snippet which
>> reproduces the error encountered. I started from the code in druntime
>> and simplified it to that point. As of your suggestions, I will
>> simplify it further and come back with another example.
>
> You cannot distinguish between ctfe/non-ctfe during semantic. Only the
> backend/glue layer differs between CTFE (interpret.d) and IR/codegen
> (e2ir).
> If you want to convert a C-API intrinsic to a template function call,
> you'd usually deal with __ctfe in druntime not in the compiler.
> DMD will always call the druntime function and if that happens during
> CTFE, it'll get interpreted.

OK, but the problem here is it indicates a problem at the call site of _d_arraycopyT, not inside the implementation. Is there an issue with the way the code is generated?

Also, as an aside: the _d_arraycopyT should probably go like this:

D _d_arraycopyT(S, D)(S from, D to) { ... }

You don't need size because it's from[0].sizeof. Correct?


Andrei

On Wednesday, 11 January 2017 at 17:10:15 UTC, Andrei Alexandrescu wrote:
> On 1/11/17 3:16 PM, Martin Nowak wrote:
>> On Wednesday, 11 January 2017 at 09:05:44 UTC, Lucia Cojocaru wrote:
>>> The dmd code (e2ir.d and expression.d are of interest):
>>> https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090
>>>
>>
>> Calls for the old dmd<->C-API are very different from template functions
>> calls, e.g. take a look at how _xOpEquals is called.
>> https://github.com/dlang/dmd/blob/538a895157acdbbfc5869791f9504f7e86b4fdd0/src/clone.d#L496
>
> Cool. That looks different from https://github.com/somzzz/dmd/commit/8bccc49ba661567c523545650aad30c01fd25090, is the latter appropriate as well? Or perhaps that's why the error with reading the variable during compilation?
>
>>> The druntime template:
>>
>> https://github.com/somzzz/druntime/commit/6cf9cbc6650697d8a038be7076e588601aefe954
>>
>>
>>> The example which doesn't compile is a standalone code snippet which
>>> reproduces the error encountered. I started from the code in druntime
>>> and simplified it to that point. As of your suggestions, I will
>>> simplify it further and come back with another example.
>>
>> You cannot distinguish between ctfe/non-ctfe during semantic. Only the
>> backend/glue layer differs between CTFE (interpret.d) and IR/codegen
>> (e2ir).
>> If you want to convert a C-API intrinsic to a template function call,
>> you'd usually deal with __ctfe in druntime not in the compiler.
>> DMD will always call the druntime function and if that happens during
>> CTFE, it'll get interpreted.
>
> OK, but the problem here is it indicates a problem at the call site of _d_arraycopyT, not inside the implementation. Is there an issue with the way the code is generated?

Don't really understand your question. What are the 2 problems you refer to?

The difference is fairly simple but huge:

- C intrinsics
  - AssignExp.semantic
    - e2ir => call RTLSYM_SYM
    - interpret => special handling

- D lowering
  - AssignExp.semantic lowered to CallExp of object._arrayCopy
    - normal function call and no special CTFE handling

> Also, as an aside: the _d_arraycopyT should probably go like this:
>
> D _d_arraycopyT(S, D)(S from, D to) { ... }
> You don't need size because it's from[0].sizeof. Correct?

Just convert the assignment to a function call, the backend deals with optimizations et.al. Also this seems to be used not only for static arrays.

NB:
  - leave aways the _d prefix it's only needed to namespace extern(C) functions with flat mangling
  - prolly should be _arrayCopy(T)(in T[] from, T[] to) as AssignExp.semantic already takes care of conversions

On 1/11/17 4:25 PM, Stefan Koch wrote:
> You should not need to special case ctfe inside the compiler for this.
> Rather the template should have if (__ctfe) inside it if those are needed.
> However I would advise against splitting code-paths, if it is not
> _strictly_ necessary.

That's what confuses me, it's the read of the temporary not the code inside the function. Would branching inside the function help with that? -- Andrei