May 19, 2022
On Thursday, 19 May 2022 at 21:55:59 UTC, Walter Bright wrote:
> On 5/19/2022 1:24 PM, H. S. Teoh wrote:
>> IME, gcc and ldc2 are well able to convert the above ?: expression into
>> the latter, without uglifying the code.  Why are we promoting (or even
>> allowing) this kind of ugly code just because dmd's optimizer is so
>> lackluster you have to manually spell things out this way?
>
> See my reply to Steven.
>
> BTW, consider auto-vectorizing compilers. A common characteristic of them is that sometimes a loop looks like it should be vectorized, but the compiler didn't, for reasons that are opaque to users. The compiler then substitutes a slow emulation to give the *appearance* of being vectorized.

Good compilers can actually print a report of why they didn't vectorize things. If they couldn't most of the time these days it's because the compiler was right and the programmer has a loop that the compiler can't reasonably assume is free of dependencies.

https://d.godbolt.org/z/djhMhMj31 has reports enabled from gcc and llvm

Intel were the cutting edge for these reports but now Intel C++ is basically dead.

These reports aren't that good for instruction selection issues, granted.

As an addendum, I would actually contend that most optimizers are actually far too aggressive when performing loop optimizations.

https://d.godbolt.org/z/Y99zs9feh See this example. Unless you give the compiler a nudge in the right direction (i.e. You can make sure you never try to compute a factorial of 100 for example), it will generate reams and reams of code.

Unless you are compiling with profile guided optimizations everything the compiler does is blind. This isn't just a question of locality but the very basics of the compilers optimizations e.g. register allocation and spill placement.

> The only way to tell what is happening is to dump the generate assembler. This is especially troublesome you're attempting to write vector code that is portable among various SIMD instruction sets. It doesn't scale, at all.

If you're writing SIMD code without dumping the assembler anyway you're not paying enough attention. If you're going to go to all that effort you're going to be profiling the code, and any good profiler will show you the disassembly alongside. Maybe it doesn't scale in some minute sense but in practice I don't think it makes that much difference because you have to either do the work anyway, or it doesn't matter.

This is still ignoring that instructions sets don't mean all that much, it's all about the microarchitecture, which once again will probably require different code. For example AMD processors present-ish and past have emulated the wider SIMD in terms of more numerous smaller execution units.

> Hence D's approach is different. You can write vector code in D. If it won't compile to the target instruction set, it doesn't replace it with emulation. It signals an error. Thus, the user knows if he writes vector code, he gets vector code. It makes it easy for him to use versioning to adjust the shape of the expressions to line up with the vector capabilities of each target.

LDC doesn't do this, GCC does. I don't think it actually matters, whereas if you're consuming a library from someone who didn't do the SIMD parts properly, it will at very least compile with LDC.

> To sum up, if you want a particular instruction mix in the output stream, a systems programming language must enable expression of that desired mix. It must not rely on undocumented and inconsistent compiler transformations.

I agree, although D is getting massively out of sync with the interesting instructions even on X86. The fun stuff is not really available unless you use inline asm (or Guillaume's intrinsics library).

For the non-x86 world (i.e. the vast majority of all processors sold) ARM has NEON but the future will be SVE2, these are variable width vector instructions. This isn't impossible to fit into the D_SIMD paradigm but will require for example types that only have a lower bound on their size.

The RISC-V vector ISA is going in a similar direction.

If I can actually get my hands on some variable-width hardware I will write D code for it ("because it's there"), but I haven't found anything cheap enough yet.
May 19, 2022
On Thursday, 19 May 2022 at 22:24:35 UTC, deadalnix wrote:
> On Thursday, 19 May 2022 at 22:14:31 UTC, kdevel wrote:
>> Free of charge I compiled and ran this for you:
>>
>>    $ dmd lcb
>>    $ ./lcb
>>    input            : A Ä
>>    bright           : a Ä
>>    toLower (std.utf): a ä
>>    input            : A Ä
>>    bright           : a ä
>>    toLower (std.utf): a ä
>>
>> See the problem?
>
> You could have use "Ali Çehreli" as a test case :)

One needs to combine U+0043 LATIN CAPITAL LETTER C + U+0327 COMBINING CEDILLA in this case. Further Reading:

[3] https://medium.com/@sthadewald/the-utf-8-hell-of-mac-osx-feef5ea42407
May 19, 2022
On Thursday, 19 May 2022 at 22:20:06 UTC, deadalnix wrote:
> In our case, we even instrumentalized valgrind to cause a CI failure when such a branch occurs and run it on every patch.

That's fun. I've never enjoyed working with the valgrind code all that much but having it around is very useful.
May 19, 2022
On Thursday, 19 May 2022 at 22:51:55 UTC, kdevel wrote:
> On Thursday, 19 May 2022 at 22:24:35 UTC, deadalnix wrote:
>> On Thursday, 19 May 2022 at 22:14:31 UTC, kdevel wrote:
>>> Free of charge I compiled and ran this for you:
>>>
>>>    $ dmd lcb
>>>    $ ./lcb
>>>    input            : A Ä
>>>    bright           : a Ä
>>>    toLower (std.utf): a ä
>>>    input            : A Ä
>>>    bright           : a ä
>>>    toLower (std.utf): a ä
>>>
>>> See the problem?
>>
>> You could have use "Ali Çehreli" as a test case :)
>
> One needs to combine U+0043 LATIN CAPITAL LETTER C + U+0327 COMBINING CEDILLA in this case. Further Reading:
>
> [3] https://medium.com/@sthadewald/the-utf-8-hell-of-mac-osx-feef5ea42407

This, or simply U+00E7 .

Which will cause a similar problem as the one you raised.
May 19, 2022

On 5/19/22 5:44 PM, Walter Bright wrote:

>

On 5/19/2022 1:01 PM, Steven Schveighoffer wrote:

>

And let the compiler come up with whatever funky stuff it wants to in order to make it fast.

You never write things like:

   a += (b < c);

? I do.

I have written these kinds of things sometimes, but also would be fine writing b < c ? 1 : 0 if required, or even int(b < c). I'd happily write that in exchange for not having this happen:

enum A : int { a }

Json j = A.a;
writeln(j); // false
>

And, as I remarked before, GPUs favor this style of coding, as does SIMD code, as does cryto code.

If the optimizer can't see through the ternary expression with 2 constants, then maybe it needs updating.

>

Hoping the compiler will transform the code into this style, if it is not specified to, is just that, hope :-/

It's just use a better compiler. And if it doesn't, so what? Just write it with casts if a) it doesn't do what you want, and b) it's critically important. I just write it the "normal" way and move on.

What if the compiler rewrites it back to the ternary expression? Either way, if you are paranoid the compiler isn't doing the right thing, you check the assembly.

>

Sometimes this style is not necessarily faster, either, even though the user may desire it for crypto reasons.

Which is exactly why I leave it to the experts.

I want to write the clearest code possible, and let the optimizer wizards do their magic.

If for some reason, the compiler isn't smart enough to figure out the right thing to do (and it bothers me to the point of investigation), then D provides so many tools to get it to spit out what you want, all the way down to inline assembly. Anyone who thinks they can predict exactly what the compiler will output for any given code is fooling themselves. If you care, check the assembly.

-Steve

May 20, 2022
On Thursday, 19 May 2022 at 23:26:57 UTC, deadalnix wrote:
>>>>    input            : A Ä
>>>>    bright           : a Ä <-- upper case
>>>>    toLower (std.utf): a ä
>>>>    input            : A Ä
>>>>    bright           : a ä
>>>>    toLower (std.utf): a ä <-- lower case
>>>>
>>>> See the problem?
>>>
>>> You could have use "Ali Çehreli" as a test case :)
>>
>> One needs to combine U+0043 LATIN CAPITAL LETTER C + U+0327 COMBINING CEDILLA in this case. Further Reading:
>>
>> [3] https://medium.com/@sthadewald/the-utf-8-hell-of-mac-osx-feef5ea42407
>
> This, or simply U+00E7 .

Not "or" but "and". The first input contains UTF-8 of the (normalized) codepoint, which is left unchanged by Walters lowercase function. The second input contains UTF-8 of the same codepoint in canonically decomposed form (NFD).

> Which will cause a similar problem as the one you raised.

The problem is that you cannot decide only from the value of a single UTF-8 codeunit (char) if it stands for an ASCII character in the string.
May 19, 2022
On 5/19/22 17:50, kdevel wrote:

> The first input contains UTF-8 of the (normalized)
> codepoint, which is left unchanged by Walters lowercase function.

Note that Walter did not write any function. He showed a piece of code that would lowercase ASCII letters inside a UTF-8 encoded Unicode string.

The code did not assume the string was ASCII and it did not claim to lowercase all Unicode characters.

Ali

May 19, 2022
On 5/19/22 15:17, deadalnix wrote:
> On Thursday, 19 May 2022 at 20:48:47 UTC, Ali Çehreli wrote:
>> In D, char is UTF-8 and ASCII is a subset of UTF-8. Walter's code
>> above is valid without making any ASCII assumption.
>>
>
> Sure, it also doesn't perform any useful operation. other than
> "Uncapitalize English, do nothing for non latin languages

It is an experimental cypher. :)

Ali

May 20, 2022
On Wednesday, 18 May 2022 at 22:24:18 UTC, Paul Backus wrote:
>
>
> So I would say that according to the spec, the answer is "yes, the example should work." Though it is rather surprising.
>

It's actually rather *unsurprising* (given D's compatability needs with C).

What is surprising, is that there's no compiler option to disable implicit type casts, or to disable them in @safe, or *at the very least*, output a record of such casts for auditing (to help minimise bugs and vulnerabilities).

May 20, 2022
On Friday, 20 May 2022 at 02:09:43 UTC, forkit wrote:
>

D suffers from the same problem as C.

// ---

module test;
@safe: // completly useless annotation here!

import std;

void main()
{
    int x = 3;
    int y = 4;
    float z =x/y;
    writeln(z); // 0

}

// -----