http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

Time to show off your leet D skilz and see how good we can do it in D!

On 29/12/2018 10:29 PM, Walter Bright wrote:
> http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
> 
> Time to show off your leet D skilz and see how good we can do it in D!

Reddit thread: https://www.reddit.com/r/programming/comments/aac4hg/modern_c_lamentations/

On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
> http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
>
> Time to show off your leet D skilz and see how good we can do it in D!

I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .

Here's another take on language complexity, related to Rust - https://graydon2.dreamwidth.org/263429.html

> Issues with “Everything is a library” C++

That kind of mentality sounds familiar, can't put my finger on it though.

Probably won't get anything as nice as what C# does:

        var triples =
            from z in Enumerable.Range(1, int.MaxValue)
            from x in Enumerable.Range(1, z)
            from y in Enumerable.Range(x, z)
            where x*x+y*y==z*z
            select (x:x, y:y, z:z);

        foreach (var t in triples.Take(100))
        {
            Console.WriteLine($"({t.x},{t.y},{t.z})");
        }

At least not creating an "object" that represents a triple in that sense that is readable and doesn't involve mixins.

On Sat, Dec 29, 2018 at 10:35:05AM +0000, JN via Digitalmars-d wrote:
> On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
> > http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
> > 
> > Time to show off your leet D skilz and see how good we can do it in D!
> 
> I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .
[...]

Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds to 2+ seconds.

Let me say that again.  Removing dependency on std.regex (by writing equivalent functionality by hand) improved compilation times from more than SEVEN seconds to just over TWO seconds. That's almost TRIPLE the compilation speed.  The mere act of using std.regex causes compilation times to TRIPLE.

Let that sink in for a moment.

These days, it has been really hard for me to boast about D compilation times with a straight face.  True, if you write C-style D code, then compilation *is* lightning fast.  But modern D has moved away from that style, and the kind of style that modern D code takes on these days is template- and CTFE-heavy, both areas of which are in the "extremely slow" category of D compilation.

This seriously needs to improve if we're going to continue boasting about compilation times, because otherwise it's becoming more and more like false advertisement when we talk about D being fast to compile. I've been saying this ever since we adopted that cringe-worthy fast-fast-fast slogan on dlang.org, and it seems not much has improved since then. These days I'm almost ashamed to talk about D compilation times. :-/

And don't get me started on dmd's ridiculous memory usage requirements, which makes it basically unusable on low-memory systems. There is not even an option to trade off compilation speed for less memory usage. We're paying mandatory memory tax yet we're still unable to improve compilation speed of std.regex to within acceptable limits. And modern C++ compilers have improved enough that on said low-memory systems, compiling C++ can be actually *faster* than D because g++ stays within available RAM and therefore isn't thrashing on swap, whereas dmd thrashes on swap like crazy because it's such a memory hog. And that's if dmd even finishes compiling at all, before it gets hit by the OOM killer, otherwise we're looking at finite compilation time vs. infinite compilation time (because it never finishes).

Don't get me wrong, I still love D for having the best power to ease of writing ratio, and I can't see myself going back to C++ in the foreseeable future (or ever again). But we seriously need to improve on these two areas before we start proclaiming how good D compilation times are, because that's no longer true.


T

-- 
A bend in the road is not the end of the road unless you fail to make the turn. -- Brian White

On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
> Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds

Ditto. (Basically). Rewriting the uri parser in cgi.d brought its build time from about 3 seconds down to 0.6. Which still feels slow, especially next to my minigui.d, which can do 0.3 since it broke off from Phobos entirely! (It was 2.5 seconds before).

> But we seriously need to improve on these two areas before we start proclaiming how good D compilation times are, because that's no longer true.

Amen. I have boxes that have infinite D build times because it is impossible to run the sloppy compiler!

On Sat, Dec 29, 2018 at 03:57:57PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
> On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
> > Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds
> 
> Ditto. (Basically). Rewriting the uri parser in cgi.d brought its build time from about 3 seconds down to 0.6. Which still feels slow, especially next to my minigui.d, which can do 0.3 since it broke off from Phobos entirely! (It was 2.5 seconds before).

IME, Phobos isn't all bad. Some Phobos modules are pretty useful, esp. std.algorithm and std.range, and despite std.algorithm being pretty huge (its submodules are pretty huge even after I split up the original std/algorithm.d -- which was so bad I couldn't run its unittests locally anymore because it ran out of memory, on a machine with 4GB RAM) it actually compiles very fast, proportional to how much you use it. The bad apples are std.regex, std.format, and maybe a few others, that rely a little too heavily on recursive templates and CTFE.


> > But we seriously need to improve on these two areas before we start proclaiming how good D compilation times are, because that's no longer true.
> 
> Amen. I have boxes that have infinite D build times because it is impossible to run the sloppy compiler!

Yeah, this is one reason I haven't dared propose using D at my day job. I'd be laughed out of the CTO's office if dmd ran out of memory on a trivial regex test. And my coworkers will hate me so much for making the already RAM-heavy build system soak up even more memory. Memory may be cheap these days, but it's still not free, and it's still a finite resource.


T

-- 
Customer support: the art of getting your clients to pay for your own incompetence.

On 12/29/18 4:29 AM, Walter Bright wrote:
> http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
> 
> Time to show off your leet D skilz and see how good we can do it in D!

Ugh, ranges really aren't a good fit for emulating nested loops, unless you write a specialized one.

I tried my best, but it kind of sucks:

    foreach(z, x, y;
    iota(size_t.max)
        .map!(a =>
             zip(StoppingPolicy.shortest, a.repeat, iota(1, a)))
        .joiner
        .map!(t =>
             zip(StoppingPolicy.shortest, t[0].repeat, t[1].repeat, iota(t[1], t[0])))
        .joiner
        .filter!(t => t[0]*t[0] == t[1]*t[1] + t[2]*t[2])
        .take(100))
    {
        writeln(x, " ", y, " ", z);
    }

Now, a specialized range looks much better and more efficient to me. This is essentially what the author wrote.

And as is typical for D, we can split the tasks into more generic pieces. For instance:

struct PossiblePythags(T)
{
    T z = 2;
    T x = 1;
    T y = 1;

    auto front() { return tuple!(T, "x", T, "y", T, "z")(x, y, z); }
    void popFront() {
        if(++y == z)
        {
            if(++x == z)
            {
                ++z;
                x = 1;
            }
            y = x;
        }
    }
    enum empty = false;
}

auto pythagrange(T = size_t)()
{
    return PossiblePythags!T()
        .filter!(p => p.x * p.x + p.y * p.y == p.z * p.z);
}

No measuring of either compilation or runtime, but I would expect the specialized range version to be pretty close to the author's measurements.

I'm wondering if some generic "emulate N nested loops" with given stopping and starting conditions might be a useful addition for std.range or std.algorithm. I'm thinking of other looping algorithms like Floyd Warshall that might benefit from such building blocks.

-Steve

On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
> On Sat, Dec 29, 2018 at 10:35:05AM +0000, JN via Digitalmars-d wrote:
>> On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
>> > http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
>> > 
>> > Time to show off your leet D skilz and see how good we can do it in D!
>> 
>> I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .
> [...]
>
> Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds to 2+ seconds.
>
> Let me say that again.  Removing dependency on std.regex (by writing equivalent functionality by hand) improved compilation times from more than SEVEN seconds to just over TWO seconds. That's almost TRIPLE the compilation speed.  The mere act of using std.regex causes compilation times to TRIPLE.
>
> Let that sink in for a moment.

I thought I'd chime in with a similar experience.  I have this small library, testlib.d, used to check text outputs for programming contest problems (used internally, no public repository right now).  As one can imagine, a regular expression is sometimes handy to parse simple alphanumeric constructs and check their format.  But after using std.regex a bit, I put the import down into one function and templated it, and mostly stopped using the function in new checkers' code, developing some not-so-nice workarounds instead.  A typical checker is very short, a couple dozen to a couple hundred lines, and this change brings compile times from 5+ seconds to 2+ seconds.

Sorry, I just can't stand having to wait 5+ seconds to compile a 50-liner, repeatedly, when the development time for a single checker is on the scale of minutes, not hours.  The C++ analog library (https://github.com/MikeMirzayanov/testlib) is currently 4500+ lines long, and so also takes 5+ seconds to compile a trivial checker.  This was actually one of the incentives for me to switch to D with a homebrew library for writing checkers.

I know there was a move to make Phobos compilation faster a few months ago, and things seemed to improve a little then, but std.regex still adds seconds of compilation time.

Ivan Kazmenko.

On 29.12.18 10:29, Walter Bright wrote:
> http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
> 
> Time to show off your leet D skilz and see how good we can do it in D!


Current D:
---
import std.range, std.algorithm, std.typecons;
import std.bigint, std.stdio;
alias then(alias a)=(r)=>map!a(r).joiner;
void main(){
    auto triples=recurrence!"a[n-1]+1"(1.BigInt)
        .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z))))
        .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2);
    triples.each!((x,y,z){ writeln(x," ",y," ",z); });
}
---

tuple-syntax branch at https://github.com/tgehr/dmd/tree/tuple-syntax
---
import std.range, std.algorithm, std.typecons;
import std.bigint, std.stdio;
alias then(alias a)=(r)=>map!a(r).joiner;
void main(){
    auto triples=recurrence!"a[n-1]+1"(1.BigInt)
        .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>(x,y,z))))
        .filter!((x,y,z)=>x^^2+y^^2==z^^2);
    triples.each!((x,y,z){ writeln(x," ",y," ",z); });
}
---