On Sun, Dec 30, 2018 at 01:58:56PM +0000, Nicholas Wilson via Digitalmars-d wrote:
> On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
> > There's also dmd's ridiculous memory usage policy, which is supposed to help compile times when you have ridiculous amounts of free RAM, but which causes anything from swap thrashing slowdowns to outright unusability on medium- to low-memory systems.
> 
> Rainer an Martin Kinkelin have been working on that for LDC, they might upstream eventually.

Recently I noticed that LDC now compiles every function into their own section and runs LTO, including GC of unreferenced sections, by default. As a result, executable sizes are back down to where equivalent C/C++ code would be, as opposed to being a MB or so larger when compiled with DMD. It more-or-less nullifies most of the ill-effects of template bloat.

Furthermore, LDC now tracks dmd releases very closely, almost on par, produces better code, has a far wider range of target archs, like Android/ARM, that I doubt DMD will ever support in the foreseeable future, and recently shows compile times pretty close to DMD (with things like std.regex or std.format making DMD dog-slow anyway, the extra time the LDC backend takes for codegen basically becomes roundoff error).

Right now, I'm very tempted to drop dmd as my primary D compiler and use LDC instead. If this memory usage thing is fixed in LDC, I probably WILL drop dmd for good and use LDC instead.  I've been itching to get D to compile on some of my low-memory systems but have been hampered until now -- it's been extremely frustrating.


T

-- 
"You are a very disagreeable person." "NO."

On 12/29/18 7:18 PM, Ivan Kazmenko wrote:
> On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
>> On Sat, Dec 29, 2018 at 10:35:05AM +0000, JN via Digitalmars-d wrote:
>>> On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
>>> > http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
>>> > > Time to show off your leet D skilz and see how good we can > do 
>>> it in D!
>>>
>>> I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .
>> [...]
>>
>> Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds to 2+ seconds.
>>
>> Let me say that again.  Removing dependency on std.regex (by writing equivalent functionality by hand) improved compilation times from more than SEVEN seconds to just over TWO seconds. That's almost TRIPLE the compilation speed.  The mere act of using std.regex causes compilation times to TRIPLE.
>>
>> Let that sink in for a moment.
> 
> I thought I'd chime in with a similar experience.  I have this small library, testlib.d, used to check text outputs for programming contest problems (used internally, no public repository right now).  As one can imagine, a regular expression is sometimes handy to parse simple alphanumeric constructs and check their format.  But after using std.regex a bit, I put the import down into one function and templated it, and mostly stopped using the function in new checkers' code, developing some not-so-nice workarounds instead.  A typical checker is very short, a couple dozen to a couple hundred lines, and this change brings compile times from 5+ seconds to 2+ seconds.
> 
> Sorry, I just can't stand having to wait 5+ seconds to compile a 50-liner, repeatedly, when the development time for a single checker is on the scale of minutes, not hours.  The C++ analog library (https://github.com/MikeMirzayanov/testlib) is currently 4500+ lines long, and so also takes 5+ seconds to compile a trivial checker.  This was actually one of the incentives for me to switch to D with a homebrew library for writing checkers.
> 
> I know there was a move to make Phobos compilation faster a few months ago, and things seemed to improve a little then, but std.regex still adds seconds of compilation time.

Hmmm, I thought Dmitry was successful at eliminating most overheads of importing and not using std.regex. Perhaps a second pass would be needed?

On 12/29/18 9:03 PM, Timon Gehr wrote:
> On 29.12.18 10:29, Walter Bright wrote:
>> http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
>>
>> Time to show off your leet D skilz and see how good we can do it in D!
> 
> 
> Current D:
> ---
> import std.range, std.algorithm, std.typecons;
> import std.bigint, std.stdio;
> alias then(alias a)=(r)=>map!a(r).joiner;
> void main(){
>      auto triples=recurrence!"a[n-1]+1"(1.BigInt)
>          .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z))))
>          .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2);
>      triples.each!((x,y,z){ writeln(x," ",y," ",z); });
> }
> ---
> 
> tuple-syntax branch at https://github.com/tgehr/dmd/tree/tuple-syntax
> ---
> import std.range, std.algorithm, std.typecons;
> import std.bigint, std.stdio;
> alias then(alias a)=(r)=>map!a(r).joiner;
> void main(){
>      auto triples=recurrence!"a[n-1]+1"(1.BigInt)
>          .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>(x,y,z))))
>          .filter!((x,y,z)=>x^^2+y^^2==z^^2);
>      triples.each!((x,y,z){ writeln(x," ",y," ",z); });
> }
> ---

The "then" abstraction is pretty awesome. Thanks!

On Sun, Dec 30, 2018 at 09:45:11AM -0500, Andrei Alexandrescu via Digitalmars-d wrote: [...]
> Hmmm, I thought Dmitry was successful at eliminating most overheads of importing and not using std.regex. Perhaps a second pass would be needed?

I think the cost of importing without actually using std.regex should have been fixed by now, but the complaint here is importing and *using* std.regex, which would be the important use case. :-D

Understandably, some amount of cost would have to be paid to actually use the module, but given our fast-fast-fast slogan, should it really add *5 seconds* to compilation time just to use a couple of near-trivial regexes?


T

-- 
Why can't you just be a nonconformist like everyone else? -- YHL

On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
>
> There's also dmd's ridiculous memory usage policy, which is supposed to help compile times when you have ridiculous amounts of free RAM, but which causes anything from swap thrashing slowdowns to outright unusability on medium- to low-memory systems.
>

Don't get me started :)
My VPS host has 512 mb RAM, and it's stressful when a D compiler can't build a few files. Upping this memory limit costs significantly more money.

On Sun, Dec 30, 2018 at 03:10:17PM +0000, Guillaume Piolat via Digitalmars-d wrote:
> On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
> > There's also dmd's ridiculous memory usage policy, which is supposed to help compile times when you have ridiculous amounts of free RAM, but which causes anything from swap thrashing slowdowns to outright unusability on medium- to low-memory systems.
> > 
> 
> Don't get me started :)
> My VPS host has 512 mb RAM, and it's stressful when a D compiler can't
> build a few files. Upping this memory limit costs significantly more
> money.

Exactly the same problem I've been facing.

Not to mention some workstations at my day job that don't have the luxury of having gobs of free RAM available.  Which is why I haven't dared to mention D at work yet... it would become the laughing stock of my coworkers if dmd were to die with an OOM error on the simplest of programs.


T

-- 
What's an anagram of "BANACH-TARSKI"?  BANACH-TARSKI BANACH-TARSKI.

On 2018-12-30 15:46, Andrei Alexandrescu wrote:
> On 12/29/18 9:03 PM, Timon Gehr wrote:
>> alias then(alias a)=(r)=>map!a(r).joiner;
> The "then" abstraction is pretty awesome. Thanks!

Isn't that usually called "flatMap"?

-- 
/Jacob Carlborg

On 12/30/18 7:27 AM, John Colvin wrote:
> On Saturday, 29 December 2018 at 22:01:58 UTC, Steven Schveighoffer wrote:
>> On 12/29/18 4:29 AM, Walter Bright wrote:
>>> http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
>>>
>>> Time to show off your leet D skilz and see how good we can do it in D!
>>
>> Ugh, ranges really aren't a good fit for emulating nested loops, unless you write a specialized one.
>>
>> I tried my best, but it kind of sucks:
>>
>>     foreach(z, x, y;
>>     iota(size_t.max)
>>         .map!(a =>
>>              zip(StoppingPolicy.shortest, a.repeat, iota(1, a)))
>>         .joiner
>>         .map!(t =>
>>              zip(StoppingPolicy.shortest, t[0].repeat, t[1].repeat, iota(t[1], t[0])))
>>         .joiner
>>         .filter!(t => t[0]*t[0] == t[1]*t[1] + t[2]*t[2])
>>         .take(100))
>>     {
>>         writeln(x, " ", y, " ", z);
>>     }
> 
> Isn't "StoppingPolicy.shortest" the default?

Maybe :) I didn't spend a lot of time examining the details.

I also like your way, it's much more readable, but I still don't like the joiners. There has to be a way to just store the three ranges and iterate them properly, something like:

struct Triples(ZRange, alias r2func, alias r3func)
{
   ZRange z;
   typeof(r2func(z.front)) x;
   typeof(r3func(z.front, x.front)) y;

   auto front() { return tuple(z.front, x.front, y.front); }
   void popFront()
   {
      y.popFront;
      if(y.empty)
      {
         scope(exit) if(!x.empty) y = r3func(z.front, x.front)
         x.popFront;
         if(x.empty)
         {
            scope(exit) if(!z.empty) x = r2func(z.front);
            z.popFront;
         }
      }
   }
   bool empty() { return y.empty }
}

auto pythags = iota(size_t.max).triples!(z => iota(1 .. z), (z, x) => iota(x, z)).filter!(t => t[1]*t[1] + t[2]*t[2] == t[0]*t[0]);

Maybe split it out into something like withloop, maybe we can make this kind of thing work:

auto pythags = iota(size_t.max)
   .withLoop!(z => iota(1, z))
   .withLoop!((z, x) => iota(x, z))
   .filter!(t => t[1]*t[1] + t[2]*t[2] == t[0]*t[0]);

Which is really similar to the loop design.

-Steve

On 12/30/18 1:13 PM, Steven Schveighoffer wrote:
> Maybe split it out into something like withloop, maybe we can make this kind of thing work:
> 

There you go, nested looping with ranges, with no need for closures or joiners:

https://run.dlang.io/is/hkqhvZ

:)

OK, spent enough time on this... I have to stop before it consumes my whole weekend.

-Steve

On 12/30/18 1:47 PM, Steven Schveighoffer wrote:
> On 12/30/18 1:13 PM, Steven Schveighoffer wrote:
>> Maybe split it out into something like withloop, maybe we can make this kind of thing work:
>>
> 
> There you go, nested looping with ranges, with no need for closures or joiners:
> 
> https://run.dlang.io/is/hkqhvZ

BTW, this would be much nicer with first-class tuples...

-Steve