Some code:
---------------------------------
struct Entity
{
  enum NumSystems = 4;
  struct SystemData
  {
    uint start, length;
  }
  SystemData[NumSystems] systemData;
  @property uint systemBits() const { return systemData[].map!(e =>
e.length).sum; }
}
Entity e;
e.systemBits(); // <- call the function, notice the codegen
---------------------------------

This property sum's 4 ints... that should be insanely fast. It should
also be something like 5-8 lines of asm.
Turns out, that call to sum() is eating 2.5% of my total perf
(significant among a substantial workload), and the call tree is quite
deep.

Basically, inliner tried, but failed to seal the deal, and leaves a call stack 7 levels deep.

Pipeline programming is hip and also *recommended* D usage. The
optimiser must do a good job. This is such a trivial workloop, and
with constant length (4).
I expect 3 integer adds to unroll and inline. A call-tree 7 levels
deep is quite a ways from the mark.

Maybe this is another instance of Walter's "phobos begat madness" observation?
The unoptimised callstack is mental. Compiling with -O trims most of
the noise in the call tree, but it fails to inline the remaining work
which ends up 7-levels down a redundant call-tree.

On Tuesday, 26 June 2018 at 02:10:17 UTC, Manu wrote:
> [snip]
>   @property uint systemBits() const { return systemData[].map!(e =>
> e.length).sum; } [snip]
>
> This property sum's 4 ints... that should be insanely fast. It should
> also be something like 5-8 lines of asm.
> Turns out, that call to sum() is eating 2.5% of my total perf
> (significant among a substantial workload), and the call tree is quite
> deep.
>
> Basically, inliner tried, but failed to seal the deal, and leaves a call stack 7 levels deep.

Last time I checked, dmd's inliner would give up as soon as it sees any type of loop, even the simplest while loop... then the unroller and optimiser later on have less to work with.

So I would expect it's the loop in `sumPairwise()` [0] or `sumKahan()` [1] that's the main source of your problems.

If we could get that to inline better using `dmd -inline` it would probably speed up quite a lot of code.

[0] https://github.com/dlang/phobos/blob/master/std/algorithm/iteration.d#L5483
[1] https://github.com/dlang/phobos/blob/master/std/algorithm/iteration.d#L5578

On Tuesday, 26 June 2018 at 02:10:17 UTC, Manu wrote:
> Some code:
> ---------------------------------
> struct Entity
> {
>   enum NumSystems = 4;
>   struct SystemData
>   {
>     uint start, length;
>   }
>   SystemData[NumSystems] systemData;
>   @property uint systemBits() const { return systemData[].map!(e =>
> e.length).sum; }
> }
> Entity e;
> e.systemBits(); // <- call the function, notice the codegen
> ---------------------------------
>
> This property sum's 4 ints... that should be insanely fast. It should
> also be something like 5-8 lines of asm.
> Turns out, that call to sum() is eating 2.5% of my total perf
> (significant among a substantial workload), and the call tree is quite
> deep.
>
> Basically, inliner tried, but failed to seal the deal, and leaves a call stack 7 levels deep.
>
> Pipeline programming is hip and also *recommended* D usage. The
> optimiser must do a good job. This is such a trivial workloop, and
> with constant length (4).
> I expect 3 integer adds to unroll and inline. A call-tree 7 levels
> deep is quite a ways from the mark.
>
> Maybe this is another instance of Walter's "phobos begat madness" observation?
> The unoptimised callstack is mental. Compiling with -O trims most of
> the noise in the call tree, but it fails to inline the remaining work
> which ends up 7-levels down a redundant call-tree.

Then use LDC! ;)

But seriously, DMD's inliner is a) in the wrong spot in the compilation pipeline (at the AST level) and b) is timid to say the least.

On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> Then use LDC! ;)

Keep LDC up to date with DMD master daily! ;)

On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>> Then use LDC! ;)
>
> Keep LDC up to date with DMD master daily! ;)

Like what GDC is doing  (almost) ;-)

Iain.

On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> > On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >>
> >> Then use LDC! ;)
> >
> > Keep LDC up to date with DMD master daily! ;)
>
> Like what GDC is doing  (almost) ;-)

Orly? Have you gotten GDC to a point where you can actively keep up to
date? You should definitely advertise that better, I think most users
still presume that GDC is a few revisions behind latest.
Do you have nightly builds that are close to the DMD nightlies?

Sadly we're working with the MSVC ABI, and I don't think GCC has a backend for that arch do they? LLVM has done a lot of work to interop with MSVC.

On Tuesday, 26 June 2018 at 18:07:56 UTC, Manu wrote:
> On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>> On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> > On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> >>
>> >> Then use LDC! ;)
>> >
>> > Keep LDC up to date with DMD master daily! ;)
>>
>> Like what GDC is doing  (almost) ;-)
>
> Orly? Have you gotten GDC to a point where you can actively keep up to
> date? You should definitely advertise that better, I think most users
> still presume that GDC is a few revisions behind latest.
> Do you have nightly builds that are close to the DMD nightlies?
>
> Sadly we're working with the MSVC ABI, and I don't think GCC has a backend for that arch do they? LLVM has done a lot of work to interop with MSVC.

D frontend was merged into GDC master. It is version 2.076 now. The older version was merged to make the switch easy and just replace the C++ frontend with the same version of the D frontend. There is a pull request to merge 2.080-beta into master:

https://github.com/D-Programming-GDC/GDC/pull/683

But some tests are still fail.

There is no nightly builds for DMD. It is currently more important to keep GDC up to date with GCC master (I'm updating GDC to to work with GCC weekly snapshots), because it would help to get GDC merged into GCC.

I can't currently promise anything but I wanted to build GDC binaries (gcc-7 or gcc-8) for me with a new frontend (2.080) and for Ubuntu, so other people can use it. Not sure how difficult will it be to update to the newer D-frontend versions (2.081 ... and so on), so not sure if I'll be able to help Iain much with that.

I should probably write some announcement about the latest work on GDC :).

On 26 June 2018 at 20:26, Eugene Wissner via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Tuesday, 26 June 2018 at 18:07:56 UTC, Manu wrote:
>>
>> On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>>
>>> On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>> > On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via > Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>> >>
>>> >> Then use LDC! ;)
>>> >
>>> > Keep LDC up to date with DMD master daily! ;)
>>>
>>> Like what GDC is doing  (almost) ;-)
>>
>>
>> Orly? Have you gotten GDC to a point where you can actively keep up to
>> date? You should definitely advertise that better, I think most users
>> still presume that GDC is a few revisions behind latest.
>> Do you have nightly builds that are close to the DMD nightlies?
>>
>> Sadly we're working with the MSVC ABI, and I don't think GCC has a backend for that arch do they? LLVM has done a lot of work to interop with MSVC.
>
>
> D frontend was merged into GDC master. It is version 2.076 now. The older version was merged to make the switch easy and just replace the C++ frontend with the same version of the D frontend. There is a pull request to merge 2.080-beta into master:
>
> https://github.com/D-Programming-GDC/GDC/pull/683
>
> But some tests are still fail.
>
> There is no nightly builds for DMD. It is currently more important to keep GDC up to date with GCC master (I'm updating GDC to to work with GCC weekly snapshots), because it would help to get GDC merged into GCC.
>
> I can't currently promise anything but I wanted to build GDC binaries (gcc-7 or gcc-8) for me with a new frontend (2.080) and for Ubuntu, so other people can use it. Not sure how difficult will it be to update to the newer D-frontend versions (2.081 ... and so on), so not sure if I'll be able to help Iain much with that.
>
> I should probably write some announcement about the latest work on GDC :).

Well, once in sync with dmd/stable. A weekly (or more frequent) check would all that would be needed.  A weekly merge would be trivial to maintain.

On 26 June 2018 at 20:07, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>
>> On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> > On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> >>
>> >> Then use LDC! ;)
>> >
>> > Keep LDC up to date with DMD master daily! ;)
>>
>> Like what GDC is doing  (almost) ;-)
>
> Orly? Have you gotten GDC to a point where you can actively keep up to
> date? You should definitely advertise that better, I think most users
> still presume that GDC is a few revisions behind latest.
> Do you have nightly builds that are close to the DMD nightlies?
>

Not quite, there are some funny bugs I'm looking at, some of which were features introduced by you. :-)

On Tue., 26 Jun. 2018, 11:45 am Iain Buclaw via Digitalmars-d, < digitalmars-d@puremagic.com> wrote:

> On 26 June 2018 at 20:07, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> > On Tue, 26 Jun 2018 at 10:43, Iain Buclaw via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >>
> >> On 26 June 2018 at 19:41, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >> > On Mon, 25 Jun 2018 at 20:50, Nicholas Wilson via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> >> >>
> >> >> Then use LDC! ;)
> >> >
> >> > Keep LDC up to date with DMD master daily! ;)
> >>
> >> Like what GDC is doing  (almost) ;-)
> >
> > Orly? Have you gotten GDC to a point where you can actively keep up to
> > date? You should definitely advertise that better, I think most users
> > still presume that GDC is a few revisions behind latest.
> > Do you have nightly builds that are close to the DMD nightlies?
> >
>
> Not quite, there are some funny bugs I'm looking at, some of which were features introduced by you. :-)
>

You love it! They're spectacular patches! :P

This is some seriously good news for GDC. Awesome stuff guys!

>