On Wednesday, 10 July 2019 at 19:16:11 UTC, bauss wrote:
> As seen in: https://www.techempower.com/benchmarks/#section=data-r18
>
> vibe.d once again ranks terrible on the list.
>
> With the power of D etc. it should be able to perform at the same level as the top frameworks but it just isn't so what are the reasons, where can it be improved and what areas do we as community need to look into and improve upon?

I guess the issue is that there are not enough people working on vibe.d

A healthier eco-system tends to solve this, as people competent enough would have jumped in and measure why it performs so bad and come up with fixes.

Maybe D can have another challenger that is simpler in design and performs better than vibe.d in this benchmark?

On Thu, Jul 11, 2019 at 10:30 AM bauss via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> ...
> We need to just realize that D did terrible in this benchmark and
> perhaps the problem is something internally in vibe.d and then we
> need to focus on what exactly the issue is, locate it and fix
> that.
>

This is what I am trying to do for some time now (few months), but vibe.d
code is not so easy to read and debug. So far I have found out that
vibe-core seems to not be the main problem (there are still some
potentional speed improvment), but it seems the issue is somewhere in
vibe-d:http.
There is some work on new version of this package (
https://github.com/vibe-d/vibe-http) , so maybe the performance of this new
attempt will be better. I am goint to try this new vibe-http, but I am not
sure if it is already in usable state.

Am 10.07.2019 um 21:16 schrieb bauss:
> As seen in: https://www.techempower.com/benchmarks/#section=data-r18
> 
> vibe.d once again ranks terrible on the list.
> 
> With the power of D etc. it should be able to perform at the same level as the top frameworks but it just isn't so what are the reasons, where can it be improved and what areas do we as community need to look into and improve upon?

The problem is that for some reason it doesn't scale properly with the number of cores in that benchmark. I have a dual-Xeon server that I can in theory use for profiling this, but I simply still lack the time to do so.

If you or anyone else who is interested has a bit of time to spare, I would suggest to change the benchmark setup to start one process per core instead of using multiple threads. This is the preferred approach anyway, for multiple reasons, but one of them being that it ensures proper scaling, no matter if the GC kicks in, or some other kind of lock contention happens.

This would require a change to the startup script to run multiple instances, as well as removing the runTaskDist call, while keeping HTTPServerOption.reusePort.

On Thursday, 11 July 2019 at 09:50:52 UTC, Daniel Kozak wrote:
> This is what I am trying to do for some time now (few months), but vibe.d
> code is not so easy to read and debug. So far I have found out that
> vibe-core seems to not be the main problem (there are still some
> potentional speed improvment), but it seems the issue is somewhere in
> vibe-d:http.
> There is some work on new version of this package (
> https://github.com/vibe-d/vibe-http) , so maybe the performance of this new
> attempt will be better. I am goint to try this new vibe-http, but I am not
> sure if it is already in usable state.

I remember finding vibe-core's reference-counting facility, aka InterfaceProxy, very taxing on the upper layers. There is a lot of passing a round and the lack of move semantics makes each function call that involves some kind of stream or managed reference an excercise in integer mutation. In pure http throughput tests it was very noticeable, around 10-30% drop in rps.

Eventcore is another arcane library very deep on the stack, I don't think anyone ever profiled it in isolation (correct me if I'm wrong), yet it's the heart of the event loop.

On the topic itself, well, the answer is always the same: everything's in it's right place.
In the usual hierarchy of merits:

productivity > correctness > flexibility and domain coverage > performance

vibe-d stack is still wrestling with the first three components, and rightfully so.

> There is some work on new version of this package ( https://github.com/vibe-d/vibe-http) , so maybe the performance of this new attempt will be better. I am goint to try this new vibe-http, but I am not sure if it is already in usable state.

The vibe-http repository is where HTTP/2 support is being worked on but it still
lacks some of the modules to allow for a complete vibe-d:http replacement.
The idea behind this repository is to rewrite parts of the vibe-d:http module
to follow these guidelines:

https://github.com/vibe-d/vibe-http/wiki

I'm currently having little spare time to finish the modifications, but I'm working on some benchmarks which will be focused on HTTP/1 and HTTP/2 as soon as it is ready. At the moment the server supports HTTP/1 and HTTP/2 request handling and response delivery, some functionalities are still missing (e.g. the vibe-d:http fileserver module).

On Thursday, 11 July 2019 at 08:29:12 UTC, bauss wrote:
> On Wednesday, 10 July 2019 at 20:50:16 UTC, Exil wrote:
>> On Wednesday, 10 July 2019 at 19:16:11 UTC, bauss wrote:
>>> As seen in: https://www.techempower.com/benchmarks/#section=data-r18
>>>
>>> vibe.d once again ranks terrible on the list.
>>>
>>> With the power of D etc. it should be able to perform at the same level as the top frameworks but it just isn't so what are the reasons, where can it be improved and what areas do we as community need to look into and improve upon?
>>
>> There's more to performance than the language you choose.
>
> Yes there is BUT for a web framework to properly scale you need performance.
>
> D is supposed to be powerful, it's supposed to have performance on pair with C / C++, so there's no excuse for why it ranks even worse than .NET.
>
> Of course the problem here is most likely not D but vibe.d's infrastructure.
>
> My post is not about "bashing" D. I have been using D primarily for all my projects for the past 6 years.
>
> My post is about highlighting an issue and figuring out what we can do as a community to fix it and have D rank higher.

The problem is how much developer time there is, and what their focus is. It doesn't look like there are that many people working on the project. They are probably working on it in their free time too. I doubt they are focusing on optimizing it to the extreme but rather bug fixes, features, and maybe some obvious optimizations.

On Thursday, 11 July 2019 at 11:23:42 UTC, Sönke Ludwig wrote:
> Am 10.07.2019 um 21:16 schrieb bauss:
>> As seen in: https://www.techempower.com/benchmarks/#section=data-r18
>> 
>> vibe.d once again ranks terrible on the list.
>> 
>> With the power of D etc. it should be able to perform at the same level as the top frameworks but it just isn't so what are the reasons, where can it be improved and what areas do we as community need to look into and improve upon?
>
> The problem is that for some reason it doesn't scale properly with the number of cores in that benchmark. I have a dual-Xeon server that I can in theory use for profiling this, but I simply still lack the time to do so.
>
> If you or anyone else who is interested has a bit of time to spare, I would suggest to change the benchmark setup to start one process per core instead of using multiple threads. This is the preferred approach anyway, for multiple reasons, but one of them being that it ensures proper scaling, no matter if the GC kicks in, or some other kind of lock contention happens.
>
> This would require a change to the startup script to run multiple instances, as well as removing the runTaskDist call, while keeping HTTPServerOption.reusePort.

Thank you for the insight! And of course thank you for vibe.d overall!

That was the kind of respond I was looking for, like some area to explore and if I find time myself I will check it out and see what I can work out.

Kind of on a limited time-basis right now but any spare time I might find in the next two weeks I will see if I can benchmark and figure out some things as well.

On Thu, Jul 11, 2019 at 1:25 PM Sönke Ludwig via Digitalmars-d < digitalmars-d@puremagic.com> wrote:

> The problem is that for some reason it doesn't scale properly with the number of cores in that benchmark. I have a dual-Xeon server that I can in theory use for profiling this, but I simply still lack the time to do so.
>
>
Unfortunately that is not true anymore. Some time ago it was true, on my 4
core system vibed generaly has been one of the most fastest.
But now it is not true. So I am not sure the main issue is with number of
cores.


> If you or anyone else who is interested has a bit of time to spare, I would suggest to change the benchmark setup to start one process per core instead of using multiple threads. This is the preferred approach anyway, for multiple reasons, but one of them being that it ensures proper scaling, no matter if the GC kicks in, or some other kind of lock contention happens.
>

I have already try this and there is no (countable) difference.

As I already said I have spent a lot of time to figure it out, but it seems the main issue right now is with vibe-d:http. But as Boris-Barboris mentioned maybe the main issue is with RC.

Am 12.07.2019 um 00:33 schrieb Daniel Kozak:
> On Thu, Jul 11, 2019 at 1:25 PM Sönke Ludwig via Digitalmars-d <digitalmars-d@puremagic.com <mailto:digitalmars-d@puremagic.com>> wrote:
> 
>     The problem is that for some reason it doesn't scale properly with the
>     number of cores in that benchmark. I have a dual-Xeon server that I can
>     in theory use for profiling this, but I simply still lack the time
>     to do so.
> 
> 
> Unfortunately that is not true anymore. Some time ago it was true, on my 4 core system vibed generaly has been one of the most fastest.
> But now it is not true. So I am not sure the main issue is with number of cores.
> 
>     If you or anyone else who is interested has a bit of time to spare, I
>     would suggest to change the benchmark setup to start one process per
>     core instead of using multiple threads. This is the preferred approach
>     anyway, for multiple reasons, but one of them being that it ensures
>     proper scaling, no matter if the GC kicks in, or some other kind of
>     lock
>     contention happens.
> 
> 
> I have already try this and there is no (countable) difference.
> 
> As I already said I have spent a lot of time to figure it out, but it seems the main issue right now is with vibe-d:http. But as Boris-Barboris mentioned maybe the main issue is with RC.

The issue with InterfaceProxy is definitely true, this has a considerable performance hit and the new vibe-http module will get rid of that. However, even with InterfaceProxy, the performance is still well within the same order of magnitude.

But the Techempower benchmark runs on a 14-core machine (plus HT) - even if a single core would only yield 15 kreq/s, that should give at least >200 kreq/s and not just 60 kreq/s. So the main issue here is definitely scaling AFAICS.

It could of course still be that the GC is somehow involved and the system is RAM bandwidth limited due to constant scanning. The GC pressure should be pretty low, but who knows. Once I get a little bit more time available and have worked a bit on the backlog, I'll run some tests on my 2x8 core box, which should be able to reproduce the issue.

On Friday, 12 July 2019 at 07:47:23 UTC, Sönke Ludwig wrote:
> Am 12.07.2019 um 00:33 schrieb Daniel Kozak:
>> [...]
>
> The issue with InterfaceProxy is definitely true, this has a considerable performance hit and the new vibe-http module will get rid of that. However, even with InterfaceProxy, the performance is still well within the same order of magnitude.
>
> [...]

In recent dmd a bug was fixed which maybe is related to your scaling observation:

https://issues.dlang.org/show_bug.cgi?id=19861

Kind regards
Andre