I run Linux.

Atila

On Friday, 7 March 2014 at 13:31:06 UTC, Rikki Cattermole wrote:
> On Friday, 7 March 2014 at 08:23:09 UTC, Atila Neves wrote:
>> It was already far above the competition in the throughput benchmark anyway. What exactly doesn't feel right to you?
>>
>> On Friday, 7 March 2014 at 05:44:16 UTC, Rikki Cattermole wrote:
>>> On Thursday, 6 March 2014 at 17:17:12 UTC, Atila Neves wrote:
>>>> Well, I found out the other day that vibe.d compiles with gdc now so I went back to see if it made any difference to the benchmarks I had.
>>>>
>>>> In throughput it made none.
>>>>
>>>> In the latency one it was about 5-10% faster with gdc compared to dmd, which is good, but it still didn't change the relative positions of the languages.
>>>>
>>>> So that was anti-climatic. :P
>>>>
>>>> Atila
>>>
>>> I'm suspecting that Vibe's performance is heavily based upon the systems state i.e. hdd. Not so much on the code generation.
>>> I don't know where we can get more performance out of it. But something doesn't quite feel right.
>
> Mostly related to how heavy of an effect a systems IO can have on performance i.e. hdd. Avast makes things a lot worse as well. Thanks to its file system shield. Could possibly get a performance gain by utilising Window's event manager. At Least for Windows.

On Friday, 7 March 2014 at 08:45:22 UTC, Bienlein wrote:
> On Friday, 7 March 2014 at 08:23:09 UTC, Atila Neves wrote:
>
>>> I'm suspecting that Vibe's performance is heavily based upon the systems state i.e. hdd. Not so much on the code generation.
>>> I don't know where we can get more performance out of it. But something doesn't quite feel right.
>
> Robert Pike, the Go lead developer, some days ago published this tweet:
>
> "Just looked at a Google-internal Go server with 139K goroutines serving over 68K active network connections. Concurrency wins."

68K connections is nothing. I'll start getting interested when his benchmarks are 200K+.  Event-based systems in C can handle millions of concurrent connections if implemented properly.  I'd like to believe vibe.d can approach this as well.


> In that way your MQTT benchmarks falls short with a maximum of 1k connections. You need to repeat it with 50k and 100k connections. Then Go and Erlang will rock and leave D behind. If you want to be fair with Erlang you need to make a benchmark run with 1.000k connections and more, see https://www.erlang-solutions.com/about/news/erlang-powered-whatsapp-exceeds-200-million-monthly-users

Does Erlang really scale that well for network IO?  I love their actor model, but their network programming model kind of stinks.


> I don't like Go's simplistic nature, either, but Go is not about the language. It is about making concurrency much simpler and allowing for many many threads. IMHO this is what gives Go the attention. Except for Erlang no other system/language than Go can get something similar accomplished (except Rust maybe when it is finished, but it is not clear whether it will have good built times like Go or D).
>
> If you want to give D a boost, put Go-style CSP and green threads into it as well. Then D will start to fly. Otherwise it will have to continue competing against C++ as its sole application area where it will always remain a niche player, because of the market dominance of C++.

vibe.d already works this way. And there's a pull request in place to make std.concurrency support green threads. I think we're really pretty close. I do need to set aside some time to start on IPC though.

On Fri, 2014-03-07 at 12:23 +0000, Atila Neves wrote:
[…]
> I suspect you might have missed the point of my original blog post. Yes, it shows D beating Erlang and Go, and that's something I obviously like. But that wasn't the point I was trying to make. My point was that just by writing it in Go doesn't mean magical performance benefits because of its CSP, and that vibe.d's fibers would do just fine in a direct competition. The data seem to support that.

That doesn't mean a CSP and dataflow implementations for D (à la DataRush, GPars, Go, PythonCSP, PyCSP) shouldn't attempted.  Sadly I think I do not have the time to drive such an endeavour, but I wish I could contribute to it if someone else could drive.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 3/7/14, 12:45 AM, Bienlein wrote:
> If you want to give D a boost, put Go-style CSP and green threads into
> it as well. Then D will start to fly. Otherwise it will have to continue
> competing against C++ as its sole application area where it will always
> remain a niche player, because of the market dominance of C++.

Interesting you should mention that. Walter has been mulling over a possible DIP on that.

Andrei

On Fri, 2014-03-07 at 16:53 +0000, Sean Kelly wrote:
[…]
> 68K connections is nothing. I'll start getting interested when his benchmarks are 200K+.  Event-based systems in C can handle millions of concurrent connections if implemented properly.  I'd like to believe vibe.d can approach this as well.

There used to be a 100k problem, i.e maintaining more than 100k active, that means regularly causing traffic, not just being dormant for a few centuries, but so many frameworks can now support that , that it has become a non-metric. I don't know if Spring, JavaEE, can handle this but on the JVM Vert.x certainly, I suspect Node.js can as well. Vert.x is caliming to be able to handle millions of active connections.

I suspect it is now at the stage that the OS is the bottle neck not the language of the framework.

> > I don't like Go's simplistic nature, either, but Go is not about the language. It is about making concurrency much simpler and allowing for many many threads. IMHO this is what gives Go the attention. Except for Erlang no other system/language than Go can get something similar accomplished (except Rust maybe when it is finished, but it is not clear whether it will have good built times like Go or D).
> >
> > If you want to give D a boost, put Go-style CSP and green threads into it as well. Then D will start to fly. Otherwise it will have to continue competing against C++ as its sole application area where it will always remain a niche player, because of the market dominance of C++.
> 
> vibe.d already works this way. And there's a pull request in place to make std.concurrency support green threads. I think we're really pretty close. I do need to set aside some time to start on IPC though.

I agree that as a stripped down C, Go sucks. But as a strongly typed language, unlike C, it is not bad. But as everyone agrees (I hope), Go's USP is CSP (*). The whole goroutines thing (and the QML capability) keeps me using Go. And to be honest the whole interfaces model and statically typed but duck typed is great fun.

I think what Go and Erlang do is to use green threads (or equivalent, goroutines in Go) for the applications side and a kernel thread pool within the runtime doing "work stealing" on the green threads. This is more or less (ish) what the Java Fork/Join framework of Doug Lea does as well. The upshot is that you appear to be able to have thousands of threads in your program but maybe only a few actual kernel threads doing the work.


(*) Rob Pike reports that he and co-workers came up with the Go model
independently of Hoare's CSP, via the Newsqueak, Alef, Limbo, Go
sequence. I see no reason to disbelieve him. Whatever the truth, Go is
now marketed as realizing CSP, not the Hoare variant of 1978 but CSP
with amendments introduced over time. It's just a pity no-one yet has a
realization of π-calculus as well – other than the programming language
Pict, and the Scala library PiLib.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 3/7/14, 12:45 AM, Bienlein wrote:
> If you want to give D a boost, put Go-style CSP and green threads into
> it as well. Then D will start to fly. Otherwise it will have to continue
> competing against C++ as its sole application area where it will always
> remain a niche player, because of the market dominance of C++.

One question - doesn't Vibe.d already use green threads?

Andrei

On Friday, 7 March 2014 at 18:58:18 UTC, Russel Winder wrote:
> I suspect it is now at the stage that the OS is the bottle neck not the
> language of the framework.

I think specialized operating systems devoted to single service will be the future of high load web projects similar to current realities of hard real-time services.

On Friday, 7 March 2014 at 19:01:34 UTC, Andrei Alexandrescu
wrote:
> On 3/7/14, 12:45 AM, Bienlein wrote:
>> If you want to give D a boost, put Go-style CSP and green threads into
>> it as well. Then D will start to fly. Otherwise it will have to continue
>> competing against C++ as its sole application area where it will always
>> remain a niche player, because of the market dominance of C++.
>
> One question - doesn't Vibe.d already use green threads?
>
> Andrei

It does. Bienlein has a very vague knowledge of topics he
comments about.

On Friday, 7 March 2014 at 18:58:18 UTC, Russel Winder wrote:

> It's just a pity no-one yet has a
> realization of π-calculus as well – other than the programming language Pict, and the Scala library PiLib.

JoCaml, an extension of Ocaml, also comes to mind. It's join-calculus, not pi-calculus, but I understand that each can be encoded in the other.

Graham

On Fri, 2014-03-07 at 19:03 +0000, Dicebot wrote:
> On Friday, 7 March 2014 at 18:58:18 UTC, Russel Winder wrote:
> > I suspect it is now at the stage that the OS is the bottle neck
> > not the
> > language of the framework.
> 
> I think specialized operating systems devoted to single service will be the future of high load web projects similar to current realities of hard real-time services.

I guess we just have to look at Bitcoin mining to appreciate how slowly Web server technology actually moves.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder