https://medium.com/@robin.verlangen/billions-of-request-per-day-meet-go-1-5-362bfefa0911

We then started analyzing the behavior of our Go application. On average the application spent ~ 2ms per request, which was great! It gave us 98 milliseconds to spare for network overhead, SSL handshake, DNS lookups and everything else that makes the internet work.

Unfortunately the standard deviation of the latency was high, about 100 milliseconds. Meeting our SLA became a major gamble. With the “runtime” package of Go we started profiling the entire application and found out that garbage collection was the cause, resulting in 95-percentile latencies of 279 milliseconds…

We decided to rewrite big chunks of the application to generate minimal or no garbage at all. This effectively helped reduce the interval at which garbage collection froze the rest of application to do its cleanup magic. But we were still having issues, so we decided to add more nodes to stay within our SLA. With over 80K requests per second at peak times, even minimal garbage can become a serious issue.

...

Yesterday evening (19 August), the moment had finally arrived. A stable version 1.5 of Go was released, claiming:

The “stop the world” phase of the collector will almost always be under 10 milliseconds and usually much less.
Just a few hours after the release we rebuilt the application with the new version of Go 1.5 and ran our unit and functional tests; they all passed. It seemed too good to be true, so we put some effort in manually verifying the functionality. After a few hours we decided it was safe to release it to a single production node.

Yes, Go has sacrificed some compute performance in favour of latency and convenience. They have also released GC improvement plans for 1.6:

https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6ofZXX3yPziAf0V27A64Mo/edit

It is rather obvious that  a building a good concurrent GC is a time consuming effort.

On Fri, 2015-08-21 at 10:47 +0000, via Digitalmars-d-learn wrote:
> Yes, Go has sacrificed some compute performance in favour of latency and convenience. They have also released GC improvement plans for 1.6:
> 
> https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6ofZXX3yPziA f0V27A64Mo/edit
> 
> It is rather obvious that  a building a good concurrent GC is a time consuming effort.

But one that Google are entirely happy to fully fund.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 22 August 2015 at 06:48:48 UTC, Russel Winder wrote:
> But one that Google are entirely happy to fully fund.

Yes, they have made Go fully supported on Google Cloud now, so I think it is safe to say that Google management is backing Go fully.

I'm kinda hoping for Go++...

On Sat, 2015-08-22 at 06:54 +0000, via Digitalmars-d-learn wrote:
> On Saturday, 22 August 2015 at 06:48:48 UTC, Russel Winder wrote:
> > But one that Google are entirely happy to fully fund.
> 
> Yes, they have made Go fully supported on Google Cloud now, so I think it is safe to say that Google management is backing Go fully.
> 
> I'm kinda hoping for Go++...

I think Go 2 is a long way off, and even then generics will not be part of the plan.

Go UK 2015 was held yesterday. It was less a conference and more a Google "rah rah" event. It was though very clear that Google are looking for new idioms and practices to come from users other than Google, rather than what has happened to date, which is the Go central team dictating everything to everyone else.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 22 August 2015 at 06:48:48 UTC, Russel Winder wrote:
> On Fri, 2015-08-21 at 10:47 +0000, via Digitalmars-d-learn wrote:
>> Yes, Go has sacrificed some compute performance in favour of latency and convenience. They have also released GC improvement plans for 1.6:
>> 
>> https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6ofZXX3yPziA f0V27A64Mo/edit
>> 
>> It is rather obvious that  a building a good concurrent GC is a time consuming effort.
>
> But one that Google are entirely happy to fully fund.

because Go is not a general purpose language.
A concurrent GC for D would kill D. Go programs saw a 25-50% performance decrease across the board for the lower latencies.

D could make some very minor changes and be capable of a per-thread GC with none of these performance drawbacks, but nobody seems very interested in it.

On Sat, 2015-08-22 at 07:30 +0000, rsw0x via Digitalmars-d-learn wrote:
> […]
> 
> because Go is not a general purpose language.

Not entirely true. Go is a general purpose language, it is a successor to C as envisioned by Rob Pike, Russ Cox, and others (I am not sure how much input Brian Kernighan has had). However, because of current traction in Web servers and general networking, it is clear that that is where the bulk of the libraries are. Canonical also use it for Qt UI applications. I am not sure of Google real intent for Go on Android, but there is one.

> A concurrent GC for D would kill D. Go programs saw a 25-50% performance decrease across the board for the lower latencies.

They also saw a 100% increase in performance when it was rewritten, and a 20% fall with this latest rewrite. I anticipate great improvement for the 1.6 rewrite.  I am surprised they are retaining having only a single garbage collector: different usages generally require different garbage collection strategies. Having said that Java is moving from having four collectors, to having one, it is going to be interesting to see if G1 meets the needs of all JVM usages.

> D could make some very minor changes and be capable of a per-thread GC with none of these performance drawbacks, but nobody seems very interested in it.

Until some organization properly funds a suite of garbage collectors for different performance targets, you have what there is.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Saturday, 22 August 2015 at 09:16:32 UTC, Russel Winder wrote:
> On Sat, 2015-08-22 at 07:30 +0000, rsw0x via Digitalmars-d-learn wrote:
>> [...]
>
> Not entirely true. Go is a general purpose language, it is a successor to C as envisioned by Rob Pike, Russ Cox, and others (I am not sure how much input Brian Kernighan has had). However, because of current traction in Web servers and general networking, it is clear that that is where the bulk of the libraries are. Canonical also use it for Qt UI applications. I am not sure of Google real intent for Go on Android, but there is one.
>
>> [...]
>
> They also saw a 100% increase in performance when it was rewritten, and a 20% fall with this latest rewrite. I anticipate great improvement for the 1.6 rewrite.  I am surprised they are retaining having only a single garbage collector: different usages generally require different garbage collection strategies. Having said that Java is moving from having four collectors, to having one, it is going to be interesting to see if G1 meets the needs of all JVM usages.
>
>> [...]
>
> Until some organization properly funds a suite of garbage collectors for different performance targets, you have what there is.

The performance decrease has been there since 1.4 and there is no way to remove it - write barriers are the cost you pay for concurrent collection. Go was already much slower than other compiled languages, now it probably struggles to keep up with mono.

On Saturday, 22 August 2015 at 09:16:32 UTC, Russel Winder wrote:
> On Sat, 2015-08-22 at 07:30 +0000, rsw0x via Digitalmars-d-learn wrote:
>> […]
>> 
>> because Go is not a general purpose language.
>
> Not entirely true. Go is a general purpose language, it is a successor to C as envisioned by Rob Pike, Russ Cox, and others (I am not sure how much input Brian Kernighan has had). However, because of current traction in Web servers and general networking, it is clear that that is where the bulk of the libraries are. Canonical also use it for Qt UI applications. I am not sure of Google real intent for Go on Android, but there is one.
>
>> A concurrent GC for D would kill D. Go programs saw a 25-50% performance decrease across the board for the lower latencies.
>
> They also saw a 100% increase in performance when it was rewritten, and a 20% fall with this latest rewrite. I anticipate great improvement for the 1.6 rewrite.  I am surprised they are retaining having only a single garbage collector: different usages generally require different garbage collection strategies. Having said that Java is moving from having four collectors, to having one, it is going to be interesting to see if G1 meets the needs of all JVM usages.
>
>> D could make some very minor changes and be capable of a per-thread GC with none of these performance drawbacks, but nobody seems very interested in it.
>
> Until some organization properly funds a suite of garbage collectors for different performance targets, you have what there is.

I didn't mean to start again the whole GC and Go vs D thing.  Just that one ought to know the lay of the land as it develops.

Out of curiosity, how much funding is required to develop the more straightforward kind of GCs ?  Or to take what's been done and  make it possible for others to use?  It needn't be a single organisation I would think if there are many that would benefit and one doesn't get bogged down in a mentality of people worrying about possibly spurious free rider problems.  Since the D Foundation seems under way, it seems worth asking the question first and thinking about goals without worrying for now about what seems realistic.

On 8/22/2015 10:47 PM, Laeeth Isharc wrote:
> On Saturday, 22 August 2015 at 09:16:32 UTC, Russel Winder wrote:
>> On Sat, 2015-08-22 at 07:30 +0000, rsw0x via Digitalmars-d-learn wrote:
>>> […]
>>>
>>> because Go is not a general purpose language.
>>
>> Not entirely true. Go is a general purpose language, it is a successor
>> to C as envisioned by Rob Pike, Russ Cox, and others (I am not sure
>> how much input Brian Kernighan has had). However, because of current
>> traction in Web servers and general networking, it is clear that that
>> is where the bulk of the libraries are. Canonical also use it for Qt
>> UI applications. I am not sure of Google real intent for Go on
>> Android, but there is one.
>>
>>> A concurrent GC for D would kill D. Go programs saw a 25-50%
>>> performance decrease across the board for the lower latencies.
>>
>> They also saw a 100% increase in performance when it was rewritten,
>> and a 20% fall with this latest rewrite. I anticipate great
>> improvement for the 1.6 rewrite.  I am surprised they are retaining
>> having only a single garbage collector: different usages generally
>> require different garbage collection strategies. Having said that Java
>> is moving from having four collectors, to having one, it is going to
>> be interesting to see if G1 meets the needs of all JVM usages.
>>
>>> D could make some very minor changes and be capable of a per-thread
>>> GC with none of these performance drawbacks, but nobody seems very
>>> interested in it.
>>
>> Until some organization properly funds a suite of garbage collectors
>> for different performance targets, you have what there is.
>
> I didn't mean to start again the whole GC and Go vs D thing. Just that
> one ought to know the lay of the land as it develops.
>
> Out of curiosity, how much funding is required to develop the more
> straightforward kind of GCs ?  Or to take what's been done and  make it
> possible for others to use?  It needn't be a single organisation I would
> think if there are many that would benefit and one doesn't get bogged
> down in a mentality of people worrying about possibly spurious free
> rider problems.  Since the D Foundation seems under way, it seems worth
> asking the question first and thinking about goals without worrying for
> now about what seems realistic.

I believe the hardest part is finding somebody can and willing to work on it.
For example I'm willing but I don't know how and there are people willing with a job and can do it. But cannot dedicated time because of money.

Really it comes down to having a budget and if somebody says hey I'll do x, y and z features to pay them for their time as they do it.
Even if they only do one small feature which takes a week.