On 3/27/2015 1:41 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> On Friday, 27 March 2015 at 08:25:26 UTC, Walter Bright wrote:
>> The MMU makes it pointless. The virtual address space allows for 4 billion
>> goroutines with 4 billion bytes each of stack.
>
> If you fragment the memory space you cannot use recursive page tables? If you
> want to address more than 512GB you need to move to 1MiB pages.
>
> http://wiki.osdev.org/Page_Tables

So what? The point is there is PLENTY of virtual address space. You can "allocate" absurd amounts of address space for each goroutine, and still have plenty without physically moving anything.

On Thu, 2015-03-26 at 18:47 -0700, Walter Bright via Digitalmars-d-announce wrote:
> On 3/26/2015 12:40 PM, Russel Winder via Digitalmars-d-announce wrote:
> > (Almost) All publicity is good publicity.
> 
> 
> I attended a presentation at NWCPP on Go last week. I have never written a Go program, so filter my opinion on that.
> 
> It seems to me that every significant but one feature of Go has a pretty much direct analog in D, i.e. you can write "Go" code in D much like you can write "C" code in D.

That is almost certainly true. I suspect you can write Go in C++ as well. D and C++ are definitely supersets of Go.

> The one difference was Go's support for green threads. There's no technical reason why D can't have green threads, it's just that nobody has written the library code to do it.


I think the way go handles interfaces and their composition would require a few tricks in D and C++, but I am sure it can be done.


Aren't "green threads" now given the label fibres? And I think Vibe.d has an implementation for these – but I do not know for certain.

The dataflow way of working and the special case CSP, also actors are clearly the right abstraction for most parallel computation, using as we all agree, a kernel thread pool animating tasks. std.parallelism has something along these lines but focussed on data parallelism.

Given the existence of C++CSP2 (http://www.cs.kent.ac.uk/projects/ofa/c ++csp/) D can get a CSP implementation for free, especially if the recent innovation on C++ working come to fruition.

As work on GPars and Quasar in the JVM arena, and Erlang and Go since they were created, show, lightweight processes with communication channels is the next step in providing good abstractions for CPU and IO bound processing.

Anthony Williams (Just Software, https://www.justsoftwaresolutions.co.uk/)) has been at work trying to put actors and dataflow on top of C++11 threads, with some success.

D needs to corral all the bits, which I think are there, to create a
good offering. However, I cannot see this happening purely on volunteer,
hobbyist resource. We need to find an organization or three willing to
resource this activity.


-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On 3/27/2015 2:57 AM, Russel Winder via Digitalmars-d-announce wrote:
> I think the way go handles interfaces and their composition would
> require a few tricks in D and C++, but I am sure it can be done.

Interfaces can be done with D templates. It'll be compile time polymorphism rather than run time, but the same effect will result (and of course it'll be faster).

It's pretty much how Ranges work in D.


> Aren't "green threads" now given the label fibres?

My understanding of fibers is they are all in one thread. Go's green threads can be in multiple threads, the same thread, and even moved from one thread to another.

> And I think Vibe.d
> has an implementation for these – but I do not know for certain.

I don't know, either.


> D needs to corral all the bits, which I think are there, to create a
> good offering.

Yes.

> However, I cannot see this happening purely on volunteer,
> hobbyist resource. We need to find an organization or three willing to
> resource this activity.

I don't think it's that hard of a problem.

On Friday, 27 March 2015 at 09:44:27 UTC, Walter Bright wrote:
> On 3/27/2015 1:41 AM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>> On Friday, 27 March 2015 at 08:25:26 UTC, Walter Bright wrote:
>>> The MMU makes it pointless. The virtual address space allows for 4 billion
>>> goroutines with 4 billion bytes each of stack.
>>
>> If you fragment the memory space you cannot use recursive page tables? If you
>> want to address more than 512GB you need to move to 1MiB pages.
>>
>> http://wiki.osdev.org/Page_Tables
>
> So what? The point is there is PLENTY of virtual address space. You can "allocate" absurd amounts of address space for each goroutine, and still have plenty without physically moving anything.

If you don't care about performance, bloated page tables and laying waste to memory:

1. Page tables are hierarchical, to save space nodes can point back to themselves. IFF the node is similar. Throwing shit all over the memory space makes this impossible. So you get BLOATed page tables.

2. All dirty pages maps to physical memory. So one recursion chain on a fiber will create a big physical mem allocation. To get rid of it you would have make a slow syscall.

3. You have no way to go below the page size for a stack.

4. Operating system calls like mmap are slow.

5. Trapped pages are slow and so are modifying page tables.

6. You cannot expect to get more than 47-53 bits of address space from the OS. It's not like 64 bits CPUs provide 64 bits address space. There is a big hole in the middle.

Have you actually thought about these issues or done performance tests?

On Fri, 2015-03-27 at 10:14 +0000, via Digitalmars-d-announce wrote: […]
> 
> Have you actually thought about these issues or done performance tests?

The Go team certainly have, and have changed their goroutine model twice because of it. No matter what they do in Go 0.0 →1.4, 1.5 onwards will be different since the who system is being revised: Go implemented Go toolchain, new GC, new runtime. I suspect Go 1.6 will be the watershed for this, but that will likely be 2016.

The question is though what should happen in D. If Vibe.d fibres are a single threaded system, then they are not suitable for the actor, dataflow, CSP implementation needed in D since that must sit on a kernel thread pool where each lightweight thread is animated by whichever work stealing kernel thread comes along. Erlang, Go, GPars, Quasar, etc. all have different solutions to the problem of thread pools and task switching since the context is all important.


-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Fri, 2015-03-27 at 03:11 -0700, Walter Bright via Digitalmars-d-announce wrote:
> On 3/27/2015 2:57 AM, Russel Winder via Digitalmars-d-announce wrote:
[…]
> > However, I cannot see this happening purely on volunteer,
> > hobbyist resource. We need to find an organization or three willing to
> > resource this activity.
> 
> I don't think it's that hard of a problem.

If no-one is resourced to write the code, it will not get written.

-- 
Russel. ============================================================================= Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net 41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

On Friday, 27 March 2015 at 10:37:01 UTC, Russel Winder wrote:
> The question is though what should happen in D. If Vibe.d fibres are a
> single threaded system, then they are not suitable for the actor,
> dataflow, CSP implementation needed in D since that must sit on a kernel
> thread pool where each lightweight thread is animated by whichever work
> stealing kernel thread comes along. Erlang, Go, GPars, Quasar, etc. all
> have different solutions to the problem of thread pools and task
> switching since the context is all important.

Yes, I agree that  the question is what should happen in D. But the claim was that D provides everything Go does and there is only a tiny scheduler that is missing. I don't think D benefits from these claims. Benchmark D thoroughly against Go before making claims or just give Go credit for being better in some areas.

If it was up to me then co-routines would be ripped out of the language. They are a cross cutting feature that makes significant optimizations and improvements difficult or impossible.

On Friday, 27 March 2015 at 03:53:36 UTC, Laeeth Isharc wrote:
>>> That kind of articles are bad for the image of the D community
>
> Nick S:
>> No. Just...no.
>>
>> I'm honestly *really* tired of general society's (seemingly?) increasing intolerance FOR intolerance.
>>
>> Some things ARE bad. Some ideas are dumb ideas (ie without merit). Some features are bad features. Some products really are crappy products. Calling it out when you see it, using a frank explanation of your reasoning, isn't bad, it's productive.
>
> Excellence is incompatible with tolerating mediocrity or what is appalling, and what I have seen is that there are aesthetic aspects to creative endeavours not conventionally thought of as having an aesthetic element, and it is in the nature of such things that one cannot and should not tolerate what one perceives to be ugly in a creative endeavour.  If one is driven mostly by ROI rather than high feelings, one doesn't get to excellence.  So it is my belief that dealing with creative people means dealing with a certain ... intensity.
>
> That (on the aesthetic aspects of technical fields) is not just my opinion, but also (I think) that of a certain Mr W Bright, judging by his comments on how good code should look and on good aircraft design, although he presented this in his usual low-key manner.  I was looking for a language that was beautiful, as well as powerful, and for whatever it is worth, this was a factor of high appeal with D.
>
> It's also the view of Feynman, not to mention many great minds of the past.  Ie it is limiting to insist on data before forming a strong opinion about something (which is not to say that one may not change one's mind in the face of contrary data).
>
> "You can recognize truth by its beauty and simplicity. When you get it right, it is obvious that it is right—at least if you have any experience—because usually what happens is that more comes out than goes in. ...The inexperienced, the crackpots, and people like that, make guesses that are simple, but you can immediately see that they are wrong, so that does not count. Others, the inexperienced students, make guesses that are very complicated, and it sort of looks as if it is all right, but I know it is not true because the truth always turns out to be simpler than you thought." - Feynman via Wikiquote (but the same idea comes across in his books).
>
>> To discourage dissent, objections, or complaints is to rob ourselves of potential improvement. *That's* what critique and complaints and objections ARE: Recognition of the potential for improvement. There *cannot* be progress and improvement without first identifying existing faults. If nobody ever identified and voiced criticism of punchcards, for example, we'd all still be stuck in the world of 1950's computing.
>
> Excellently put.   (And, I would add, a constructive draw towards what is generative - not just fault-finding).
>
>> It's not as if "the D crowd" doesn't critique itself and it's own language just plenty, so it's not like there's any hypocrisy here. And I'm certainly not willing to accept that programmers should be viewed as being part of distinct mutually-exclusive factions based on some single-language allegiance. I'm a D guy. I also happen to be a fan of Nemerle. And both languages have things I hate. So scratch the "it's the D crowd" idea.
>
> Interesting - what should I read about Nemerle, and what is it best at ?
>>
>> And seriously, the article in question barely mentions D at all.
>>
>> So no, this is NOT some sort of "D community piece attacking another language" as some comments seem to imply. It is merely an isolated critique of one language by someone who happens to be *using* the given language.
>
> There are some very interesting psychological dynamics in the reaction to this kind of piece.  For me it was key that although it was clearly written in a humorous tone, and hurriedly, he seemed to speak from the heart - it is refreshing to see such work even when one doesn't agree with it.
>
> BTW since when has linking to something been an endorsement of it?

Interesting. Have you read Oscar Wilde? Your comments remind me of him somehow. I was just thinking yesterday how working with D makes me happy whereas working with other (lower) languages makes me grumpy. Going down to the punchcard level (PHP, JS etc.) is boring and doesn't do justice to the human mind. Whenever I use D, I am confident that I can map a complicated reality onto a machine, it inspires me and it challenges me. Primitive languages discourage me. So there's more to productivity than meets the eye when looking at numbers. Numbers are insignificant, they can prove anything you want, and you can tweak them any way you want. "Eat shit, a million flies can't be wrong!", as they say.

It's one thing to be productive in terms of maintaining and selling apps and another in terms of being innovative. You can sell a million records by sticking to well-trodden paths (dum-dum-dum-di-dum) or you can be a Mozart, a Beethoven, a Miles Davis or a Hendrix and just say "I'm gonna do my own thing!". Sure, it involves what is commonly perceived as "arrogance", but it's not.

I don't think it's such a good idea to dump on another language too much. My reaction to Go is that it doesn't have what I want in it, and that's about it. People can write Go if they want to, and I won't.

I think I'd prefer to just present a good tool. If it's good enough for a particular job people will use it.

Am 27.03.2015 um 11:11 schrieb Walter Bright:
> On 3/27/2015 2:57 AM, Russel Winder via Digitalmars-d-announce wrote:
>> Aren't "green threads" now given the label fibres?
>
> My understanding of fibers is they are all in one thread. Go's green
> threads can be in multiple threads, the same thread, and even moved from
> one thread to another.
>
>> And I think Vibe.d
>> has an implementation for these – but I do not know for certain.
>
> I don't know, either.

It has, that is more or less the original selling point. It also keeps an internal thread pool where each thread has a dynamic set of reusable fibers to execute tasks. Each fiber is bound to a certain thread, though, and they have to, because otherwise things like thread local storage or other thread specific code (e.g. the classic OpenGL model, certain COM modes etc.) would break.

Apart from these concerns, It's also not clear to me that moving tasks between threads is necessarily an improvement. There are certainly cases where that leads to a better distribution across the cores, but in most scenarios the number of concurrent tasks should be high enough to keep all cores busy anyhow. There are also additional costs for moving fibers (synchronization, cache misses).