On Sun, 27 Dec 2009 15:32:52 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.

I probably wouldn't contribute much, but I'm satisfied as long as I can view the discussion. I have learned a lot from the D newsgroups.

I would strongly prefer a newsgroup to a mailing list. I wish every mailing list on the Internet would convert to a newsgroup.

The rest of this message is just a dump of my limited experience with threading in case its useful for you to see the perspective of someone without a lot of threading experience.

My only major experience with concurrency involved a program which was split into a GUI (using C#/WPF) and a C++ DLL with three always-on threads plus a thread from C# coming in occasionally using PInvoke. The three threads in the C++ side were in while(true) loops which checked volatile boolean flags for work and slept briefly when there was nothing to do. (See below for a general idea of what the code looked like.) I didn't use any synchronization except the volatile flags and global volatile sets of parameters. This worked for me since only one thread handled any one piece of functionality. So some built-in way of low overhead cross-thread function calling would be nice. Or if there was a better way to do it, a design that would nudge me in the right direction would be good.

Actually, after looking at the code below, I'm not convinced I got it right since TakeData could possibly be called before thread1_buffer got filled.

globals:
volatile bool take_data_flag = false;
volatile char* take_data_buffer = null; // constant buffer size (something like 32KiB)

thread1:
void TakeData(char* buffer) {
	take_data_buffer = buffer;
	take_data_flag = true;
	take_data_called = true;
}

bool take_data_called = false;
char thread1_buffer[BUFFER_SIZE];
while (true) {
	if (!take_data_called) {
		// Add data to thread1_buffer
		if (<buffer is full>) {
			TakeData(thread1_buffer);
		}
	} else {
		if (take_data_flag == false) {
			// ok to use buffer again
			take_data_called = false;
		}
	}
	sleep(50);
}

thread2:
char thread2_buffer[BUFFER_SIZE];
while (true) {
	if (take_data_flag) {
		// copy take_data_buffer to thread2_buffer
		take_data_flag = false;
		// process thread2_buffer
	}
	else
		sleep(50);
}

Michel Fortin wrote:
> On 2009-12-27 15:32:52 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:
> 
>> I think we are now in the position of defining a solid set of concurrency primitives for D. This follows many months of mulling over models and options.
>>
>> It would be great to open the participation to the design as broadly as possible, but I think it's realistic to say we won't be able to get things done on the newsgroup. When we discuss a topic around here, there's plenty of good ideas but also the inevitable bikeshed discussions, explanations being asked, explanations being given, and other sources of noise. We simply don't have the time to deal with all that - the time is short and we only have one shot at this.
>>
>> That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.
> 
> I think it should be as open as possible. If done in a separate smaller group, it may be a good idea to post reports to the general newsgroup more or less regularly so that those who cannot participate in the detailed discussions have an idea of where it's going, and also to get more general input.

That's obviously the best way to go, but there are a couple of circumstances that make that more difficult.

1. Chapter drafts will be the basis for discussion, but understandably the publisher does not allow me to freely distribute them.

2. Time. There are regulars on this group that have a "when in doubt, make them sweat" policy. I think it's a very good and gainful attitude for everyone involved, and I generally enjoy discussing this or that idea because it helps me and others gain a better understanding, but this time there won't be much time for discussions of the form:

a) Poster: "Subtle issue X sounds like a bad idea. I don't agree with it."

b) <Long argumentation back and forth.>

c) Poster: "I stay unconvinced." or "That makes sense."

There will be very little time for anything like this, particularly if explaining X requires a fair amount of background building.

Building a shared vision is very difficult among only a small group of people, and doing so for a larger group will be an enormous drag. I feel very lucky that Walter and I share views most of the time (except, of course, when he's wrong :o)).

> About the bikeshed issue, I'm not sure how much those bikeshed discussions are slowing down the more important ones, but they often start from legitimate real, often syntactic, issues. Those discussions shouldn't be avoided just because everyone has an opinion. But perhaps regular reports to the general newsgroup would help confining them there.
> 
> I'd be in favor of creating a newsgroup for concurrency, and I'll probably want to participate a little too, although I'm not sure how much yet.

OK, let's see what Walter thinks.


Andrei

Phil Deets, el 28 de diciembre a las 11:09 me escribiste:
> On Sun, 27 Dec 2009 15:32:52 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> 
> >That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.
> 
> I probably wouldn't contribute much, but I'm satisfied as long as I can view the discussion. I have learned a lot from the D newsgroups.
> 
> I would strongly prefer a newsgroup to a mailing list. I wish every mailing list on the Internet would convert to a newsgroup.

Then you will looove Gmane, because it is exactly that a NNTP gateway for
almost any ML:
http://www.gmane.org/

(you can add missing mailing lists)

BTW, I think all the discussion should be here, if there is no time to discuss, there is no point in creating another ML/NG, just discuss it privately and be done with it (I really think it would be a very bad idea though).

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
"Somos testigos de Jaimito, venimos a traer la salvación, el mundo va a
desaparecer, somos testigos de Jaimito!". Nos enyoguizamos... Así que
"somos testigos"? Te dejo el culo hecho un vino, y la conch'itumá, y la
conch'itumá!
	-- Sidharta Kiwi

Mon, 28 Dec 2009 10:20:53 -0600, Andrei Alexandrescu wrote:

> Michel Fortin wrote:
>> On 2009-12-27 15:32:52 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:
>> 
>>> I think we are now in the position of defining a solid set of concurrency primitives for D. This follows many months of mulling over models and options.
>>>
>>> It would be great to open the participation to the design as broadly as possible, but I think it's realistic to say we won't be able to get things done on the newsgroup. When we discuss a topic around here, there's plenty of good ideas but also the inevitable bikeshed discussions, explanations being asked, explanations being given, and other sources of noise. We simply don't have the time to deal with all that - the time is short and we only have one shot at this.
>>>
>>> That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.
>> 
>> I think it should be as open as possible. If done in a separate smaller group, it may be a good idea to post reports to the general newsgroup more or less regularly so that those who cannot participate in the detailed discussions have an idea of where it's going, and also to get more general input.
> 
> That's obviously the best way to go, but there are a couple of circumstances that make that more difficult.
> 
> 1. Chapter drafts will be the basis for discussion, but understandably the publisher does not allow me to freely distribute them.
> 
> 2. Time. There are regulars on this group that have a "when in doubt, make them sweat" policy. I think it's a very good and gainful attitude for everyone involved, and I generally enjoy discussing this or that idea because it helps me and others gain a better understanding, but this time there won't be much time for discussions of the form:
> 
> a) Poster: "Subtle issue X sounds like a bad idea. I don't agree with it."
> 
> b) <Long argumentation back and forth.>
> 
> c) Poster: "I stay unconvinced." or "That makes sense."
> 
> There will be very little time for anything like this, particularly if explaining X requires a fair amount of background building.
> 
> Building a shared vision is very difficult among only a small group of people, and doing so for a larger group will be an enormous drag. I feel very lucky that Walter and I share views most of the time (except, of course, when he's wrong :o)).

I think a shared vision is easier to achieve in a small group. If the group size is 1, you only need to convince yourself, if 2, you have only one other person who needs to agree and so forth. In groups like c++ standards committee every player in the software industry wants to be heard. That's why it's not gonna be c++0x anymore, maybe c++1x (c++0x0a) or c++2x.

I don't like how in D community new features are added. We often first get a binary dmd release containing the new feature, after that some obscure half-official explanation about the corner cases that will be fixed in the next release. Only after that Walter might answer 'Yes.' to some well laid questions and finally after 2-3 years a very terse informal text is added to the specification.

What kind of experts are you exactly seeking now? Guys who build massive and scalable native/green thread pools to serve http requests? Guys who compute weather forecasts with 10.000 node MPI setups? Is it going to be a multi-threaded, multi-core, or multi-computer architecture?

On 29/12/2009 3:48 a.m., Michel Fortin wrote:
> On 2009-12-27 15:32:52 -0500, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> said:
>
>> I think we are now in the position of defining a solid set of
>> concurrency primitives for D. This follows many months of mulling over
>> models and options.
>>
>> It would be great to open the participation to the design as broadly
>> as possible, but I think it's realistic to say we won't be able to get
>> things done on the newsgroup. When we discuss a topic around here,
>> there's plenty of good ideas but also the inevitable bikeshed
>> discussions, explanations being asked, explanations being given, and
>> other sources of noise. We simply don't have the time to deal with all
>> that - the time is short and we only have one shot at this.
>>
>> That's why I'm thinking of creating a mailing list or maybe another
>> group for this. Any ideas on what would be the best approach? I also
>> want to gauge interest from threading experts who'd like to
>> participate. Please advise: (a) whether you would like to participate
>> to the design; (b) keep discussions on the general group; (c) create a
>> separate newsgroup; (d) create a mailing list. The latter would have
>> open enrollment.
>
> I think it should be as open as possible. If done in a separate smaller
> group, it may be a good idea to post reports to the general newsgroup
> more or less regularly so that those who cannot participate in the
> detailed discussions have an idea of where it's going, and also to get
> more general input.
>
> About the bikeshed issue, I'm not sure how much those bikeshed
> discussions are slowing down the more important ones, but they often
> start from legitimate real, often syntactic, issues. Those discussions
> shouldn't be avoided just because everyone has an opinion. But perhaps
> regular reports to the general newsgroup would help confining them there.
>
> I'd be in favor of creating a newsgroup for concurrency, and I'll
> probably want to participate a little too, although I'm not sure how
> much yet.
>

I agree with the open as possible +1. A closed model would likely only permit fully committed and those that have no say without letting anything in between where as the open allows some feedback.

As for off topic: This can be avoided by using another form of communication. Maybe just add a few announcements here and put all proposals with for/against arguments on the wiki. Or anything that supports some sort of moderation/editing but I think wiki is perfect for this.

On Mon, 28 Dec 2009 12:46:25 -0500, Leandro Lucarella <llucax@gmail.com> wrote:

>
> Then you will looove Gmane, because it is exactly that a NNTP gateway for
> almost any ML:
> http://www.gmane.org/
>
> (you can add missing mailing lists)
>

Thanks for the link. It looks awesome.

On 2009-12-28 11:20:53 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> Michel Fortin wrote:
> 
>> I think it should be as open as possible. If done in a separate smaller group, it may be a good idea to post reports to the general newsgroup more or less regularly so that those who cannot participate in the detailed discussions have an idea of where it's going, and also to get more general input.
> 
> That's obviously the best way to go, but there are a couple of circumstances that make that more difficult.
> 
> 1. Chapter drafts will be the basis for discussion, but understandably the publisher does not allow me to freely distribute them.

I see. I find that unfortunate, because this seems to imply it'll need to be a closed group, not only closed regarding the participants but also regarding external observers (because the basis of the discussion will be hidden from them).

Often you can evaluate how good something is not only by looking at the final result, but by looking at the process that led to it. Having an archive of the past is often very useful to check why some things have been made that way. But that works best if the discussion are done in the open.


> 2. Time. There are regulars on this group that have a "when in doubt, make them sweat" policy. I think it's a very good and gainful attitude for everyone involved, and I generally enjoy discussing this or that idea because it helps me and others gain a better understanding, but this time there won't be much time for discussions of the form:
> 
> a) Poster: "Subtle issue X sounds like a bad idea. I don't agree with it."
> 
> b) <Long argumentation back and forth.>
> 
> c) Poster: "I stay unconvinced." or "That makes sense."
> 
> There will be very little time for anything like this, particularly if explaining X requires a fair amount of background building.
> 
> Building a shared vision is very difficult among only a small group of people, and doing so for a larger group will be an enormous drag. I feel very lucky that Walter and I share views most of the time (except, of course, when he's wrong :o)).

I think it'd be a good idea to set an objective for the group for that limited time frame. This way if the discussion deviates towards something off-topic it's easy to suggest posting to the general or the D.learn newsgroup instead, or if the discussion is draining taking too much time in explanations it's easy to justify postponing that to later in order to fulfill the more pressing objectives. Just be careful not setting the goal too broadly.

Also keep in mind that we don't really need a shared vision among everyone. What's needed is someone who takes the decisions. Discussion is only needed to help that person take the right decisions. Although consensus among all members certainly boosts the decider self-confidence, it is not required, and not necessarily desirable either. A consensus among only a few key people is all that is needed, and this has little to do with who is allowed to raise issues and propose solutions.

And those few key people are the implementors, because if they don't agree the result isn't gonna be pretty.

(This reminds me pretty much of how HTML5 is being developed right now: Ian Hickson, the spec editor, decides what goes in and out. But he always seeks the broadest consensus among the major browser vendors because in the end it's them who decide what they want to implement, and if they refuse to implement the spec, the spec is useless. This doesn't prevent the WHATWG mailing list from being open to all. I did some research a few years ago, posted the results on the list and after some discussion it was used to make the <figure> element what it is now. I wouldn't have bothered if the list wasn't open.)

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Sun, 27 Dec 2009 12:32:52 -0800, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> I think we are now in the position of defining a solid set of concurrency primitives for D. This follows many months of mulling over models and options.
>
> It would be great to open the participation to the design as broadly as possible, but I think it's realistic to say we won't be able to get things done on the newsgroup. When we discuss a topic around here, there's plenty of good ideas but also the inevitable bikeshed discussions, explanations being asked, explanations being given, and other sources of noise. We simply don't have the time to deal with all that - the time is short and we only have one shot at this.
>
> That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.
>
>
> Andrei

I'd like to participate and prefer newsgroups to mailing lists. I think a separate newsgroup, although some work to setup, would make it easier to follow the discussions.

Andrei Alexandrescu wrote:
> I think we are now in the position of defining a solid set of concurrency primitives for D. This follows many months of mulling over models and options.
> 
[snip]
> 
> That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.
> 
> 
> Andrei


Andrei

Will you be inviting Bartosz to participate, or have you already has discussions with him ? He has written a number of  blogs around this issue.

Nick B

Nick B wrote:
> Andrei Alexandrescu wrote:
>> I think we are now in the position of defining a solid set of concurrency primitives for D. This follows many months of mulling over models and options.
>>
> [snip]
>>
>> That's why I'm thinking of creating a mailing list or maybe another group for this. Any ideas on what would be the best approach? I also want to gauge interest from threading experts who'd like to participate. Please advise: (a) whether you would like to participate to the design; (b) keep discussions on the general group; (c) create a separate newsgroup; (d) create a mailing list. The latter would have open enrollment.
>>
>>
>> Andrei
> 
> 
> Andrei
> 
> Will you be inviting Bartosz to participate, or have you already has discussions with him ? He has written a number of  blogs around this issue.
> 
> Nick B

I'm not sure about Bartosz' and Walter's thoughts. I personally think that past experience suggests that collaboration would be difficult. No less than three attempts to work together on threads ranging from design to implementation have failed, in spite of everybody's best intentions.

Andrei