Steve Schveighoffer wrote:
>
> It's too bad that tail-const wouldn't work.  I don't think tail-shared presents the same problems.  Please don't dismiss all the points I wrote just because of past failures.  They are new ideas that don't apply to const at all.
>
> 

They do apply. It is not about the semantic difference between const and shared, it is about the question "is this expression shared or not shared? const or not const?" How do I declare a type to be tail-shared? tail-const? Those questions must be unambiguously answered before you can apply semantics.

In this context, tail-shared and tail-const are exactly the same problem.

Le 2010-01-08 ? 15:04, Walter Bright a ?crit :

> Steve Schveighoffer wrote:
>> The only caveat is, we would *need* to change the semantics of shared so that you can have a non-shared reference to a shared class instance easily.
> 
> That's the so-called "tail-shared" problem. We tried (hard) to make "tail-const" work, where the reference is mutable but the contents were const. It sounds simple, but just does not work in a language with implicit reference semantics.

I wrote something using immutable objects once. I had to use Rebindable (after fixing it) for almost all members of type object. It was quite annoying. I predict the same thing will happen for shared.

I'm don't really want to reopen the const debate, but I feel the same issue is going to repeat itself with shared and it makes me feel uneasy too. Are we going to need Rebindable!(TailShared!(const shared T))? All I can see that could work is some sort of tail modifier, like "const! shared! T" (change "!" for your preferred character). It isn't very intuitive, but neither is Rebindable!(TailShared!(const shared T)) and at least it looks better and should play better with overloading.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

Le 2010-01-08 ? 15:00, Walter Bright a ?crit :

> Your proposal requires yet a new type constructor, "shared immutable", with a new set of rules and overloads. I don't think it's worth it. Currently, "shared immutable" == "immutable", i.e. the two are indistinguishable.

Since I don't think you'll change your mind at this point I'll rest my case. Let's forget about those per-thread GCs. ;-(

This leaves us with the following definitions to check our designs again:

* shared: applies to memory that can always be made visible to other threads
* immutable: applies to memory that cannot be changed and can always be made visible to other threads (and thus implicitly shared)

It's not exactly what I hoped, but it's still better than what I've seen in other languages, which practically means nothing.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

Michel Fortin wrote:
>
>
> This leaves us with the following definitions to check our designs again:
>
> * shared: applies to memory that can always be made visible to other threads
> * immutable: applies to memory that cannot be changed and can always be made visible to other threads (and thus implicitly shared)
>
> It's not exactly what I hoped, but it's still better than what I've seen in other languages, which practically means nothing.
>
> 

I think it's a lot better than better than nothing!

This all sounds excellent to me.

A question:
How will the message-passing API avoid exposing "shared"? Surely the
message channel itself is a shared mutable object, even though the
messages passing through it are immutable. If it isn't shared, aren't we
cheating by hiding the fact that multiple threads are accessing it?


Walter Bright wrote:
>
> There's another aspect here. Consider all the problems we have getting across the idea of an immutable type. What hope is there for shared? I see mass confusion everywhere. Frankly, I see little hope of any but a handful of programmers ever being able to grok shared and use it correctly for concurrent programs. The notion that one can just slap 'shared' on a data type and have it work correctly across threads without further thought is a pipe dream.
>
> So what to do?
>
> I want to pin the mainstream concurrency on message passing. The message passing user never sees shared, never has to deal with locks, never has to deal with memory barriers. It just works. Message passing should be a robust, scalable solution for most users. I believe the Erlang experience validates this. Go and Scala also rely entirely on message passing (but they don't have immutable data, so their models are unsafe and I predict many rude surprises).
>
> So why bother with shared at all?
>
> Because message passing does not cover all the bases, and D is supposed to be a systems programming language. So we need a paradigm for synchronization and shared data structures. What shared provides is:
>
> 1. A way to identify shared data. This is incredibly important. A lot of sharing bugs come about because of inadvertant unrecognized sharing of data. This should be pretty much impossible in D. Furthermore, if you do have a sharing bug in your code, you look at the 1% of the data tagged as shared, rather than every freakin' line of code and every piece of data. Half the battle in debugging code is figuring out where to look for the problem. Shared pares that problem down to a reasonable size.
>
> 2. Shared comes with a collection of static typing rules and guarantees that will head off a number of concurrency bugs, such as sequential consistency.
>
> I view shared as sort of like the latest electric arc welders which automatically adjust the current and wire feed for you. They dramatically shorten (but don't eliminate) the learning curve for people trying to master the art of welding. D is the only language to even attempt this. C++ leaves you completely on your own, Java offers no help, Erlang, Scala and Go throw in the towel and won't allow anything but message passing.
>
> As for a shared gc vs thread local gc, I just see an awful lot of
> strange irreproducible bugs when someone passes data from one to the
> other. I doubt it's worth it, unless it can be done with compiler
> guarantees, which seem doubtful.
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

The send and receive routines for message passing will just be template functions, so their implementation is totally hidden from the user.  It shouldn't matter in the least ifthe underlying implementation uses shared or not.  They may be able to optimize passing shared data better by avoiding a copy but this would just happen.  My hope is that the typical application will use shared in it's own code little to not at all.

Sent from my iPhone

On Jan 10, 2010, at 3:16 PM, Graham St Jack <graham.stjack at internode.on.net
 > wrote:

> This all sounds excellent to me.
>
> A question:
> How will the message-passing API avoid exposing "shared"? Surely the
> message channel itself is a shared mutable object, even though the
> messages passing through it are immutable. If it isn't shared,
> aren't we cheating by hiding the fact that multiple threads are
> accessing it?
>
>
> Walter Bright wrote:
>>
>> There's another aspect here. Consider all the problems we have getting across the idea of an immutable type. What hope is there for shared? I see mass confusion everywhere. Frankly, I see little hope of any but a handful of programmers ever being able to grok shared and use it correctly for concurrent programs. The notion that one can just slap 'shared' on a data type and have it work correctly across threads without further thought is a pipe dream.
>>
>> So what to do?
>>
>> I want to pin the mainstream concurrency on message passing. The message passing user never sees shared, never has to deal with locks, never has to deal with memory barriers. It just works. Message passing should be a robust, scalable solution for most users. I believe the Erlang experience validates this. Go and Scala also rely entirely on message passing (but they don't have immutable data, so their models are unsafe and I predict many rude surprises).
>>
>> So why bother with shared at all?
>>
>> Because message passing does not cover all the bases, and D is supposed to be a systems programming language. So we need a paradigm for synchronization and shared data structures. What shared provides is:
>>
>> 1. A way to identify shared data. This is incredibly important. A lot of sharing bugs come about because of inadvertant unrecognized sharing of data. This should be pretty much impossible in D. Furthermore, if you do have a sharing bug in your code, you look at the 1% of the data tagged as shared, rather than every freakin' line of code and every piece of data. Half the battle in debugging code is figuring out where to look for the problem. Shared pares that problem down to a reasonable size.
>>
>> 2. Shared comes with a collection of static typing rules and guarantees that will head off a number of concurrency bugs, such as sequential consistency.
>>
>> I view shared as sort of like the latest electric arc welders which automatically adjust the current and wire feed for you. They dramatically shorten (but don't eliminate) the learning curve for people trying to master the art of welding. D is the only language to even attempt this. C++ leaves you completely on your own, Java offers no help, Erlang, Scala and Go throw in the towel and won't allow anything but message passing.
>>
>> As for a shared gc vs thread local gc, I just see an awful lot of
>> strange irreproducible bugs when someone passes data from one to
>> the other. I doubt it's worth it, unless it can be done with
>> compiler guarantees, which seem doubtful.
>> _______________________________________________
>> dmd-concurrency mailing list
>> dmd-concurrency at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

Sean Kelly wrote:
> The send and receive routines for message passing will just be template functions, so their implementation is totally hidden from the user.  It shouldn't matter in the least ifthe underlying implementation uses shared or not.  They may be able to optimize passing shared data better by avoiding a copy but this would just happen.  My hope is that the typical application will use shared in it's own code little to not at all.
I'm happy with things being carefully wrapped and auto-magic. However, surely the message channel (or whatever it is called) object itself will have be shared (and its externally accessible methods synchronized), otherwise we are cheating. We have a object whose sole purpose in life is to be shared by multiple threads, so surely if anything is shared, this one has to be.

If this is the case, then we need a mechanism to stop "shared" from leaking out into the whole code base.

Since no-one ever responds to this line of questioning, it looks like I am missing something fundamental. Can anyone help me out here?

>
> Sent from my iPhone
>
> On Jan 10, 2010, at 3:16 PM, Graham St Jack <graham.stjack at internode.on.net> wrote:
>
>> This all sounds excellent to me.
>>
>> A question:
>> How will the message-passing API avoid exposing "shared"? Surely the
>> message channel itself is a shared mutable object, even though the
>> messages passing through it are immutable. If it isn't shared, aren't
>> we cheating by hiding the fact that multiple threads are accessing it?
>>
>>
>> Walter Bright wrote:
>>>
>>> There's another aspect here. Consider all the problems we have getting across the idea of an immutable type. What hope is there for shared? I see mass confusion everywhere. Frankly, I see little hope of any but a handful of programmers ever being able to grok shared and use it correctly for concurrent programs. The notion that one can just slap 'shared' on a data type and have it work correctly across threads without further thought is a pipe dream.
>>>
>>> So what to do?
>>>
>>> I want to pin the mainstream concurrency on message passing. The message passing user never sees shared, never has to deal with locks, never has to deal with memory barriers. It just works. Message passing should be a robust, scalable solution for most users. I believe the Erlang experience validates this. Go and Scala also rely entirely on message passing (but they don't have immutable data, so their models are unsafe and I predict many rude surprises).
>>>
>>> So why bother with shared at all?
>>>
>>> Because message passing does not cover all the bases, and D is supposed to be a systems programming language. So we need a paradigm for synchronization and shared data structures. What shared provides is:
>>>
>>> 1. A way to identify shared data. This is incredibly important. A lot of sharing bugs come about because of inadvertant unrecognized sharing of data. This should be pretty much impossible in D. Furthermore, if you do have a sharing bug in your code, you look at the 1% of the data tagged as shared, rather than every freakin' line of code and every piece of data. Half the battle in debugging code is figuring out where to look for the problem. Shared pares that problem down to a reasonable size.
>>>
>>> 2. Shared comes with a collection of static typing rules and guarantees that will head off a number of concurrency bugs, such as sequential consistency.
>>>
>>> I view shared as sort of like the latest electric arc welders which automatically adjust the current and wire feed for you. They dramatically shorten (but don't eliminate) the learning curve for people trying to master the art of welding. D is the only language to even attempt this. C++ leaves you completely on your own, Java offers no help, Erlang, Scala and Go throw in the towel and won't allow anything but message passing.
>>>
>>> As for a shared gc vs thread local gc, I just see an awful lot of
>>> strange irreproducible bugs when someone passes data from one to the
>>> other. I doubt it's worth it, unless it can be done with compiler
>>> guarantees, which seem doubtful.
>>> _______________________________________________
>>> dmd-concurrency mailing list
>>> dmd-concurrency at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>>
>> _______________________________________________
>> dmd-concurrency mailing list
>> dmd-concurrency at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency

Le 2010-01-10 ? 22:47, Graham St Jack a ?crit :

> I'm happy with things being carefully wrapped and auto-magic. However, surely the message channel (or whatever it is called) object itself will have be shared (and its externally accessible methods synchronized), otherwise we are cheating. We have a object whose sole purpose in life is to be shared by multiple threads, so surely if anything is shared, this one has to be.
> 
> If this is the case, then we need a mechanism to stop "shared" from leaking out into the whole code base.

When your browser requests a web page to a server, it sends messages and receives back replies. But it doesn't need the browser need to have a "shared" object with the server. Instead, you have a socket on each side and both get connected through the operating system's TCP/IP stack auto-magic.

In the same way, a message-passing API, especially one that intends to go beyond threads, doesn't need and shouldn't expose any shared object. That doesn't mean the implementation won't use shared for thread-to-thread communications, but the user of the API shouldn't need to see that.


-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

Graham St Jack wrote:
> I'm happy with things being carefully wrapped and auto-magic. However, surely the message channel (or whatever it is called) object itself will have be shared (and its externally accessible methods synchronized), otherwise we are cheating. We have a object whose sole purpose in life is to be shared by multiple threads, so surely if anything is shared, this one has to be.
>
> If this is the case, then we need a mechanism to stop "shared" from leaking out into the whole code base.
>
> Since no-one ever responds to this line of questioning, it looks like I am missing something fundamental. Can anyone help me out here?
>

Messages are values, not references. Values have copy semantics, so the message is a copy, so the receiving thread gets a copy, not a reference.

Immutables can be passed by reference, because immutables have value semantics.

Mutable objects are reference types, and so cannot be passed as messages.

An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-concurrency/attachments/20100112/f074a59c/attachment.htm>