January 20, 2010 [dmd-concurrency] draft 5 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Robert Jacques | Robert Jacques wrote:
> On Wed, 20 Jan 2010 01:36:10 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
>
>> Kevin Bealer wrote:
>>> So the caller calls setMaxMailboxSize with a callee's Tid. Then that
>>> caller will not send messages unless the callee's mailbox is smaller
>>> than that amount?
>>> Will all callers obey the limit, or should they each set their own
>>> limit? If the latter, then the design is as I thought.
>>
>> Each thread has a mailbox. That mailbox has a capacity. That capacity can be set from any thread. It's that simple.
>>
>> Andrei
>
> Bike shed but, what about: tid.capacity = N; ?
Yah but sometimes one needs to also set the contingency action atomically.
Andrei
|
January 20, 2010 [dmd-concurrency] shutting down | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | On Wed, Jan 20, 2010 at 8:50 PM, Andrei Alexandrescu <andrei at erdani.com>wrote: > Michel Fortin wrote: > >> Le 2010-01-20 ? 18:37, Andrei Alexandrescu a ?crit : >> >> Sean Kelly wrote: >>> >>>> On Jan 20, 2010, at 1:00 PM, Andrei Alexandrescu wrote: >>>> >>>>> I'm thinking, there are too many programs that hang when I want to end them just because some thread inside is waiting for a socket. I think that any operation that could block for an arbitrary amount of time should be disallowed during shutdown. >>>>> >>>> What if the program is logging to a remote host and wants to log a shutdown event? >>>> >>> I think it should do so in main. Nevertheless, my intuition is that we can use a judgment call a la "file operations are considered fast so we won't kill them" even though technically a file could be opened over a network connection. >>> >> >> This is the kind of assumption I'd gladly do without. >> >> While typically you might consider the network slow, there are cases where the software is designed to work with some specific hardware. When you have a guarantied 1000 Mbps Ethernet crossover wire connecting directly two devices, you know the network interface isn't slow. In fact, the latency is probably much better on an 1000 Mbps Ethernet link than with a rotating hard drive. >> >> Also if you're writing a program that works on one of these two interdependent devices, sending a shutdown notice to the other system is one thing you definitely want to do. >> >> This is not a made-up example. >> > > I agree, but the disk is usually encased with the machine. The network is a connection with an independent machine. That's a fundamental difference. I'm aware that many things that seem to be disks and files are also really connections with independent machines. > > Nevertheless, I'm open to any idea that makes things better. In short nobody stops people in special situations from writing code that does the appropriate synchronization. I just want to figure out some reasonable defaults, and to me it seems like taking down sockets but not files is a reasonable default. Do you know a better one? > > > Andrei I think it's bad to have a special shutdown mode that everyone has to use for their primary shutdown approach. But I think the good news is, this isn't necessarily a "make everyone do X" scenario. Assuming the OOB exposes enough power to allow users to do most of this themselves, then it's a matter of having a selection of policies and letting people pick one off the shelf or (in special cases) send their own messages of doom. I want a way to send messages to all actors and in general to implement my own scheme for this kind of policy. If I can do my own all-points-bulletin with tools that are just as powerful as what your version is using, then I can just do whatever I want from main() and let your end-of-main code be plan B. In that case, I love your plan because it's a reasonable default for people who don't have specific requirements and for most smallish applications. Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.puremagic.com/pipermail/dmd-concurrency/attachments/20100120/9dd461c0/attachment-0001.htm> |
January 21, 2010 [dmd-concurrency] draft 5 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | On Wed, 20 Jan 2010 21:27:29 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
> Robert Jacques wrote:
>> On Wed, 20 Jan 2010 01:36:10 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
>>
>>> Kevin Bealer wrote:
>>>> So the caller calls setMaxMailboxSize with a callee's Tid. Then that
>>>> caller will not send messages unless the callee's mailbox is smaller
>>>> than that amount?
>>>> Will all callers obey the limit, or should they each set their own
>>>> limit? If the latter, then the design is as I thought.
>>>
>>> Each thread has a mailbox. That mailbox has a capacity. That capacity can be set from any thread. It's that simple.
>>>
>>> Andrei
>> Bike shed but, what about: tid.capacity = N; ?
>
> Yah but sometimes one needs to also set the contingency action atomically.
>
> Andrei
I was actually think capacity would be a property and therefore support atomicity.
|
January 20, 2010 [dmd-concurrency] draft 5 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Robert Jacques | Robert Jacques wrote:
> On Wed, 20 Jan 2010 21:27:29 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
>
>> Robert Jacques wrote:
>>> On Wed, 20 Jan 2010 01:36:10 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
>>>
>>>> Kevin Bealer wrote:
>>>>> So the caller calls setMaxMailboxSize with a callee's Tid. Then
>>>>> that caller will not send messages unless the callee's mailbox is
>>>>> smaller than that amount?
>>>>> Will all callers obey the limit, or should they each set their own
>>>>> limit? If the latter, then the design is as I thought.
>>>>
>>>> Each thread has a mailbox. That mailbox has a capacity. That capacity can be set from any thread. It's that simple.
>>>>
>>>> Andrei
>>> Bike shed but, what about: tid.capacity = N; ?
>>
>> Yah but sometimes one needs to also set the contingency action atomically.
>>
>> Andrei
>
> I was actually think capacity would be a property and therefore support atomicity.
I mean: you want to set capacity _and_ contingency action in one atomic unit.
Andrei
|
January 21, 2010 [dmd-concurrency] draft 5 | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | On Thu, 21 Jan 2010 00:24:40 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
> Robert Jacques wrote:
>> On Wed, 20 Jan 2010 21:27:29 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
>>
>>> Robert Jacques wrote:
>>>> On Wed, 20 Jan 2010 01:36:10 -0500, Andrei Alexandrescu <andrei at erdani.com> wrote:
>>>>
>>>>> Kevin Bealer wrote:
>>>>>> So the caller calls setMaxMailboxSize with a callee's Tid. Then
>>>>>> that caller will not send messages unless the callee's mailbox is
>>>>>> smaller than that amount?
>>>>>> Will all callers obey the limit, or should they each set their own
>>>>>> limit? If the latter, then the design is as I thought.
>>>>>
>>>>> Each thread has a mailbox. That mailbox has a capacity. That capacity can be set from any thread. It's that simple.
>>>>>
>>>>> Andrei
>>>> Bike shed but, what about: tid.capacity = N; ?
>>>
>>> Yah but sometimes one needs to also set the contingency action atomically.
>>>
>>> Andrei
>> I was actually think capacity would be a property and therefore
>> support atomicity.
>
> I mean: you want to set capacity _and_ contingency action in one atomic unit.
>
> Andrei
You're right. I forgot about the contingency action. Still, member functions generally are nicer than free functions.
|
January 21, 2010 [dmd-concurrency] shutting down | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu |
----- Original Message ----
> From: Andrei Alexandrescu <andrei at erdani.com>
> To: Discuss the concurrency model(s) for D <dmd-concurrency at puremagic.com>
> Sent: Wed, January 20, 2010 4:00:22 PM
> Subject: Re: [dmd-concurrency] shutting down
>
> Michel Fortin wrote:
> > Le 2010-01-20 ? 15:24, Andrei Alexandrescu a ?crit :
> >
> >> I think threads shouldn't be able to do much during shutdown. For example I
> don't think they should suddenly open sockets. How about opening files? Or calling system()? Any ideas on a unified approach?
> >
> > Why do you think that?
>
> I'm thinking, there are too many programs that hang when I want to end them just because some thread inside is waiting for a socket. I think that any operation that could block for an arbitrary amount of time should be disallowed during shutdown.
>
> What sockets library do today is, they offer a global close_library function. Once that is called, all pending socket connections and all new socket connections fail immediately.
You're getting closer to a function that effectively and artificially kills all threads. I don't really like this scenario, a thread should be able to do whatever it wants, even during shutdown. For instance, let's say you log that a thread is ending via a finally clause at the end of the thread. This works no problem with default logging configuration since the log is to disk, but one enterprising user has all his applications log to a network socket, so he configures the logging library of your app to use a network socket to store data. All of a sudden, the logs go away because doing the log causes an exception. I think this is unacceptable and hard to plan for. You can't possibly know all the uses of sockets in underlying functions that a thread may call on shutdown. I think all those should be allowed to proceed.
Not to mention you are relying on the network library providing such a function which might not exist (killing all socket communication is not a standard socket function). Someone who wants this behavior can implement it manually in environments where it is provided -- I don't think it should be a standard practice.
Normal practice is to not block on a socket indefinitely (i.e. wait for data in a loop with a timeout, checking for shutdown every loop), or wait for both socket data *and* another event. In some OSes, the latter is not possible, and I think Phobos should try to use the least common denominator.
On the other hand, it's perfectly valid to compose such a loop in a standard function without relying on OS ability:
ubyte[] readWithShutdown(Socket s, int period, int timeout, ubyte[] buffer);
what this means is, read from socket s in a loop, checking for shutdown every period, returning with no data after timeout elapses. Increasing the period improves CPU usage, but makes shutdowns less responsive.
I'd prefer such an approach to infiltrating the socket library using assumptions that might not be true.
BTW, I agree that all threads should exit when main exits. Any multithreadded app I've written always follows this rule. I haven't ever really had a need to use a daemon thread.
-Steve
|
January 21, 2010 [dmd-concurrency] shutting down | ||||
---|---|---|---|---|
| ||||
Posted in reply to Steve Schveighoffer | Steve Schveighoffer wrote: > You're getting closer to a function that effectively and artificially kills all threads. I don't really like this scenario, a thread should be able to do whatever it wants, even during shutdown. For instance, let's say you log that a thread is ending via a finally clause at the end of the thread. This works no problem with default logging configuration since the log is to disk, but one enterprising user has all his applications log to a network socket, so he configures the logging library of your app to use a network socket to store data. All of a sudden, the logs go away because doing the log causes an exception. I think this is unacceptable and hard to plan for. You can't possibly know all the uses of sockets in underlying functions that a thread may call on shutdown. I think all those should be allowed to proceed. I understand your concern, but I also like the following scenario when I'm at the airport: 1. I start Firefox 2. There is no wireless available 3. All my many pages are spinning waiting to load 4. I close Firefox It's instant. I want to prevent threads started by arbitrary libraries for various purposes from being able to delay application shutdown indefinitely. > Not to mention you are relying on the network library providing such a function which might not exist (killing all socket communication is not a standard socket function). Someone who wants this behavior can implement it manually in environments where it is provided -- I don't think it should be a standard practice. That function always exists (see my previous post). > Normal practice is to not block on a socket indefinitely (i.e. wait for data in a loop with a timeout, checking for shutdown every loop), or wait for both socket data *and* another event. In some OSes, the latter is not possible, and I think Phobos should try to use the least common denominator. Normal practice is to close the socket library, and it will instantly fail all pending and new calls. That makes both threads and the main program much easier to write. Andrei |
January 21, 2010 [dmd-concurrency] shutting down | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | Fixin that situation is easy--just close the sockets.
Sent from my iPhone
On Jan 21, 2010, at 9:08 AM, Andrei Alexandrescu <andrei at erdani.com> wrote:
> Steve Schveighoffer wrote:
>> You're getting closer to a function that effectively and artificially kills all threads. I don't really like this scenario, a thread should be able to do whatever it wants, even during shutdown. For instance, let's say you log that a thread is ending via a finally clause at the end of the thread. This works no problem with default logging configuration since the log is to disk, but one enterprising user has all his applications log to a network socket, so he configures the logging library of your app to use a network socket to store data. All of a sudden, the logs go away because doing the log causes an exception. I think this is unacceptable and hard to plan for. You can't possibly know all the uses of sockets in underlying functions that a thread may call on shutdown. I think all those should be allowed to proceed.
>
> I understand your concern, but I also like the following scenario when I'm at the airport:
>
> 1. I start Firefox
>
> 2. There is no wireless available
>
> 3. All my many pages are spinning waiting to load
>
> 4. I close Firefox
>
> It's instant. I want to prevent threads started by arbitrary libraries for various purposes from being able to delay application shutdown indefinitely.
>
>> Not to mention you are relying on the network library providing such a function which might not exist (killing all socket communication is not a standard socket function). Someone who wants this behavior can implement it manually in environments where it is provided -- I don't think it should be a standard practice.
>
> That function always exists (see my previous post).
>
>> Normal practice is to not block on a socket indefinitely (i.e. wait for data in a loop with a timeout, checking for shutdown every loop), or wait for both socket data *and* another event. In some OSes, the latter is not possible, and I think Phobos should try to use the least common denominator.
>
> Normal practice is to close the socket library, and it will instantly fail all pending and new calls. That makes both threads and the main program much easier to write.
>
>
> Andrei
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
|
January 21, 2010 [dmd-concurrency] shutting down | ||||
---|---|---|---|---|
| ||||
Posted in reply to Andrei Alexandrescu | Le 2010-01-21 ? 12:08, Andrei Alexandrescu a ?crit : > I understand your concern, but I also like the following scenario when I'm at the airport: > > 1. I start Firefox > > 2. There is no wireless available > > 3. All my many pages are spinning waiting to load > > 4. I close Firefox > > It's instant. I want to prevent threads started by arbitrary libraries for various purposes from being able to delay application shutdown indefinitely. I think you're trying to fix the problem at the wrong layer. Putting restrictions to the shutdown protocol means that people will bypass the shutdown protocol (write their own) when it gets in the way, and then libraries concerned with properly closing things will each implement their own shutdown protocol you'll need to call manually at the end of main. Forcefully closing sockets when the application closes isn't much different than calling exit(0), but limited to sockets. In either case, it's a major pain to workaround when you really need to do something before termination. And I'll add that in a GUI application, often it's the GUI that is sluggish to terminate itself, taking its time to close windows, closing files, closing everything. Only once that's done is the main thread terminated, at which point little time remains to be gained. All this talk makes me think of a funny feature Apple added to Mac OS X 10.6 (Snow Leopard): Sudden Termination. When you shutdown the computer, the OS literally kills every process having the sudden termination flag set. Applications are free to set and unset this flag as they run. Typically, an application with no document open will set its sudden termination flag. But the flag is not set by default so applications can do proper cleanup. Perhaps this could work for threads too. -- Michel Fortin michel.fortin at michelf.com http://michelf.com/ |
January 21, 2010 [dmd-concurrency] shutting down | ||||
---|---|---|---|---|
| ||||
Posted in reply to Michel Fortin | Michel Fortin wrote: > I think you're trying to fix the problem at the wrong layer. Putting restrictions to the shutdown protocol means that people will bypass the shutdown protocol (write their own) when it gets in the way, and then libraries concerned with properly closing things will each implement their own shutdown protocol you'll need to call manually at the end of main. I think putting restrictions on the shutdown protocol means that people will make sure their programs end orderly. > Forcefully closing sockets when the application closes isn't much different than calling exit(0), but limited to sockets. In either case, it's a major pain to workaround when you really need to do something before termination. Sockets must be closed when main exits. If you want to wait for the sockets to timeout, you wait in main. I don't see how that is a limitation. It just clarifies the application exit sequence. > And I'll add that in a GUI application, often it's the GUI that is sluggish to terminate itself, taking its time to close windows, closing files, closing everything. Only once that's done is the main thread terminated, at which point little time remains to be gained. In a properly designed application, indeed. But sockets timeout is (by default) 60 seconds. Nobody would be happy if there was 60 seconds extra delay when e.g. restarting their browser. > All this talk makes me think of a funny feature Apple added to Mac OS X 10.6 (Snow Leopard): Sudden Termination. When you shutdown the computer, the OS literally kills every process having the sudden termination flag set. Applications are free to set and unset this flag as they run. Typically, an application with no document open will set its sudden termination flag. But the flag is not set by default so applications can do proper cleanup. > > Perhaps this could work for threads too. This is an interesting feature, and I can definitely see its motivation and its usefulness in Phobos' threads. Andrei |
Copyright © 1999-2021 by the D Language Foundation