It's the user's responsibility, just as with any of the other 'unsafe' functions. Also, it's the runtime, not the standard library. I think there is a significant difference between those two terms. You shouldn't touch the runtime in general unless you're prepared to go low-level.

Regards,
Alex

On Wed, May 16, 2012 at 11:48 PM, Sean Kelly <sean@invisibleduck.org> wrote:
> On May 16, 2012, at 2:39 PM, Alex Rønne Petersen wrote:
>
>> What do you mean? This might hold true if you're executing in a signal handler, but if the code you execute after suspending/killing a bunch of threads does not touch any state that the suspended/killed threads owned/accessed, you'll be fine.
>
> But how do you know what the threads own or are accessing?  This is the standard library.  The user could be doing absolutely anything.
> _______________________________________________
> D-runtime mailing list
> D-runtime@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

I'm sure it could be done outside of core.thread, but then anyone who legitimately knows what they're doing and needs it would have to reinvent the wheel.

Regards,
Alex

On Wed, May 16, 2012 at 11:47 PM, Sean Kelly <sean@invisibleduck.org> wrote:
> Right.  This is why I don't want the functionality in core.thread.  There are potentially legitimate reasons to kill a thread, but exposing it in the library effectively sanctions it as acceptable behavior, when this absolutely isn't the case.  Unless there were some reason why it absolutely couldn't be done without support in core.thread, I don't think inherently unsafe operations should be exposed to the user.
>
> On May 16, 2012, at 2:36 PM, Alex Rønne Petersen wrote:
>
>> In my particular case, the threads I'm going to kill are executing isolated managed code, so going into an undefined state can't really happen, but you're certainly right that it's entirely possible in normal C and D code. In fact, even .NET has a zillion warnings about using Thread.Abort(): http://msdn.microsoft.com/en-us/library/ty8d3wta.aspx
>>
>> Regards,
>> Alex
>>
>> On Wed, May 16, 2012 at 11:32 PM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>>> On Wednesday, May 16, 2012 17:19:02 David Simcha wrote:
>>>> For the sake of argument, what are the most non-obvious reasons why killing threads is bad? The ones I can think of are because the thread may be in the middle of doing something important and bad things will happen if it's interrupted and because the thread might hold resources that will never get freed if it's killed before it gets to free them.
>>>>
>>>> I was thinking at one point that I wanted a kill() primitive when I was designing std.parallelism, but I don't remember why I wanted it and obviously I've managed to do without it. Is it ok to kill threads if they're not doing useful work at the time and they're not holding onto any resources that will never get freed if you kill them?
>>>
>>> It's the same reason that you don't want to continue executing after an assertion fails. Your program is by definition in an undefined state. In this case, it would be a single thread which was then in an undefined state. If it were sufficiently isolated (e.g. doesn't use shared _at all_), it _might_ mean that the rest of the program is okay, but there's no question that anything relating to that thread is then in an undefined state, and odds are that that means that the rest of the program is also in an undefined state, though the impact is likely to be much less if the thread was well isolated. If you were using shared much though, there's not really any question that your whole program is then in an undefined state, and who knows what could happen if you tried to continue. And even without using shared much, the threading stuff is still built on top of C primitives which _are_ shared (well, __gshared), so by definition, there's at least _some_ portion of the rest of your program which is in an undefined state.
>>>
>>> - Jonathan M Davis
>>> _______________________________________________
>>> D-runtime mailing list
>>> D-runtime@puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/d-runtime
>> _______________________________________________
>> D-runtime mailing list
>> D-runtime@puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/d-runtime
>
> _______________________________________________
> D-runtime mailing list
> D-runtime@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On May 16, 2012, at 2:51 PM, Alex Rønne Petersen wrote:

> It's the user's responsibility, just as with any of the other 'unsafe' functions. Also, it's the runtime, not the standard library. I think there is a significant difference between those two terms. You shouldn't touch the runtime in general unless you're prepared to go low-level.


I think that there are three important aspects of library design.  The first is to make it useful by exposing any functionality that is commonly needed by users of the library, or in some cases that is not commonly needed but which can't be done without library support.  The second is that this should be done with the understanding that the interface of a library provides a framework for how the library is to be used.  This is the really tricky part, because a library could expose the same functionality in two different ways and the applications built upon each would be fundamentally different.  The third is that the library should be no more complex than absolutely necessary.  Doing so tends to expose too many assumptions about how the library will be used, and tends to confuse things without really adding much of use.

What I've found is that when people thing of something useful for their program, if it's something regarding a library they tend to want to build that functionality into the library even if it doesn't need to be there.  The problem is that they may be the only person in the world that needs that functionality, and adding it to the library may increase maintenance cost to no one's actual benefit but that one user.  There have been a number of requests like this for Druntime, and some of the few that were accepted have just been deleted after having been deprecated for years, without a single complaint.  Assuming that people are actually using Druntime, I'll admit that I prefer to see it shrinking rather than growing.  It means that it's being distilled down to only containing the functionality that people actually care about.

Regarding Druntime's "expert only" status… Druntime still follows the core tenet of D which is that the design should encourage people to do things the "right way" by making that approach inherently easy.  For example, the GC needs to suspend and scan threads so it can collect unused memory.  This functionality is absolutely needed.  But an important observation is that the GC only ever needs to suspend and scan all threads in bulk--never just one at a time.  So rather than exposing a Thread.suspend() routine as in Java, Druntime only provides a way to suspend all threads at once.  This makes it far more likely that the functionality will only be used as was intended--by the GC--rather than by users as a means of implementing synchronization.  And if you don't think people use suspend() in this way you need look no farther than Doug Lea's containers library.  He's clearly an expert and so Gets it Right, and he's admittedly doing it by necessity since Java doesn't have semaphores or whatever, but I think it's a fair point all the same.  What should have happened was for semaphores to be added as a separate API so Doug didn't have to cowboy it with suspend().
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On May 16, 2012, at 2:51 PM, Alex Rønne Petersen wrote:

> I'm sure it could be done outside of core.thread, but then anyone who legitimately knows what they're doing and needs it would have to reinvent the wheel.

But it's trivial in this case:

class KillableThread : Thread {
    this(void function() fn) {
        m_fn = fn;
        super(run);
    }

    public kill() {
        pthread_cancel(m_thisThread);
    }

    private void run() {
        m_thisThread = pthread_self();
        fn();
    }

    private pthread_t m_thisThread;
    private void function() m_fn;
}


Which I suppose could be made even easier by adding a protected method to Thread that returns the thread handle.
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

The trouble is that they have to have version blocks for every platform and every threading library supported. The fact that core.thread supports pthreads should not even be known to code that extends the Thread class; it's an implementation detail (other threading libraries exist, especially on Linux). I believe there is currently active work to make GDC support other C libraries, and I imagine that involves other threading libraries on rather exotic platforms too.

The way I've always seen core.thread is as a fundamental building block for threading; it should hide the native threading interface completely so people shouldn't have to mess with those at all.

Regards,
Alex

On Thu, May 17, 2012 at 12:36 AM, Sean Kelly <sean@invisibleduck.org> wrote:
> On May 16, 2012, at 2:51 PM, Alex Rønne Petersen wrote:
>
>> I'm sure it could be done outside of core.thread, but then anyone who legitimately knows what they're doing and needs it would have to reinvent the wheel.
>
> But it's trivial in this case:
>
> class KillableThread : Thread {
>    this(void function() fn) {
>        m_fn = fn;
>        super(run);
>    }
>
>    public kill() {
>        pthread_cancel(m_thisThread);
>    }
>
>    private void run() {
>        m_thisThread = pthread_self();
>        fn();
>    }
>
>    private pthread_t m_thisThread;
>    private void function() m_fn;
> }
>
>
> Which I suppose could be made even easier by adding a protected method to Thread that returns the thread handle.
> _______________________________________________
> D-runtime mailing list
> D-runtime@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On Wednesday, May 16, 2012 14:44:12 Sean Kelly wrote:
> If the killed thread even has a chance of allocating memory ir could be killed while inside the GC and lock all threads out of the GC forever.

Ouch.

- Jonathan M Davis
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

Again, this boils down to knowing what you're doing. You should NOT be killing a thread if there is any chance whatsoever that the thread could be doing GC allocation while you do it.

That said, I'm not convinced that this is a problem that we cannot work around; I think that if kill() would acquire the global thread lock, this problem could be prevented from happening.

Regards,
Alex

On Thu, May 17, 2012 at 12:52 AM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> On Wednesday, May 16, 2012 14:44:12 Sean Kelly wrote:
>> If the killed thread even has a chance of allocating memory ir could be killed while inside the GC and lock all threads out of the GC forever.
>
> Ouch.
>
> - Jonathan M Davis
> _______________________________________________
> D-runtime mailing list
> D-runtime@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On May 16, 2012, at 3:42 PM, Alex Rønne Petersen wrote:

> The trouble is that they have to have version blocks for every platform and every threading library supported. The fact that core.thread supports pthreads should not even be known to code that extends the Thread class; it's an implementation detail (other threading libraries exist, especially on Linux). I believe there is currently active work to make GDC support other C libraries, and I imagine that involves other threading libraries on rather exotic platforms too.
> 
> The way I've always seen core.thread is as a fundamental building block for threading; it should hide the native threading interface completely so people shouldn't have to mess with those at all.

I tend to agree, though for some of the really risky stuff I think it may be fair to expect the user to be familiar with the platform specifics of what he's trying to do.  Kill threads, for example, works differently on Posix vs. Win32 and is (surprise surprise) actually a bit safer.  Regarding killing threads in general though, I wonder if there isn't perhaps a safer application-specific approach that you could use.  Could these threads periodically check some global "cancel" state and exit if set?  Is it truly necessary to have an external thread violently kill them?
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

On Thu, May 17, 2012 at 12:28 AM, Sean Kelly <sean@invisibleduck.org> wrote:
> On May 16, 2012, at 2:51 PM, Alex Rønne Petersen wrote:
>
>> It's the user's responsibility, just as with any of the other 'unsafe' functions. Also, it's the runtime, not the standard library. I think there is a significant difference between those two terms. You shouldn't touch the runtime in general unless you're prepared to go low-level.
>
>
> I think that there are three important aspects of library design.  The first is to make it useful by exposing any functionality that is commonly needed by users of the library, or in some cases that is not commonly needed but which can't be done without library support.  The second is that this should be done with the understanding that the interface of a library provides a framework for how the library is to be used.  This is the really tricky part, because a library could expose the same functionality in two different ways and the applications built upon each would be fundamentally different.  The third is that the library should be no more complex than absolutely necessary.  Doing so tends to expose too many assumptions about how the library will be used, and tends to confuse things without really adding much of use.

The problem is that druntime's core.thread is supposed to hide the native threading mechanism being used. With the example in your other email, the abstraction is completely broken, and portability becomes a *giant* pain (especially because compilers don't set any flags indicating what threading library the runtime is built for). So, I do believe kill() would be an important primitive to have, as unsafe as it is.

>
> What I've found is that when people thing of something useful for their program, if it's something regarding a library they tend to want to build that functionality into the library even if it doesn't need to be there.  The problem is that they may be the only person in the world that needs that functionality, and adding it to the library may increase maintenance cost to no one's actual benefit but that one user.  There have been a number of requests like this for Druntime, and some of the few that were accepted have just been deleted after having been deprecated for years, without a single complaint.  Assuming that people are actually using Druntime, I'll admit that I prefer to see it shrinking rather than growing.  It means that it's being distilled down to only containing the functionality that people actually care about.

What were some of those (out of curiosity)?

I agree that keeping druntime small is a good idea - but IMO that shouldn't mean that we can't have the abstractions that actually make druntime useful. Obviously, I'm biased here, since this is a feature I, specifically, want. But I'm not convinced that it would never be used by anyone else. D is a systems language intended to be a competitor to C and C++. A virtual machine is just one example of a system that would need the functionality to destructively kill threads, regardless of whether all hell breaks loose as a result.

>
> Regarding Druntime's "expert only" status… Druntime still follows the core tenet of D which is that the design should encourage people to do things the "right way" by making that approach inherently easy.  For example, the GC needs to suspend and scan threads so it can collect unused memory.  This functionality is absolutely needed.  But an important observation is that the GC only ever needs to suspend and scan all threads in bulk--never just one at a time.  So rather than exposing a Thread.suspend() routine as in Java, Druntime only provides a way to suspend all threads at once.  This makes it far more likely that the functionality will only be used as was intended--by the GC--rather than by users as a means of implementing synchronization.  And if you don't think people use suspend() in this way you need look no farther than Doug Lea's containers library.  He's clearly an expert and so Gets it Right, and he's admittedly doing it by necessity since Java doesn't have semaphores or whatever, but I think it's a fair point all the same.  What should have happened was for semaphores to be added as a separate API so Doug didn't have to cowboy it with suspend().

I should make it clear that I don't consider druntime "expert only". I just consider it a place where you should *seriously RTFM* before you use anything it exposes.

I agree that exposing a suspend primitive for individual threads is a very bad idea; all sorts of Bad Things (TM) could happen if the user starts interfering with individual suspend counts. Keep in mind that what I'm asking for will: 1) acquire the global thread lock on every call 2) kill the thread completely and remove it from the global thread list. This makes it much less error-prone than a simple suspend().

> _______________________________________________
> D-runtime mailing list
> D-runtime@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime

Regards,
Alex
_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime

That forces the GC to expose functionality for acquiring any relevant locks for something that really has nothing to do with memory allocation., and still only takes care of one potential deadlock, albeit a common one.  Critical sections would help as well (that's basically how pthread_cancel() works), but applying them to this case would be tricky.

On May 16, 2012, at 3:54 PM, Alex Rønne Petersen wrote:

> Again, this boils down to knowing what you're doing. You should NOT be killing a thread if there is any chance whatsoever that the thread could be doing GC allocation while you do it.
> 
> That said, I'm not convinced that this is a problem that we cannot work around; I think that if kill() would acquire the global thread lock, this problem could be prevented from happening.
> 
> Regards,
> Alex
> 
> On Thu, May 17, 2012 at 12:52 AM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>> On Wednesday, May 16, 2012 14:44:12 Sean Kelly wrote:
>>> If the killed thread even has a chance of allocating memory ir could be killed while inside the GC and lock all threads out of the GC forever.
>> 
>> Ouch.
>> 
>> - Jonathan M Davis
>> _______________________________________________
>> D-runtime mailing list
>> D-runtime@puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/d-runtime
> _______________________________________________
> D-runtime mailing list
> D-runtime@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/d-runtime

_______________________________________________
D-runtime mailing list
D-runtime@puremagic.com
http://lists.puremagic.com/mailman/listinfo/d-runtime