On 10 October 2013 03:13, Michel Fortin <michel.fortin@michelf.ca> wrote:

> On 2013-10-09 16:51:03 +0000, Manu <turkeyman@gmail.com> said:
>
>  On 10 October 2013 01:40, Johannes Pfau <nospam@example.com> wrote:
>>
>>  But if someone really wants to strip the GC _completely_ there's a huge
>>> issue with memory management of Exceptions.
>>>
>>
>> Exceptions have a pretty well defined lifetime... can't they be manually cleaned up by the exception handler after the catching scope exits?
>>
>
> Exceptions don't need a well-defined lifetime for things to work.
>
> D exceptions are classes and are heap-allocated. So if everything becomes reference-counted, exceptions would be reference-counted too. The exception handler would be the one decrementing the reference count once it has done with the exception (all this under the hood, managed by the compiler).
>
> Alternatively an exception handler could return the exception to the parent function (as a return value), store the exception elsewhere, or throw it again, in which case the decrement operation would be balanced by an increment, and both increment and decrement should be elided by the compiler as they're cancelling each other.
>
> I fail to see an issue.


I was talking about using exceptions without any sort of GC at all.
I think it's the only critical language feature left that relies on a GC in
some form.
If would be nice if at least the fundamental language concepts were usable
without a GC of any sort.
Most other things can be reasonably worked around at this point.

On Wednesday, October 09, 2013 20:04:50 Jacob Carlborg wrote:
> On 2013-10-09 19:10, Andrei Alexandrescu wrote:
> > Again: USER CODE SHOULD HAVE NO BUSINESS CASTING CASUALLY TO GET WORK DONE.
> 
> I agree. Doesn't "new shared(X)", or something like that, already work?

That depends. It works with objects I think (for both shared and immutable), but you definitely have to cast to immutable if you want an immutable array or AA.

Also, casting _away_ shared is going to be a very common operation due to how shared works. In order to use shared, you basically have to protect the variable with a mutex or synchronized block, cast away shared, and then use it as thread-local for whatever you're doing until you release the lock (in which case, you have to be sure that there are no more thread-local references to the shared object). As such, while it might be possible to construct stuff directly as shared, it's going to have to be cast to thread-local just to use it in any meaningful way. So, at this point, I don't think that it even vaguely flies to try and make it so that casting away shared is something that isn't typically done. It's going to be done about as often as shared is used for anything other than very basic stuff.

As things stand, I don't think that it's even vaguely tenable to claim that casting should be abnormal with regards to immutable and shared. There will definitely have to be language changes if we want casting to be abnormal, and I think that the two main areas which would have to change would be the initialization of immutable arrays and AAs (since that outright requires casting at this point), and shared would probably have to have a major redesign, because it's outright unusable without casting it away.

- Jonathan M Davis

On Wednesday, October 09, 2013 20:55:07 Jonathan M Davis wrote:
> On Wednesday, October 09, 2013 20:04:50 Jacob Carlborg wrote:
> > On 2013-10-09 19:10, Andrei Alexandrescu wrote:
> > > Again: USER CODE SHOULD HAVE NO BUSINESS CASTING CASUALLY TO GET WORK DONE.
> > 
> > I agree. Doesn't "new shared(X)", or something like that, already work?
> 
> That depends. It works with objects I think (for both shared and immutable), but you definitely have to cast to immutable if you want an immutable array or AA.
> 
> Also, casting _away_ shared is going to be a very common operation due to how shared works. In order to use shared, you basically have to protect the variable with a mutex or synchronized block, cast away shared, and then use it as thread-local for whatever you're doing until you release the lock (in which case, you have to be sure that there are no more thread-local references to the shared object). As such, while it might be possible to construct stuff directly as shared, it's going to have to be cast to thread-local just to use it in any meaningful way. So, at this point, I don't think that it even vaguely flies to try and make it so that casting away shared is something that isn't typically done. It's going to be done about as often as shared is used for anything other than very basic stuff.
> 
> As things stand, I don't think that it's even vaguely tenable to claim that casting should be abnormal with regards to immutable and shared. There will definitely have to be language changes if we want casting to be abnormal, and I think that the two main areas which would have to change would be the initialization of immutable arrays and AAs (since that outright requires casting at this point), and shared would probably have to have a major redesign, because it's outright unusable without casting it away.

And given that std.concurrency requires casting to and from shared or immutable in order to pass objects across threads, it seems ilke most of D's concurrency model requires casting to and/or from shared or immutable. The major exception is structs or classes which are shared or synchronized rather than a normal object which is used as shared, and I suspect that that's done fairly rarely at this point. In fact, it seems like the most common solution is to ignore shared altogether and use __gshared, which is far worse than casting to and from shared IMHO.

So, it's my impression that being able to consider casting to or from shared as abnormal in code which uses shared is a bit of a pipe dream at this point. The current language design pretty much requires casting when doing much of anything with concurrency.

- Jonathan M Davis

On Thursday, 10 October 2013 at 03:06:43 UTC, Manu wrote:
> I was talking about using exceptions without any sort of GC at all.
> I think it's the only critical language feature left that relies on a GC in
> some form.
> If would be nice if at least the fundamental language concepts were usable
> without a GC of any sort.
> Most other things can be reasonably worked around at this point.

You can free your exception when done with it. It doesn't seem very complicated.

Again from the other thread. The are a few problems with mangling the type;
It breaks when you need to interact with libraries.
It's incompatible with struct alignment, and changes the struct size. These
are very carefully managed properties of structures.
It obscures/complicates generic code.
It doesn't deal with circular references, which people keep bringing up as
a very important problem.

What happens when a library receives a T* arg? Micro managing the ref-count
at library boundaries sounds like a lot more trouble than manual memory
management.
On 10 Oct 2013 05:20, "Walter Bright" <newshound2@digitalmars.com> wrote:

> On 10/9/2013 7:30 AM, Manu wrote:
>
>> ARC. I've been here years now, and I see absolutely no evidence that the
>> GC is
>> ever going to improve. I can trust ARC, it's predictable, I can control
>> it.
>> Also, proper support for avoiding the GC without severe inconvenience as
>> constantly keeps coming up. But I don't think there's any debate on that
>> one.
>> Everyone seems to agree.
>>
>
> I think we can get pretty close to ARC by using RefCounted, but you've indicated you disagree.
>
> (Note that ObjC does not have a template system, and so using a library system was not possible for ObjC.)
>

On 2013-10-10 02:22, Sean Kelly wrote:

> Only that this would have to be communicated to the user, since moving data later is problematic. Today, I think it's common to construct an object as unshared and then cast it.

What is the reason to not create it as "shared" in the first place?

-- 
/Jacob Carlborg

On 2013-10-10 05:55, Jonathan M Davis wrote:

> That depends. It works with objects I think (for both shared and immutable),
> but you definitely have to cast to immutable if you want an immutable array or
> AA.
>
> Also, casting _away_ shared is going to be a very common operation due to how
> shared works. In order to use shared, you basically have to protect the
> variable with a mutex or synchronized block, cast away shared, and then use it
> as thread-local for whatever you're doing until you release the lock (in which
> case, you have to be sure that there are no more thread-local references to
> the shared object). As such, while it might be possible to construct stuff
> directly as shared, it's going to have to be cast to thread-local just to use
> it in any meaningful way. So, at this point, I don't think that it even
> vaguely flies to try and make it so that casting away shared is something that
> isn't typically done. It's going to be done about as often as shared is used
> for anything other than very basic stuff.

What's the reason for casting away "shared", is it to pass it to a function that doesn't accept "shared"? The it should be safe if you synchronize around the call? But that function could put away, the now , unshared data, which is actually shared and cause problem?


-- 
/Jacob Carlborg

On 2013-10-10 06:24, Jonathan M Davis wrote:

> And given that std.concurrency requires casting to and from shared or
> immutable in order to pass objects across threads, it seems ilke most of D's
> concurrency model requires casting to and/or from shared or immutable. The
> major exception is structs or classes which are shared or synchronized rather
> than a normal object which is used as shared, and I suspect that that's done
> fairly rarely at this point. In fact, it seems like the most common solution
> is to ignore shared altogether and use __gshared, which is far worse than
> casting to and from shared IMHO.

Isn't the whole point of std.concurrency that is should only accept "shared" for reference types? If you want to use std.concurrency create a "shared" object in the first place?

> So, it's my impression that being able to consider casting to or from shared
> as abnormal in code which uses shared is a bit of a pipe dream at this point.
> The current language design pretty much requires casting when doing much of
> anything with concurrency.

There must be a better way to solve this.

-- 
/Jacob Carlborg

On 10/9/2013 11:34 PM, Jacob Carlborg wrote:
> On 2013-10-10 02:22, Sean Kelly wrote:
>
>> Only that this would have to be communicated to the user, since moving data
>> later is problematic. Today, I think it's common to construct an object as
>> unshared and then cast it.
>
> What is the reason to not create it as "shared" in the first place?

1. Shared data cannot be passed to regular functions.

2. Functions that create data structures would have to know in advance that they'll be creating a shared object. I'm not so sure this would not be an invasive change.

3. Immutable data is implicitly shared. But it is not created immutable - it is created as mutable data, then set to some state, then cast to immutable.

On Thursday, October 10, 2013 08:38:31 Jacob Carlborg wrote:
> On 2013-10-10 05:55, Jonathan M Davis wrote:
> > That depends. It works with objects I think (for both shared and immutable), but you definitely have to cast to immutable if you want an immutable array or AA.
> > 
> > Also, casting _away_ shared is going to be a very common operation due to how shared works. In order to use shared, you basically have to protect the variable with a mutex or synchronized block, cast away shared, and then use it as thread-local for whatever you're doing until you release the lock (in which case, you have to be sure that there are no more thread-local references to the shared object). As such, while it might be possible to construct stuff directly as shared, it's going to have to be cast to thread-local just to use it in any meaningful way. So, at this point, I don't think that it even vaguely flies to try and make it so that casting away shared is something that isn't typically done. It's going to be done about as often as shared is used for anything other than very basic stuff.
> 
> What's the reason for casting away "shared", is it to pass it to a function that doesn't accept "shared"? The it should be safe if you synchronize around the call? But that function could put away, the now , unshared data, which is actually shared and cause problem?

Pretty much nothing accepts shared. At best, templated functions accept shared. Certainly, shared doesn't work at all with classes and structs unless the type is specifically intended to be used as shared, because you have to mark all of its member functions shared to be able to call them. And if you want to use that class or struct as both shared and unshared, you have to duplicate all of its member functions.

That being the case, the only way in general to use a shared object is to protect it with a lock, cast it to thread-local (so that it can actually use its member functions or be passed to other functions to be used), and then use it. e.g.

synchronized
{
     auto tl = cast(T)mySharedT;
     auto result = tl.foo();
     auto result2 = bar(tl);
}

Obviously, you then have to make sure that there are no thread-local references to the shared object when the lock is released, but without casting away shared like that, you can't do much of anything with it. So, similar to when you cast away const, it's up to you to guarantee that the code doesn't violate the type system's guarantees - i.e. that a thread-local variable is not accessed by multiple threads. So, you use a lock of some kind to protect the shared variable while it's treated as a thread-local variable in order to ensure that that guarantee holds. Like with casting away const or with @trusted, there's obviously risk in doing this, but there's really no other way to use shared at this point - certainly not without it being incredibly invasive to your code and forcing code duplication.

- Jonathan M Davis