On Thursday, 10 October 2013 at 21:15:39 UTC, Sean Kelly wrote:
> On Thursday, 10 October 2013 at 20:50:10 UTC, Walter Bright wrote:
>>
>> I'm curious why NNTP would be blocked. I've been able to access it from any wifi hotspots I've tried it from.
>
> My only guess is that usenet may be perceived as a illegal file sharing resource.  But it's been a while since I've tried, so I'll give it another shot.

No luck.  I can get to the SSL port (563) on usenet hosts that offer it, but not port 119 from anywhere I care to check news from.

On Thursday, October 10, 2013 11:11:12 Sean Kelly wrote:
> On Oct 10, 2013, at 10:55 AM, Andrei Alexandrescu
<SeeWebsiteForEmail@erdani.org> wrote:
> > On 10/10/13 12:33 AM, Jonathan M Davis wrote:
> >> Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting.
> > 
> > no
> 
> Yeah, I'd want to see this claim backed up by some examples. The only data I share globally in my own apps is the occasional container. Configuration data, a user database, whatever. I'll also frequently move data between threads while dispatching tasks, but otherwise everything is thread-local. I imagine there are other reasonable methods for using shared data, but I don't know what they are.

Yeah, but it's that moving data between threads while dispatching tasks which requires casting. Pretty much anything which isn't a value type has to be cast to either immutable or shared in order to pass it across threads, and then it needs to then be cast back to thread-local mutable on the other side for to be useable. Unless you're only passing really basic stuff like int, or the types that your passing are only used for being passed across threads (and thus are designed to work as shared), you end up having to cast.

The fact that you can only pass shared or immutable objects across combined with the fact that shared objects are generally unusable makes it so that you're at minimum going to have to either cast the object once it gets to the other thread, even if it was constructed as shared. And since shared is so useless means that if you need to do anything more than simply construct the object before passing it across, you're going have to have it as thread-local in the originating thread as well.

I just don't see how you could avoid casting when passing ownership of an object from one thread to another without having a way to pass an object across threads without having to make it shared or immutable to pass it.

- Jonathan M Davis

On Thursday, October 10, 2013 11:28:02 Walter Bright wrote:
> On 10/10/2013 10:54 AM, Andrei Alexandrescu wrote:
> > On 10/10/13 12:18 AM, Walter Bright wrote:
> >> 1. Shared data cannot be passed to regular functions.
> > 
> > I don't understand this. If a function/method accepts "shared", then it can be passed shared data.
> 
> I meant regular functions as in they are not typed as taking shared arguments. Shared cannot be implicitly cast to unshared. I say regular because very, very few functions are typed as accepting shared arguments.

Yeah. The only times that something is going to accept shared is when it was specifically designed to work as shared (which most code isn't), or if it's templated and the template happens to work with shared. Regular functions just aren't going to work with shared without casting away shared, because that would usually mean either templating everything or duplicating functions all over the place.

- Jonathan M Davis

On Thursday, 10 October 2013 at 23:33:17 UTC, Jonathan M Davis wrote:
>
> I just don't see how you could avoid casting when passing ownership of an
> object from one thread to another without having a way to pass an object
> across threads without having to make it shared or immutable to pass it.

Well, the restriction to only pass immutable and shared data is simply enforced statically by the API.  So if there were an assumeUnique analog, the check could be modified to accept that as well, and then the class would arrive as unshared.  This could be accomplished pretty easily.  It would be yet another step towards not having thread-local pools though.  I was initially pretty conservative in what was an acceptable type to send, because it's always easier to loosen restrictions than tighten them.

On Thursday, 10 October 2013 at 23:33:27 UTC, Jonathan M Davis wrote:
>
> Yeah. The only times that something is going to accept shared is when it was
> specifically designed to work as shared (which most code isn't), or if it's
> templated and the template happens to work with shared. Regular functions just
> aren't going to work with shared without casting away shared, because that
> would usually mean either templating everything or duplicating functions all over the place.

I think that's pretty reasonable though.  Shared data needs to be treated differently, explicitly, or things go downhill fast.

On 2013-10-10, 20:28, H. S. Teoh wrote:

> On Thu, Oct 10, 2013 at 07:36:06PM +0200, Joseph Rushton Wakeling wrote:
>> On 10/10/13 19:31, Jonathan M Davis wrote:
>> >I'm honestly surprised that Andrei is rejecting the idea of casting
>> >to/from shared or immutable being normal given how it's required by
>> >our current concurrency model. And changing that would be a _big_
>> >change.
>>
>> I'm starting to incline towards the view that type qualifications of
>> _any_ kind become problematic once you start working with any types
>> other than built-in, and not just in the context of concurrency.
>> See e.g.:
>> http://d.puremagic.com/issues/show_bug.cgi?id=11148
>> http://d.puremagic.com/issues/show_bug.cgi?id=11188
>>
>> I'd really appreciate advice on how to handle issues like these,
>> because it's becoming a serious obstacle to my work on std.rational.
>
> I left some comments on these bugs. Basically, BigInt should not be
> implicitly castable from const/immutable to unqual, because unlike the
> built-in types, it's *not* a value type:
[snip]
> What you need to do is to use inout for functions that need to handle
> both built-in ints and BigInts, e.g.:
[snip]

Here's a COW reference type that I can easily pass to a function
requiring a mutable version of the type:

  struct S {
    immutable(int)[] arr;
  }

And usage:

  void foo(S s) {}

  void main() {
    const S s;
    foo(s);
  }


This compiles and works beautifully. Of course, no actual COW is
happening here, but COW is what the type system says has to happen.
Another example COW type:

  string;

Now, my point here is that BigInt could easily use an immutable
buffer internally, as long as it's purely COW. It could, and it should.
If it did, we would not be having this discussion, as bugs #11148 and
#11188 would not exist. Inventing rules like 'you should use inout'
does not help - it's obscuring the problem.

TLDR: Do not use inout(T). Fix BigInt.

-- 
  Simen

On Thursday, October 10, 2013 10:55:49 Andrei Alexandrescu wrote:
> On 10/10/13 12:33 AM, Jonathan M Davis wrote:
> > I honestly don't think we can solve it a different way without completely redesigning shared. shared is specifically designed such that you have to either cast it way to do anything with it
> 
> no
> 
> > or write all of your code to
> > explicitly work with shared, which is not something that generally makes
> > sense to do unless you're creating a type whose only value is in being
> > shared across threads.
> 
> yes

Really? Do you honestly expect the average use of shared to involve creating structs or classes which are designed specifically to be used as shared? That definitely has its use, but I would expect it to be far more common that someone would want to share the exact same types that they're using in their thread-local code. In fact, if anything, the normal responses to discussions on shared go in the complete opposite direction of creating classes which are designed to work as shared. It seems like the normal thing to do is simply avoid shared altogether and use __gshared so that you don't have to deal with any of the problems that shared causes. Granted, I obviously haven't seen everyone's code, but I don't believe that I have ever seen anyone create a type designed to be used as shared, and that's certainly not what people discuss doing when shared comes up. TDPL discusses that - and again, I do think that that has its place - but I've never seen it done, and I've never run into any place in my own code where I would have even considered it. Usually, you want to share an object of the same type that you're using in your thread-local code.

And even if a struct or class is set up so that its member functions work great as shared, very little code seems to be written with shared in mind (since thread-local is the default), so the only functions which will work with it are its member functions, functions written specifically to work with that type, and templated functions that happen to work with shared. As such, I fully expect casting away shared to be a very common idiom. Without that, the number of things you can do with a shared object is very limited.

> > Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting.
> 
> no

What else do you expect to be doing with std.concurrency? That's what it's _for_. Unless all of the stuff that you're passing across threads are value types or are designed to work as immutable or shared (which most types aren't), the objects which get passed across need to be cast to thread-local mutable on the target thread in order to be used there, and if you have to do much of anything with the object other than constructing it before passing it across, then you're going to have to have it as thread-local on the originating thread as well, because most functions are going to be unusable with shared.

- Jonathan M Davis

On 10/10/13 10:36 AM, Joseph Rushton Wakeling wrote:
> On 10/10/13 19:31, Jonathan M Davis wrote:
>> I'm honestly surprised that Andrei is rejecting the idea of casting
>> to/from
>> shared or immutable being normal given how it's required by our current
>> concurrency model. And changing that would be a _big_ change.
>
> I'm starting to incline towards the view that type qualifications of
> _any_ kind become problematic once you start working with any types
> other than built-in, and not just in the context of concurrency.  See e.g.:
> http://d.puremagic.com/issues/show_bug.cgi?id=11148
> http://d.puremagic.com/issues/show_bug.cgi?id=11188
>
> I'd really appreciate advice on how to handle issues like these, because
> it's becoming a serious obstacle to my work on std.rational.

I'll look into this soon.

Andrei

On Friday, 11 October 2013 at 00:30:35 UTC, Simen Kjaeraas wrote:
> Here's a COW reference type that I can easily pass to a function
> requiring a mutable version of the type:
>
>   struct S {
>     immutable(int)[] arr;
>   }
>
> And usage:
>
>   void foo(S s) {}
>
>   void main() {
>     const S s;
>     foo(s);
>   }
>
>
> This compiles and works beautifully. Of course, no actual COW is
> happening here, but COW is what the type system says has to happen.
> Another example COW type:
>
>   string;
>
> Now, my point here is that BigInt could easily use an immutable
> buffer internally, as long as it's purely COW. It could, and it should.
> If it did, we would not be having this discussion, as bugs #11148 and
> #11188 would not exist. Inventing rules like 'you should use inout'
> does not help - it's obscuring the problem.
>
> TLDR: Do not use inout(T). Fix BigInt.

Good catch. immutable(T)[] is special.

Do the same with a contained associative array and you'll be my hero.

On Friday, October 11, 2013 02:05:19 Sean Kelly wrote:
> It would be yet another step
> towards not having thread-local pools though.

At this point, I don't see how we can have thread-local pools unless casting to and from shared has hooks for managing that. Otherwise, it's far too likely that an object is going to be in the wrong pool, because it's being used as shared when it was constructed as thread-local or vice versa. And we may need some sort of hook with std.concurrency.send which understands that the object being sent is being transferred from one thread to another and would tell the GC to migrate the object from one pool to another (though to do that, it would probably have to not be typed as shared but rather as thread-local, which would jive better with what you're talking about doing with std.concurrency).

Certainly, with how shared currently works, it's hard to see how we could get away with having thread-local GC pools as great as that would be. So, if we want that, something about how shared works is going to have to change.

- Jonathan M Davis