On Wed, 09 Oct 2013 22:30:05 +0200, Adam D. Ruppe wrote:

> On Wednesday, 9 October 2013 at 20:10:40 UTC, Justin Whear wrote:
>> Related to the latter, it would be really nice to be able to prove that a section of code makes no heap allocations/GC collections.
> 
> As a quick temporary thing, how about gc_throw_on_next(); ?
> 
> That'd just set a thread local flag that gc_malloc checks and if it is set, immediately resets it and throws an AllocAssertError. My thought is this could be quickly and easily implemented pending a better solution and in the mean time can be used in unit tests to help check this stuff.

So user-code would look like this?

    // Set up code, GC is fine here
    ...

    // Entering critical loop (which may run for months at a time)
    debug GC.throw_on_next(true);
    while (true)
    {
       ...
    }

    // Tear-down code, GC is fine here
    // (though unnecessary as the process is about to exit)
    debug GC.throw_on_next(false);
    ...

Something like this would make testing simpler and is probably much more feasible than deep static analysis.

On 10/10/13 19:50, Sean Kelly wrote:
> Configuration data, for example.  On app start you might load a config file, generate information about the user, and so on, before real processing begins.  This data needs to be visible everywhere and it rarely if ever changes as the program runs, so you fill the data structures and then make them immutable.  Assuming, of course, that the data structures have immutable versions of all the necessary functions (which is unfortunately a pretty big assumption).

Yup, you're right, it's a big assumption.  In my case I was interested in loading a graph (network) and running many simulations on it in parallel.  The graph itself was static, so could readily be made immutable.  However, I found that it was difficult to write code that would accept both immutable and mutable graphs as input, without impacting performance.  So, I opted for threads to receive an immutable graph and cast it to mutable, even though it was never actually altered.

My experience was no doubt partially due to issues with the overall design I chose, and maybe I could have found a way around it, but it just seemed easier to use this flawed approach than to re-work everything.

On 10/10/13 19:46, Sean Kelly wrote:
> As for const / immutable, I guess I don't see this as such an issue because I've been dealing with it in C++ for so long.  You either have to commit 100% to using const attributes or not use them at all.  Anything in between is fraught with problems.

Well, the problem is essentially that you can have a function like:

    void foo(int i) { ... }

... and if you pass it an immutable or const int, this is not a problem, because you're passing by value.

But now try

    void foo(BigInt i) { ... }

... and it won't work when passed a const/immutable variable, even though again you're passing by value.  That's not nice, not intuitive, and generally speaking makes working with complex data types annoying.

It's why, for example, std.math.abs currently works with BigInt but not with const or immutable BigInt -- which is very irritating indeed.

On Oct 10, 2013, at 11:17 AM, Joseph Rushton Wakeling <joseph.wakeling@webdrake.net> wrote:

> On 10/10/13 19:50, Sean Kelly wrote:
>> Configuration data, for example.  On app start you might load a config file, generate information about the user, and so on, before real processing begins.  This data needs to be visible everywhere and it rarely if ever changes as the program runs, so you fill the data structures and then make them immutable.  Assuming, of course, that the data structures have immutable versions of all the necessary functions (which is unfortunately a pretty big assumption).
> 
> Yup, you're right, it's a big assumption.  In my case I was interested in loading a graph (network) and running many simulations on it in parallel.  The graph itself was static, so could readily be made immutable.  However, I found that it was difficult to write code that would accept both immutable and mutable graphs as input, without impacting performance.  So, I opted for threads to receive an immutable graph and cast it to mutable, even though it was never actually altered.
> 
> My experience was no doubt partially due to issues with the overall design I chose, and maybe I could have found a way around it, but it just seemed easier to use this flawed approach than to re-work everything.

That's kind of the issue I ran into with shared in Druntime.  It seemed like what I had to do was have a shared method that internally cast "this" to unshared and then called the real function, which I knew was safe but the type system hated.  But this seemed like a horrible approach and so I didn't ever qualify anything as shared.

On 2013-10-10 17:34:47 +0000, Sean Kelly <sean@invisibleduck.org> said:

> On Oct 10, 2013, at 4:17 AM, Michel Fortin <michel.fortin@michelf.ca>
> wrote:
> 
>> http://michelf.ca/blog/2012/mutex-synchonization-in-d/
> 
> Good article.  But why didn't you mention core.sync?  It has both a
> Mutex and a ReadWriteMutex (ie. shared_mutex).

Because that would have required a ton of explanations about why you need casts everywhere to remove shared, and I don't even know where to begin to explain shared semantics. Shared just doesn't make sense to me the way it works right now.

The examples in C++ are much clearer than anything I could have done in D2. I don't want to have to explain why I have to bypass the type system every time I need to access a variable. I'll add that I'm coding in C++ right now so it's much easier to come up with C++ examples.

That said, it might be a good idea to add a note at the end about core.sync in case someone wants to try that technique in D.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Thursday, 10 October 2013 at 17:23:20 UTC, Joseph Rushton Wakeling wrote:
> On 09/10/13 06:25, Andrei Alexandrescu wrote:
>> The way I see it we must devise a robust solution to that, NOT consider the
>> state of the art immutable (heh, a pun).
>
> Must say I have had a miserable experience with immutability and any kind of complex data structure, particularly when concurrency is involved.

I feel your pain. See also this thread in D.learn:

http://forum.dlang.org/post/sdefkajobwcfikkelxbr@forum.dlang.org

On 10/10/2013 10:54 AM, Andrei Alexandrescu wrote:
> On 10/10/13 12:18 AM, Walter Bright wrote:
>> On 10/9/2013 11:34 PM, Jacob Carlborg wrote:
>>> On 2013-10-10 02:22, Sean Kelly wrote:
>>>
>>>> Only that this would have to be communicated to the user, since
>>>> moving data
>>>> later is problematic. Today, I think it's common to construct an
>>>> object as
>>>> unshared and then cast it.
>>>
>>> What is the reason to not create it as "shared" in the first place?
>>
>> 1. Shared data cannot be passed to regular functions.
>
> I don't understand this. If a function/method accepts "shared", then it can be
> passed shared data.

I meant regular functions as in they are not typed as taking shared arguments. Shared cannot be implicitly cast to unshared. I say regular because very, very few functions are typed as accepting shared arguments.



>> 2. Functions that create data structures would have to know in advance
>> that they'll be creating a shared object. I'm not so sure this would not
>> be an invasive change.
>
> There is no other way around it. And this is not a change - it's fixing something.

I'm not convinced of that at all.


>> 3. Immutable data is implicitly shared. But it is not created immutable
>> - it is created as mutable data, then set to some state, then cast to
>> immutable.
>
> That all must happen in the runtime, NOT in user code.
>
>
> Andrei
>

On Oct 10, 2013, at 11:21 AM, Joseph Rushton Wakeling <joseph.wakeling@webdrake.net> wrote:

> On 10/10/13 19:46, Sean Kelly wrote:
>> As for const / immutable, I guess I don't see this as such an issue because I've been dealing with it in C++ for so long.  You either have to commit 100% to using const attributes or not use them at all.  Anything in between is fraught with problems.
> 
> Well, the problem is essentially that you can have a function like:
> 
>    void foo(int i) { ... }
> 
> ... and if you pass it an immutable or const int, this is not a problem, because you're passing by value.
> 
> But now try
> 
>    void foo(BigInt i) { ... }
> 
> ... and it won't work when passed a const/immutable variable, even though again you're passing by value.  That's not nice, not intuitive, and generally speaking makes working with complex data types annoying.
> 
> It's why, for example, std.math.abs currently works with BigInt but not with const or immutable BigInt -- which is very irritating indeed.


Isn't BigInt a struct?  I'd expect it to work via copying just like concrete types.

On Thu, Oct 10, 2013 at 07:36:06PM +0200, Joseph Rushton Wakeling wrote:
> On 10/10/13 19:31, Jonathan M Davis wrote:
> >I'm honestly surprised that Andrei is rejecting the idea of casting to/from shared or immutable being normal given how it's required by our current concurrency model. And changing that would be a _big_ change.
> 
> I'm starting to incline towards the view that type qualifications of
> _any_ kind become problematic once you start working with any types
> other than built-in, and not just in the context of concurrency.
> See e.g.:
> http://d.puremagic.com/issues/show_bug.cgi?id=11148
> http://d.puremagic.com/issues/show_bug.cgi?id=11188
> 
> I'd really appreciate advice on how to handle issues like these, because it's becoming a serious obstacle to my work on std.rational.

I left some comments on these bugs. Basically, BigInt should not be implicitly castable from const/immutable to unqual, because unlike the built-in types, it's *not* a value type:

	BigInt x = 123;
	BigInt y = x;  // creates an alias to x's data.

Allowing implicit conversion to unqual would break immutability:

	immutable(BigInt) x = 123;
	const(BigInt) sneaky = x; // sneaky aliases x
	BigInt y = sneaky; // now y aliases sneaky, which aliases x (oops)

Of course, the way BigInt is implemented, any operation on it causes new data to be created (essentially it behaves like a copy-on-write type), so it's not as though you can directly modify immutable this way, but it breaks the type system and opens up possible loopholes.

What you need to do is to use inout for functions that need to handle both built-in ints and BigInts, e.g.:

	inout(Num) abs(Num)(inout(Num) x) {
		return (x >= 0) ? x : -x;
	}

This *should* work (I think -- I didn't check :-P).

Arguably, a *lot* of generic code involving numerical operations is broken, because they assume built-in types' behaviour of being implicitly convertible to/from immutable (due to being value types).

I don't know about shared, though. Last I heard, shared was one big mess so I'm not even going to touch it.


T

-- 
If the comments and the code disagree, it's likely that *both* are wrong. -- Christopher

On 10/10/13 20:28, Sean Kelly wrote:
> Isn't BigInt a struct?  I'd expect it to work via copying just like concrete types.

Yes, it's a struct, but somewhere inside its internals I think it contains arrays.  I'm not sure how that affects copying etc., but suffice to say that if you try the following:

    BigInt a = 2;
    BigInt b = a;
    b = 3;
    assert(a != b);
    assert(a !is b);

... then it passes.  So it behaves at least in this extent like a value type.

But suffice to say that it was an unpleasant surprise that I couldn't just take it and pass to a function accepting an unqualified BigInt argument.