On 2014-04-23 04:33:00 +0000, Walter Bright <newshound2@digitalmars.com> said:

> On 4/22/2014 12:42 PM, Michel Fortin wrote:
>> On 2014-04-22 19:02:05 +0000, Walter Bright <newshound2@digitalmars.com> said:
>> 
>>> Memory safety is not a strawman. It's a critical feature for a modern
>>> language, and will become ever more important.
>> 
>> What you don't seem to get is that ARC, by itself, is memory-safe.
> 
> I repeatedly said that it is not memory safe because you must employ escapes from it to get performance.

It wasn't that clear to me you were saying that, but now it makes sense.

In Objective-C, the performance-sensitive parts are going to be implemented in C, that's true. But that's rarely going to be more than 5% of your code, and probably only a few isolated parts where you're using preallocated memory blocks retained by ARC while you're playing with the content.

If you're writing something that can't tolerate a GC pause, then it makes perfect sense to make this performance-critical code unsafe so you can write the remaining 95% of your app in a memory-safe environment with no GC pause.

D on the other hand forces you to have those GC pauses or have no memory management at all. It's a different tradeoff and it isn't suitable everywhere, but I acknowledge it makes it easier to make performance-sensitive code @safe, something that'd be a shame to lose.


>> Objective-C isn't memory safe because it lets you play with raw pointers too. If
>> you limit yourself to ARC-managed pointers (and avoid undefined behaviours
>> inherited from C) everything is perfectly memory safe.
> 
> Allow me to make it clear that IF you never convert an ARC reference to a raw pointer in userland, I agree that it is memory safe. But this is not practical for high performance code.

Framing the problem this way makes it easier to find a solution.

I wonder, would it be acceptable if ARC was used everywhere by default but could easily be disabled inside performance-sensitive code by allowing the user choose between safe GC-based memory management or unsafe manual memory management? I have an idea that'd permit just that. Perhaps I should write a DIP about it.


>> I'm pretty confident that had I continued my work on D/Objective-C we'd now be
>> able to interact with Objective-C objects using ARC in @safe code. I was
>> planning for that. Objective-C actually isn't very far from memory safety now
>> that it has ARC, it just lacks the @safe attribute to enable compiler verification.
> 
> I wish you would continue that work!

I wish I had the time too.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Wed, 23 Apr 2014 05:14:44 -0400, Manu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

> On 23 April 2014 04:28, Steven Schveighoffer via Digitalmars-d
>> ARC does not equal guaranteed memory safety. So NO, it cannot replace the GC
>> for D @safe code. That doesn't make it useless.
>
> Why not? Assuming that direct access to the refcount is not @safe, why
> would ARC be unsafe? What makes it less safe than the GC?

Arguably, it is safe, as long as you only use ARC pointers. I don't know that I would ever want or use that in D (or even Objective-C). So it's not that it's not safe, it's that it cannot be a drop-in-replacement for the GC in existing D @safe code.

For example, you could never use slices or ranges, or these would have to be rewritten to keep references to the full object.

-Steve

On Wed, 23 Apr 2014 02:11:38 -0400, Jacob Carlborg <doob@me.com> wrote:

> On 22/04/14 20:48, Steven Schveighoffer wrote:
>
>> I mean not like I can't because I don't want to or don't have time, but
>> can't as in I lack the skill set :) It's interesting to debate, and I
>> get the concepts, but I am not a CPU/cache guy, and these things are
>> really important to get right for performance, since ref counting would
>> be used frequently.
>
> That's the worst kind of excuses :) I don't remember the last time I started working on a project and know what I was doing/had the right skill set. I mean, that's how you learn.

Sure, but there are things I CAN do with my limited time, that I do have the expertise for. I've already been schooled by the likes of you and Michel Fortin on my knowledge of ref counting implementation.

BTW, this is how RedBlackTree (dcollections) came into existence, I had no idea what I was doing, just the API that I wanted (and back then, I had more time). The code is actually a slightly-hand-optimized copy of my CLR book's red-black algorithm.

-Steve

On 2014-04-23 17:57, Steven Schveighoffer wrote:

> Sure, but there are things I CAN do with my limited time, that I do have
> the expertise for. I've already been schooled by the likes of you and
> Michel Fortin on my knowledge of ref counting implementation.

That's completely different. I've felt the same for a long time. Instead of working on the compiler I built tools and libraries for D. Then I finally couldn't keep my hands off and now I have D/Objective-C working for 64bit :)

-- 
/Jacob Carlborg

On 4/23/2014 6:10 AM, Jacob Carlborg wrote:
> That conversation started out from the D/Objective-C conversations. To have ARC
> in D and be compatible with the one in Objective-C you don't have many choices.
> I'm not sure but I don't think your proposal was not compatible with ARC in
> Objective-C.

Too many double negatives for me to be sure what you're saying. But it is clear to me that with Michel's experience with ARC in iOS combined with Manu's enthusiasm for it suggests that they are the right team to come up with a workable proposal, where mine failed.

On 23/04/14 19:12, Walter Bright wrote:

> Too many double negatives for me to be sure what you're saying. But it
> is clear to me that with Michel's experience with ARC in iOS combined
> with Manu's enthusiasm for it suggests that they are the right team to
> come up with a workable proposal, where mine failed.

Sorry, now that I read it out loud it is confusing. Here's another try:

You're proposal wasn't compatible with ARC in Objective-C.

I'm not sure if I remember correctly.

-- 
/Jacob Carlborg

Walter Bright:

> http://wiki.dlang.org/DIP60
>
> Start on implementation:
>
> https://github.com/D-Programming-Language/dmd/pull/3455

Currently this code doesn't compile because the lambda allocates the closure on the heap:

void main() @nogc {
    import std.algorithm: map;
    int[3] data = [1, 2, 3];
    immutable int x = 3;
    auto result = data[].map!(y => y * x);
}


test.d(1,6): Error: function D main @nogc function allocates a closure with the GC

Such kind of code is common, so a good amount of range-based code can't be @nogc.

-----------

In the meantime the good Kenji has created a patch for missing semantics:
https://github.com/D-Programming-Language/dmd/pull/3493

Bye,
bearophile

On 4/24/2014 6:35 AM, bearophile wrote:
> Currently this code doesn't compile because the lambda allocates the closure on
> the heap:

Pointing out these issues is exactly what @nogc is designed to do.

> void main() @nogc {
>      import std.algorithm: map;
>      int[3] data = [1, 2, 3];
>      immutable int x = 3;
>      auto result = data[].map!(y => y * x);
> }
>
>
> test.d(1,6): Error: function D main @nogc function allocates a closure with the GC
>
> Such kind of code is common,

True.

> so a good amount of range-based code can't be @nogc.

"Can't" is a bit strong of a word. Needing a workaround that is perhaps a bit ugly is more accurate. For your example,

    enum int x = 3;

will solve the issue.

Walter Bright:

> Pointing out these issues is exactly what @nogc is designed to do.

Right.


> "Can't" is a bit strong of a word. Needing a workaround that is perhaps a bit ugly is more accurate. For your example,
>
>     enum int x = 3;
>
> will solve the issue.

In most cases that "x" is a run-time value, as in my example.

Bye,
bearophile

On 4/24/2014 11:49 AM, bearophile wrote:
> Walter Bright:
>> "Can't" is a bit strong of a word. Needing a workaround that is perhaps a bit
>> ugly is more accurate. For your example,
>>
>>     enum int x = 3;
>>
>> will solve the issue.
>
> In most cases that "x" is a run-time value, as in my example.

You can make it a static and it'll work. Ugly, but it'll work.