On Thu, 12 Jan 2017 20:02:38 -0800, Jonathan M Davis via Digitalmars-d-announce wrote:
> I don't see how it possibly could given how dynamic arrays work in D. It would have to have some sort of reference counting mechanism, which would likely be a nightmare with slicing

On that topic, D's arrays would play nicer with both refcounting *and* modern garbage collectors if they were structured as base, offset, length instead of start, length. You could put metadata just before the start of the array, including the reference count.

On Friday, January 13, 2017 05:33:07 Chris Wright via Digitalmars-d-announce wrote:
> On Thu, 12 Jan 2017 20:02:38 -0800, Jonathan M Davis via
>
> Digitalmars-d-announce wrote:
> > I don't see how it possibly could given how dynamic arrays work in D. It would have to have some sort of reference counting mechanism, which would likely be a nightmare with slicing
>
> On that topic, D's arrays would play nicer with both refcounting *and* modern garbage collectors if they were structured as base, offset, length instead of start, length. You could put metadata just before the start of the array, including the reference count.

It's actually really nice as well as performant for D's dynamic arrays to be the way that they are. Adding any kind of reference count into them would add overhead as well as not play nicely when you're doing stuff like slicing pointers or static arrays, which should definitely not be ref-counted. If you're willing to use the GC, they way that D's dynamic arrays work right now is fantastic. And even if you're not willing to use the GC, the way they work is great if you have other code managing their memory appropriately and just don't use the concatentation or appending operations. Plenty of code would not want any ref-counting to be going on when passing a dynamic array around.

Having a ref-counted object that's like an array in addition to D's current dynamic arrays would be fine and great for some programs, but I sure wouldn't want to lose what we have now.

- Jonathan M Davis

On Friday, 13 January 2017 at 05:33:07 UTC, Chris Wright wrote:
> On Thu, 12 Jan 2017 20:02:38 -0800, Jonathan M Davis via Digitalmars-d-announce wrote:
>> I don't see how it possibly could given how dynamic arrays work in D. It would have to have some sort of reference counting mechanism, which would likely be a nightmare with slicing
>
> On that topic, D's arrays would play nicer with both refcounting *and* modern garbage collectors if they were structured as base, offset, length instead of start, length. You could put metadata just before the start of the array, including the reference count.

I know that you mean for the general case, but the structure you describe is the same as `std.experimental.ndclice`s Slice!(1,T*): pointer, length, stride triplet. This would also work with higher dimensional Slices as well.

Combine that with prefix allocator ét voilá.

On Friday, 13 January 2017 at 05:33:07 UTC, Chris Wright wrote:
> On that topic, D's arrays would play nicer with both refcounting *and* modern garbage collectors if they were structured as base, offset, length instead of start, length.

That might be slower sometimes as slices wouldn't fit in two registers then.

> You could put metadata just before the start of the array, including the reference count.

Yes, but GC arrays already do that with GC metadata (alloc size) without having offset, so that technique could in theory be done with RC too. It's a bit mysterious how the base address is found, would be nice to have some clear docs on this to point to.

On Friday, 13 January 2017 at 12:53:16 UTC, Nick Treleaven wrote:
> On Friday, 13 January 2017 at 05:33:07 UTC, Chris Wright wrote:
>> On that topic, D's arrays would play nicer with both refcounting *and* modern garbage collectors if they were structured as base, offset, length instead of start, length.
>
> That might be slower sometimes as slices wouldn't fit in two registers then.

Although for custom data structures, offset & length could often share one register. Then the magic looking up of the base address can be avoided, saving time.

On Friday, 13 January 2017 at 04:02:38 UTC, Jonathan M Davis wrote:
> On Thursday, January 12, 2017 21:57:37 Andrew Browne via Digitalmars-d- announce wrote:
>> On Wednesday, 4 January 2017 at 19:22:33 UTC, Andrei Alexandrescu
>>
>> wrote:
>> > We release a brief Vision document summarizing the main goals we plan to pursue in the coming six months. This half we are focusing on three things: safety, lifetime management, and static introspection.
>> >
>> > https://wiki.dlang.org/Vision/2017H1
>> >
>> >
>> > Andrei
>>
>> Is there a design document for how D will achieve safety with
>> nogc?
>> How does D plan to prevent leaks, use-after-free, double-free
>> bugs when not using the GC?
>
> Part of the reason that we have the GC in D is because of the safety guarantees that you can have with a GC that you can't have with mechanisms like malloc and free. Some amount of @nogc code can be @safe, but some of it will never be able to be @safe. e.g. the very nature of malloc and free makes @safe impossible in the general case.

This is why I ask. Because the vision document says

"2. @nogc: Use of D without a garbage collector, most likely by using reference counting and related methods Unique/Weak references) for reclamation of resources. This task is made challenging by the safety requirement."

It sounds like @safe @nogc code is the goal. I know it is challenging, I'd like to know if there is a plan/design for how to achieve this.

> It's trivial for a piece of code to free something that's currently in use by other code. If they're constrained within a ref-counting system, then @safety becomes more possible, but even then, when you have to start worrying about stuff like weak references in order to get around circular reference problems, it gets dicey if not impossible to make it fully @safe.

Yep, circular references are a difficulty of ref-counting. Is there a plan/design for how to handle this?

> It might be possible to guarantee safety if you have a whole bunch of extra constraints like Rust does with its borrowing stuff, but we're not going to add something like that to D, because it's too complicated on top of everything else that we already have.

So that level of safety is just not a goal of D? If I want memory safe code without a GC, should I just use Rust?

> I fully expect that certain idioms will be in place to help maintain @safety in @nogc code, but to some extent, by abandoning the GC, you're abandoning @safety - or at least you're making a lot more of your code need to be @trusted, and you can rely less on the compiler to guarantee @safety for you. Taking the freeing of memory out of the hands of the programmer like happens with the GC is _huge_ in guaranteeing the memory safety of code.
>
>> Will @nogc also have first class support in the language?
>
> And what do you mean my first class support? Some features require the GC, and I wouldn't expect it to ever be otherwise. Giving up the GC means giving up on certain features. We don't want that list to be longer that it needs to be, but some stuff fundamentally needs the GC to do what it does.

That is what I meant. The next questions are examples of what I meant.

>> Afaik the GC is currently needed for language features like array concatenation. Will features like array concatentation still work with @nogc?
>
> I don't see how it possibly could given how dynamic arrays work in D. It would have to have some sort of reference counting mechanism, which would likely be a nightmare with slicing and certainly does not at all play well with how low level D arrays are. We may very well get some sort of ref-counted array type that has concatenation, but it would be a library construct rather than in the language, because it doesn't need to be in the language, and the built-in arrays would not be affected by it.
>
>> GC allocations have a keyword 'new' (afaik 'new' currently never means anything other than GC allocation). Will we be able to do @nogc allocations by the 'new' keyword?
>
> I very much doubt it. Constructing objects into memory is done via emplace, which is a library construct, and there's really no need for it to be in the language. As it is, if we were doing things from scratch, new probably wouldn't even be a keyword. It would likely be a library construct in druntime, because D is powerful enough that new doesn't need to be in the language to do what it does. And in general, at this point, Walter and Andrei don't want to put stuff in the language unless it actually needs to be there. If it can be done with a library, it will be done with a library. The only reason that they decided that we needed some sort of ref-counting mechanism in the language is because they decided that it wasn't actually possible to make it fully @safe without it being part of the language. And even then, I'm not sure that the intention is that the ref-counting mechanism use anything other than the GC. It's not yet clear what it's going to look like, but previously, the talk was using the GC to take care of circular references, which would mean that the memory was still GC-allocated even if it were ref-counted. We'll have to wait and see though.
>
>> Is the same code always expected to work with/without @nogc?
>
> That would depend entirely on the code. std.experimental.allocator has a GC allocator. So, code that is designed around it could work with the GC or without. But the whole mechanism of newing something up and then not worrying about ownership which happens by default with the GC doesn't play at all nicely with how memory has to be managed via other mechanisms like malloc and free. I don't think that it's at all reasonable to expect that code that is written with the idea that its memory will be managed by the GC will also work without the GC. Code that isn't managing memory directly shouldn't care - and that's a lot of code (especially once lazy ranges are in the mix) - but I think that it's pretty clear that there's plenty of code that simply can't just swap out its allocation mechanism and still work properly. In general, code needs to be written with that in mind for it to work, and even then, there are limits given how different various memory management mechanisms are.
>
> - Jonathan M Davis

Yep, I know there are a lot of things that become hard to make safe without a GC.
That is why, I'd like to know what is a plan/design for achieving @safe @nogc code? What is not planned to be supported?

On Wednesday, 4 January 2017 at 19:22:33 UTC, Andrei Alexandrescu wrote:
> We release a brief Vision document summarizing the main goals we plan to pursue in the coming six months. This half we are focusing on three things: safety, lifetime management, and static introspection.
>
> https://wiki.dlang.org/Vision/2017H1


I guess the 2017H1 document is quite final now. Why not turn that into a blog post? It could be official (by the leaders) or informal (interview with Andrei).