On Monday, 27 November 2017 at 10:13:41 UTC, codephantom wrote:
> But in a discussion about GC, some technical details might prove to be very useful to those of us following this discussion.

Precise scanning of pointers makes sense when you have many cachelines on the GC with no pointers in them. But if you mostly have pointers (a large graph or a tree) then it makes little difference.

You need to add a more extensive whole program type analysis where you prove that the GC memory heap isnt reachable from a type... I.e. "pointers reachable through class T can provably never point into the a GC heap in this specific program, so therefore we can ignore all pointers to T".

On Monday, 27 November 2017 at 07:03:01 UTC, Ola Fosheim Grostad wrote:
> On Monday, 27 November 2017 at 06:47:00 UTC, Dmitry Olshansky wrote:
>> Last time I check shared_ptr can be safely shared across threads, hence RC is takling synchronization and most likely atomics since locks won’t be any better.
>
> The controlblock can, but it is crazy to use shared_ptr for anything more than high level ownership. It is a general solution with weak pointers and extra indirection, not a typical RC implementation for datastructures.

Then watch Herb’s Sutter recent talk “Leak freedom by default”. Now THAT guy must be out of his mind :)

I have no idea what are your typical C++ programmers.

>
>>> In C++ sync is manual, which is the only efficient way to do
>>
>> ??? shared_ptr is nowhere manual.
>
> There is an upcoming atomic_shared_ptr, but it is not in the standard yet.

atomic_shared_pointer is nothing but what you seem to imply. It’s not manual sync for one.

>
>> My post is about particular primitive in C++ std, what could be done instead or in addition to is not important.
>
> Oh, but it is.

You keep spreading FUD on this forum, I’m still not sure of your reasons though.

>
> 1. D currently does not provide what you says it does.

RefCounted!T

>
> 2. Sane C++ programmers rarely use shared_ptr for more than exchsnging ownership (suitable for sharing things like bitmap textures).

Including across threads btw.

> There are plenty of other RC implementations for tracking memory.

Like what? In standard C++ all you’ve got is unique_ptr and shared_ptr (and weak but that besides the point). There is maybe a ton of reimplementations of said things.

On Monday, 27 November 2017 at 14:35:03 UTC, Dmitry Olshansky wrote:
> Then watch Herb’s Sutter recent talk “Leak freedom by default”. Now THAT guy must be out of his mind :)

He could be, I havent seen it... Shared_ptr isnt frequently used, it is a last resort,

> atomic_shared_pointer is nothing but what you seem to imply. It’s not manual sync for one.

Err... That was my point... Only assignment and reset is protected in shared_ptr, all other methods require manual sync.

> You keep spreading FUD on this forum, I’m still not sure of your reasons though.

And there we go ad hominem as usual... With no argument to back it up... Bye.

Btw, it would improve the discourse if people tried to distinguish between language constructs and library constructs...

On Monday, 27 November 2017 at 15:56:09 UTC, Ola Fosheim Grostad wrote:
> On Monday, 27 November 2017 at 14:35:03 UTC, Dmitry Olshansky wrote:
>> Then watch Herb’s Sutter recent talk “Leak freedom by default”. Now THAT guy must be out of his mind :)
>
> He could be, I havent seen it... Shared_ptr isnt frequently used, it is a last resort,

Ahhhaaahhha.
Really, shared_ptr is the most contagious primitive of modern C++.

To quote MS STL guy “I’m surprised we had no non-inteusive ref-counted ptr in std lib for so long”.
Going Native videos are full of questions on that.

>
>> atomic_shared_pointer is nothing but what you seem to imply. It’s not manual sync for one.
>
> Err... That was my point... Only assignment and reset is protected in shared_ptr, all other methods require manual sync.

For the benefit of others, let me destroy that:

- shared_ptr allows to share T with thread-safe ownership, ref-counts are accounted atomically (sharing copies of shared_ptr pointing to the same block). Copy of shared_ptr is thread safe and does the count.

- atomic_shared_prt would also allow one to initialize a shared variable (eg global) of type shared_ptr safely from multiple threads

The manual synchronization part comes in if you try to work with payload T itself. THAT is manual.

Since C++ doesn’t see a difference at type level of shared vs local, there is no thread-local variation of shared_ptr. It would be too unsafe even for those guys, contrary to what Ola responses imply.

>
>> You keep spreading FUD on this forum, I’m still not sure of your reasons though.
>
> And there we go ad hominem as usual... With no argument to back it up... Bye.

Well I can clearly see misinformation and describe it as such. See your point about atomic_shared_ptr.

On Monday, November 27, 2017 15:56:09 Ola Fosheim Grostad via Digitalmars-d wrote:
> On Monday, 27 November 2017 at 14:35:03 UTC, Dmitry Olshansky
>
> wrote:
> > Then watch Herb’s Sutter recent talk “Leak freedom by default”. Now THAT guy must be out of his mind :)
>
> He could be, I havent seen it... Shared_ptr isnt frequently used, it is a last resort,

I don't understand this. I would expect most modern C++ programs to be using shared_ptr as the default for most pointers and thus use it heavily. Any time that I've been able to use at least TR1 for C++, I've used it everywhere, and when I've been stuck with C++98, I've used a home-grown equivalent. shared_ptr is a _huge_ boon for avoiding memory problems.

- Jonathan M Davis

On Monday, 27 November 2017 at 17:16:50 UTC, Dmitry Olshansky wrote:
> Really, shared_ptr is the most contagious primitive of modern C++.

Not really. Unique_ptr is, though.

> To quote MS STL guy “I’m surprised we had no non-inteusive ref-counted ptr in std lib for so long”.
> Going Native videos are full of questions on that.

Yeah, they tend to pretend that C++ is a high level language, it makes for good talks.

Let me put it this way, C++ is an acceptable low level language, but it is a rather poor high level language.

So if you talk with people who use C++ as a high level language you probably see different usage  patterns.

>> Err... That was my point... Only assignment and reset is protected in shared_ptr, all other methods require manual sync.
>
> For the benefit of others, let me destroy that:

Oh well, looks like reset and assignment isn't required per the spec to be protected against races either. That doesn't weaken my point, the control block is protected but not the methods of shared_ptr.

I hardly ever use shared_ptr. In practical programming you usually need to exchange more than a single entity so you need manual sync anyway.


> - shared_ptr allows to share T with thread-safe ownership, ref-counts are accounted atomically (sharing copies of shared_ptr pointing to the same block). Copy of shared_ptr is thread safe and does the count.

Not sure if I follow that description. Shared shared_ptr's are not thread safe, a shared control block is. (Technically you share a pointer, not an object of type T, but that is a minor detail.)

shared_ptr is pointing to a shared control block that in turn has a pointer that points to the resource. This control block  contains two counters. These counters are incremented and decremented atomically.

If you access the same shared_ptr from two threads then you have a potential race condition per the spec.

> - atomic_shared_prt would also allow one to initialize a shared variable (eg global) of type shared_ptr safely from multiple threads

The main point is that the methods of atomic_shared_ptr are protected against races.

It is needed because you usually would have have ownership pointers embedded in a another shared object.

So having a protected control-block is not sufficient outside the trivial toy-program.

> The manual synchronization part comes in if you try to work with payload T itself. THAT is manual.

No, none of the methods on shared_ptr guarantees that races won't happen. And in general you usually want to access more than one unit in a critical section.

So in C++ you typically need manual sync.

There are no suitable language features to get around that in C++ and library features are inadequate in the general sense.

Go and Pony have language features that are helpful. C++ doesn't.

> Since C++ doesn’t see a difference at type level of shared vs local, there is no thread-local variation of shared_ptr. It would be too unsafe even for those guys, contrary to what Ola responses imply.

Sigh. I implied that shared_ptr on a single thread is mostly useless.

But this:

struct OurSharedCache {
   shared_ptr<T> something;
   shared_ptr<T> somethingelse
}

Not safe.


> Well I can clearly see misinformation and describe it as such. See your point about atomic_shared_ptr.

What about it?

«If multiple threads of execution access the same shared_ptr without synchronization and any of those accesses uses a non-const member function of shared_ptr then a data race will occur»

http://en.cppreference.com/w/cpp/memory/shared_ptr

«The class template atomic_shared_ptr provides thread-safe atomic pointer operations over a std::shared_ptr.»

http://en.cppreference.com/w/cpp/experimental/atomic_shared_ptr

On Monday, 27 November 2017 at 18:00:59 UTC, Jonathan M Davis wrote:
> I don't understand this. I would expect most modern C++ programs to be using shared_ptr as the default for most pointers and thus use it heavily.

You get this:

shared_ptr -> control_block -> object

Instead of this:

unique_ptr -> object
my_ref_ptr ->object


> shared_ptr is a _huge_ boon for avoiding memory problems.

Yes, if you don't mind the inefficiency, i.e. if you are doing high-level programming in C++.

On Monday, 27 November 2017 at 18:29:56 UTC, Ola Fosheim Grøstad wrote:
> On Monday, 27 November 2017 at 17:16:50 UTC, Dmitry Olshansky wrote:
>> Really, shared_ptr is the most contagious primitive of modern C++.
>
> Not really. Unique_ptr is, though.
>
>> To quote MS STL guy “I’m surprised we had no non-inteusive ref-counted ptr in std lib for so long”.
>> Going Native videos are full of questions on that.
>
> Yeah, they tend to pretend that C++ is a high level language, it makes for good talks.
>
> Let me put it this way, C++ is an acceptable low level language, but it is a rather poor high level language.
>
> So if you talk with people who use C++ as a high level language you probably see different usage  patterns.

I’ve seen a tech giant that works on uber high-performance things making heavy use of STL, and being fond of C++14 “high-level” features. I almost took their job offer, it is a nice place.

Comparatively I’ve worked at Google where none of the new goodies are in use. There are some great things in their “std”, but most have horrible interfaces and people routinelly discuss how having at least STL-like primitives would be a huge boon.

>>> Err... That was my point... Only assignment and reset is protected in shared_ptr, all other methods require manual sync.
>>
>> For the benefit of others, let me destroy that:
>
> Oh well, looks like reset and assignment isn't required per the spec to be protected against races either. That doesn't weaken my point, the control block is protected but not the methods of shared_ptr.
>
> I hardly ever use shared_ptr. In practical programming you

That must be why you seriously have no idea how people use it. Otherwise you’d know that nobody shares a reference to shared pointer but rather a copy. The whole point of smart pointer is to avoid naked references.

>> - shared_ptr allows to share T with thread-safe ownership, ref-counts are accounted atomically (sharing copies of shared_ptr pointing to the same block). Copy of shared_ptr is thread safe and does the count.
>
> Not sure if I follow that description. Shared shared_ptr's are not thread safe

They are if you use them as intended - value types, that pretend to be pointers.

> a shared control block is. (Technically you share a pointer, not an object of type T, but that is a minor detail.)

No, you actually share an object and the last one to decrement the ref will destroy it. Now that is actually thread-safe thanks to atomic counter.

> shared_ptr is pointing to a shared control block that in turn has a pointer that points to the resource. This control block  contains two counters. These counters are incremented and decremented atomically.

That is my original point, which you now violently agree with :)

> If you access the same shared_ptr from two threads then you have a potential race condition per the spec.

Just underlines that you don’t understand how it is supposed to be used.

>> - atomic_shared_prt would also allow one to initialize a shared variable (eg global) of type shared_ptr safely from multiple threads
>
> The main point is that the methods of atomic_shared_ptr are protected against races.
>
> It is needed because you usually would have have ownership pointers embedded in a another shared object.
>
> So having a protected control-block is not sufficient outside the trivial toy-program.

Sufficient for what? Smart pointers are about ownership not protecting from races.

I also don’t understand what are you trying to prove. My point was: C++ has to do atomic counting, because it has no concept of shared vs local.

All of things you’ve said just prove it or are talking about something I’m not discussing here.

>
>> The manual synchronization part comes in if you try to work with payload T itself. THAT is manual.
>
> No, none of the methods on shared_ptr guarantees that races won't happen.

Just as you quoted below - all const are, including copy & destructor.

>> Since C++ doesn’t see a difference at type level of shared vs local, there is no thread-local variation of shared_ptr. It would be too unsafe even for those guys, contrary to what Ola responses imply.
>
> Sigh. I implied that shared_ptr on a single thread is mostly useless.

You did imply it’s useless on single thread and not used in concurrency because you need manual sync. I’d argue you don’t need a sync to access shared_ptr, you’d need it for object it points to.

It still solves the ownership and deterministic destruction in the presense of concurrent shared_ptrs of the same object.

But again - you countered a different point all together, see above.

>
> But this:
>
> struct OurSharedCache {
>    shared_ptr<T> something;
>    shared_ptr<T> somethingelse
> }

Not that I talked about it. This has nothing to do with shared_ptr and/or things I stated originally.

>> Well I can clearly see misinformation and describe it as such. See your point about atomic_shared_ptr.
>
> What about it?
>
> «If multiple threads of execution access the same shared_ptr without synchronization and any of those accesses uses a non-const member function of shared_ptr then a data race will occur»

Copy & destruction is actually fine, which you seem to ignore. Also accessing const methods of payload is fine. New C++ implies const == thread safe btw, at least in all of STL.

On Monday, 27 November 2017 at 20:13:35 UTC, Dmitry Olshansky wrote:
> I’ve seen a tech giant that works on uber high-performance things making heavy use of STL, and being fond of C++14 “high-level” features.

Look, I am not against "high level" features, but shared_ptr is nothing like the thread local ref-counting you were talking about which was at least introduced by Walter and Andrei as a competing solution to tracking of borrowed pointers in Rust.


> That must be why you seriously have no idea how people use it. Otherwise you’d know that nobody shares a reference to shared pointer but rather a copy. The whole point of smart pointer is to avoid naked references.

I have an idea of how people use it… why all the ad hominem?

I just don't find it very useful. In practice I'm usually better off exchanging through a facade and having the exchanged resource wrapped up in a custom type.

It is more maintainable and easy to deal with in terms of correctness.


> They are if you use them as intended - value types, that pretend to be pointers.

Sure. Not that this is how they are defined to be used. It is a very limited use case.


>> a shared control block is. (Technically you share a pointer, not an object of type T, but that is a minor detail.)
>
> No, you actually share an object and the last one to decrement the ref will destroy it. Now that is actually thread-safe thanks to atomic counter.

Ok, sure, you share an ownership relation to an object, but it isn't general enough, which was what I tried to suggest. For instance, if you have a resource on a GPU with an integer ID that is also a resource, but shared_ptr does not work (unless you accept extra indirections).

So, I'd rather see an owned<T>, which also can handle non-pointed-to-objects. Then one might ask, why not just have "unique<owningptr<T>>" or "shared<owningptr<T>>" that also would work with "unique<Resource>" and "shared<Resource>"?

Then you can build generic ADTs with a generic notion of ownership .

The basic idea is that an owned resource that has to go through a pointer isn't really necessary or as generic a type as it could have been.


> That is my original point, which you now violently agree with :)

Ok, then I didn't get your original point.


> Just underlines that you don’t understand how it is supposed to be used.

I know how it can be used. That doesn't mean that it is as generally useful as it should be to be useful for what I want to do.


> I also don’t understand what are you trying to prove. My point was: C++ has to do atomic counting, because it has no concept of shared vs local.

That I disagree with. In C++ correct syncing is entirely dependent on the programmer. Even if atomic_shared_ptr provide some guarantees these guarantees will not hold if you depend on more than one entity. So there is no mechanism in C++ that provide something to ensure correctness when it comes to concurrency. I.e. you totally depend on the programmer's ability.

You can create a concept of D-style local/shared by just creating a wrapper type and use a lint-style verification. This is the route C++ is heading in general for various properties related to pointer types it seems. (e.g. wrap with borrowed, owned, nullable etc).

Which is also basically what D has, except shared is a builtin, but semantically I see no difference. To get a semantic difference that matter you need concurrency to be part of the language itself. If you cannot express propositions about concurrency in the language, then there is a very limited ability to do something interesting with it.


> You did imply it’s useless on single thread and not used in concurrency because you need manual sync. I’d argue you don’t need a sync to access shared_ptr, you’d need it for object it points to.

Listen, if you build a shared graph with shared_ptrs then you are in trouble. So you would need atomic_shared_ptrs. That's a perfectly reasonable use case.

You are presuming that the shared_ptr objects are thread local and that they are not embedded in other objects that are reachable by another thread.

Those a very limiting presumptions.


> It still solves the ownership and deterministic destruction in the presense of concurrent shared_ptrs of the same object.

But it doesn't solve correctness. You still rely on the programmer to get correctness. Which is a tall order when you use encapsulation.

That only works if you never embed shared_ptr in a class.


> Copy & destruction is actually fine, which you seem to ignore. Also accessing const methods of payload is fine. New C++ implies const == thread safe btw, at least in all of STL.

As far as I can see from the description at cppreference.com there is no guarantee of freedom for races for anything if both threads reference the same shared_ptr object (not the control block but the object pointing to the control block).

You can obviously transmit it to another thread using a mutex-like/messaging-like setup if that is what you mean.