On Wednesday, 24 September 2014 at 14:58:34 UTC, Andrei Alexandrescu wrote:
> Thanks for this work! -- Andrei

BTW: If I want to construct my network once and destroy it all in one pass, I should probably use a region-based allocator from std.allocator to allocate the strings that are larger than maxSmall. Is the template RCXString parameter realloc sufficient for my needs here?

On Monday, 15 September 2014 at 02:26:19 UTC, Andrei Alexandrescu wrote:
> So, please fire away. I'd appreciate it if you used RCString in lieu of string and note the differences. The closer we get to parity in semantics, the better.

Further,

import std.container: Array;
import rcstring;

unittest
{
    Array!RCString x;
}

fails as

/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/conv.d(4010,17): Error: expression hasElaborateAssign!(RCXString!(immutable(char), 23LU, realloc)) of type void does not have a boolean value
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/conv.d(3970,31): Error: template instance std.conv.emplaceInitializer!(RCXString!(immutable(char), 23LU, realloc)) error instantiating
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/conv.d(4064,18):        instantiated from here: emplaceImpl!(string)
/home/per/Work/justd/rcstring.d(428,16):        instantiated from here: emplace!(RCXString!(immutable(char), 23LU, realloc), string)
/home/per/Work/justd/rcstring.d(13,18):        instantiated from here: RCXString!(immutable(char), 23LU, realloc)
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/algorithm.d(1577,16): Error: template instance std.traits.hasElaborateAssign!(RCXString!(immutable(char), 23LU, realloc)) error instantiating
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/container/array.d(85,26):        instantiated from here: initializeAll!(RCXString!(immutable(char), 23LU, realloc)[])
t_rcstring_array.d(8,5):        instantiated from here: Array!(RCXString!(immutable(char), 23LU, realloc))
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/container/array.d(276,24): Error: template instance std.algorithm.move!(RCXString!(immutable(char), 23LU, realloc)) error instantiating
t_rcstring_array.d(8,5):        instantiated from here: Array!(RCXString!(immutable(char), 23LU, realloc))
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/conv.d(4064,18): Error: template instance std.conv.emplaceImpl!(RCXString!(immutable(char), 23LU, realloc)).emplaceImpl!(RCXString!(immutable(char), 23LU, realloc)) error instantiating
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/container/array.d(186,20):        instantiated from here: emplace!(RCXString!(immutable(char), 23LU, realloc), RCXString!(immutable(char), 23LU, realloc))
/home/per/opt/x86_64-unknown-linux-gnu/dmd/linux/bin64/src/phobos/std/container/array.d(356,21):        instantiated from here: __ctor!(RCXString!(immutable(char), 23LU, realloc))
t_rcstring_array.d(8,5):        instantiated from here: Array!(RCXString!(immutable(char), 23LU, realloc))

Any clue what's missing in RCXString?

On 25 September 2014 00:55, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 9/24/14, 3:31 AM, Dmitry Olshansky wrote:
>>
>> 23-Sep-2014 19:13, Andrei Alexandrescu пишет:
>>>
>>> On 9/23/14, 12:17 AM, Dmitry Olshansky wrote:
>>>>
>>>> In my imagination it would be along the lines of
>>>> @ARC
>>>> struct MyCountedStuff{ void opInc(); void opDec(); }
>>>
>>>
>>> So that would be a pointer type or a value type? Is there copy on write somewhere? -- Andrei
>>
>>
>> It would be an intrusively counted type with pointer somewhere in the body. To put it simply MyCountedStuff is a kind of smart pointer.
>
>
> Then that would be confusing seeing as structs are value types. What you're saying is that a struct with opInc() and opDec() has pointer semantics whereas one with not has value semantics. That design isn't going to fly.

I think the way I imagine refcounting is the opposite of what you're saying here.

RC would be a type of pointer. An RC struct pointer can be handled
implicitly without problem.
That said, it's potentially very useful to support RC value types too,
and the way to do that would be for the struct to have opInc/opDec. In
that event, the compiler can generate calls to those operators on
assignment.

Something like (whatever syntax you like):

int^ rcInt; // refcounted pointer to an int
MyStruct^ rcStruct; // refcounted pointer to a struct
MyStruct s; // normal value-type struct, but if the struct has
opInc/opDec, the RC handling code in the compiler can implicitly
generate calls to opInc/opDec on assignment, which will allow the
struct to manage itself.

Not sure how to express an RC dynamic array... int[^] rcArray? Anyway, syntax is whatever it is, I think this approach is what makes sense to me though.

On 9/24/14, 12:49 PM, "Nordlöw" wrote:
> On Wednesday, 24 September 2014 at 14:58:34 UTC, Andrei Alexandrescu wrote:
>>> I didn't test speed.
>
> So the pro must be (de)allocation speed then, I suppose?

The pro is twofold:

1. Code using RC will be more compact about using memory if strings are created and then discarded. Depending on a variety of factors, that may lead to better cache friendliness.

2. If a GC cycle will occur, that will add to the total run time of GC code, meaning RC code will gain an advantage.

Code that doesn't create/discard a lot of strings and doesn't get to run the GC is likely to be slower with RCString.


Andrei

On Wednesday, 24 September 2014 at 20:22:05 UTC, Andrei Alexandrescu wrote:
> 1. Code using RC will be more compact about using memory if strings are created and then discarded. Depending on a variety of factors, that may lead to better cache friendliness.
>
> 2. If a GC cycle will occur, that will add to the total run time of GC code, meaning RC code will gain an advantage.
>
> Code that doesn't create/discard a lot of strings and doesn't get to run the GC is likely to be slower with RCString.

Ok, thanks.

On 9/24/14, 1:15 PM, Manu via Digitalmars-d wrote:
> Something like (whatever syntax you like):
>
> int^ rcInt; // refcounted pointer to an int
> MyStruct^ rcStruct; // refcounted pointer to a struct
> MyStruct s; // normal value-type struct, but if the struct has
> opInc/opDec, the RC handling code in the compiler can implicitly
> generate calls to opInc/opDec on assignment, which will allow the
> struct to manage itself.

You're getting confused here, postblit and destructor take care of that.

> Not sure how to express an RC dynamic array... int[^] rcArray? Anyway,
> syntax is whatever it is, I think this approach is what makes sense to
> me though.

Whatever syntax I like? Awesome! How about:

RefCounted!int rcInt; // refcounted pointer to an int
RefCounted!MyStruct rcStruct; // refcounted pointer to a struct
RefCounted!(int[]) rcArray; // refcounted array

The irony is the first two already work with the semantics you need, but apparently I've had difficulty convincing you to try them and report back. My work on RCString has also gone ignored by you, although it's exactly stuff that you're asking for.


Andrei

On 25 September 2014 07:17, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 9/24/14, 1:15 PM, Manu via Digitalmars-d wrote:
>>
>> Something like (whatever syntax you like):
>>
>> int^ rcInt; // refcounted pointer to an int
>> MyStruct^ rcStruct; // refcounted pointer to a struct
>> MyStruct s; // normal value-type struct, but if the struct has
>> opInc/opDec, the RC handling code in the compiler can implicitly
>> generate calls to opInc/opDec on assignment, which will allow the
>> struct to manage itself.
>
>
> You're getting confused here, postblit and destructor take care of that.

No they don't. It's not a ref counting mechanism, the compiler can't
elide those calls.
It's what I use now, and it's as good at C++, but we can do much
better than that.


>> Not sure how to express an RC dynamic array... int[^] rcArray? Anyway, syntax is whatever it is, I think this approach is what makes sense to me though.
>
>
> Whatever syntax I like? Awesome! How about:
>
> RefCounted!int rcInt; // refcounted pointer to an int RefCounted!MyStruct rcStruct; // refcounted pointer to a struct RefCounted!(int[]) rcArray; // refcounted array
>
> The irony is the first two already work with the semantics you need, but apparently I've had difficulty convincing you to try them and report back. My work on RCString has also gone ignored by you, although it's exactly stuff that you're asking for.

It all feels backwards to me. You've completely alienated me from the discussion. I've given up, but I am watching with interest.

I'm not interested in a library solution until I know if, and how the
compiler will optimise it... in which case it's not a library solution
anymore, so why make the implementation a lib?
@nogc users will use this stuff EXCLUSIVELY. There is already more
than enough attribution in D, I *WILL NOT* wrap everything in my
program with RefCounted!(). I will continue to invent my own solutions
in every instance, and we will live the same reality as C++; where
everyone has their own implementations, and none of them are
compatible.

Call it a bikeshed, whatever. I'm certain this is the practical reality.

I have tried RefCounted extensively in the past. Mangling my types
like that caused lots of problems, heaps of is(T == RefCounted!U, U)
started appearing throughout my code, and incompatibility with
existing libraries.
Perhaps the most annoying thing about a library implementation though
is the debuginfo baggage. It's extremely frustrating dealing with
things that are wrapped up like RefCounted while debugging. You can't
evaluate the thing anymore, stepping through your code leads you
through countless library stubs and indirections. You lose the ability
to just read a symbol name without fuss, you need to make your
callstack window half the screen wide to see what you're looking at.

I'm also not convinced meaningful refcounting can be implemented before we have scope(T) working properly. I think we should be addressing that first.

On 9/24/14, 3:34 PM, Manu via Digitalmars-d wrote:
> On 25 September 2014 07:17, Andrei Alexandrescu via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On 9/24/14, 1:15 PM, Manu via Digitalmars-d wrote:
>>>
>>> Something like (whatever syntax you like):
>>>
>>> int^ rcInt; // refcounted pointer to an int
>>> MyStruct^ rcStruct; // refcounted pointer to a struct
>>> MyStruct s; // normal value-type struct, but if the struct has
>>> opInc/opDec, the RC handling code in the compiler can implicitly
>>> generate calls to opInc/opDec on assignment, which will allow the
>>> struct to manage itself.
>>
>>
>> You're getting confused here, postblit and destructor take care of that.
>
> No they don't. It's not a ref counting mechanism, the compiler can't
> elide those calls.

It can.

> It's what I use now, and it's as good at C++, but we can do much
> better than that.

D's copy semantics are different from C++'s.

>>> Not sure how to express an RC dynamic array... int[^] rcArray? Anyway,
>>> syntax is whatever it is, I think this approach is what makes sense to
>>> me though.
>>
>>
>> Whatever syntax I like? Awesome! How about:
>>
>> RefCounted!int rcInt; // refcounted pointer to an int
>> RefCounted!MyStruct rcStruct; // refcounted pointer to a struct
>> RefCounted!(int[]) rcArray; // refcounted array
>>
>> The irony is the first two already work with the semantics you need, but
>> apparently I've had difficulty convincing you to try them and report back.
>> My work on RCString has also gone ignored by you, although it's exactly
>> stuff that you're asking for.
>
> It all feels backwards to me.

So it's not whatever syntax I like? Make up your mind.

> You've completely alienated me from the
> discussion. I've given up, but I am watching with interest.

How do you mean that?

> I'm not interested in a library solution until I know if, and how the
> compiler will optimise it... in which case it's not a library solution
> anymore, so why make the implementation a lib?

This can't be serious. So you're not looking at any prototype because it's not optimized up to wazoo?

> @nogc users will use this stuff EXCLUSIVELY. There is already more
> than enough attribution in D, I *WILL NOT* wrap everything in my
> program with RefCounted!(). I will continue to invent my own solutions
> in every instance, and we will live the same reality as C++; where
> everyone has their own implementations, and none of them are
> compatible.

But in C++ you'd use shared_ptr<T> all the time. Your position is completely irrational. Frankly it looks you're bullshitting your way through the whole conversation. Whenever anyone puts you in the position of making a positive contribution, you run away for something else to whine about.

> Call it a bikeshed, whatever. I'm certain this is the practical reality.
>
> I have tried RefCounted extensively in the past. Mangling my types
> like that caused lots of problems, heaps of is(T == RefCounted!U, U)
> started appearing throughout my code, and incompatibility with
> existing libraries.

Existing libraries won't be helped by adding new functions.

What issues did you have with RefCounted? Why do you keep on coming back to "it's mangling my types"? Why do you need heaps of is(...)? What is it that you're trying to solve, that only a solution you're unable to specify can fix?

> Perhaps the most annoying thing about a library implementation though
> is the debuginfo baggage. It's extremely frustrating dealing with
> things that are wrapped up like RefCounted while debugging.

How do you debug code using shared_ptr?

> You can't
> evaluate the thing anymore, stepping through your code leads you
> through countless library stubs and indirections. You lose the ability
> to just read a symbol name without fuss, you need to make your
> callstack window half the screen wide to see what you're looking at.
>
> I'm also not convinced meaningful refcounting can be implemented
> before we have scope(T) working properly. I think we should be
> addressing that first.

That may as well be true.


Andrei

On 9/24/14, 3:54 PM, Andrei Alexandrescu wrote:
> Frankly it looks you're bullshitting your way through the whole
> conversation.

s/bullshitting/fumbling/

Yah, I've done it. Apologies.


Andrei

On 25 September 2014 08:54, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 9/24/14, 3:34 PM, Manu via Digitalmars-d wrote:
>>
>> On 25 September 2014 07:17, Andrei Alexandrescu via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>> On 9/24/14, 1:15 PM, Manu via Digitalmars-d wrote:
>>>>
>>>>
>>>> Something like (whatever syntax you like):
>>>>
>>>> int^ rcInt; // refcounted pointer to an int
>>>> MyStruct^ rcStruct; // refcounted pointer to a struct
>>>> MyStruct s; // normal value-type struct, but if the struct has
>>>> opInc/opDec, the RC handling code in the compiler can implicitly
>>>> generate calls to opInc/opDec on assignment, which will allow the
>>>> struct to manage itself.
>>>
>>>
>>>
>>> You're getting confused here, postblit and destructor take care of that.
>>
>>
>> No they don't. It's not a ref counting mechanism, the compiler can't elide those calls.
>
>
> It can.

Elaborate? I could be doing anything to the effect of refcounting, and it could be interleaved with anything else performed by those functions...


>> It's what I use now, and it's as good at C++, but we can do much better than that.
>
>
> D's copy semantics are different from C++'s.

I don't see how that influences this.


>>>> Not sure how to express an RC dynamic array... int[^] rcArray? Anyway, syntax is whatever it is, I think this approach is what makes sense to me though.
>>>
>>>
>>>
>>> Whatever syntax I like? Awesome! How about:
>>>
>>> RefCounted!int rcInt; // refcounted pointer to an int RefCounted!MyStruct rcStruct; // refcounted pointer to a struct RefCounted!(int[]) rcArray; // refcounted array
>>>
>>> The irony is the first two already work with the semantics you need, but
>>> apparently I've had difficulty convincing you to try them and report
>>> back.
>>> My work on RCString has also gone ignored by you, although it's exactly
>>> stuff that you're asking for.
>>
>>
>> It all feels backwards to me.
>
>
> So it's not whatever syntax I like? Make up your mind.

This isn't syntax distinction, it's a whole approach.


>> You've completely alienated me from the
>> discussion. I've given up, but I am watching with interest.
>
>
> How do you mean that?

I think it's safe to say I've been the most vocal participant on this
topic for more than half a decade now (6 years maybe?).
Suddenly there is motion, and it goes precisely the opposite direction
that I have been hoping for all this time.
That's fine, I'm very happy there is motion. I'm just clearly not the customer.

I'll be extremely happy to be proven wrong, but I don't feel like I
have an awful lot to add.
I hope it gets to a place I'm happy with, but I find it hard to get on
this train, I'm just not excited by this direction at all.

I specifically don't want shared_ptr<> and std::string, which is where it seems to be going.


>> I'm not interested in a library solution until I know if, and how the compiler will optimise it... in which case it's not a library solution anymore, so why make the implementation a lib?
>
>
> This can't be serious. So you're not looking at any prototype because it's not optimized up to wazoo?

I'm completely serious, but not for the reason you paraphrased.
You're suggesting that this is supported by the language somehow? I
have no idea how that works, and that's what I'm interested in.
We all already have RC libs of our own.


>> @nogc users will use this stuff EXCLUSIVELY. There is already more than enough attribution in D, I *WILL NOT* wrap everything in my program with RefCounted!(). I will continue to invent my own solutions in every instance, and we will live the same reality as C++; where everyone has their own implementations, and none of them are compatible.
>
>
> But in C++ you'd use shared_ptr<T> all the time.

I would never use shared_ptr<> for all the same reasons.


> Your position is completely irrational. Frankly it looks you're bullshitting your way through the whole
> conversation. Whenever anyone puts you in the position of making a positive
> contribution, you run away for something else to whine about.

Whatever. I'll resist the urge to respond similarly ad hominem.

I've made my position on shared_ptr<> clear on so many occasions. I'm
not interested in a D implementation of shared_ptr<>.
I'm not alone, it's not a tool that people use in my industry; most
companies I know even have rigid policies against stl and/or boost.

What contribution am I supposed to make?
I'm not a language developer, I don't want to be. I'm a customer/end
user, that's the relationship I want. I have code to write and I'm
sick of C++, that's why I'm here.
I'm also a strong advocate for D, giving lots of demonstrations,
lecture/presentations, and time/energy within my industry and local
community.
I have in the past had significant involvement in this NG and IRC.
I also influenced a major commercial company to adopt D.
I don't feel my contribution is worthless, or can be summarised as you
so nicely did.

I complain about things that hinder my work. That's what I care about. You'll notice that I rarely get involved or excited about fancy futuristic language constructs or library stuff, I am almost entirely task oriented, and if something is causing chronic friction between me and getting my work done, then I raise that as an issue.

In terms of 'positive contribution', whatever that means exactly in
this context, all I have ever really managed to achieve around here is
to be a catalyst for action on a few particularly important issues
that affect myself and my industry.
Many of these have been addressed, and that's awesome. I find the
frequency of language related problems has reduced significantly; most
of my current problems are practical (tooling, etc).


>> Call it a bikeshed, whatever. I'm certain this is the practical reality.
>>
>> I have tried RefCounted extensively in the past. Mangling my types like that caused lots of problems, heaps of is(T == RefCounted!U, U) started appearing throughout my code, and incompatibility with existing libraries.
>
>
> Existing libraries won't be helped by adding new functions.
>
> What issues did you have with RefCounted? Why do you keep on coming back to "it's mangling my types"?

It's unsightly, interferes with the type system (every time I want T, I need to check if it's wrapped, and then unwrap it), compromises debugging experience, runs particularly poorly in non-optimised builds.

For instance: __traits(allMembers, T) == tuple("RefCountedStore",
"_refCounted", "refCountedStore", "__ctor", "__postblit", "__dtor",
"opAssign", "refCountedPayload")
That's not what the user expects. In such an imaginary context, there
is probably already a 'static if(isPointer!T)' branch, which recurses
with PointerTarget!T. Such is likely to exist in libs, and I expect it
would be handled transparent if RC were a type of pointer.


> Why do you need heaps of is(...)?

Because that's how you find T from RefCounted!T. It's not uncommon to
want to know what type you're dealing with.
I expect things like PointerTarget, isPointer, etc, would be enhanced
to support RC pointers in the event they were to exist.
They don't support RefCounted, and I'm not even sure that really makes
sense, because as I've said before, RefCounted!T is conceptually
backwards; ie, T is a kind of RefCounted, rather than refcounted being
a kind of pointer.

> What is it that you're trying to solve, that only a solution you're unable to specify can fix?

I'd like RC pointers parallel to GC pointers, since I think we've established in the past that there is no chance we will ever replace the GC with something like GC backed ARC across the board.

I don't recall trying to specify a solution, just suggesting an approach that feels natural to me, and is similarly implemented in other languages.


>> Perhaps the most annoying thing about a library implementation though is the debuginfo baggage. It's extremely frustrating dealing with things that are wrapped up like RefCounted while debugging.
>
>
> How do you debug code using shared_ptr?

I don't, for the same reasons.


>> You can't
>> evaluate the thing anymore, stepping through your code leads you
>> through countless library stubs and indirections. You lose the ability
>> to just read a symbol name without fuss, you need to make your
>> callstack window half the screen wide to see what you're looking at.
>>
>> I'm also not convinced meaningful refcounting can be implemented before we have scope(T) working properly. I think we should be addressing that first.
>
>
> That may as well be true.

Shall we declare this a decisive area of development and start a conversation?
I particularly liked the proposal in the other thread, until it
transformed into a storage class.