On 9/24/14, 7:16 PM, Manu via Digitalmars-d wrote:
[snip]

Thanks for answering. I've figured I have no meaningful answer to this, but allow me to elaborate why out of respect.

Apologies for letting go of the horses a little. An explanation (but not an excuse) is that Walter and I are humans and as much as we do our best to keep levelheaded at all times we get a tad frustrated.

These are heady days for D. We're more optimistic than ever about future prospects and possibilities, which we hope to confirm publicly soon. We have a very strong vision (C++ and GC, C++ and GC...) and we have designs realizing it that we believe we can make work. This is literally the best time in the history of D for the community to help us. Amid this feverish preparation (we talk some 15-45 minutes on the phone every day), we find ourselves stumped by the occasional idle chatter on inconsequential matters that sometimes descends into a spiral of gratuitous negativity. Or wine party, as graciously dubbed by Walter.

I have difficulty communicating with you, to the extent that I very candidly have no idea what you're trying to accomplish, and how you propose language technology to help you. But I think it's possible to improve on that.

One simple rule of thumb is to pop one level up and describe the task you're trying to accomplish, instead of describing at low level what you believe would be obviously the language-based solution. Two examples:

1. You say "if ref were part of a type" and not only say it, but also build on the good presumed consequence of it. That can't be done in D, simple as that. We can't flip a switch and do it. The ripples throughout the entire fabric of the language would essentially transform it in a whole different language, with most rules subtly different from today's. You yourself confessed "I'm not a language developer, I don't want to be." Then the best route is to focus on the high-level task as opposed on what you believe would be the language change fixing it. Please take this as kindly as I mean it: most language-space solutions you propose are alien and unworkable.

2. You wrongly believe language solutions are innately better than engineering solutions. Please understand that no amount of notation would save you from the issues you encounter with RefCounted. Also, your notion that optimization technology only kicks in for language-baked artifacts is wrong. Please trust us on this: YES, we can define increment and decrement in libraries in such a way they're elided if they cancel themselves out. I find it very difficult to sympathize with you completely dismissing library engineering solutions for vague reasons, which, from what I can tell, no amount of built-in notation can save you.

What I hope to come out of this is a clear idea of what you're trying to accomplish (not "I want ref part of the type", but "I want to write a framework that does ..."), and how you find the current offering in the language + standard library + your own custom libraries wanting. Can we get that kind of dialog going?


Andrei

On 9/24/2014 1:15 PM, Manu via Digitalmars-d wrote:
> Something like (whatever syntax you like):
>
> int^ rcInt; // refcounted pointer to an int
> MyStruct^ rcStruct; // refcounted pointer to a struct
> MyStruct s; // normal value-type struct, but if the struct has
> opInc/opDec, the RC handling code in the compiler can implicitly
> generate calls to opInc/opDec on assignment, which will allow the
> struct to manage itself.

I think Microsoft's C++/CLI tried that mixed pointer approach, and it was a disaster. I don't have personal knowledge of this.

I suspect I know why. C++ on DOS had mixed pointer types - the near and far thing. It was simply unworkable in C++. Sure, it technically worked, but was so awful trying to deal with "is my ref type near or far" that people just couldn't write comprehensible code with it.

Note that a ^ pointer will be a different size than a * pointer. Things go downhill from there.

On 9/22/2014 11:19 PM, deadalnix wrote:
> I think a library solution + intrinsic for increment/decrement (so they can be
> better optimized) would be the best option.

Intrinsics are unnecessary. The compiler is perfectly capable of recognizing certain code patterns and replacing them with single instructions.

I'm afk (on a phone) for 2 days, but I'll get back to this.
On 25 Sep 2014 15:30, "Andrei Alexandrescu via Digitalmars-d" <
digitalmars-d@puremagic.com> wrote:

> On 9/24/14, 7:16 PM, Manu via Digitalmars-d wrote:
> [snip]
>
> Thanks for answering. I've figured I have no meaningful answer to this, but allow me to elaborate why out of respect.
>
> Apologies for letting go of the horses a little. An explanation (but not an excuse) is that Walter and I are humans and as much as we do our best to keep levelheaded at all times we get a tad frustrated.
>
> These are heady days for D. We're more optimistic than ever about future prospects and possibilities, which we hope to confirm publicly soon. We have a very strong vision (C++ and GC, C++ and GC...) and we have designs realizing it that we believe we can make work. This is literally the best time in the history of D for the community to help us. Amid this feverish preparation (we talk some 15-45 minutes on the phone every day), we find ourselves stumped by the occasional idle chatter on inconsequential matters that sometimes descends into a spiral of gratuitous negativity. Or wine party, as graciously dubbed by Walter.
>
> I have difficulty communicating with you, to the extent that I very candidly have no idea what you're trying to accomplish, and how you propose language technology to help you. But I think it's possible to improve on that.
>
> One simple rule of thumb is to pop one level up and describe the task you're trying to accomplish, instead of describing at low level what you believe would be obviously the language-based solution. Two examples:
>
> 1. You say "if ref were part of a type" and not only say it, but also build on the good presumed consequence of it. That can't be done in D, simple as that. We can't flip a switch and do it. The ripples throughout the entire fabric of the language would essentially transform it in a whole different language, with most rules subtly different from today's. You yourself confessed "I'm not a language developer, I don't want to be." Then the best route is to focus on the high-level task as opposed on what you believe would be the language change fixing it. Please take this as kindly as I mean it: most language-space solutions you propose are alien and unworkable.
>
> 2. You wrongly believe language solutions are innately better than engineering solutions. Please understand that no amount of notation would save you from the issues you encounter with RefCounted. Also, your notion that optimization technology only kicks in for language-baked artifacts is wrong. Please trust us on this: YES, we can define increment and decrement in libraries in such a way they're elided if they cancel themselves out. I find it very difficult to sympathize with you completely dismissing library engineering solutions for vague reasons, which, from what I can tell, no amount of built-in notation can save you.
>
> What I hope to come out of this is a clear idea of what you're trying to accomplish (not "I want ref part of the type", but "I want to write a framework that does ..."), and how you find the current offering in the language + standard library + your own custom libraries wanting. Can we get that kind of dialog going?
>
>
> Andrei
>
>

On 25 Sep 2014 15:50, "Walter Bright via Digitalmars-d" < digitalmars-d@puremagic.com> wrote:
>
> On 9/24/2014 1:15 PM, Manu via Digitalmars-d wrote:
>>
>> Something like (whatever syntax you like):
>>
>> int^ rcInt; // refcounted pointer to an int
>> MyStruct^ rcStruct; // refcounted pointer to a struct
>> MyStruct s; // normal value-type struct, but if the struct has
>> opInc/opDec, the RC handling code in the compiler can implicitly
>> generate calls to opInc/opDec on assignment, which will allow the
>> struct to manage itself.
>
>
> I think Microsoft's C++/CLI tried that mixed pointer approach, and it was
a disaster. I don't have personal knowledge of this.

C++ doesn't have any notion of borrowing. Scope can (will?) fix a whole lot of existing problems, and also allow RC -> raw pointers work nicely.

Also, I have no experience that leads me to believe it was any sort of disaster. It would have been nice if it didn't imply com specifically, that's probably the real problem that got the bad press; ^ pointers are implicitly com, which I don't recommend repeating ;)

> I suspect I know why. C++ on DOS had mixed pointer types - the near and
far thing. It was simply unworkable in C++. Sure, it technically worked, but was so awful trying to deal with "is my ref type near or far" that people just couldn't write comprehensible code with it.

I don't think this is the same thing. There's a practical and functional difference. People would definitely use it if they have it.

I don't think the MS problem was in any way related.

> Note that a ^ pointer will be a different size than a * pointer. Things
go downhill from there.

Dynamic arrays are a different size from pointers, that never caused any problems. I can't imagine any issues from this.

Anyway, it hasn't been explored at length. If it has any chance then I'll put extensive thought into it.

Your double posting is baaaaack!

Seriously, get a better newsreader/poster. You're the only one with this issue!

On 9/25/2014 12:10 AM, Manu via Digitalmars-d wrote:
>  > I think Microsoft's C++/CLI tried that mixed pointer approach, and it was a
> disaster. I don't have personal knowledge of this.
> C++ doesn't have any notion of borrowing. Scope can (will?) fix a whole lot of
> existing problems, and also allow RC -> raw pointers work nicely.

Consider that people complain a lot about annotations. See the other thread. Adding the scope annotations everywhere is a LOT of annotations. Do you think people will be happy with that? I don't.

I remember reading a paper about someone adding pointer annotations to Java. It was a technical success, and a usability failure. People just couldn't be bothered to add the annotations.

Walter Bright:

> Consider that people complain a lot about annotations.

It's much better to try to quantify how much this "a lot" means, even roughly. Otherwise you can't reason on anecdote. (I think D annotations are a burden, and their return of investment is currently not large, but so far they are acceptable for me. We can discuss possible ways to improve the situation).


> See the other thread. Adding the scope annotations everywhere
> is a LOT of annotations. Do you think people will be happy with that?
> I don't.

How much is that "a LOT"? I'd like you to give this idea a chance to show its advantages in practice. Removing problems like this, and at the same time giving a help to garbage collection (reference counting is another kind of garbage collection. And it has some costs) is good:

int* foo() @safe {
    int[10] a;
    int[] b = a[];
    return &b[1];
}
void main() {}


So it's a matter of weighting the costs of a system to track ownership of memory areas compared to the costs in correctness and performance of the current situation (or the current situation plus an additional garbage collection strategy).

Lately for me correctness has become very important (more than for the author of the video about video game language design), I am willing to work more when I think about the code and adding some annotations if this avoids me the wasted time to search bugs or avoids me troubles with too much GC activity when the program gets larger (as shown by Maxime Chevalier-Boisvert).

Bye,
bearophile

On Thursday, 25 September 2014 at 05:46:35 UTC, Walter Bright wrote:
> On 9/24/2014 1:15 PM, Manu via Digitalmars-d wrote:
>> Something like (whatever syntax you like):
>>
>> int^ rcInt; // refcounted pointer to an int
>> MyStruct^ rcStruct; // refcounted pointer to a struct
>> MyStruct s; // normal value-type struct, but if the struct has
>> opInc/opDec, the RC handling code in the compiler can implicitly
>> generate calls to opInc/opDec on assignment, which will allow the
>> struct to manage itself.
>
> I think Microsoft's C++/CLI tried that mixed pointer approach, and it was a disaster. I don't have personal knowledge of this.
>
> I suspect I know why. C++ on DOS had mixed pointer types - the near and far thing. It was simply unworkable in C++. Sure, it technically worked, but was so awful trying to deal with "is my ref type near or far" that people just couldn't write comprehensible code with it.
>
> Note that a ^ pointer will be a different size than a * pointer. Things go downhill from there.

Why it was a disaster? Microsoft is still using it.

It is how C++/CX works, recently introduced with Windows 8 new COM based runtime.

The different to C++/CLI being that ^ (handle types) are not GC pointers, but rather compiler managed COM (AddRef/Release) instances.

For those that wish to stay away from C++/CX, there is the more complex Windows Runtime C++ Template Library with ComPtr<>(), but then bye compiler support for increment/decrement removals.

http://msdn.microsoft.com/en-us/library/hh699870.aspx

http://msdn.microsoft.com/en-us/library/windows/apps/jj822931.aspx

Example for those lazy to follow the link,

using namespace Platform;

Person^ p = ref new Person("Clark Kent");
p->AddPhoneNumber("Home", "425-555-4567");
p->AddPhoneNumber("Work", "206-555-9999");
String^ workphone = p->PhoneNumbers->Lookup("Work");



--
Paulo

On Thursday, 25 September 2014 at 02:16:32 UTC, Manu via Digitalmars-d wrote:
> On 25 September 2014 08:54, Andrei Alexandrescu via Digitalmars-d
>>> It's what I use now, and it's as good at C++, but we can do much
>>> better than that.
>>
>>
>> D's copy semantics are different from C++'s.
>
> I don't see how that influences this.

For one, it allows to turn copy+destruction into a move, e.g. for tail calls.

>>> I'm also not convinced meaningful refcounting can be implemented
>>> before we have scope(T) working properly. I think we should be
>>> addressing that first.
>>
>>
>> That may as well be true.
>
> Shall we declare this a decisive area of development and start a conversation?
> I particularly liked the proposal in the other thread, until it
> transformed into a storage class.

Please note that I'm not opposed to turn it back into a type modifier. I just don't understand your problems enough to form an opinion on it, and as long as that is the case, it's not a good idea to switch it back and forth, especially since it brings along its own problems. Please post a create example where a storage class fails (in the other thread).