On Sunday, 09 April 2017 at 23:14:08 UTC, Jack Stoufer wrote:
>
>You're missing the forrest for the trees. D needs a general solution to the problem of GC code in Phobos. This >tackles one specific area via special case but leaves every other GC allocation in Phobos, with no way to make >it @safe @nogc. These will either require the holistic approach eventually or more special cases.

On Tuesday, 11 April 2017 at 11:39:20 UTC, rjframe wrote:
>
> I have no problem with this specific change, but this method of solving problems is going to turn D into a horrible language with all kinds of weird edge cases. I left Python for D mostly because the language was becoming one hack built on top of another; it would be nice if D could avoid that path. The occasional kludge may be necessary, but it shouldn't be normal or the first thought.
>
> --Ryan

I'm personally belongs to the camp of perfectionists and holists. But there is bitter experience I want to share, that to be too purist with such approach just make unable things to happen. "Release early, release often" isn't just a good words, but real need, because of lack of early feedback your "perfect" solution just "SUDDENLY" may not match real use cases, and you don't have use cases before USE cases :)

So if something going to happen and solve bunch of problem it should be done (in such way that not prevent more general holistic solution to be gone into life at some point). The real issue is only bake transitional process in such way that it addresses the problems of majority.

So what could be done:
1. Implement this feature; but apply it only if special compiler flag is specified (the issue I see for this particular feature, that you should have versioned function declaration, IIRC there is no possibility to version of an attribute), emit all warnings, that may you current code clashes with upcoming feature
2. Introduce new flag that disable this feature and remove previous flag for enabling (from now it would be enabled by default), make the warnings errors.
3. Remove disabling flag.
(All this describes situation if feature is accepted _as is_ by language users).


Furthermore it seems to me that D need something like official binary nightmare releases. It introduces new features in the manner I have described, but also have a flag like `-disable-all-fresh-feature` which affect all features at stage 2, for those of us who need industrial compilation. This "nightmare releases" require however much more frequent major release cycle (like each 2-3 months), and that isn't bad _per se_.

On 4/11/2017 1:54 AM, Jonathan M Davis via Digitalmars-d wrote:
>> No. D can catch C++ exceptions.
>
> What does this look like? I recall you discussing planning adding the
> ability for D to catch C++ exceptions, but I was unaware that it had been
> implemented yet. Certainly, I couldn't find it in the documentation when I
> went looking a month or two back. But I wouldn't think that catching
> Exception would work if it's a C++ exception - even if it's a
> std::exception, since std::exception doesn't have the same functions as
> Exception or Throwable. So, how would you catch a C++ exception in D?

You define a C++ class in D that matches a C++ one, throw a pointer to that class in C++, and catch it in D.

It currently works on Elf and MachO, but not on Windows.

On Sunday, 9 April 2017 at 20:14:24 UTC, Walter Bright wrote:
> On 4/9/2017 1:16 AM, Daniel N wrote:
>> ... but why not go all the way, making it "always" refcounted? (for any "new E",
>> not emplace).
>
> Backwards compatibility, for one. For another, a general mechanism for safe refcounting of classes has eluded us.

Hi guys. Hey Walter. So, about this point. On the lifetime study thread, http://forum.dlang.org/post/56301A8C.1060808@erdani.com , the following two problems were stated by Andrei, but I don't think they were adequately addressed in the subsequent posts:

===
Widget global;
@rc class Widget {
   int x;
   void fun() {
     global = null;
     ++x;
   }
}

void main() {
   global = new Widget;
   global.fun();
}

In this example, if global has a refcount==1 upon entering fun(), the assignment "global = null" deletes the Widget object and ++x accesses dangling memory.

I should add here another pattern that turned problematic for our older attempts in DIP74:

C c = new C();
foo(c);

int foo(scope C d) {
     c = new C();    // c's old instance gets deleted
     return d.i;        // oops! d is invalid
}
===

So here's my analysis of both these problems.

When calling a function, the reference count for an object must increase by the total number of aliases created by the call — *minus the ones lost*. Globals are special in this regard, because access to them is not lost in the called function. Local variables, however, are lost to the called function — they cannot be accessed except through the new alias they receive as a parameter. Thus, only globals must modify the reference count when passed.

Thinking about this is made easier if we turn all accessible aliases into function parameters. For this, we need to consider the set of globals as a hidden parameter to the function. Any global is therefore already passed to all functions. If we pass it again via parameter, that amounts to two aliases, thus two references. The same logic applies, 1. to duplicating any given variable in the argument list, e.g. "fun(c, c);", 2. to duplicating the hidden 'this' parameter, 3. to recognizing and outer function's stack frame as an additional alias.

All these can be verified at compile time. Also, it is optimistic regarding the most common case, i.e. passing a local does not require increasing the reference count.

--Zach

On 4/11/2017 4:39 AM, rjframe wrote:
> I have no problem with this specific change, but this method of solving
> problems is going to turn D into a horrible language with all kinds of
> weird edge cases. I left Python for D mostly because the language was
> becoming one hack built on top of another; it would be nice if D could
> avoid that path. The occasional kludge may be necessary, but it shouldn't
> be normal or the first thought.

On the other hand, overly principled languages tend to not be as successful, because what people need to do with programs is often dirty.

Monads, and "functional reactive programming", are obtuse things that come about when a functional programming language requires 100% purity and immutability.

Back in the 80's, like everyone else, I went about creating a GUI user interface library. I discovered something interesting - what is orthogonal and consistent to a computer is anything but when dealing with people. What people view as orthogonal and consistent is a rat's nest of exceptions in the code to implement it. This is what makes a user interface library fiendishly difficult to pull off.

Language design is like that, too.

I hear what you're saying, and agree in principle. That is why this feature comes with no new syntax, and existing code should "just work" with it to the largest extent possible.

On Monday, 10 April 2017 at 21:44:32 UTC, Jonathan Marler wrote:
> "throw"     operator (throw a Throwable object)
> "new"       operator (create a GC object)
> "throw new" operator (create and throw a reference-counted Throwable object)
>

There is no need for this, the compiler already understands the notion of unique for newed objects and object returned from pure functions.

On Tuesday, 11 April 2017 at 17:43:20 UTC, Walter Bright wrote:
> On the other hand, overly principled languages tend to not be as successful, because what people need to do with programs is often dirty.
>
> Monads, and "functional reactive programming", are obtuse things that come about when a functional programming language requires 100% purity and immutability.
>

Monads and "functional reactive programming" are not a manifestation of being principled but being ideological.

Javascript is principled.

Language that historically are very unprincipled such as PHP and C++, first didn't succeeded because they were unprincipled, but because they were quick and dirty hack solving people's problems where no other solution was available, and are moving toward being more principled with each version.

On Tuesday, 11 April 2017 at 11:08:34 UTC, Dukc wrote:
> This idea could be generalized:
>
> -DRuntime would add an interface "ReferenceCountable".
> -Throwable would implement it.
> -When a new expression of ReferenceCountable type is used to assign to a scope variable or argument, it's guaranteed to be @nogc.

scope c = new Object;

This should be recognised as a scope class instance. It doesn't need reference counting, as it should always be destroyed at the end of current scope. The compiler can allocate on the stack (if the instance isn't too big):

https://forum.dlang.org/post/mailman.1045.1471418257.3131.digitalmars-d-announce@puremagic.com

If it is too big, the compiler could use a @nogc heap allocator.

On 4/11/2017 10:24 AM, MysticZach wrote:
> Hi guys. Hey Walter. So, about this point. On the lifetime study thread,
> http://forum.dlang.org/post/56301A8C.1060808@erdani.com , the following two
> problems were stated by Andrei, but I don't think they were adequately addressed
> in the subsequent posts:

The way they will be addressed is to increment the reference count in the call to the function that takes a reference to an RC object.

On Wednesday, 12 April 2017 at 15:08:44 UTC, Nick Treleaven wrote:
> This should be recognised as a scope class instance. It doesn't need reference counting, as it should always be destroyed at the end of current scope.

The reason it needs:

{   scope Object ob = new RefCountableType("foo");
    scope ob2 = ob;
    ob = new RefCountableType("bar");
}

The "foo" instance would leak if the destruction would be done by calling at end of scope. You also cannot call the destructor when a new object is assigned to it, because then ob2 would become dangling.

What could act without GC or RC, is an immutable scoped object. But that's a different matter altogether, because throwing lifetimed objects did not work according to Walter.

On Wednesday, 12 April 2017 at 19:01:25 UTC, Walter Bright wrote:
> On 4/11/2017 10:24 AM, MysticZach wrote:
>> Hi guys. Hey Walter. So, about this point. On the lifetime study thread,
>> http://forum.dlang.org/post/56301A8C.1060808@erdani.com , the following two
>> problems were stated by Andrei, but I don't think they were adequately addressed
>> in the subsequent posts:
>
> The way they will be addressed is to increment the reference count in the call to the function that takes a reference to an RC object.

Makes sense, and I thought the same thing until after I wrote my post — but now I realize it's not quite good enough. Merely incrementing the refcount (and decrementing it afterwards) has the following problem:

@rc class RC;

void fun(ref RC a, RC b) {
   a = new RC; // (1)
   // b...
}

void main() {
   auto r = new RC; // (2)
   --> r.opInc(); // compiler insert
   fun(r, r);
   --> r.opDec();
}

Let's assume the reference counting scheme involves the methods opInc and opDec, inserted automatically by the compiler as the result of detecting a duplicate parameter. If you read closely, you'll realize that at mark 1 above, the program will leak the data acquired at mark 2. The assign statement of the refcounted object will decrement the data it points to before it is reassigned. But since the data at (2)'s refcount was prematurely incremented, it will fall to 1 and never get deleted. Furthermore, the opDec in main() will decrement and delete the data acquired at mark 1, thinking it was the mark 2 data.

The problem is that the calling context at "fun(r,r);" fails to keep a real reference to the mark 2 data. If one of the parameters is sent by reference, we can't assume it points to the same data upon returning as when it was sent. And the mark 2 data can't be deleted before fun() returns, or it will invalidate 'b' in fun(). This suggests we need to save a real, separate reference to 'r' before sending two or more versions of it.

That said, I believe the following compiler inserts in main() would do the trick:

void main() {
   auto r = new RC; // (2)
   --> RC __rsave = r; // compiler insert
   --> scope(exit) __rsave = null;
   fun(r, r);
}

This solution uses only the assign method of RC to inc and dec the refcount, suggesting that opInc and opDec are ultimately unnecessary, except as (very rare) optimizations. But you still need for the compiler to distinguish a refcounted class from a regular one, so maybe the presence of an opDec() could indicate that, opDec being the quasi-destroyer that turns a refcounted class variable into an RAII type. Otherwise, you might need something like an @rc attribute.

At any rate, your original claim was, "a general mechanism for safe refcounting of classes has eluded us." Is this still the case, considering the above?

As far as the original post, even if a general mechanism were found, it doesn't mean you'd have to use it in the case of Exception anyway. To generalize the expression 'throw new ...', you'd have to redefine 'new' to be different with refcounted classes than with regular ones. Exceptions are probably worth a more specialized solution like the one you proposed. Otherwise everyone will have to change their 'throw new' code to accommodate '@nogc'. Adam D. Ruppe gave us his solution here:

http://arsdnet.net/exception.d

tl;dr Everyone would now have to say things like 'throw emplace!...' or 'raise!"my_exception"(...)'.

I guess the proposed hack of 'throw new Exception' is simply the shadow of D's original philosophy of wanting the GC to do too much.