On Thu, 30 Jun 2005 15:20:46 +0100, Stewart Gordon <smjg_1998@yahoo.com> wrote:
> Regan Heath wrote:
>> On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998@yahoo.com>  wrote:
> <snip>
>>> Of the claims by various people that we "need" overloading of the assignment operator, I'm yet to see one that's true. AFAIC if you want something to the effect of custom assignment behaviour, you can create your own method with whatever name that does it.  Can anyone supply an example, let alone a real-world one, that doesn't work unless the aforementioned syntactic sugar is available?
>>  The problem, as I see it, is that it's not just syntactic sugar. You cannot prevent reference assignment in cases where the idea/technique requires it. As in these 2 cases. (reference counting, auto pointer)
>
> I guess that if you use either of these, you're supposed to know what you're doing and that you have to be careful.  Remember that because of the way D is designed, most applications will be using the built-in GC rather than this.  As such, a facility for disabling the assignment operator for select types would probably be seen as an unnecessary complexity.

Just because you're being clever doesn't mean you can't use all the help you can get. Being able to write an idiot proof reference counting implementation would be an enormous boon for everybody. I don't care how it's done (to a degree), I'm not sure disabling assignment is the solution.

>> I've tried to write a reference counting implementation and also an auto pointer implementation using the recommended practices found on the D website. Both work, both use a "copy constructor" instead of opAssign (as suggested by the D spec/website). The problem is this:
>>  auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p owns  object. (or "pwns" for those of you..)
>> auto AutoPtr s;
>>  s = new AutoPtr(p); //no problem, the copy constructor transfers ownership.
>> s = p; //problem, s now refers to p, both 'own' "object".
>
> http://www.digitalmars.com/d/attribute.html#auto
> "Assignment to an auto, other than initialization, is not allowed."
>
> I just had a go at it with GDC 0.10.  It appears from the destruction order that if an auto variable is initialised to another, then the new reference becomes the owning one.  However, there seems to be a bug whereby it accepts assignments after initialisation; however, doing this doesn't seem to have any effect on what gets destructed when.

Indeed, it's broken then. I wasn't aware of the rule:
  "Assignment to an auto, other than initialization, is not allowed."

in fact, I suggested it after trying to write the implementation in response to Andrew pointing out what I just showed you. Funny that.

> Here's the program I just tried:
>
> ----------
> import std.stream;
> import std.stdio;
>
> class Thing {
>      int value;
>      this(int v) { value = v; }
>      ~this() {
>          writefln("destructed %d", value);
>      }
> }
>
> void main() {
>      auto Thing o = new Thing(1);
>      auto Thing p = new Thing(2);
> 	
>      char[] line = std.stream.stdin.readLine();
>      auto Thing q = line[0] == 'y' ? o : new Thing(3);
> 		
>      if (line[1] == 'y') o = q;
>      writefln(q.value);
>      writefln(o.value);
> }
> ----------
>
>> The same problem occurs with the reference counting implementation. s becomes a 2nd reference to object but the reference count does not increase.
>>  We can 'ask' people not to use a straight reference assignment but we  cannot enforce it.
> <snip>
>
> True.  But I guess what we really need is a clear spec of what happens if one auto is assigned to another.

That is step 1.

> If only by some means (not sure how easy or not it would be to implement) the auto stayed with the original owner, then such reference assignment could be OK (and indeed more efficient) as long as it isn't going to outlive the original.

I think having 2 references to a reference counter without incrementing the counter is still illegal and should be prevented. Same goes for an auto pointer really.

Preventing the assignment of an 'auto' and preventing assigning an auto to another reference would solve the problem, from what I can see.

Regan

On Thu, 30 Jun 2005 08:37:56 -0400, Ben Hinkle <ben.hinkle@gmail.com> wrote:
> "Regan Heath" <regan@netwin.co.nz> wrote in message
> news:opss6mi5iy23k2f5@nrage.netwin.co.nz...
>> On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle@gmail.com>
>> wrote:
>>> "Regan Heath" <regan@netwin.co.nz> wrote in message
>>> news:opss0boj0r23k2f5@nrage.netwin.co.nz...
>>>> I got little or no comment on this idea/code I posted to another (rather
>>>> large) thread here, I figured I'd post it again and see what people
>>>> thought.
>>>>
>>>> After reading this thread (the one I mention above) and also this one
>>>> Andrew dug up:
>>>>   http://www.digitalmars.com/d/archives/7988.html
>>>>
>>>> It appears you need a type that:
>>>>
>>>> 1 - is stack based.
>>>> 2 - has deterministic destruction.
>>>> 3 - has opAssign.
>>>>
>>>> It seems (from recent posts) that Walter plans to put auto classes on
>>>> the
>>>> stack, so they'd then fulfil requirements 1 and 2.
>>>>
>>>> But what about 3? To me, opAssign isn't generally a good idea for
>>>> classes
>>>> (assigning a value to a reference seems illogical) but would be a good
>>>> for
>>>> structs (they're value types, assignment assigns a value).
>>>>
>>>> Can you do reference counting without opAssign?
>>>
>>> See the section in the Memory Management doc about reference counting. It
>>> says the add/release must be explicit.
>>
>> Indeed. This strikes me as being rather bug intolerant i.e. open for
>> mistakes and bugs to creep in.
>> I am fine with it being this explicit:
>>
>> RefCnt r = new RefCnt(object); //explicit
>> RefCnt s;
>> s = new RefCent(r); //explicit
>>
>> However the problem is actually this:
>>
>> s = r; //needs to be disallowed or to inc refcnt.
>
> People will want to be able to get a reference
> without changing the ref
> count and = does that. There are advantages and disadvantages to using =. To
> me the main advantage is that = does the right thing and matches the user's
> expectations of what = does and the main disadvantage is that people who
> don't know they need to care about ref counting will just use = and get into
> trouble. To me the advantages outweigh the disadvantages.

Do you mean get a reference to the "reference counting object", or a reference to the "object being counted"?

= gives the former, a reference to the "reference counting object".

Getting a reference the the "reference counting object" without incrementing the reference count is not dangerous, provided the new reference does not outlive the original (as the object could be free'd then). I can see how you might want one in order to log something or whatever, but the general case you want to increment the count. that suggests to me that the generally expected behaviour is to increment the count, and the specialist behaviour is not to, in which case the obvious thing = should do the general case.

>>> Perhaps Walter could expound on that design decision somewhere to
>>> address people's concerns.
>>
>> Yes please.
>>
>>> Personally I have no beef with making non-trivial memory mangement
>>> explicit since it should be
>>> rarely needed. And in those cases having it explicit will alert code
>>> maintainers to what is going on.
>>
>> I agree.
>>
>>> Also it avoids having to add special "give me a reference without
>>> manipulating the count forperformance" issue.
>>
>> Ah, but that is the problem.
>
> I'm not sure what you mean. I don't think it's unusual to have a
> ref-counting API that has a way to get a reference without changing the
> count.

See above.

>>> On the other hand by making it explicit one has to advertise the fact it
>>> uses fancy memory management so that users know to increment and
>>> decrement the count.
>>
>> So, you're saying it should be done:
>>
>> RefCnt r = new RefCnt(object);
>> r.inc(); //perhaps optional?;
>>
>> RefCnt s = r;
>> s.inc();
>>
>> the room for error here is rather larger than I'd like.
>
> Actually I wouldn't even bother having a separate RefCnt class. The ref
> counting can be mixed into the class of object, much like it is for COM
> objects. So for example
>   Foo f = new Foo(); // creates with refcount of 1
>   Foo g = f.incRef; // inc ref count and return this
>   g.decRef; // dec ref count

The point of a seperate object is that it does this leg work for you. Meaning, you cannot get it wrong, meaning it's more "bug preventative", robust, and in the end simply easier to use.

>>> Currently D errs on the side of simplicity and making code explicit -
>>> which is a fairly common tactic in D in general.
>>
>> I am a big believer in simple code and making odd things explicit. What I
>> want in this case is something simple, that is bug tolerant. I don't mind
>> if it can be abused/broken with intent, it just needs to be accident(bug)
>> tolerant.
>
> I agree but I think in this case there are a number of factors that make the decision non-trivial. Sometimes it's best to keep everything simple and
> allow people to shoot themselves in the foot instead of making things
> complicated to prevent them from shooting themselves in the foot. The
> weights one gives to all the factors can tip the design in one way or
> another. The current situation is a reasonable set of compromises IMHO.
> Adding opAssign would have an impact beyond ref-counting so it's a pretty
> big step to take.

I'm not asking for opAssign!! I'm asking for a solution to the problem, everyone assumes the only solution is opAssign (which might be because it is, I don't know).

I'm saying, here is the problem, find me a solution (because I've exhausted my abilities trying). I'm starting to understand Andrew's frustration. Everyone here seems incapable of exploring the landscape for a solution, they'd all much rather sit in their dirt huts and bandage their bleeding foot. (just ignore me here, this is frustration talking..)

Regan

Regan Heath wrote:
> On Thu, 30 Jun 2005 15:20:46 +0100, Stewart Gordon <smjg_1998@yahoo.com>  wrote:
<snip>
>> I guess that if you use either of these, you're supposed to know what  you're doing and that you have to be careful.  Remember that because of the way D is designed, most applications will be using the built-in GC  rather than this.  As such, a facility for disabling the assignment operator for select types would probably be seen as an unnecessary complexity.
> 
> Just because you're being clever doesn't mean you can't use all the help you can get.

True.  But there's certainly a limit to the extent to which language features to protect you during with uncommon D&D tasks are worth it.

> Being able to write an idiot proof reference counting implementation would be an enormous boon for everybody.

Everybody?  Even those of us who are perfectly happy with D's own GC?

> I don't care how it's done (to a degree), I'm not sure disabling assignment is the solution.

That makes a change.

>> If only by some means (not sure how easy or not it would be to implement) the auto stayed with the original owner, then such reference assignment could be OK (and indeed more efficient) as long as it isn't going to outlive the original.
> 
> I think having 2 references to a reference counter without incrementing the counter is still illegal and should be prevented. Same goes for an auto pointer really.

I refer you back to this:

http://www.digitalmars.com/d/garbage.html

"Reference counting is a common solution to solve explicit memory allocation problems. The code to implement the increment and decrement operations whenever assignments are made is one source of slowdown. Hiding it behind smart pointer classes doesn't help the speed."

So you're saying it should be forbidden to partly get around this issue by circumventing the reference count even when you know it's safe?

> Preventing the assignment of an 'auto' and preventing assigning an auto to another reference would solve the problem, from what I can see.

Up to a point.  But autos can be passed into functions, which generally don't know that they're dealing with auto objects and so can arbitrarily create more references to them.  But arguably this is a good thing, as such ability is useful for Windows GDI stuff and the like.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on the 'group where everyone may benefit.

On Fri, 01 Jul 2005 11:23:44 +0100, Stewart Gordon <smjg_1998@yahoo.com> wrote:
>> Being able to write an idiot proof reference counting implementation would be an enormous boon for everybody.
>
> Everybody?  Even those of us who are perfectly happy with D's own GC?

Reference counting is not an "alternative" to the GC, it's an "additional" tool. I can imagine using the GC for 80-90% of my memory management and reference counting for those remaining situations where it is beneficial.

>> I don't care how it's done (to a degree), I'm not sure disabling assignment is the solution.
>
> That makes a change.

?

>>> If only by some means (not sure how easy or not it would be to implement) the auto stayed with the original owner, then such reference assignment could be OK (and indeed more efficient) as long as it isn't going to outlive the original.
>>  I think having 2 references to a reference counter without incrementing the counter is still illegal and should be prevented. Same goes for an auto pointer really.
>
> I refer you back to this:
>
> http://www.digitalmars.com/d/garbage.html
>
> "Reference counting is a common solution to solve explicit memory allocation problems. The code to implement the increment and decrement operations whenever assignments are made is one source of slowdown. Hiding it behind smart pointer classes doesn't help the speed."

True.

> So you're saying it should be forbidden to partly get around this issue by circumventing the reference count even when you know it's safe?

As long as the new reference does not outlive the original it's probably safe, but that sort of thing is not the general or common usage case. Meaning, ideally, it shouldn't be easy to do by accident. Currently it is:

AutoPtr p = new AutoPtr(object);
AutoPtr s;
s = p; //oops

>> Preventing the assignment of an 'auto' and preventing assigning an auto to another reference would solve the problem, from what I can see.
>
> Up to a point.  But autos can be passed into functions, which generally don't know that they're dealing with auto objects and so can arbitrarily create more references to them.  But arguably this is a good thing, as such ability is useful for Windows GDI stuff and the like.

But again, not a general or comman usage case. So ideally, shouldn't be easy to do accidentally. It sounds like it is currently.

Regan

"David Medlock" <noone@nowhere.com> wrote in message news:da0s3i$1309$1@digitaldaemon.com...
> Andrew Fedoniouk wrote:
>> "Regan Heath" <regan@netwin.co.nz> wrote in message news:opss5jwkvj23k2f5@nrage.netwin.co.nz...
>>
>>>On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone@nowhere.com> wrote:
>>>
>>>>Regan Heath wrote:
>>>>
>>>>
>>>>>Regan
>>>>
>>>>I thought the stack based types were to be useful for speed gains, not managing memory.
>>>
>>>Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.
>>>
> So why not simply grab the reference once per frame(as in frames per second) and draw all the needed objects?  Dont pass the context to the objects, pass the objects to the context owner.
>
> I've written code of this type (managing texture memory in OpenGL for one) and its much easier managed on a per-frame basis than per object.
>
> (Sorry if I am not understanding this correctly, couldnt find the example you mention)

window::draw()
{
   auto Graphics g = new Graphics( this );
   draw_content(...);
}

Graphics is contains single memeber HDC (on win32).
destructor contains : ReleaseDC(...).

Heap allocation of Graphics here is far from optimal.


>>>
>>>>Why do you need reference counting?
>>>
>>>I don't, yet. Andrew does, now.
>>
>>
>> Reference counting is one of most famous design patterns.
>> RC as a memory management has its own limitations though ( e.g. cyclic
>> refs ).
>> Its power in those areas where GC is weak. And vice versa.
>> So be able to use both will benefit the whole system twice.
>>
>> Good example of RC is in implementations of COW strings and shared_ptr in C++.
>>
>>
> Those are required in C++ because there is no garbage collector.  In C++ Copy on write is a programming discipline/design.  Its still up to the programmer though, const just helps to keep the programmers honest.
>
> With (non-transactional)concurrent programming, there is no way to use reference counting without locks at each assignment/deref.
>
> Consider:
> A resource we call A has 2 references to it, owned by a variable in both
> Thread T and U.
>
> Thread T calls deref, but is blocked before it can check the reference count of A.
>
> Thread U calls deref as well, sees the reference count is zero, and deallocates the reference.
>
> Thread T awakens, sees the reference count is  zero, calls destructor....OOPS.
>
> Not to mention modern architectures are very cache-sensitive. When you call destructors throughout your code, you 'thrash the cache' jumping around in memory a lot more than a gc calling the same destructors several times in sequence.
>
> I reject the assertion that managing references at each object is superior to an external collector(of any type, not simply memory). All those AddRef/Decref/Ref checks are not free.

Absolutely true.

Refcounting as a sole method of managing memory is bad.

The point is simple:  Nor GC nor RC are silver bullets. Together
they can be considered as a suboptimal solution.
To be short - I would like to have a choice in each particular case.

The thing is - in D there is no physical way to implement ligthweight RC.

>
> Managed resources should be managed by the code, not the runtime.  I'm sure some will disagree with some(or all) of this, but I have yet to see evidence to the contrary.

I agree.

>
> -DavidM

"Andrew Fedoniouk" <news@terrainformatica.com> wrote in message news:da40ce$1olo$1@digitaldaemon.com...
>
> "David Medlock" <noone@nowhere.com> wrote in message news:da0s3i$1309$1@digitaldaemon.com...
>> Andrew Fedoniouk wrote:
>>> "Regan Heath" <regan@netwin.co.nz> wrote in message news:opss5jwkvj23k2f5@nrage.netwin.co.nz...
>>>
>>>>On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone@nowhere.com> wrote:
>>>>
>>>>>Regan Heath wrote:
>>>>>
>>>>>
>>>>>>Regan
>>>>>
>>>>>I thought the stack based types were to be useful for speed gains, not managing memory.
>>>>
>>>>Correct. We need/want the reference counter or auto pointer to be as fast as possible because they're typically instantiated and destroyed with abandon. Andrew posted a good example of this, where his graphics library needs to obtain ownership of a graphics context on every call to the 'paint' method, it needs to get it, use it and release it, this function is called very often.
>>>>
>> So why not simply grab the reference once per frame(as in frames per second) and draw all the needed objects?  Dont pass the context to the objects, pass the objects to the context owner.
>>
>> I've written code of this type (managing texture memory in OpenGL for one) and its much easier managed on a per-frame basis than per object.
>>
>> (Sorry if I am not understanding this correctly, couldnt find the example you mention)
>
> window::draw()
> {
>   auto Graphics g = new Graphics( this );
>   draw_content(...);
> }
>
> Graphics is contains single memeber HDC (on win32).
> destructor contains : ReleaseDC(...).
>
> Heap allocation of Graphics here is far from optimal.

As another datapoint MinWin uses a free list to manage Graphics instances. The draw() method grabs a Graphics from the free list and returns it after finishing drawing. It's the only class in MinWin with custom memory management (so far).