On 01/12/2013 03:27 AM, Era Scarecrow wrote:
> On Saturday, 12 January 2013 at 10:58:23 UTC, thedeemon wrote:
>> So the runtime is aware of weak pointers and clears them to "empty"
>> state when pointed value dies. I don't see yet how it can be
>> implemented in D without patching its GC.
>
> There's that, but also what about what if the pointer wasn't a pointer
> to the data at all? What it if was a part of a float that determined a
> calculation for your taxes? Or a series of flags that happen to make
> that magic number? If they get 'nulled' by the GC, I'd call that a
> definite bug and problem; Unless the GC can tell apart what is and isn't
> a pointer to data.

You really shouldn't break the type constraints.  If you do, then you must expect problems.  But weak pointers do not themselves break type constraints.  They do, however, require that the garbage collector cooperate with them.  User implemented weak pointers which the garbage collector is unaware of are almost guaranteed to cause problems.  The garbage collector needs to know to null them when it frees the storage.  And if you store non-pointer data into a pointer, you are asking for trouble in ANY language, but in languages with dynamic storage management you are practically guaranteed to get the trouble you asked for.  I languages like C/C++ you can often get away with it...but it's still a "really bad idea"(tm).

P.S.:  Perhaps you're thinking of unions with a combination of floats and pointers in them.  Legal, I think, but still a really bad idea.

On 01/12/2013 02:04 AM, Era Scarecrow wrote:
> On Saturday, 12 January 2013 at 04:41:00 UTC, Charles Hixson wrote:
>> Thanks. That looks quite useful. OTOH, I don't see how the pointer
>> allows an item to be freed. You probably meant that this was a
>> framework to start development from.
>
> Correct. To create an object if it wasn't there when trying to access it
> basically. That's what I originally thought you wanted; nothing about
> freeing it.
>
>> And it does show a way to auto initialize a null pointer (but that's
>> not where I'm hung up...rather on how to automatically null the
>> pointer when the last valid reference is discarded. It looks as if
>> reference counting is the only feasible way, which isn't what I want.
>> I'd need something that the garbage collector cooperated with. If it
>> hasn't been built already, then a customized approach is
>> better...which doesn't involve weak pointers.)
>>
>> But others points are well taken. A weak reference wouldn't be enough
>> by itself, I'd also need to have the garbage collector prefer to
>> collect stale objects. So what I'll probably do is accompany each item
>> in the structure (Not a struct, but also not just a class. More a
>> collection of structs and classes that work together.) with a sequence
>> counter that's episodically advanced. Then at some point I decide that
>> anything that hasn't been touched since some particular sequence is to
>> be freed. And *THEN* I go back through the entire RAM resident
>> structure and either free the items or subtract the current sequence
>> counter value from them, and then reset the sequence counter to the
>> largest remaining value + 1. When freeing, check if the state needs to
>> be saved.
>
> Hmmmm... I'd just stay with reference counting. If you leave scope with
> a struct it's destructor is handled (and refcounting), if it's a class,
> then it remains 'stale' until the GC collects it, then refcounting is
> updated as everything else is destroyed.
I don't think reference counting would work for my purposes, which is why I wanted a "weak pointer".  There will usually be many live references at the time I need to release an item.  If I'd had weak pointers I could have made almost all of them weak pointers.  I want to release things that are stale, not things that are unreferenced.  I was trying to avoid needing to cycle through the entire data structure.

>
>> That's the rough idea. Weak pointers will make it easier, but no big
>> deal either way. I may follow your guidance on implementing them, but
>> what I was really hoping for was weak references. Used sort of like:
>>
>> Item item = weak(new Item(params));
>
> Best if you don't... A sorta nice idea, but we don't need to be
> duplicating more of C++'s (boost's?) mistakes.
Well, it doesn't exist, so no problem.  But what about that is a mistake?  I haven't used enough C++ to know.  (My recent background is Python, Java, and Ruby...C is way in the background, and the last time I used C++ heavily was before the STL.)

>
>> With Item being a class. Clearly what weak() returned would need to be
>> an instance of a descendant class of Item. Then I could simply
>> maintain a LRU cache of Items, and clear out the old ones, but not
>> free them before the garbage collector decided it was time. The
>> approach I'm now planning on using frees things in a way that is much
>> less sensitive (I suspect) to current memory pressures. OTOH, I'd need
>> to ensure that Items saved their state (if necessary) before being
>> freed anyway. Handling it "externally" allows this to be ensured by
>> something more definite in timing than a destructor.
>>
>> Still, your design for a weak pointer makes it quite easy for an
>> instance to be created if missing. That may be enough advantage that
>> it would be the best way to proceed.
>>
>> Also, I'm really not enamored of template functions, though I am
>> certainly aware of their power. So the design you offered is one that
>> I would prefer over a templated one. (Besides, each class/struct that
>> I want to have a weak pointer to would have a different way of
>> constructing missing versions. So the template would need so much
>> customization, that it wouldn't be much benefit.)
>
> The template one, (should it have worked) would have had a signature of:
> T* weakPtr(T, V...)(ref T* ptr, V args); //V being default arguments (if
> any)
>
> How it would need to be more customizable I don't know; Alternatively a
> delegate could have been included making only a lamba needed in key
> locations.
Some of the items have heavy data inclusions, and they would get passed around as pointers, some of them are light, and they'd be structs.  Some could be automatically calculated from the creation parameters, others need to be read in from files.  And these cause different values returned from functions doing abstractly the same job.

>
>> The only things I dislike about you design are things I dislike about
>> all designs that use pointers...and I *did* ask for such a design.
>
> Yes you did. But since you can't use 'ref' as part of a variable's
> signature (outside of function declarations) you'd have to to use
> pointers instead. Just try to be as safe as possible. With pointer
> arithmetic being unsafe and mostly unused (unneeded due to the arrays)
> it's just a matter of allocating and accessing the pointer that's safe
> (or as much as using classes anyways).
If I'd been able to have actual weak pointers, that would be worthwhile.  As it is...I'm going to use references (i.e., class instances) as the external interfaces.  Internally, things will often be quite a bit different.  (Yeah, maybe I should say handle rather than reference.  I'm not sure.  I *am* sure I like the syntax of class instances a lot better than I do pointers to structs.)

On Sunday, 13 January 2013 at 00:37:07 UTC, Charles Hixson wrote:
> P.S.:  Perhaps you're thinking of unions with a combination of floats and pointers in them.  Legal, I think, but still a really bad idea.

 No; Unless you flag every relevant portion as pointer/non pointer, then a float can get confused as a pointer. To my understanding, things aren't tagged that way.

On Sunday, 13 January 2013 at 00:56:59 UTC, Charles Hixson wrote:
> I don't think reference counting would work for my purposes, which is why I wanted a "weak pointer".  There will usually be many live references at the time I need to release an item.  If I'd had weak pointers I could have made almost all of them weak pointers.  I want to release things that are stale, not things that are unreferenced.  I was trying to avoid needing to cycle through the entire data structure.

 Just off hand I think I'm recalling that in C++ the weak pointers were related and part of the main reference counting; An allocated (heap) control block handled the references and memory. If then you were to extended a second number to represent weak pointers, they would be for when the control block could free itself (vs the data with the strong pointers).

 But the GC wouldn't be part of it most likely unless the control block itself was leaked in it's entirety. That might be safe (mostly); In those cases you'd have to test the control block which would tell you if the pointer was still valid or not.

 Well I got my hands full so I won't be the to touch this possible implementation; At least for a while.

On 11-01-2013 19:15, Charles Hixson wrote:
> I was looking for a way to create a weak reference to either a struct or
> a class.  I need to be able to use it to automatically generate an
> active reference on access.  (I intend to do this by rolling in data
> from a file.)
>
> Any guidance as to where I should look?

https://github.com/lycus/mci/blob/master/src/mci/core/weak.d

It has some caveats (see the comments).

-- 
Alex Rønne Petersen
alex@alexrp.com / alex@lycus.org
http://lycus.org

On Saturday, 12 January 2013 at 11:27:03 UTC, Era Scarecrow wrote:
> On Saturday, 12 January 2013 at 10:58:23 UTC, thedeemon wrote:
>> So the runtime is aware of weak pointers and clears them to "empty" state when pointed value dies. I don't see yet how it can be implemented in D without patching its GC.
>
>  There's that, but also what about what if the pointer wasn't a pointer to the data at all?

In OCaml, which I was talking about, GC is precise, so there are no false pointers. This is done by tagging: pointers have least significant bit being 0 and non-pointers all have it 1, so native ints in OCaml are one bit smaller than usual (31-bit on x86).

On 01/12/2013 09:24 PM, Alex Rønne Petersen wrote:
> On 11-01-2013 19:15, Charles Hixson wrote:
>> I was looking for a way to create a weak reference to either a struct or
>> a class. I need to be able to use it to automatically generate an
>> active reference on access. (I intend to do this by rolling in data
>> from a file.)
>>
>> Any guidance as to where I should look?
>
> https://github.com/lycus/mci/blob/master/src/mci/core/weak.d
>
> It has some caveats (see the comments).
>

It certainly does have some caveats.  And according to at least one of them it may break badly in the future from a plausible compiler change.

Well, it's what I asked for, but using something that can be expected to break without warning.  I want to write code that I can forget after I test.

It would have been a great convenience, but that's a bit too high a price to pay for a convenience.

On 14-01-2013 00:18, Charles Hixson wrote:
> On 01/12/2013 09:24 PM, Alex Rønne Petersen wrote:
>> On 11-01-2013 19:15, Charles Hixson wrote:
>>> I was looking for a way to create a weak reference to either a struct or
>>> a class. I need to be able to use it to automatically generate an
>>> active reference on access. (I intend to do this by rolling in data
>>> from a file.)
>>>
>>> Any guidance as to where I should look?
>>
>> https://github.com/lycus/mci/blob/master/src/mci/core/weak.d
>>
>> It has some caveats (see the comments).
>>
>
> It certainly does have some caveats.  And according to at least one of
> them it may break badly in the future from a plausible compiler change.
>
> Well, it's what I asked for, but using something that can be expected to
> break without warning.  I want to write code that I can forget after I
> test.
>
> It would have been a great convenience, but that's a bit too high a
> price to pay for a convenience.
>

Well, I'm not going to lie: You cannot implement weak references in D any other way. The GC just isn't helpful enough.

The only way this code could break is if D ever gets a copying or compacting GC. The chances of that happening, ever, are practically nil because such GCs are extremely impractical in natively-compiled systems languages.

Otherwise all you have to be aware of in your code is that the referenced object cannot be used as a mutex in synchronized statements and cannot have custom dispose events (finalizers).

(Note, the last part there doesn't mean that the referenced object can't have a finalizer declared via `~this()` - it just means that you should not dynamically attach finalizers to it with rt_attachDisposeEvent().)

-- 
Alex Rønne Petersen
alex@alexrp.com / alex@lycus.org
http://lycus.org

On 01/13/2013 09:01 PM, Alex Rønne Petersen wrote:
> On 14-01-2013 00:18, Charles Hixson wrote:
>> On 01/12/2013 09:24 PM, Alex Rønne Petersen wrote:
>>> On 11-01-2013 19:15, Charles Hixson wrote:
>>>> I was looking for a way to create a weak reference to either a
>>>> struct or
>>>> a class. I need to be able to use it to automatically generate an
>>>> active reference on access. (I intend to do this by rolling in data
>>>> from a file.)
>>>>
>>>> Any guidance as to where I should look?
>>>
>>> https://github.com/lycus/mci/blob/master/src/mci/core/weak.d
>>>
>>> It has some caveats (see the comments).
>>>
>>
>> It certainly does have some caveats. And according to at least one of
>> them it may break badly in the future from a plausible compiler change.
>>
>> Well, it's what I asked for, but using something that can be expected to
>> break without warning. I want to write code that I can forget after I
>> test.
>>
>> It would have been a great convenience, but that's a bit too high a
>> price to pay for a convenience.
>>
>
> Well, I'm not going to lie: You cannot implement weak references in D
> any other way. The GC just isn't helpful enough.
>
> The only way this code could break is if D ever gets a copying or
> compacting GC. The chances of that happening, ever, are practically nil
> because such GCs are extremely impractical in natively-compiled systems
> languages.
O.  To me that sounded like a likely thing for a garbage collector to have added.
>
> Otherwise all you have to be aware of in your code is that the
> referenced object cannot be used as a mutex in synchronized statements
> and cannot have custom dispose events (finalizers).
Ok, but that's a thing that's already in place.  I was worried about code that I wrote breaking with a new version of the compiler, perhaps a few years after I'd stopped thinking about it.
>
> (Note, the last part there doesn't mean that the referenced object can't
> have a finalizer declared via `~this()` - it just means that you should
> not dynamically attach finalizers to it with rt_attachDisposeEvent().)
Thanks.  That's an important clarification, as I would otherwise have misunderstood it.
>