I was looking for a way to create a weak reference to either a struct or a class.  I need to be able to use it to automatically generate an active reference on access.  (I intend to do this by rolling in data from a file.)

Any guidance as to where I should look?

On Friday, 11 January 2013 at 18:22:30 UTC, Charles Hixson wrote:
> I was looking for a way to create a weak reference to either a struct or a class.  I need to be able to use it to automatically generate an active reference on access.  (I intend to do this by rolling in data from a file.)
>
> Any guidance as to where I should look?

 Looks like you'll have to create the access yourself, you can forward access to the new ptr via a property call by the function you name. This works.

  struct S {
    int x;
    S* _s;  //for weak ptr.

    //constructor details so we can show it's
    //differences in the lower prints.
    S* s(int x) {
      if (_s is null)
        _s = new S(x);

      return _s;
    }

    void print(){writeln(this);}
  }

  S test;
  test.print();
  test.s(10).print();
  test.print();


 I've tried making a weakPtr template function, however it complains about a nested function call (perhaps cause it was within a unittest); Shows the problem(s) on that side.

 It's possible to write a mixin that could do those details for you.

On Friday, 11 January 2013 at 21:12:40 UTC, Era Scarecrow wrote:
>  I've tried making a weakPtr template function, however it complains about a nested function call (perhaps cause it was within a unittest); Shows the problem(s) on that side.

Honest question: How can you have a weak pointer in a language that is garbage collected?

If you mean weak pointer to manually allocated data, wouldn't that imply also already having a shared pointer? AFAIK D doesn't really have that (though it does have RefCounted).

On Friday, 11 January 2013 at 22:07:45 UTC, monarch_dodra wrote:
> Honest question: How can you have a weak pointer in a language that is garbage collected?

I beleive the OP means something like pythons weakref (http://docs.python.org/2/library/weakref.html) that is qoute:

A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else.

I would like to have no how to do something like this in D aswell as it could be handy.

 I had the impression in the original text you wanted to auto allocate memory when accessing the field, not as described here. Might be time to re-watch the remainder of the computer science lectures.

On Friday, 11 January 2013 at 23:51:04 UTC, Nekroze wrote:
> A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else.

 How very C++'ish of you. With a weak reference as that, memory safety becomes a much larger issue (and you likely can only make @system code, I surely wouldn't mark it @trusted or @safe).

 It sorta sounds like what slices are in D; Although D's GC won't free memory unless it's sure it isn't in use (rather than the other way around). You might get the desired effect by removing areas from GC's scanning (for active references), but use at your own risk.

 With that in mind I have the impression you won't find weak pointers in D (or phobos anyways); Or if they are present, then it simply lets the object go once the strong pointers are gone and lets the GC pick it up later (which then weak pointers are pointless as GC does that anyways).

On Saturday, January 12, 2013 00:51:02 Nekroze wrote:
> On Friday, 11 January 2013 at 22:07:45 UTC, monarch_dodra wrote:
> > Honest question: How can you have a weak pointer in a language that is garbage collected?
> 
> I beleive the OP means something like pythons weakref (http://docs.python.org/2/library/weakref.html) that is qoute:
> 
> A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else.
> 
> I would like to have no how to do something like this in D aswell as it could be handy.

That doesn't fly with the GC, because then you can have a reference to memory that's been freed - though I suppose that it could work if the weak reference were set to null when the normal reference was collected.

There's no reason that you couldn't do that with ref-counted objects, so perhaps something like that could be done with std.typecons.RefCounted, but we don't currently have anything like that, and you'd still need to make the weak ref null when the normal ref is destroyed, or you'd have memory safety issues.

- Jonathan M Davis

On 01/11/2013 01:12 PM, Era Scarecrow wrote:
> On Friday, 11 January 2013 at 18:22:30 UTC, Charles Hixson wrote:
>> I was looking for a way to create a weak reference to either a struct
>> or a class. I need to be able to use it to automatically generate an
>> active reference on access. (I intend to do this by rolling in data
>> from a file.)
>>
>> Any guidance as to where I should look?
>
> Looks like you'll have to create the access yourself, you can forward
> access to the new ptr via a property call by the function you name. This
> works.
>
> struct S {
> int x;
> S* _s; //for weak ptr.
>
> //constructor details so we can show it's
> //differences in the lower prints.
> S* s(int x) {
> if (_s is null)
> _s = new S(x);
>
> return _s;
> }
>
> void print(){writeln(this);}
> }
>
> S test;
> test.print();
> test.s(10).print();
> test.print();
>
>
> I've tried making a weakPtr template function, however it complains
> about a nested function call (perhaps cause it was within a unittest);
> Shows the problem(s) on that side.
>
> It's possible to write a mixin that could do those details for you.

Thanks. That looks quite useful.  OTOH, I don't see how the pointer allows an item to be freed.  You probably meant that this was a framework to start development from.  And it does show a way to auto initialize a null pointer (but that's not where I'm hung up...rather on how to automatically null the pointer when the last valid reference is discarded.  It looks as if reference counting is the only feasible way, which isn't what I want.  I'd need something that the garbage collector cooperated with.  If it hasn't been built already, then a customized approach is better...which doesn't involve weak pointers.)

But others points are well taken.  A weak reference wouldn't be enough by itself, I'd also need to have the garbage collector prefer to collect stale objects.  So what I'll probably do is accompany each item in the structure (Not a struct, but also not just a class. More a collection of structs and classes that work together.) with a sequence counter that's episodically advanced.  Then at some point I decide that anything that hasn't been touched since some particular sequence is to be freed.  And *THEN* I go back through the entire RAM resident structure and either free the items or subtract the current sequence counter value from them, and then reset the sequence counter to the largest remaining value + 1.  When freeing, check if the state needs to be saved.

That's the rough idea.  Weak pointers will make it easier, but no big deal either way. I may follow your guidance on implementing them, but what I was really hoping for was weak references.  Used sort of like:

Item item = weak(new Item(params));

With Item being a class.  Clearly what weak() returned would need to be an instance of a descendant class of Item.  Then I could simply maintain a LRU cache of Items, and clear out the old ones, but not free them before the garbage collector decided it was time.  The approach I'm now planning on using frees things in a way that is much less sensitive (I suspect) to current memory pressures.  OTOH, I'd need to ensure that Items saved their state (if necessary) before being freed anyway. Handling it "externally" allows this to be ensured by something more definite in timing than a destructor.

Still, your design for a weak pointer makes it quite easy for an instance to be created if missing.  That may be enough advantage that it would be the best way to  proceed.

Also, I'm really not enamored of template functions, though I am certainly aware of their power.  So the design you offered is one that I would prefer over a templated one.  (Besides, each class/struct that I want to have a weak pointer to would have a different way of constructing missing versions.  So the template would need so much customization, that it wouldn't be much benefit.)  The only things I dislike about you design are things I dislike about all designs that use pointers...and I *did* ask for such a design.

On Saturday, 12 January 2013 at 04:41:00 UTC, Charles Hixson wrote:
> Thanks. That looks quite useful.  OTOH, I don't see how the pointer allows an item to be freed.  You probably meant that this was a framework to start development from.

 Correct. To create an object if it wasn't there when trying to access it basically. That's what I originally thought you wanted; nothing about freeing it.

> And it does show a way to auto initialize a null pointer (but that's not where I'm hung up...rather on how to automatically null the pointer when the last valid reference is discarded. It looks as if reference counting is the only feasible way, which isn't what I want.  I'd need something that the garbage collector cooperated with.  If it hasn't been built already, then a customized approach is better...which doesn't involve weak pointers.)
>
> But others points are well taken.  A weak reference wouldn't be enough by itself, I'd also need to have the garbage collector prefer to collect stale objects.  So what I'll probably do is accompany each item in the structure (Not a struct, but also not just a class. More a collection of structs and classes that work together.) with a sequence counter that's episodically advanced.  Then at some point I decide that anything that hasn't been touched since some particular sequence is to be freed.  And *THEN* I go back through the entire RAM resident structure and either free the items or subtract the current sequence counter value from them, and then reset the sequence counter to the largest remaining value + 1. When freeing, check if the state needs to be saved.

 Hmmmm... I'd just stay with reference counting. If you leave scope with a struct it's destructor is handled (and refcounting), if it's a class, then it remains 'stale' until the GC collects it, then refcounting is updated as everything else is destroyed.

> That's the rough idea.  Weak pointers will make it easier, but no big deal either way. I may follow your guidance on implementing them, but what I was really hoping for was weak references.  Used sort of like:
>
> Item item = weak(new Item(params));

 Best if you don't... A sorta nice idea, but we don't need to be duplicating more of C++'s (boost's?) mistakes.

> With Item being a class.  Clearly what weak() returned would need to be an instance of a descendant class of Item.  Then I could simply maintain a LRU cache of Items, and clear out the old ones, but not free them before the garbage collector decided it was time.  The approach I'm now planning on using frees things in a way that is much less sensitive (I suspect) to current memory pressures.  OTOH, I'd need to ensure that Items saved their state (if necessary) before being freed anyway. Handling it "externally" allows this to be ensured by something more definite in timing than a destructor.
>
> Still, your design for a weak pointer makes it quite easy for an instance to be created if missing.  That may be enough advantage that it would be the best way to  proceed.
>
> Also, I'm really not enamored of template functions, though I am certainly aware of their power.  So the design you offered is one that I would prefer over a templated one.  (Besides, each class/struct that I want to have a weak pointer to would have a different way of constructing missing versions.  So the template would need so much customization, that it wouldn't be much benefit.)

 The template one, (should it have worked) would have had a signature of:
 T* weakPtr(T, V...)(ref T* ptr, V args); //V being default arguments (if any)

 How it would need to be more customizable I don't know; Alternatively a delegate could have been included making only a lamba needed in key locations.

> The only things I dislike about you design are things I dislike about all designs that use pointers...and I *did* ask for such a design.

 Yes you did. But since you can't use 'ref' as part of a variable's signature (outside of function declarations) you'd have to to use pointers instead. Just try to be as safe as possible. With pointer arithmetic being unsafe and mostly unused (unneeded due to the arrays) it's just a matter of allocating and accessing the pointer that's safe (or as much as using classes anyways).

On Saturday, 12 January 2013 at 02:49:40 UTC, Jonathan M Davis wrote:

> That doesn't fly with the GC, because then you can have a reference to memory
> that's been freed - though I suppose that it could work if the weak reference
> were set to null when the normal reference was collected.

That does fly with GC in languages that are aware of weak pointers. Like OCaml which has a moving GC. From its manual:
"A weak pointer is a value that the garbage collector may erase whenever the value is not used any more (through normal pointers) by the program. Note that finalisation functions are run after the weak pointers are erased.
A weak pointer is said to be full if it points to a value, empty if the value was erased by the GC."
http://caml.inria.fr/pub/docs/manual-ocaml/libref/Weak.html

So the runtime is aware of weak pointers and clears them to "empty" state when pointed value dies. I don't see yet how it can be implemented in D without patching its GC.

On Saturday, 12 January 2013 at 10:58:23 UTC, thedeemon wrote:
> So the runtime is aware of weak pointers and clears them to "empty" state when pointed value dies. I don't see yet how it can be implemented in D without patching its GC.

 There's that, but also what about what if the pointer wasn't a pointer to the data at all? What it if was a part of a float that determined a calculation for your taxes? Or a series of flags that happen to make that magic number? If they get 'nulled' by the GC, I'd call that a definite bug and problem; Unless the GC can tell apart what is and isn't a pointer to data.