Hi all, I'm a C++ programmer trying to decide whether to switch my main focus to D, and so I'm working on a pet project using it. So far I really like some of the practical aspects of the language (built-in contracts are great, the metaprogramming is very accessible, and I can't enough of these compile speeds!), but I keep finding myself frustrated by what seem like expressiveness limitations (unless, as I hope, they are just examples of my newbieness shining through). Case in point:

In an attempt to work around one apparent limitation (previously asked about here http://forum.dlang.org/thread/eizmagtimvetoganawrr@forum.dlang.org) I came up with an idea which would require storing internal points in a struct. A very stripped-down but illustrative example would be something like this:

[code]
struct Foo
{
    invariant
    {
        assert(m_this == &this);
    }

    @disable(this);

    this(/* arguments to populate stuff */)
    {
        m_this = &this;
        /* ... populate stuff ... */
    }

    this(this)
    {
        m_this = &this;
        /* ... do more stuff ... */
    }

    private:
        Foo* m_this;
        /* ... stuff ... */
}
[/code]

This is just a piece of what I am doing, if you are wondering why I am bothering to save a pointer to this. However, I was doing some reading on D and came across a section in TDPL which said internal pointers are verboten because objects must be relocatable. Does this mean my example is invalid (e.g., the invariant will not hold in all circumstances)? If it is invalid, does that mean there are circumstances under which the post-blit constructor can be elided when performing a copy or copy-like operation (such as a move)? I've been treating it like a sort of copy-constructor that lacks visibility on the copied-from object, but maybe that's a mistake...

On 01/29/2016 05:07 PM, Matt Elkins wrote:

>      this(/* arguments to populate stuff */)
>      {
>          m_this = &this;
>          /* ... populate stuff ... */
>      }

> a section in TDPL which said internal pointers are
> verboten because objects must be relocatable. Does this mean my example
> is invalid

Yes, D explicitly bans internal pointers.

> does that mean there are circumstances under which the
> post-blit constructor can be elided when performing a copy or copy-like
> operation (such as a move)?

Definitely so. Rvalues are moved around all the time. The following program has two rvalue moves without calling post-blits or destructors.

struct Foo {
    this(this) {
        assert(false);    // not expected to be called in this program
    }
}

Foo makeFoo() {
    return Foo();
}

void takesFoo(Foo foo) {
}

void main() {
    Foo foo;
    foo = makeFoo();    // post-blit not called
    takesFoo(Foo());    // post-blit not called
}

Ali

On Saturday, 30 January 2016 at 01:18:33 UTC, Ali Çehreli wrote:
> Definitely so. Rvalues are moved around all the time. The following program has two rvalue moves without calling post-blits or destructors.

Oi, that makes life tough. Ok, I'll figure something else out, then...

Thanks for the response!

On Sat, Jan 30, 2016 at 01:21:27AM +0000, Matt Elkins via Digitalmars-d-learn wrote:
> On Saturday, 30 January 2016 at 01:18:33 UTC, Ali Çehreli wrote:
> >Definitely so. Rvalues are moved around all the time. The following program has two rvalue moves without calling post-blits or destructors.
> 
> Oi, that makes life tough. Ok, I'll figure something else out, then...
[...]

Keep in mind that D structs are conceptually different from C++ structs (even if they are similarly implemented). D structs are supposed to be value types with POD-like semantics; so when passing structs around they are bit-copied into the destination and then the postblit method (this(this)) is called to "patch up" the copy. This is unlike in C++ where you have copy ctors and dtors and operator=() to manage copying.

Because there are no copy ctors, having internal pointers can be dangerous, since structs can move around in memory without any warning (e.g., returning a struct from a function generally involves copying it from the callee's stack frame into a local variable in the caller's stack frame).

If you need something with internal pointers, you might want to consider classes instead. Either that, or be sure to allocate your structs on the heap instead, and work with pointers instead of the struct values directly. (Note that this is still risky, since somebody might dereference the pointer and get a stack copy of the struct, which will cause problems when it then gets passed around.)


T

-- 
I am a consultant. My job is to make your job redundant. -- Mr Tom

On Saturday, 30 January 2016 at 01:28:54 UTC, H. S. Teoh wrote:
> On Sat, Jan 30, 2016 at 01:21:27AM +0000, Matt Elkins via Digitalmars-d-learn wrote:
>> On Saturday, 30 January 2016 at 01:18:33 UTC, Ali Çehreli wrote:
>> >Definitely so. Rvalues are moved around all the time. The following program has two rvalue moves without calling post-blits or destructors.
>> 
>> Oi, that makes life tough. Ok, I'll figure something else out, then...
> [...]
>
> Keep in mind that D structs are conceptually different from C++ structs (even if they are similarly implemented). D structs are supposed to be value types with POD-like semantics; so when passing structs around they are bit-copied into the destination and then the postblit method (this(this)) is called to "patch up" the copy. This is unlike in C++ where you have copy ctors and dtors and operator=() to manage copying.
>
> Because there are no copy ctors, having internal pointers can be dangerous, since structs can move around in memory without any warning (e.g., returning a struct from a function generally involves copying it from the callee's stack frame into a local variable in the caller's stack frame).
>
> If you need something with internal pointers, you might want to consider classes instead. Either that, or be sure to allocate your structs on the heap instead, and work with pointers instead of the struct values directly. (Note that this is still risky, since somebody might dereference the pointer and get a stack copy of the struct, which will cause problems when it then gets passed around.)
>
>
> T

Yeah, but the whole point of what I am doing is to avoid using the heap; I can think of several ways to implement this if I relax that restriction :). I'm basically trying to make C++'s std::unique_ptr for resource handles, a thin wrapper which ensures resource cleanup and allows moving the handle. Since I'm putting it in my lowest-level/most-generic library with no visibility on how it gets used, I want it very lightweight (ideally zero-cost, like I can do in C++11, or at least low-cost [sans heap] like I could do in C++98) so that I can use it with the broadest range of higher-level applications.

On 1/29/16 8:07 PM, Matt Elkins wrote:
> [snip]
> on D and came across a section in TDPL which said internal pointers are
> verboten because objects must be relocatable. Does this mean my example
> is invalid (e.g., the invariant will not hold in all circumstances)? If
> it is invalid, does that mean there are circumstances under which the
> post-blit constructor can be elided when performing a copy or copy-like
> operation (such as a move)? I've been treating it like a sort of
> copy-constructor that lacks visibility on the copied-from object, but
> maybe that's a mistake...

No, you cannot have internal pointers. But...

I figured out a way to have them. You just have to guarantee you don't copy the actual "pointer" out of the struct:

https://forum.dlang.org/post/mk5k4l$s5r$1@digitalmars.com

-Steve

On Saturday, 30 January 2016 at 02:09:55 UTC, Steven Schveighoffer wrote:
> I figured out a way to have them. You just have to guarantee you don't copy the actual "pointer" out of the struct:
>
> https://forum.dlang.org/post/mk5k4l$s5r$1@digitalmars.com

Unfortunately, that won't work for what I was trying to do. The stuff I elided in the comments were more pointers to other Foo instances, used to create a linked-list (of stack-allocated objects); these would still break under the conditions Ali described. I was only storing the this pointer so that blitted objects could deduce where they came from (trying to turn the post-blit constructor into a copy-constructor).

Thanks, though. I'm thinking that maybe D just can't express these semantics without substantial overhead. While somewhat disappointing (I came into D with stars in my eyes :)), it's not enough by itself to make me go back to C++, at least not just yet. Not when I can just use a few static ifs to do what previously required careful template crafting that I wouldn't understand 3 months later. On the other hand, I'm falling behind on my library books since I no longer have any time for reading during compilations ;).

On 1/29/16 9:35 PM, Matt Elkins wrote:
> On Saturday, 30 January 2016 at 02:09:55 UTC, Steven Schveighoffer wrote:
>> I figured out a way to have them. You just have to guarantee you don't
>> copy the actual "pointer" out of the struct:
>>
>> https://forum.dlang.org/post/mk5k4l$s5r$1@digitalmars.com
>
> Unfortunately, that won't work for what I was trying to do. The stuff I
> elided in the comments were more pointers to other Foo instances, used
> to create a linked-list (of stack-allocated objects); these would still
> break under the conditions Ali described. I was only storing the this
> pointer so that blitted objects could deduce where they came from
> (trying to turn the post-blit constructor into a copy-constructor).

Ah, so you were actually counting on the postblit to have an *invalid* pointer to begin with :) Yeah, that isn't going to work. In D, it's legal to do something like memcpy struct data (with no postblit), and this is done quite often in many places because of that.

> Thanks, though. I'm thinking that maybe D just can't express these
> semantics without substantial overhead. While somewhat disappointing (I
> came into D with stars in my eyes :)), it's not enough by itself to make
> me go back to C++, at least not just yet. Not when I can just use a few
> static ifs to do what previously required careful template crafting that
> I wouldn't understand 3 months later. On the other hand, I'm falling
> behind on my library books since I no longer have any time for reading
> during compilations ;).

There are some really smart people who frequent these forums, if you post your actual use case, you may get an answer that you hadn't thought of.

I saw you were trying to implement something like std::unique_ptr? There is http://dlang.org/phobos/std_typecons.html#.Unique

Not sure if it helps.

-Steve

On Saturday, 30 January 2016 at 03:00:11 UTC, Steven Schveighoffer wrote:
> There are some really smart people who frequent these forums, if you post your actual use case, you may get an answer that you hadn't thought of.

Yeah, I tried that first (on the general forum, since at the time I didn't know about this one).

> I saw you were trying to implement something like std::unique_ptr? There is http://dlang.org/phobos/std_typecons.html#.Unique

std.typecons.Unique seems to require heap allocation, which makes it a far cry from std::unique_ptr.

On 1/29/16 10:13 PM, Matt Elkins wrote:
> On Saturday, 30 January 2016 at 03:00:11 UTC, Steven Schveighoffer wrote:
>> There are some really smart people who frequent these forums, if you
>> post your actual use case, you may get an answer that you hadn't
>> thought of.
>
> Yeah, I tried that first (on the general forum, since at the time I
> didn't know about this one).

Ah, didn't notice that. You are likely to get more eyes there, so that is good.

>
>> I saw you were trying to implement something like std::unique_ptr?
>> There is http://dlang.org/phobos/std_typecons.html#.Unique
>
> std.typecons.Unique seems to require heap allocation, which makes it a
> far cry from std::unique_ptr.

I admit I'm not familiar with either's implementation. I just know that they have similar functions.

-Steve