View mode: basic / threaded / horizontal-split · Log in · Help
January 23, 2012
Re: C++ pimpl
On Mon, 23 Jan 2012 01:33:47 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:

> On 01/23/2012 01:07 AM, so wrote:
>> On Mon, 23 Jan 2012 01:39:23 +0200, so <so@so.so> wrote:
>>
>>> I have been asking that for some time now, i am afraid you won't get
>>> much of an audience.
>>> You can get rid of both additional allocation and indirection but it
>>> is not pretty. We could definitely use some help/sugar on this.
>>>
>>> http://www.artima.com/cppsource/backyard3.html
>>
>> http://www.digitalmars.com/d/archives/digitalmars/D/Implementation_hiding_139625.html
>>
>>
>> There is another issue Walter forgot to mention in the article.
>> I think there might be a way but looks like we also loose the  
>> "destructor".
>> Which means we are all the way back to the
>> http://en.wikipedia.org/wiki/Opaque_pointer.
>>
>> Walter, is there a way to get around destructor limitation?
>
> This seems to work.
>
> a.di:
>
> final class A{
>      private this();
>      static A factory();
>      T1 publicMember1(int x);
>      T2 publicMember2(float y);
>      T3 publicField;
>      // ...
> }
>
> a.d:
>
> class A{
>      static A factory(){return new A;}
>      T1 publicMember1(int x){ ... }
>      T2 publicMember2(float y){ ... }
>      T3 publicField;
>      // ...
> private:
>      T1 field1;
>      T2 field2;
> }
>
This will even work with plain new/this, as the allocation is done
in the constructor. The difficulty is that you can't build inheritable  
pimpls.
January 23, 2012
Re: C++ pimpl
On Mon, 23 Jan 2012 02:49:39 +0200, Martin Nowak <dawg@dawgfoto.de> wrote:

> This will even work with plain new/this, as the allocation is done
> in the constructor. The difficulty is that you can't build inheritable  
> pimpls.

No, you can't define a struct without providing all the fields.
Inheritance has zero importance on this one. If you will use inheritance  
why would you need things like this in the first place?
January 23, 2012
Re: C++ pimpl
On 01/23/2012 01:49 AM, Martin Nowak wrote:
> On Mon, 23 Jan 2012 01:33:47 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>> On 01/23/2012 01:07 AM, so wrote:
>>> On Mon, 23 Jan 2012 01:39:23 +0200, so <so@so.so> wrote:
>>>
>>>> I have been asking that for some time now, i am afraid you won't get
>>>> much of an audience.
>>>> You can get rid of both additional allocation and indirection but it
>>>> is not pretty. We could definitely use some help/sugar on this.
>>>>
>>>> http://www.artima.com/cppsource/backyard3.html
>>>
>>> http://www.digitalmars.com/d/archives/digitalmars/D/Implementation_hiding_139625.html
>>>
>>>
>>>
>>> There is another issue Walter forgot to mention in the article.
>>> I think there might be a way but looks like we also loose the
>>> "destructor".
>>> Which means we are all the way back to the
>>> http://en.wikipedia.org/wiki/Opaque_pointer.
>>>
>>> Walter, is there a way to get around destructor limitation?
>>
>> This seems to work.
>>
>> a.di:
>>
>> final class A{
>> private this();
>> static A factory();
>> T1 publicMember1(int x);
>> T2 publicMember2(float y);
>> T3 publicField;
>> // ...
>> }
>>
>> a.d:
>>
>> class A{
>> static A factory(){return new A;}
>> T1 publicMember1(int x){ ... }
>> T2 publicMember2(float y){ ... }
>> T3 publicField;
>> // ...
>> private:
>> T1 field1;
>> T2 field2;
>> }
>>
> This will even work with plain new/this, as the allocation is done
> in the constructor. The difficulty is that you can't build inheritable
> pimpls.

Ah, good to know. One solution would be to just add some dummy space to 
the class declaration (private void[256] dummy=void) so that the 
implementation can use it for private members. (union{void[256] dummy; 
struct{/*private_members*/}}) Of course, this is potentially wasteful.
January 23, 2012
Re: C++ pimpl
> 
> What you're asking for is dynamic typing. You can achieve something
> equivalent using opDispatch and std.variant.

I am not sure I understood you correctly. It is dynamic typing more or less, 
just as polymorphic types like classes implement dynamic types, because you 
can change the implementation at runtime, by assigning an object of another 
derived class. 

What I am asking for is hiding the implemenation of class completely, not only 
to the programmer but also to the compiler.
January 23, 2012
Re: C++ pimpl
> Ah, good to know. One solution would be to just add some dummy space to  
> the class declaration (private void[256] dummy=void) so that the  
> implementation can use it for private members. (union{void[256] dummy;  
> struct{/*private_members*/}}) Of course, this is potentially wasteful.

This is what I did for the thread pimpl, but it requires you to manually
keep the size in sync with the implementation.
January 23, 2012
Re: C++ pimpl
On Monday, January 23, 2012 02:19:35 AM Timon Gehr wrote:
> On 01/23/2012 01:49 AM, Martin Nowak wrote:
> > On Mon, 23 Jan 2012 01:33:47 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:
> >> On 01/23/2012 01:07 AM, so wrote:
> >>> On Mon, 23 Jan 2012 01:39:23 +0200, so <so@so.so> wrote:
> >>>> I have been asking that for some time now, i am afraid you won't
> >>>> get
> >>>> much of an audience.
> >>>> You can get rid of both additional allocation and indirection but
> >>>> it
> >>>> is not pretty. We could definitely use some help/sugar on this.
> >>>> 
> >>>> http://www.artima.com/cppsource/backyard3.html
> >>> 
> >>> http://www.digitalmars.com/d/archives/digitalmars/D/Implementation_h
> >>> iding_139625.html
> >>> 
Thanks for the links, it was a good read.
> >>> 
> >>> 
> >>> There is another issue Walter forgot to mention in the article.
> >>> I think there might be a way but looks like we also loose the
> >>> "destructor".
> >>> Which means we are all the way back to the
> >>> http://en.wikipedia.org/wiki/Opaque_pointer.
> >>> 
> >>> Walter, is there a way to get around destructor limitation?
What's the destructor limitation?
> >> 
> >> This seems to work.
> >> 
> >> a.di:
> >> 
> >> final class A{
> >> private this();
> >> static A factory();
> >> T1 publicMember1(int x);
> >> T2 publicMember2(float y);
> >> T3 publicField;
> >> // ...
> >> }
> >> 
> >> a.d:
> >> 
> >> class A{
> >> static A factory(){return new A;}
> >> T1 publicMember1(int x){ ... }
> >> T2 publicMember2(float y){ ... }
> >> T3 publicField;
> >> // ...
> >> private:
> >> T1 field1;
> >> T2 field2;
> >> }
> > 
> > This will even work with plain new/this, as the allocation is done
> > in the constructor. The difficulty is that you can't build inheritable
> > pimpls.
Hmm, I see some problems with this approach: It is not only impossible to 
derive from this class without using the di file with the full implementation 
(which might be acceptable)but you can't even use derived classes by an A 
reference, because the compiler would not use the vtbl because it assumes 
there are no derived classes. Which is not acceptable, in my opinion.

> 
> Ah, good to know. One solution would be to just add some dummy space to
> the class declaration (private void[256] dummy=void) so that the
> implementation can use it for private members. (union{void[256] dummy;
> struct{/*private_members*/}}) Of course, this is potentially wasteful.

I also thought of such variants, but as you already noted, they are either 
wasteful, or limited, or both.

The simplest and most efficient variant I could think of, was the one I 
already explained, but I am illustrating it here with some code, to make it 
more clear:
// base.d
import std.container;
class Base {
	this() {
	// ...
	}
	SList!int overrideMe() {
		//....
	}
@private_impl:
	int foo;
	float bar;
}

//////

Generate two .di files, with --private-impl compiler switch:
dmd -H -c --private-impl base.d

The --private-impl switch would cause the following:
1. It creates two ".di" files, see below.
2. It creates a hidden static "size" field in the object file:

If you don't pass the --private-impl switch, just one standard .di file is 
generated and the "@private_impl:" is interpreted as plain "private:".

// base.d generated code:
import std.container;
class Base {
	this() {
	// ...
	}
	SList!int overrideMe() {
		//....
	}
@private_impl:
	int foo;
	float bar;
	static size_t __size_of_object=actual_base_object_size;
}

// base.d -- This is the public interface:
import std.container;
//Non public imports are still necessary, because the compiler needs to check 
//that client code uses the same SList implementation.
// but imports only needed by method bodies, should be omitted, but how to 
tell this the compiler?
class Base {
	this(); // No inline, we are hiding the implementation!
	SList!int overrideMe(); 
@private_impl: // Omit private data, just a hint to the compiler that there 
are private hidden fields (and that there exists a hidden size field)
}

// base_private.di -- the second .di file, just the normally generated one.

If I want to create a derived class and I want to be independent of the base 
class implementation, I would simple import the public base.di file:

// derived.d
import base;
class D : Base {
	SList!int overrideMe() {
		// override it with something
		// access  my_field:
		my_field=2;
		// compiler would generate:
		int* p=cast(int*)
((cast(void*)this)+Base.__size_of_object+offset_of_my_field);
		*p=2;
		// instead of:
		int* p=cast(int*)
((cast(void*)this)+offset_of_my_field_including_statically_known_base_size);
		*p=2;
		// A class deriving from D, would also have to add Ds size and so on, 
but as these offsets don't change during the life time of a program, the 
calculated offsets could easily be cached. 
	}
@private_impl:
	int my_field;
}

If I don't care about implementation hiding, I would simple import the 
base_private.di file, and I would have statically calculated offsets. Derived 
classes in the same library would usually do that.

// main.d

import derived;
import std.container;

void main() {
	D d=new D;
	// I am using the public .di file, thus new has to look up the objects size 
at runtime in the library and allocate the needed space, then it triggers the 
now non inlined initialization sequence:
	// - initialize all base members
	// - initialize all derived members
	// - call base class constructor
	// - call derived class constructor
	// -> can be optimized if derived class uses the non public interface.

	// use d.
	// d gets destroyed, simply call the destructors, and free the used space.
}

If you don't rename the private implementations, but simply use another module 
lookup path, you can switch implementations by no code change at all, simply 
comple main.d with the private module search path and you get the old 
behaviour.
--> client code can choose, without a code change in the library, if you don't 
mind the static size fields, not even a recompile of the library is necessary.

I hope it is now more clear what my basic approach is. I think the benefits 
are overwhelming and it seems to be easy, maybe too easy. So I am pretty sure 
it is not that simple, but what is it I am missing?

Best regards,

Robert
January 23, 2012
Re: C++ pimpl
On Mon, 23 Jan 2012 18:09:58 +0200, Robert Caravani <jfanatiker@gmx.at>  
wrote:

> Thanks for the links, it was a good read.

I think it is the best answer to the problem.

> What's the destructor limitation?

struct S {
  static S* make(); // constructor
  static void purge(S*); // destructor - you have to provide this as well
}

Limitation is that just like constructor, we also lose the destructor.
Which means we can't use "delete", and the tools designed for it.
As a result we don't have something new. It is exactly like C impl. hiding.

struct S;
S* make();
void purge(S*);
February 10, 2012
Re: C++ pimpl
On Monday, 23 January 2012 at 17:09:30 UTC, so wrote:
> On Mon, 23 Jan 2012 18:09:58 +0200, Robert Caravani 
> <jfanatiker@gmx.at> wrote:
>
>> Thanks for the links, it was a good read.
>
> I think it is the best answer to the problem.
>
>> What's the destructor limitation?
>
> struct S {
>  static S* make(); // constructor
>  static void purge(S*); // destructor - you have to provide 
> this as well
> }
>
> Limitation is that just like constructor, we also lose the 
> destructor.
> Which means we can't use "delete", and the tools designed for 
> it.
> As a result we don't have something new. It is exactly like C 
> impl. hiding.
>
> struct S;
> S* make();
> void purge(S*);

I think i finally got a solution.
(Sorry for C++ code)

--- pimpl.hpp
template<typename T>
struct pimpl : noncopyable
{
 virtual ~pimpl() {}
};

--- lib.hpp
struct lib : public pimpl<lib>
{
 void fun(...);
 ...
 static lib* make();
};

--- lib.cpp
struct lib_impl : lib
{
 ... // data
};

void lib::fun(...)
{
 auto& r = *static_cast<lib_impl*>(this);
 ...
}

lib* lib::make()
{
 return new lib_impl;
}
February 19, 2012
Re: C++ pimpl
On Friday 10 February 2012 20:23:59 so wrote:
> I think i finally got a solution.
> (Sorry for C++ code)
Never mind, it is my mother tongue.
> 
> --- pimpl.hpp
> template<typename T>
> struct pimpl : noncopyable
> {
>   virtual ~pimpl() {}
> };
> 
> --- lib.hpp
> struct lib : public pimpl<lib>
> {
>   void fun(...);
>   ...
>   static lib* make();
> };
> 
> --- lib.cpp
> struct lib_impl : lib
> {
>   ... // data
> };
> 
> void lib::fun(...)
> {
>   auto& r = *static_cast<lib_impl*>(this);
>   ...
> }
> 
> lib* lib::make()
> {
>   return new lib_impl;
> }

Well it is not bad, but how do you support derived classes with this scheme? 
Derived classes would have to derive from the implementation, so it would have 
to be public and compiled code using the derived class would break on changes 
of the private fields. Am I misinterpreting something?

What's the purpose of the pimpl template?

What do you think about my proposal? Is it sane and feasible at all?

Best regards,

Robert
February 19, 2012
Re: C++ pimpl
On Sunday, 19 February 2012 at 17:25:51 UTC, Robert Caravani 
wrote:
> On Friday 10 February 2012 20:23:59 so wrote:
>> I think i finally got a solution.
>> (Sorry for C++ code)
> Never mind, it is my mother tongue.
>> 
>> --- pimpl.hpp
>> template<typename T>
>> struct pimpl : noncopyable
>> {
>>   virtual ~pimpl() {}
>> };
>> 
>> --- lib.hpp
>> struct lib : public pimpl<lib>
>> {
>>   void fun(...);
>>   ...
>>   static lib* make();
>> };
>> 
>> --- lib.cpp
>> struct lib_impl : lib
>> {
>>   ... // data
>> };
>> 
>> void lib::fun(...)
>> {
>>   auto& r = *static_cast<lib_impl*>(this);
>>   ...
>> }
>> 
>> lib* lib::make()
>> {
>>   return new lib_impl;
>> }
>
> Well it is not bad, but how do you support derived classes with 
> this scheme? Derived classes would have to derive from the 
> implementation, so it would have to be public and compiled code 
> using the derived class would break on changes of the private 
> fields. Am I misinterpreting something?

Whole purpose of this is hiding implementation from user with 
"zero" cost.
Derived classes has no places here. Using anything about derived 
classes means "zero" cost is not your first concern. So just use 
pure classes / interfaces, they are much cleaner and to the point.

> What's the purpose of the pimpl template?

template<typename T>
struct pimpl : noncopyable
{
  virtual ~pimpl() {}
};

"virtual ~pimpl() {}"

This is to get rid of leaking, you know we need to provide a 
"virtual destructor" for an interface otherwise a base class has 
no way of knowing it is a "base" class and when you delete a 
pointer you would free only the memory base class allocated. But 
i guess you are asking why it should be like this, another way is:

#define PIMPL(x) \
	private: \
		x(const x&); \
		const x& operator=(const x&); \
	protected: \
		x() {} \
	public: \
		virtual	~x() {}

now we can just use:

struct lib
{
   PIMPL(lib)
   ...
   ...
};

I don't understand why this didn't get much attention either, i 
am now using it in my framework and it rocks!

> What do you think about my proposal? Is it sane and feasible at 
> all?

I am having trouble understanding the need for supporting class 
hierarchies.
Deriving from a private implementation feels quite wrong. A pimpl 
IMO should be a black box.
1 2 3
Top | Discussion index | About this forum | D home