So I am working on persistence, and being a newbie, its tough. I have this method:

void Set(Object o) {
	file.Stream.writeBlock(cast(ubyte*)o, o.classinfo.init.length);
}

One specific question I have is, is classinfo populated with the info for the class that the object really is? For example:

MyObject m = new MyObject;
ObjectDatabase o = Dope.Open("filename");
o.Set(m);

Of course I would like to have it work that way. Any tips, pointers, smacks on the head for being an idiot are welcome!

Kyle

"Kyle Furlong" <ky220@umail.ucsb.edu> wrote in message news:d4scbl$1ik8$1@digitaldaemon.com...
> One specific question I have is, is classinfo populated with the info for the class that the object really is? For example:

Yep.

Jarrett Billingsley wrote:
> "Kyle Furlong" <ky220@umail.ucsb.edu> wrote in message news:d4scbl$1ik8$1@digitaldaemon.com...
> 
>>One specific question I have is, is classinfo populated with the info for the class that the object really is? For example:
> 
> 
> Yep. 
> 
> 

So the classinfo.init.length property is the current size of the object in memory?

Kyle Furlong wrote:
> Jarrett Billingsley wrote:
> 
>> "Kyle Furlong" <ky220@umail.ucsb.edu> wrote in message news:d4scbl$1ik8$1@digitaldaemon.com...
>>
>>> One specific question I have is, is classinfo populated with the info for the class that the object really is? For example:
>>
>>
>>
>> Yep.
>>
> 
> So the classinfo.init.length property is the current size of the object in memory?

But... the object doesn't change it's size in memory... it's data can be allocated dynamically, yet init.length doesn't change because only pointers/references are assigned some values.
Note that you wouldn't like to serialize the object just plain writing its bytes to a file. Sure, you'd store some data, but whilst loading it in another program run, you'd load pointers/refernces that would no longer be valid.
You might take a look at the Mango library, which has some serialization support.
I've written my own lib, yet it needs some readme/tutorial...

-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/

Tom S wrote:
> Kyle Furlong wrote:
> 
>> Jarrett Billingsley wrote:
>>
>>> "Kyle Furlong" <ky220@umail.ucsb.edu> wrote in message news:d4scbl$1ik8$1@digitaldaemon.com...
>>>
>>>> One specific question I have is, is classinfo populated with the info for the class that the object really is? For example:
>>>
>>>
>>>
>>>
>>> Yep.
>>>
>>
>> So the classinfo.init.length property is the current size of the object in memory?
> 
> 
> But... the object doesn't change it's size in memory... it's data can be allocated dynamically, yet init.length doesn't change because only pointers/references are assigned some values.
> Note that you wouldn't like to serialize the object just plain writing its bytes to a file. Sure, you'd store some data, but whilst loading it in another program run, you'd load pointers/refernces that would no longer be valid.
> You might take a look at the Mango library, which has some serialization support.
> I've written my own lib, yet it needs some readme/tutorial...
> 

I have looked at mango, but it seems that Kris only provides the IPickle  interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code?

Kyle

Kyle, I'm afraid this is what I meant when I talked about "reflection" and "serialization" and how D didn't support them like Java and C#.  You are going to have a real fun time making it completely transparent, as I don't know that you can.

Now, if classes are self-aware and can serialize/unserialize itself with methods written for each class, that's more than possible.  But if you want to automate that part transparently... well, that's specifically why some languages have reflection.

Your method would work fine for any class that had no arrays, pointers, or other classes as members.  Failing that, you'd have to "find" the pointers in a way much the same as garbage collecting, I suppose, and serialize those too.

-[Unknown]


> I have looked at mango, but it seems that Kris only provides the IPickle  interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code?
> 
> Kyle

>> But... the object doesn't change it's size in memory... it's data can be
>> allocated dynamically, yet init.length doesn't change because only
>> pointers/references are assigned some values.
>> Note that you wouldn't like to serialize the object just plain writing
>> its bytes to a file. Sure, you'd store some data, but whilst loading it
>> in another program run, you'd load pointers/refernces that would no
>> longer be valid.
>> You might take a look at the Mango library, which has some serialization
>> support.
>> I've written my own lib, yet it needs some readme/tutorial...
>>
>
> I have looked at mango, but it seems that Kris only provides the IPickle interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code?
>
> Kyle

One of the tools I want to eventually use the "DMD front end starter kit"
for was a program (I've been calling it "deflect") that takes a class source
file (say A) and generates a "metadata" class output source (for say MetaA)
that can be compiled and linked with A so that code that wants to serialize
and deserialize A can ask MetaA to do it. A meta class is a bit like the
ClassInfo class but it has more hooks - eg the entire reflection API in Java
would be possible. You can ask MetaA for the fields in A, their offsets,
their types etc etc.
If you are feeling ambitious you can grab the DMD Front End starter kit from
http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. Note that
such a program would do more than support serialization so it might be too
much for your particular needs.

-Ben

Ben Hinkle wrote:
>>>But... the object doesn't change it's size in memory... it's data can be allocated dynamically, yet init.length doesn't change because only pointers/references are assigned some values.
>>>Note that you wouldn't like to serialize the object just plain writing its bytes to a file. Sure, you'd store some data, but whilst loading it in another program run, you'd load pointers/refernces that would no longer be valid.
>>>You might take a look at the Mango library, which has some serialization support.
>>>I've written my own lib, yet it needs some readme/tutorial...
>>>
>>
>>I have looked at mango, but it seems that Kris only provides the IPickle interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code?
>>
>>Kyle
> 
> 
> One of the tools I want to eventually use the "DMD front end starter kit" for was a program (I've been calling it "deflect") that takes a class source file (say A) and generates a "metadata" class output source (for say MetaA) that can be compiled and linked with A so that code that wants to serialize and deserialize A can ask MetaA to do it. A meta class is a bit like the ClassInfo class but it has more hooks - eg the entire reflection API in Java would be possible. You can ask MetaA for the fields in A, their offsets, their types etc etc.
> If you are feeling ambitious you can grab the DMD Front End starter kit from http://home.comcast.net/~benhinkle/dmdfe/ and take a crack at it. Note that such a program would do more than support serialization so it might be too much for your particular needs.
> 
> -Ben 
> 
> 

I saw dmdfe, but was unsure how to incorporate it, could you give examples, or instructions on how to go about getting at that meta class you talk about? Like I said, I am very new to the C/D paradigm, growing up as a programmer in Java/C#.

Kyle

Unknown W. Brackets wrote:
> Kyle, I'm afraid this is what I meant when I talked about "reflection" and "serialization" and how D didn't support them like Java and C#.  You are going to have a real fun time making it completely transparent, as I don't know that you can.
> 
> Now, if classes are self-aware and can serialize/unserialize itself with methods written for each class, that's more than possible.  But if you want to automate that part transparently... well, that's specifically why some languages have reflection.
> 
> Your method would work fine for any class that had no arrays, pointers, or other classes as members.  Failing that, you'd have to "find" the pointers in a way much the same as garbage collecting, I suppose, and serialize those too.
> 
> -[Unknown]
> 
> 
>> I have looked at mango, but it seems that Kris only provides the IPickle  interface, and leaves it to the user to implement the actually serializing code. I want to make a lib which can take any object passed to it without knowing what class it is, push it to a file, and be able to reconstitute it. I understand that the pointers wont be valid accross sessions, but there should be a way to substitute virtual pointers when writing to file, so that when trying to deserialize, you would take the root object, follow any of your "virtual pointers" to other objects stored, construct those objects in the same way, and pass back the "real" pointer to the newly created object to the root object. Does that scheme sound plausible? What I am looking for is something other than a class specific implementation, where you for example write a serialize method for each class. Maybe this isnt possible in D currently, but any other pointers anyone has, feel free to speak out. BTW Tom, what does your library actually do? Could you post some sample code?
>>
>> Kyle

For the scheme I described above to work, each class would have to provide a method to tell the library which fields are which types. I can get at the fields in memory, but I cant tell what type they are. So you are right, it cannot be completely transparent without some outside metadata.

Kyle

Kyle Furlong wrote:
> BTW Tom, what does your library actually do? Could you post some sample code?
> 
> Kyle

I guess it's not what you are looking for, since the user of my lib has
to implement a 'describe' function to enable serialization, e.g.:

#	void describe(Serializer s)
#	{
#		super.describe(s);
#		s.str!(Texture)(diffuseTex);
#		s.str!(Texture)(lightmapTex);
#	}

or

#	void describe(Serializer s)
#	{
#		if (Serializer.SMode.R == s.mode)
#		{
#			ISceneNode[] chLst;
#			s.objArr!(ISceneNode)(chLst);
#			
#			foreach(ISceneNode ch; chLst)
#				attachChild(ch);
#				
#			delete chLst;
#		}
#		else
#		{
#			s.objArr!(ISceneNode)(m_children);
#		}
#		
#		s.arr!(char)(m_name);
#	}

or

#	void describe(Serializer s)
#	{
#		s.strArr!(vec3)(vertices);
#		s.strArr!(vec3)(normals);
#		s.strArr2!(vec2)(texCoords);
#		s.strArr!(vec3ub)(colors);
#		s.arr!(uint)(indices);
#		s.obj!(Effect)(effect);
#	}

And classes have to be registered in a factory:

#	static this()
#	{
#		new factory.type!(DirectionalLight);
#		new factory.type!(PointLight);
#	}

Yet for simple structs it's enough to use a mixin:

#	mixin Serializer.RawStruct;


... and it's capable of serializing whole complicated object hierarchies
just like this:

#	ISceneNode level = Pickle!(ISceneNode).read("level.sg");

#	Pickle!(ISceneNode).read("optimizedLevel.sg").setParent(root);

Note: Circular referencing classes will serialize just fine, as class instances are serialized just once. But this is only implemented for classes. If you have structs that recursively reference each other, you'd get a nice infinite loop. I didn't need that functionality yet, but it's gonna get there. The API is probably also going to change.

Anyways, if you want to take a look ( to get some ideas or find out how not to do it ;) ), the sources are here:
http://codeinsane.info/listDir.spy?viewType=normal&path=code/heresy

The file is called serializer.d but I've given you a link to a wider code base because serializer.d is dependent on some other modules.

-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/