I looked at the JSON format and it seems very inefficient at loading arrays
as it isn't limited to one type per array.
This is nice when you want to save a small array with different typed
elements but for my purposes this is kind of a performance problem.

This is why I will try and get suggestions again about the D-styled format I tried to suggest a few threads ago :)

Let me suggest a simple example:
(Please tell me when something isn't obvious :)

---file.dat
//comment
int number = 10;

float [3][2]  simpleArray = [
[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6]];
--

---main.d
static import ddata;
void main()
{
char[][] file = read (`file.dat`);
int i;
char c;

ddata.get(file,'number',i);
ddata.get(file,'number',c); //type fail, thus returns -1 or throws an
exception

float [3][2] a;

ddata.get(file,'simpleArray',a);
ddata.write(file,'simpleArrayCopy', a);

a[0][0] =3.0;

ddata.write(file,'simpleArray', a);

write('file.dat', file);
}
--

resulting data file:

---file.dat
//comment
int number = 10;

float [3][2]  simpleArray = [
[3.0, 0.2, 0.3],
[0.4, 0.5, 0.6]];

float [3][2]  simpleArrayCopy = [
[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6]];
-- 

"Saaa" <empty@needmail.com> wrote in message news:gtlrs3$1b9s$1@digitalmars.com...
>
> I looked at the JSON format and it seems very inefficient at loading
> arrays as it isn't limited to one type per array.
> This is nice when you want to save a small array with different typed
> elements but for my purposes this is kind of a performance problem.
>
> This is why I will try and get suggestions again about the D-styled format I tried to suggest a few threads ago :)
>
> Let me suggest a simple example:
> (Please tell me when something isn't obvious :)
>
> ---file.dat
> //comment
> int number = 10;
>
> float [3][2]  simpleArray = [
> [0.1, 0.2, 0.3],
> [0.4, 0.5, 0.6]];
> --
>
> ---main.d
> static import ddata;
> void main()
> {
> char[][] file = read (`file.dat`);
> int i;
> char c;
>
> ddata.get(file,'number',i);
> ddata.get(file,'number',c); //type fail, thus returns -1 or throws an
> exception
>
> float [3][2] a;
>
> ddata.get(file,'simpleArray',a);
> ddata.write(file,'simpleArrayCopy', a);
>
> a[0][0] =3.0;
>
> ddata.write(file,'simpleArray', a);
>
> write('file.dat', file);
> }
> --
>
> resulting data file:
>
> ---file.dat
> //comment
> int number = 10;
>
> float [3][2]  simpleArray = [
> [3.0, 0.2, 0.3],
> [0.4, 0.5, 0.6]];
>
> float [3][2]  simpleArrayCopy = [
> [0.1, 0.2, 0.3],
> [0.4, 0.5, 0.6]];
> -- 
>
>

I've run into similar problems when I wanted to save lots of stuff to a
file, and be able to load them again.
Huge multi-dimensional arrays seem to be an issue with JSON and similar..

Such a file writer as you propose would be very handy to have.

Saaa wrote:

> 
> I looked at the JSON format and it seems very inefficient at loading
arrays
> as it isn't limited to one type per array.
> This is nice when you want to save a small array with different typed
> elements but for my purposes this is kind of a performance problem.
> 
> This is why I will try and get suggestions again about the D-styled format
I
> tried to suggest a few threads ago :)
> 
> Let me suggest a simple example:
> (Please tell me when something isn't obvious :)
> 

I guess you will have to write this one yourself, it will be to D what JSON is to javascript ;)

I wonder how much of an performance improvement you will get though when loading data at runtime. You still have to parse and check everything since it remains a text based format.

I know of one other solution but that's again not available for D: google
protobuf: http://code.google.com/p/protobuf/
This uses a text-based format for describing the structure of the data, but
the actual data can be in an optimized binary format.

>
> I guess you will have to write this one yourself, it will be to D what
> JSON
> is to javascript ;)
:-)

>
> I wonder how much of an performance improvement you will get though when loading data at runtime.
As I couldn't even figure out how to save an multidimensional array in JSON we may never know :D

> You still have to parse and check everything since
> it remains a text based format.
I will be focussed on (safe) speed and not flexibility because as you
mention
otherwise it might be useless for the large arrays I wish to get/write


>
> I know of one other solution but that's again not available for D: google
> protobuf: http://code.google.com/p/protobuf/
> This uses a text-based format for describing the structure of the data,
> but
> the actual data can be in an optimized binary format.
>
For me the data should be readable.
You may write the binary extension if you like :)

My first stab at the get function.
As you might see, I need help :D
Thanks!

How do I make the function take a variadic argument and get its type?

//I guess the void pointer isn't the correct way, but I don't know any other
way
void get(in char[][] file, in char[] indentifier, void* var)
{

TypeInfo type = typeof(var);
int row;

// Is it possible to check it is an array, no matter the depth?
if ( !( type ==  typeid(int) || type == typeid(int[][])  ) ) // || etc
{
  throw new exception;
  //which kind of exception should I throw?
}

if (file.length == 0 || file.length >int.max)
{
  throw new exception;
  //ditto
}

//gets the row number of the first correct occurence for which the following
is true
//( file[row][0..len] ==  type.stringof ~ ' ' ~ identifier )
row = getRow(file, type, identifier);
if (row < 0 || row >=line.length)
{
  throw new exception;
  //same question :)
}
//parses the file from row
//what is the prototype of a function which has a variadic return type?
var = parse(file, row, type, identifier);
//can all types hold null?
if (var == null)
{
  throw new exception;
  //again
}

return;
}

Saaa wrote:
> My first stab at the get function.
> As you might see, I need help :D
> Thanks!
> 
> How do I make the function take a variadic argument and get its type?

void get(in char[][] file, in char[] identifier, ...)
{
	TypeInfo type = _arguments[0];
	void* var = _argptr;
	// etc
}

> void get(in char[][] file, in char[] indentifier, void* var)
> {
> 
> TypeInfo type = typeof(var);
> int row;
> 
> // Is it possible to check it is an array, no matter the depth?

If you use tango:

import tango.core.RuntimeTraits;
if (isArray(type)) {}

Otherwise, just copy the code from RuntimeTraits. The functions you would need are: realType, isStaticArray, isDynamicArray, isArray

> if ( !( type ==  typeid(int) || type == typeid(int[][])  ) ) // || etc
> {
>   throw new exception;
>   //which kind of exception should I throw?

Define your own exception type.

> //what is the prototype of a function which has a variadic return type?

"Variadic" means that the function can take any number of arguments.

Use Variant, from std.variant or tango.core.Variant. It's typesafe. You could also return a void*, but that will require allocation.

> var = parse(file, row, type, identifier);
> //can all types hold null?

If you use Variant, you can check isEmpty / hasValue or some such.

Thanks!
With this I can go on for a bit :)

"Christopher Wright" <dhasenan@gmail.com> wrote in message news:gtrpo7$2h7j$1@digitalmars.com...
> Saaa wrote:
>> My first stab at the get function.
>> As you might see, I need help :D
>> Thanks!
>>
>> How do I make the function take a variadic argument and get its type?
>
> void get(in char[][] file, in char[] identifier, ...)
> {
> TypeInfo type = _arguments[0];
> void* var = _argptr;
> // etc
> }
>
>> void get(in char[][] file, in char[] indentifier, void* var)
>> {
>>
>> TypeInfo type = typeof(var);
>> int row;
>>
>> // Is it possible to check it is an array, no matter the depth?
>
> If you use tango:
>
> import tango.core.RuntimeTraits;
> if (isArray(type)) {}
>
> Otherwise, just copy the code from RuntimeTraits. The functions you would need are: realType, isStaticArray, isDynamicArray, isArray
>
>> if ( !( type ==  typeid(int) || type == typeid(int[][])  ) ) // || etc
>> {
>>   throw new exception;
>>   //which kind of exception should I throw?
>
> Define your own exception type.
>
>> //what is the prototype of a function which has a variadic return type?
>
> "Variadic" means that the function can take any number of arguments.
>
> Use Variant, from std.variant or tango.core.Variant. It's typesafe. You could also return a void*, but that will require allocation.
>
>> var = parse(file, row, type, identifier);
>> //can all types hold null?
>
> If you use Variant, you can check isEmpty / hasValue or some such.
>