Jump to page: 1 2
Thread overview
D-styled data file
Apr 27, 2009
Saaa
Apr 28, 2009
Saaa
Apr 28, 2009
Saaa
Apr 29, 2009
Georg Wrede
Apr 29, 2009
Saaa
Apr 29, 2009
Georg Wrede
Apr 29, 2009
Saaa
Apr 29, 2009
Georg Wrede
Apr 29, 2009
Saaa
Apr 29, 2009
Georg Wrede
Apr 30, 2009
Saaa
Apr 30, 2009
Christopher Wright
Apr 30, 2009
Saaa
Apr 30, 2009
Christopher Wright
Apr 30, 2009
Saaa
Apr 29, 2009
Lutger
Apr 29, 2009
Saaa
Apr 30, 2009
Saaa
Apr 30, 2009
Saaa
April 27, 2009
I would like to be able to read and write variables which are human
readable.
It should be as fast as possible and easy to use.

Something like this:

-- file.dat
char[] header = `test header`;

int[6] numbers = [ 1, 2, 3, 4, 5, 6];

// not sure about the array layout yet
// the following layout makes human reading a lot easier if you'd had
something like float[10][10][10]
// also comments, or any line not starting with a type is ignored
bool[][][] map =
[
[
[true, true],
[false, false],
[false, false]
],
[true, true],
[false, false],
[false, false]
]
]

bool map = bool; // this is ignored to speed up finding a variable

--


--  main.d
module main;

import std.stdio;
import std.file;

void main()
{
char[][] file = read (`file.dat`);

//I'm not sure how this part should work :)
TypeFile file_TF = getTF (file);
// a TypeFile holds known information about the file
// list of : name, type, line number
// It also holds a copy of the file but maybe there can be a less safe
version
// getTF_unsafe(file);
// which only keeps a reference to the file.


char[] name = file_TF.header; //probably needs a better function name
//parses only the requested variable
//a lot of room for optimizations (indexing until found etc.)

writefln(file_TFnumbers.length); // prints 6
int[] numbers = file_TF.numbers;

numbers[3] = 30;

file_TF.numbers = numbers;
//updates the copy of the char[][]

write(file.dat, file_TF.stringof); // the only keyword //comments in the original file would be preserved this way }
--

Any comments, questions or pointers?
Maybe all this is already available?
I would think that the parsing code is already available somewhere.


April 28, 2009
Maybe add another keyword to force a full index parse :)


April 28, 2009
I changed the prototype to:

void get(in char[] varName, ...)

I'm not totally happy with it because I only need one variadic argument and :

How do I get the .stringof of an variadic type? _arguments[0].stringof doesn't work :)

How do I mutate the original argument? (ref)

Please tell me if I'm on the totally wrong track here.

---
module ddata.main;

import std.file;
import std.stdio;
import std.string;


void main()
{
 char[] filename = `data.dat`;
 char[][] file;

 try{
  file = splitlines( cast(char[])read(filename) );
 }
 catch{
  throw new Exception("Couldn't load : " ~ filename);
 }

 DData file_dd = new DData(file);
 int i;
 file_dd.get(`i`, i);
}


class DData
{

 private char[][] _file;

 this(char[][] file)
 {
  _file = file.dup;
 }

 void get(in char[] varName, ...)
 {
  char[] type = _arguments[0].stringof;
  writefln (type);
  foreach(int i, char[] line; _file)
  {
   if(line[0..type.length] == type)
   {
   writefln (i);
   break;
   }
   writefln (`failed`);
  }
 }
}
--- 


April 29, 2009
Saaa wrote:
> I changed the prototype to:
> 
> void get(in char[] varName, ...)
> 
> I'm not totally happy with it because I only need one variadic argument and :
> 
> How do I get the .stringof of an variadic type?
> _arguments[0].stringof doesn't work :)
> 
> How do I mutate the original argument? (ref)
> 
> Please tell me if I'm on the totally wrong track here.

What you're doing is called serialization.

I often find it good to get some background info. That often helps me rethink and maybe redefine the problem. It takes time to read stuff, but in the end it often saves even more time. Or effort. A couple of pointers:

http://en.wikipedia.org/wiki/Serialization_(computing)#Human-readable_serialization

Thinking about what your file might look like could also help:

http://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example
April 29, 2009
Thanks for your reply

"Georg Wrede" <georg.wrede@iki.fi> wrote in message news:gt96lc$dig$1@digitalmars.com...
> Saaa wrote:
>> I changed the prototype to:
>>
>> void get(in char[] varName, ...)
>>
>> I'm not totally happy with it because I only need one variadic argument and :
>>
>> How do I get the .stringof of an variadic type? _arguments[0].stringof doesn't work :)
>>
>> How do I mutate the original argument? (ref)
>>
>> Please tell me if I'm on the totally wrong track here.
>
> What you're doing is called serialization.
Isn't serialization the other way around?
Of course a d-styled fileformat would be very handy when it comes to
serialization,
but I started because of the inverse: I wanted to read data which would be
easily put into D types.

>
> I often find it good to get some background info. That often helps me rethink and maybe redefine the problem. It takes time to read stuff, but in the end it often saves even more time. Or effort. A couple of pointers:
>
> http://en.wikipedia.org/wiki/Serialization_(computing)#Human-readable_serialization
>
> Thinking about what your file might look like could also help:
>
> http://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example
The problem with this format (to me) would be that multidimensional arrays aren't standard.

Because the compiler can (of course) already read d styled data files I know
the code
must be there already.

I'm still dangling between a full parsing at load and parse on demand.
A parse on demand (get) would do the following steps:
Get the type of the variable in which the data must be stored (in string
format)
Search for this type in every line of the char[][], when found check whether
the name is the same
and then convert the chars to that type and place the data in the variable.


April 29, 2009
Saaa wrote:
> Thanks for your reply
> 
> "Georg Wrede" <georg.wrede@iki.fi> wrote in message news:gt96lc$dig$1@digitalmars.com...
>> Saaa wrote:
>>> I changed the prototype to:
>>>
>>> void get(in char[] varName, ...)
>>>
>>> I'm not totally happy with it because I only need one variadic argument and :
>>>
>>> How do I get the .stringof of an variadic type?
>>> _arguments[0].stringof doesn't work :)
>>>
>>> How do I mutate the original argument? (ref)
>>>
>>> Please tell me if I'm on the totally wrong track here.
>> What you're doing is called serialization.
> Isn't serialization the other way around?

Technically yes, deserialization. Although the whole thing comprising both of these is still called serialization.

> Of course a d-styled fileformat would be very handy when it comes to serialization,
> but I started because of the inverse: I wanted to read data which would be easily put into D types.
> 
>> I often find it good to get some background info. That often helps me rethink and maybe redefine the problem. It takes time to read stuff, but in the end it often saves even more time. Or effort. A couple of pointers:
>>
>> http://en.wikipedia.org/wiki/Serialization_(computing)#Human-readable_serialization
>>
>> Thinking about what your file might look like could also help:
>>
>> http://en.wikipedia.org/wiki/JSON#Data_types.2C_syntax_and_example
> The problem with this format (to me) would be that multidimensional arrays aren't standard.
> 
> Because the compiler can (of course) already read d styled data files I know the code
> must be there already.
> 
> I'm still dangling between a full parsing at load and parse on demand.
> A parse on demand (get) would do the following steps:
> Get the type of the variable in which the data must be stored (in string format)
> Search for this type in every line of the char[][], when found check whether the name is the same
> and then convert the chars to that type and place the data in the variable.

This sounds complicated. Can you decide the input file format? Or is it from some other program that you can't change?

Suppose you already had this get function. How would you use it? An example would help.
April 29, 2009

>> I'm still dangling between a full parsing at load and parse on demand.
>> A parse on demand (get) would do the following steps:
>> Get the type of the variable in which the data must be stored (in string
>> format)
>> Search for this type in every line of the char[][], when found check
>> whether the name is the same
>> and then convert the chars to that type and place the data in the
>> variable.
>
> This sounds complicated. Can you decide the input file format? Or is it from some other program that you can't change?

The input file format would be like D
Like file.dat from the original post.
I just often have the need to save large arrays and other variables and I
thought
why not just save them like the way I would use them in my modules.


>
> Suppose you already had this get function. How would you use it? An example would help.

I use the get functions a few posts back, in the main:

---
void main()
{
 char[] filename = `data.dat`;
 char[][] file;

 try{
  file = splitlines( cast(char[])read(filename) );
 }
 catch{
  throw new Exception("Couldn't load : " ~ filename);
 }

 DData file_dd = new DData(file);
 int i;
 file_dd.get(`i`, i);
}
---

The data.dat should have a line like this to work:
--- data.dat
int i = 10;
---

I hope it all makes sense :)


April 29, 2009
Saaa wrote:

> I would like to be able to read and write variables which are human
> readable.
> It should be as fast as possible and easy to use.
> 
...
> --
> 
> Any comments, questions or pointers?
> Maybe all this is already available?
> I would think that the parsing code is already available somewhere.

JSON is a very nice format for this and Tango has a parser for it. Yaml is even easier imho, but there are no D parsers for it yet.


April 29, 2009
"Lutger" <lutger.blijdestijn@gmail.com> wrote in message news:gt9rjm$1hpe$1@digitalmars.com...
> Saaa wrote:
>
>> I would like to be able to read and write variables which are human
>> readable.
>> It should be as fast as possible and easy to use.
>>
> ...
>> --
>>
>> Any comments, questions or pointers?
>> Maybe all this is already available?
>> I would think that the parsing code is already available somewhere.
>
> JSON is a very nice format for this and Tango has a parser for it. Yaml is even easier imho, but there are no D parsers for it yet.
>
>
But JSON doesn't do multidimensional arrrays, right?
Ow, my bad. It's recursive.
but.. but.. I liked the idea of a D styled dataformat
Let me read the tango code.


April 29, 2009
Saaa wrote:
>>> I'm still dangling between a full parsing at load and parse on demand.
>>> A parse on demand (get) would do the following steps:
>>> Get the type of the variable in which the data must be stored (in string format)
>>> Search for this type in every line of the char[][], when found check whether the name is the same
>>> and then convert the chars to that type and place the data in the variable.
>> This sounds complicated. Can you decide the input file format? Or is it from some other program that you can't change?
> 
> The input file format would be like D
> Like file.dat from the original post.
> I just often have the need to save large arrays and other variables and I thought
> why not just save them like the way I would use them in my modules.

Err, ok, lemme assume this is kind-of like a scrapbook thing. You collect nice arrays, filled data structures and the like. And the you'd want to read them into your programs.

Now, to know how to use the stuff, the program would need to not only be able to handle the data structures, but also know their names. Sure, you could have a parser that returns first the name of the thing, and then the thing itself. But then, to use it, you probably couldn't have an Universal program. Rather, you'd have to write the program different for each time you decide to use a particular item from the scrap book.

Which actually leads us to another simple solution. Why not write your scrapbook simply in total D format, and then include it in the program?

>> Suppose you already had this get function. How would you use it? An example would help.
> 
> I use the get functions a few posts back, in the main:
> 
> ---
> void main()
> {
>  char[] filename = `data.dat`;
>  char[][] file;
> 
>  try{
>   file = splitlines( cast(char[])read(filename) );
>  }
>  catch{
>   throw new Exception("Couldn't load : " ~ filename);
>  }
> 
>  DData file_dd = new DData(file);
>  int i;
>  file_dd.get(`i`, i);
> }
> ---
> 
> The data.dat should have a line like this to work:
> --- data.dat
> int i = 10;
> ---

So here main has to know that there will be a variable i in the data file, then get it, then presumably assign the value to the here defined "int i;". This way you have to do everything (and more) you would if you were to rename data.dat to data.d, include it, and then just use the variables.

> I hope it all makes sense :) 

So, either I still don't get it, or you're doing Double Thinking. A picture might clarify:

http://ocw.mit.edu/NR/rdonlyres/A166EB14-E7FD-473F-98FB-741A9817540C/0/chp_triangle.jpg

It's very often that programmers spend time attempting to do something that is possible, but when it comes down to it, it either isn't, or it takes an inordinate amount of work /compared/ to the value of the goal.



« First   ‹ Prev
1 2