Thread overview | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
February 27, 2012 Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
All, I am just starting to learn D. I am an economist - not a programmer, so I appreciate your patience with lack of knowledge. I have some financial data in a binary file that I would like to process. In C++ I have the data in a structure like this: struct TaqIdx { char symbol[10]; int tdate; int begrec; int endrec; } And I use an ifstream to cast the data to the structure in read. I'm struggling to get a handle on I/O in D. Can you give some pointers? Thanks so much! TJB |
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to tjb | On Mon, 27 Feb 2012 12:21:21 -0600, tjb <broughtj@gmail.com> wrote:
> All,
>
> I am just starting to learn D. I am an economist - not a programmer, so I appreciate your patience with lack of knowledge.
>
> I have some financial data in a binary file that I would like to process. In C++ I have the data in a structure like this:
>
> struct TaqIdx {
> char symbol[10];
> int tdate;
> int begrec;
> int endrec;
> }
>
> And I use an ifstream to cast the data to the structure in read. I'm struggling to get a handle on I/O in D. Can you give some pointers? Thanks so much!
>
> TJB
This is about the simplest way to read binary data in:
auto data = cast(TaqIdx[]) std.file.read(filename);
|
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to tjb | On Mon, 27 Feb 2012 19:21:21 +0100, tjb wrote: > All, > > I am just starting to learn D. I am an economist - not a programmer, so I appreciate your patience with lack of knowledge. > > I have some financial data in a binary file that I would like to process. In C++ I have the data in a structure like this: > > struct TaqIdx { > char symbol[10]; > int tdate; > int begrec; > int endrec; > } > > And I use an ifstream to cast the data to the structure in read. I'm struggling to get a handle on I/O in D. Can you give some pointers? Thanks so much! > > TJB Check out std.stream (http://dlang.org/phobos/std_stream.html). I'd do something like this: auto input = new File("somefile"); TagIdx tag; input.readExact( &tag, TagIdx.sizeof ); If you get funky results, the file might be using a different endianness. |
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to tjb | On 02/27/2012 10:21 AM, tjb wrote: > I have some financial data in a binary file that I would like to > process. In C++ I have the data in a structure like this: > > struct TaqIdx { > char symbol[10]; > int tdate; > int begrec; > int endrec; > } The equivalent of that C++ (and C) struct would be almost the same in D. Just replace char with 'ubyte' (or 'byte', if the 'char' were signed). Let me stress the fact that character types of D are UTF code units, not bytes. struct TaqIdx { ubyte[10] symbol; int tdate; int begrec; int endrec; } On the other hand, if symbol were indeed in ASCII, and will be treated as such in the program, then char[10] is fine too: char[10] symbol; Also, the int type is always 32 bits in D. Check whether that was the case for the system that the C++ TaqIdx was used on. You can read in binary by std.stdio.rawRead. It is mildly annoying that you still have to use an array even when reading a single TaqIdx: TaqIdx[1] taqs; file.rawRead(taqs); Then use taqs[0] if there is only one. > Can you give some pointers? Must... resist... :p Ali |
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to Justin Whear | Doesn't the struct alignment play a role here? |
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tobias Brandt | On Mon, 27 Feb 2012 19:42:36 +0100, Tobias Brandt wrote:
> Doesn't the struct alignment play a role here?
Good point. If the data is packed, you can toss an align(1) on the front of the struct declaration.
|
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to Justin Whear | On Monday, 27 February 2012 at 18:56:15 UTC, Justin Whear wrote:
> On Mon, 27 Feb 2012 19:42:36 +0100, Tobias Brandt wrote:
>
>> Doesn't the struct alignment play a role here?
>
> Good point. If the data is packed, you can toss an align(1) on the front
> of the struct declaration.
So, something like this should work:
import std.stdio : writeln, writefln;
import std.stream;
align(1) struct TaqIdx {
char[10] symbol;
int tdate;
int begrec;
int endrec;
}
void main() {
auto input = new File("T200808A.IDX");
TaqIdx taq;
input.readExact(&taq, TaqIdx.sizeof);
writefln("%s %s %s %s", taq.symbol, taq.tdate, taq.begrec, taq.endrec);
}
Thanks so much!
TJB
|
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to tjb | > So, something like this should work:
> [...]
It really depends on how you wrote the file originally. If you know that it is packed, i.e. 10+32+32+32=106 bytes per record, then yes.
If you wrote to the file with a C++ program, then I guess the compiler aligned the data so that the whole struct is 128 bytes in size. Technically, the C++ compiler is allowed to do anything short of changing the order of the struct fields.
You could just let your C++ program print out sizeof(TaqIdx) or
manually divide the file size by the number of records (if you
know it) to make sure.
|
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to Tobias Brandt | On 02/27/2012 11:27 AM, Tobias Brandt wrote: >> So, something like this should work: >> [...] > > It really depends on how you wrote the file originally. If you > know that it is packed, i.e. 10+32+32+32=106 bytes per record, > then yes. You meant 4 bytes per int. :) > If you wrote to the file with a C++ program, then I guess the > compiler aligned the data so that the whole struct is 128 bytes > in size. Technically, the C++ compiler is allowed to do > anything short of changing the order of the struct fields. That is correct for non-POD types. The C++ compiler must treat POD structs essentially as if they are C structs. Ali |
February 27, 2012 Re: Binary I/O for Newbie | ||||
---|---|---|---|---|
| ||||
Posted in reply to Ali Çehreli | >> It really depends on how you wrote the file originally. If you know that it is packed, i.e. 10+32+32+32=106 bytes per record, then yes. > > You meant 4 bytes per int. :) Yep, good catch. >> If you wrote to the file with a C++ program, then I guess the compiler aligned the data so that the whole struct is 128 bytes in size. Technically, the C++ compiler is allowed to do anything short of changing the order of the struct fields. > That is correct for non-POD types. The C++ compiler must treat POD structs essentially as if they are C structs. Correct me if I'm wrong. But as far a I know the C standard also allows arbitrary alignment. |
Copyright © 1999-2021 by the D Language Foundation