View mode: basic / threaded / horizontal-split · Log in · Help
April 15, 2005
stdio revised (writef/format/readf/unformat)
I won't have time to clean it up myself,
but if someone else wants to play with
integrated the revised stdio into Phobos
- I put the source code hacks up at :

http://www.algonet.se/~afb/d/stdio/

Since TypeInfo for all pointer types is
horribly broken (or at least missing...),
there's a pretty ugly workaround present.
I think you will notice it in the sources :-)


Anyway, the way it is meant to work is:

import std.stdio;

// writeln should be familiar to Java users
std.stdio.writeln("100% formatless");

write(100,"%"); //without format
writef("%d%%", 100); // with format

char[] name;
writeln("What is your name?");

read(&name); // without format
readf("%s", &name); // with format

writeln("Hello ", name, ", pleased to meet you");


If you need to access files instead of stdin / stdout,
there are fread and fwrite versions that take a FILE*.


For working with strings, there are:

import std.string;

char[] s;
int i,j;

s = format("%d%d",i,j); // traditional
s = format("%d",i,"%d",j); // mixed
s = format(i,j); // formatless

unformat(s, "%d%d",&i,&j); // traditional
unformat(s, "%d",&i,"%d",&j); // mixed
unformat(s, &i,&j); // formatless


The formatless are simpler for the beginner and
for the common cases, but the formats are very
handy for formatting input/output - especially
for any integers and floating point numbers.


For advanced users, there is also an "unFormat":

module std.unformat;
void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
              TypeInfo[] arguments, va_list argptr);

It is very similar to the old "doFormat" routine:

module std.format;
void doFormat(void delegate(dchar) putc,
              TypeInfo[] arguments, va_list argptr,
              bool ignorePercent = false);

Those are the workhorses of the formatting/unformatting.


unformat was written by Sean Kelly, and accepts
most of the "scanf" format characters - just as
format accepts most of the "printf" characters.
Unlike the C variants, they do check the types.

http://www.digitalmars.com/d/archives/digitalmars/D/11021.html

Such format/param errors, will throw "FormatError".
(for now, in want of a revised Exception hierarchy)
That's a change from the original version of unFormat,
also his "sreadf" has now been renamed into "unformat".

--anders


PS.
And the traditional disclaimer here: if you want
C++ << >> stream operators, look at Mango instead ?
(the IReader and IWriter interfaces, in mango.io)

import mango.io.Buffer;
import mango.io.Reader;
import mango.io.Writer;

Buffer buf = new Buffer (256);

// map same buffer into both reader and writer
IReader r = new Reader(buf);
IWriter w = new Writer(buf);

int i = 10;
long j = 20;
double d = 3.14159;
char[] c = "fred";

// write data types out
w << c << i << j << d;

// read them back again
r >> c >> i >> j >> d;

From http://mango.dsource.org/ (The Mango Tree)
April 15, 2005
Re: stdio revised (writef/format/readf/unformat)
Hoorah! An opportunity for a plug! ;-)

Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.
Here's an example pulled from the Mango docs:

#   int i = 10;
#  char[] msg = "green-bottles";
#
#  Stdout.put(i).put(msg);
#
#  Stdout << i << msg;
#
#  Stdout (i) (msg);

One nice thing about Mango.io lies in the symmetry for reading things back
again (no pointers):

#  Stdin.get(i).get(msg);
#
#  Stdin >> i >> msg;
#
#  Stdin (i) (msg);


Note that Stdout & Stdin are instances of a fundamental type in Mango:
IWriter & IReader. Because of this, you apply the same approach for file IO,
socket IO, memory buffers, etc. The 'whisper' format is the unofficial
standard for Mango ~ it's simple, since the only thing that changes between
input and output 'modes' is the verb itself (not the syntax ~ see the two
examples above).

Note also that mango.io handles arrays for all primitive types; not just
char[]. It also supports plugable encoding and decoding options (for wchar &
dchar). Everything operates bi-directionally.


Memory OutBuffer/InBuffer example:

#  Buffer b = new Buffer;
#
#  // write to buffer
#  Writer output = new Writer (b);
#  output (i) (msg);
#
#  // read back from buffer
#  Reader input = new Reader (b);
#  input (i) (msg);


File random-IO example:

#  FileConduit file = new FileConduit ("myfile.txt");
#
#  // write to file, and flush output
#  Writer output = new Writer (file);
#  output (i) (msg) ();
#
#  file.seek (0);
#
#  Reader input = new Reader (file);
#  input (i) (msg);


Socket example (we'll assume an Echo server on the other end):

#  SocketConduit socket = new SocketConduit;
#  socket.connect (new InternetAddress("some Echo server"));
#
#  // write to socket, and flush output
#  Writer output = new Writer (socket);
#  output (i) (msg) ();
#
#  Reader input = new Reader (socket);
#  input (i) (msg);


There's a printf() style wrapper as well, for doing traditional formatting.
Plus a 'layout' wrapper for ordering text strings. In fact, I should ask
Anders and Sean if mango.io can adopt their formatters/unformatters. How
about it, you guys? Is that cool?

Note the seperation between IO actions and the formatting thereof? Mango has
multiple Reader/Writer instances for doing different kinds of formatting
(text, binary, endian, pickling, etc). This is one way in which mango.io
supports extensibility. Another mechanism supported is the ability to
directly read and write your own classes ~ this maintains encapsulation at
the class level, and supports things like class serialization (mango.cluster
shuttles class instances around a network). You'll also find all the other
usual file oriented functionality in there; laid out in a clear, simple, and
non-redundant fashion.

Mango.io also has a simple, extensible, Tokenizer framework. It handles
serial  tokenizing from an input Buffer, and there are predefined Tokenizers
for splitting upon lines, commas, punctuation, regex, etc.

In general, mango.io is designed for high throughput server-style
environments. But you may find it useful for general purpose usage. Quite a
few others do.

- Kris



"Anders F Björklund" <afb@algonet.se> wrote in message
news:d3odh2$aot$1@digitaldaemon.com...
> I won't have time to clean it up myself,
> but if someone else wants to play with
> integrated the revised stdio into Phobos
> - I put the source code hacks up at :
>
> http://www.algonet.se/~afb/d/stdio/
>
> Since TypeInfo for all pointer types is
> horribly broken (or at least missing...),
> there's a pretty ugly workaround present.
> I think you will notice it in the sources :-)
>
>
> Anyway, the way it is meant to work is:
>
> import std.stdio;
>
> // writeln should be familiar to Java users
> std.stdio.writeln("100% formatless");
>
> write(100,"%"); //without format
> writef("%d%%", 100); // with format
>
> char[] name;
> writeln("What is your name?");
>
> read(&name); // without format
> readf("%s", &name); // with format
>
> writeln("Hello ", name, ", pleased to meet you");
>
>
> If you need to access files instead of stdin / stdout,
> there are fread and fwrite versions that take a FILE*.
>
>
> For working with strings, there are:
>
> import std.string;
>
> char[] s;
> int i,j;
>
> s = format("%d%d",i,j); // traditional
> s = format("%d",i,"%d",j); // mixed
> s = format(i,j); // formatless
>
> unformat(s, "%d%d",&i,&j); // traditional
> unformat(s, "%d",&i,"%d",&j); // mixed
> unformat(s, &i,&j); // formatless
>
>
> The formatless are simpler for the beginner and
> for the common cases, but the formats are very
> handy for formatting input/output - especially
> for any integers and floating point numbers.
>
>
> For advanced users, there is also an "unFormat":
>
> module std.unformat;
> void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
>                TypeInfo[] arguments, va_list argptr);
>
> It is very similar to the old "doFormat" routine:
>
> module std.format;
> void doFormat(void delegate(dchar) putc,
>                TypeInfo[] arguments, va_list argptr,
>                bool ignorePercent = false);
>
> Those are the workhorses of the formatting/unformatting.
>
>
> unformat was written by Sean Kelly, and accepts
> most of the "scanf" format characters - just as
> format accepts most of the "printf" characters.
> Unlike the C variants, they do check the types.
>
> http://www.digitalmars.com/d/archives/digitalmars/D/11021.html
>
> Such format/param errors, will throw "FormatError".
> (for now, in want of a revised Exception hierarchy)
> That's a change from the original version of unFormat,
> also his "sreadf" has now been renamed into "unformat".
>
> --anders
>
>
> PS.
> And the traditional disclaimer here: if you want
> C++ << >> stream operators, look at Mango instead ?
> (the IReader and IWriter interfaces, in mango.io)
>
> import mango.io.Buffer;
> import mango.io.Reader;
> import mango.io.Writer;
>
> Buffer buf = new Buffer (256);
>
> // map same buffer into both reader and writer
> IReader r = new Reader(buf);
> IWriter w = new Writer(buf);
>
> int i = 10;
> long j = 20;
> double d = 3.14159;
> char[] c = "fred";
>
> // write data types out
> w << c << i << j << d;
>
> // read them back again
> r >> c >> i >> j >> d;
>
>  From http://mango.dsource.org/ (The Mango Tree)
April 15, 2005
Re: stdio revised (writef/format/readf/unformat)
In article <d3odh2$aot$1@digitaldaemon.com>,
=?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= says...
>
>unformat was written by Sean Kelly, and accepts
>most of the "scanf" format characters - just as
>format accepts most of the "printf" characters.
>Unlike the C variants, they do check the types.

FWIW, unformat/readf is completely compliant with the C99 spec for scanf (as far
as I know).  The only difference is that this revised version of unformat throws
exceptions in the same instances where format throws exceptions, while scanf
obviously never throws exceptions :)  The unit tests are somewhat sparse, but
I'm not aware of any bugs in the original implementation (and I don't expect any
in this re-release, though I haven't seen the code yet).  Questions are probably
best directed to this newsgroup.


Sean
April 15, 2005
Re: stdio revised (writef/format/readf/unformat)
In article <d3p73i$15gg$1@digitaldaemon.com>, Kris says...
>
>Hoorah! An opportunity for a plug! ;-)
>
>Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.

FWIW, I love the new 'whisper' syntax, but it makes me wonder... what happens
with classes that implement both IReader and IWriter?

As for the rest, I prefer Mango over readf/writef in most instances, though I
think readf/writef do handle cases involving complex formatting rather
succinctly (even if it is nearly unreadable).  For example, here are two unit
tests for unformat:

int i; double x; char name[50];
sreadf( "56789 0123 56a72", "%2d%f%*d %[0-9]", &i, &x, &name[0] );
assert( i == 56 && x == 789.0 && !strcmp( "56", name ) );

double f1, f2, f3;
char c1, c2, c3;
sreadf( "1;-5.9:+0x1pag", "%f%c%f%c%f%c", &f1, &c1, &f2, &c2, &f3, &c3 );
assert( f1 == 1.0 && c1 == ';' && f2 == -5.9 && c2 == ':' && f3 == 1024.0 && c3
== 'g' );

I don't have enough experience with Mango to say, but I know that things like
this are a pain with C++ iostreams.


Sean
April 15, 2005
Re: stdio revised (writef/format/readf/unformat)
"Sean Kelly" <sean@f4.ca> wrote ...
> In article <d3p73i$15gg$1@digitaldaemon.com>, Kris says...
> >
> >Hoorah! An opportunity for a plug! ;-)
> >
> >Mango.io supports three related idioms: get/put, >>/<<, and 'whisper'.
>
> FWIW, I love the new 'whisper' syntax, but it makes me wonder... what
happens
> with classes that implement both IReader and IWriter?

:-)

The interfaces declare everything using get/put, and then alias the (other)
operators. It would be *interesting* to combine IReader & IWriter into one
class, but I can't think of a good reason to do so. Regardless, it's a point
well made, and well noted.

> As for the rest, I prefer Mango over readf/writef in most instances,
though I
> think readf/writef do handle cases involving complex formatting rather
> succinctly (even if it is nearly unreadable).  For example, here are two
unit
> tests for unformat:
>
> int i; double x; char name[50];
> sreadf( "56789 0123 56a72", "%2d%f%*d %[0-9]", &i, &x, &name[0] );
> assert( i == 56 && x == 789.0 && !strcmp( "56", name ) );
>
> double f1, f2, f3;
> char c1, c2, c3;
> sreadf( "1;-5.9:+0x1pag", "%f%c%f%c%f%c", &f1, &c1, &f2, &c2, &f3, &c3 );
> assert( f1 == 1.0 && c1 == ';' && f2 == -5.9 && c2 == ':' && f3 == 1024.0
&& c3
> == 'g' );
>
> I don't have enough experience with Mango to say, but I know that things
like
> this are a pain with C++ iostreams.

Aye, they are. I've tried various approaches with mango.io, and one that
results in the most cleanliness is binding of classes directly to the IO
system. That way, you can simply read in a number of class instances rather
than a long stream of individual primitives. Plus, everything is
encapsulated nicely. A silly example for illustrative purpose:

#  class Foo : IReadable {}
#  {
#      char[] name, address;
#      double salary;
#
#      void read (IReader input)
#      {
#          input (name) (address) (salary);
#      }
#  }
#
#  class Bar : IReadable
#  {
#      char[] header;
#      Foo f1, f2, f3;
#      ...
#      void read (IReader input)
#      {
#          input (header) (f1) (f2) (f3);
#      }
#  }
#
#  IReader input;
#
#  input (new Bar);


You chain these together in whatever manner works for the application. The
premise is that one generally keeps the values around for a while, so why
not encapsulate them to begin with. Output with classes is performed in a
similar manner. Mango has formatters and parsers for doing format-twiddling
similar to printf/scanf (mango.io does not use printf internally)

That aside; readf() is truly handy. I've been trying to get similar
functionality working that doesn't require the & operator (FormatReader &
FormatWriter), and it's looking good. But I'll probably also hookup the
traditional functions as an option too, if I may leverage the various works
you (and Anders) have made available?

- Kris
April 15, 2005
Re: stdio revised (writef/format/readf/unformat)
In article <d3pf1f$1bpe$1@digitaldaemon.com>, Kris says...
>
>That aside; readf() is truly handy. I've been trying to get similar
>functionality working that doesn't require the & operator (FormatReader &
>FormatWriter), and it's looking good. But I'll probably also hookup the
>traditional functions as an option too, if I may leverage the various works
>you (and Anders) have made available?

Feel free.  The code was a pain to get right, and I wrote it intending it to be
wrapped by other stuff anyway.


Sean
April 15, 2005
Re: stdio revised (writef/format/readf/unformat)
Cool!

What is the status of unformat.d?
Will it be added to standard phobos?

Andrew.


"Anders F Björklund" <afb@algonet.se> wrote in message 
news:d3odh2$aot$1@digitaldaemon.com...
>I won't have time to clean it up myself,
> but if someone else wants to play with
> integrated the revised stdio into Phobos
> - I put the source code hacks up at :
>
> http://www.algonet.se/~afb/d/stdio/
>
> Since TypeInfo for all pointer types is
> horribly broken (or at least missing...),
> there's a pretty ugly workaround present.
> I think you will notice it in the sources :-)
>
>
> Anyway, the way it is meant to work is:
>
> import std.stdio;
>
> // writeln should be familiar to Java users
> std.stdio.writeln("100% formatless");
>
> write(100,"%"); //without format
> writef("%d%%", 100); // with format
>
> char[] name;
> writeln("What is your name?");
>
> read(&name); // without format
> readf("%s", &name); // with format
>
> writeln("Hello ", name, ", pleased to meet you");
>
>
> If you need to access files instead of stdin / stdout,
> there are fread and fwrite versions that take a FILE*.
>
>
> For working with strings, there are:
>
> import std.string;
>
> char[] s;
> int i,j;
>
> s = format("%d%d",i,j); // traditional
> s = format("%d",i,"%d",j); // mixed
> s = format(i,j); // formatless
>
> unformat(s, "%d%d",&i,&j); // traditional
> unformat(s, "%d",&i,"%d",&j); // mixed
> unformat(s, &i,&j); // formatless
>
>
> The formatless are simpler for the beginner and
> for the common cases, but the formats are very
> handy for formatting input/output - especially
> for any integers and floating point numbers.
>
>
> For advanced users, there is also an "unFormat":
>
> module std.unformat;
> void unFormat( dchar delegate() getc, dchar delegate(dchar) ungetc,
>               TypeInfo[] arguments, va_list argptr);
>
> It is very similar to the old "doFormat" routine:
>
> module std.format;
> void doFormat(void delegate(dchar) putc,
>               TypeInfo[] arguments, va_list argptr,
>               bool ignorePercent = false);
>
> Those are the workhorses of the formatting/unformatting.
>
>
> unformat was written by Sean Kelly, and accepts
> most of the "scanf" format characters - just as
> format accepts most of the "printf" characters.
> Unlike the C variants, they do check the types.
>
> http://www.digitalmars.com/d/archives/digitalmars/D/11021.html
>
> Such format/param errors, will throw "FormatError".
> (for now, in want of a revised Exception hierarchy)
> That's a change from the original version of unFormat,
> also his "sreadf" has now been renamed into "unformat".
>
> --anders
>
>
> PS.
> And the traditional disclaimer here: if you want
> C++ << >> stream operators, look at Mango instead ?
> (the IReader and IWriter interfaces, in mango.io)
>
> import mango.io.Buffer;
> import mango.io.Reader;
> import mango.io.Writer;
>
> Buffer buf = new Buffer (256);
>
> // map same buffer into both reader and writer
> IReader r = new Reader(buf);
> IWriter w = new Writer(buf);
>
> int i = 10;
> long j = 20;
> double d = 3.14159;
> char[] c = "fred";
>
> // write data types out
> w << c << i << j << d;
>
> // read them back again
> r >> c >> i >> j >> d;
>
> From http://mango.dsource.org/ (The Mango Tree)
April 16, 2005
Re: stdio revised (writef/format/readf/unformat)
In article <d3phbd$1djm$1@digitaldaemon.com>, Sean Kelly says...
>
>In article <d3pf1f$1bpe$1@digitaldaemon.com>, Kris says...
>>
>>That aside; readf() is truly handy. I've been trying to get similar
>>functionality working that doesn't require the & operator (FormatReader &
>>FormatWriter), and it's looking good. But I'll probably also hookup the
>>traditional functions as an option too, if I may leverage the various works
>>you (and Anders) have made available?
>
>Feel free.  The code was a pain to get right, and I wrote it intending it to be
>wrapped by other stuff anyway.

Just a quick follow-up.  I checked the new source and a ton of changes have been
made from the version I released.  I trust it all works as advertised, but it's
sufficiently unfamiliar that you're better off asking Anders about it if you
have any specific questions.  My old version is still available in the usual
spot as well.


Sean
April 16, 2005
Re: stdio revised (writef/format/readf/unformat)
Thanks, Sean.

"Sean Kelly" <sean@f4.ca> wrote in message
news:d3pm92$1h8v$1@digitaldaemon.com...
> In article <d3phbd$1djm$1@digitaldaemon.com>, Sean Kelly says...
> >
> >In article <d3pf1f$1bpe$1@digitaldaemon.com>, Kris says...
> >>
> >>That aside; readf() is truly handy. I've been trying to get similar
> >>functionality working that doesn't require the & operator (FormatReader
&
> >>FormatWriter), and it's looking good. But I'll probably also hookup the
> >>traditional functions as an option too, if I may leverage the various
works
> >>you (and Anders) have made available?
> >
> >Feel free.  The code was a pain to get right, and I wrote it intending it
to be
> >wrapped by other stuff anyway.
>
> Just a quick follow-up.  I checked the new source and a ton of changes
have been
> made from the version I released.  I trust it all works as advertised, but
it's
> sufficiently unfamiliar that you're better off asking Anders about it if
you
> have any specific questions.  My old version is still available in the
usual
> spot as well.
>
>
> Sean
>
>
April 16, 2005
Re: stdio revised (writef/format/readf/unformat)
Kris wrote:

>>unformat was written by Sean Kelly, and accepts
>>most of the "scanf" format characters - just as
>>format accepts most of the "printf" characters.
>>Unlike the C variants, they do check the types.
>
> There's a printf() style wrapper as well, for doing traditional formatting.
> Plus a 'layout' wrapper for ordering text strings. In fact, I should ask
> Anders and Sean if mango.io can adopt their formatters/unformatters. How
> about it, you guys? Is that cool?

Sure, Sean wrote most of it - and I just massacred it to fit into a new 
API... :-) And I'm not sure I got it all right yet, since it was using 
Exceptions for flow handling and some stuff I didn't really figure out.

But there was a pretty permissive license on Seans stuff in unformat,
my own stuff is under the same license as Phobos: zlib/png or even PD.


So, yes, it's cool... (I didn't really change so much of the formatting
internals, except for changed delegate definitions and new exceptions)

In case it wasn't clear, it's not really working correctly or tested,
just yet - and in any way, the pointer stuff wants new TypeInfo first ?

--anders
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home