I've updated the three C/C++ test files (C.c, Cpp.cpp and STL.cpp), and also added two new test files (Cpp_minimal.cpp) and STL_minimal.cpp).

If you've time, please take a look at them and let me know whether (and by how much) they address your concerns.

Cheers

Matthew

Thanks for taking the comments in the right spirit.

1. Open source:
The success of the open source model lies in (competent) others being able
to (relatively) easily understand the code and hence develop confidence, use
it and further be able to enhance it. Goes perfectly well with the idiom
that programmers write code for other programmers (could be 'new' you, after
3 months!). Got to keep it simple, if can't, got to put comments next to it.
I strictly follow that 95% of code be less than 20 line routines, but don't
want to give others any advise on this issue. (Those who are sold on it,
don't need the advice; others are going to call me impractical, anyway!).
And, I have written and am maintaining reasonably big programs for a number
of years (decades). Oops, giving away my age..
So, others are going to look at your code first to see if they can
understand it .. that decides whether they will use it. So, 'internal /
worker' routines also need to be 'good'!

2. 'c' and 'stl'
With practically every platform having a c++ compiler, today I use C only
for the embedded projects, where it is not possible/often not required to
have c++. Thus, the mention of 'c' usability gave me the hopes of using this
with the smaller 8031 based 'c' projects. The small 'file system' would be a
better standard to follow than a proprietary one for every project. From
this perspective, the first look at the library is disappointing.

The thick STL book convinced me that STL beats 'hand coding' in every situation. So, it should be used without reservation. What I am looking for, however, is 'minimum' use of STL to avoid problems with various 'differently' conforming compilers. Even here, as written, my 'desired' sample program effectively hides the use of STL, so it is largely usable as-is on the smaller configurations. So, no boost, no stlsoft. Not for such a small thing..

Even then, aggressively using STL for every situation still eludes me. Hats off to guys like you who can understand the complex templates and 'meta' programming. Just keep on explaining and we will (hopefully) pick it up. But, use it for the wrong things and we will be driven away. (After saying this, I have made an attempt to use nothing else but STL, and I am aware that there must be umpteen ways to do it right!)

3. Other design choices:
This task of picking up records can be done easily (if the setup is right)
using lex/yacc, boost spirit and any other dozens of similar parsing
mechanisms, but all are overkills for separating 'name: value'. This looks
like a simple C thing. What's the big deal about it? Yes, this action needs
to be center-stage. The table - record comments, count of records,
selectively ignoring spaces are all peripheral. The present implementation
needs to segregate these in obvious ways.

Fields are smallest operable entities, each record has multiple fields (whether 2 records have same named fields is a design decision, OpenRj is attractive because it does not force us to), many records constitute a table (same size=rectangular table. Otherwise 'round'??), many tables make a database. A real database is a much bigger beast. This is a table handler, and it is useful as one. Pretend it to be anything else and it gets ridiculed. Anything slightly bigger and programmers would use Sqlite, MySql or other 'in-memory' databases capable of understanding SQL. Too much prior art there to ignore.

One class or two derived from a base? That's your choice. The possibility of testing using in-memory strings and then switching over to files is very real. Users want minimum change then. That's why the suggestion about a single class. And that is where writing a real application before the library comes into the picture. So, please don't say 'vestigial/unfinished' !! Please start with the app, first.

Also, I am talking about the use of 'foreach' in the example code. The library does not use it. This is a perfect way to illustrate a technique, in this case, the emphasis would be on the library classes 'capable' of being traversed in this way. In my illustrative design, the same ones may be traversed in the 'usual' begin-end way!! I needed your opinion on this as a researcher of iterations over containers. Are you aware of an implementation of 'foreach' for c++? I would find it hard to believe that what I have been using for quite some time (even before I started using STL) has not been in circulation...

Already, we have discussed too much regarding this ;-) Only excusable, if we
consider the principles in general, not for this project.
The only reason I have still not posted any code is to make sure that this
is read! I know I would rush to the code first and ignore everything ;-)

Regards,
- Rajiv



"Matthew" <admin.hat@stlsoft.dot.org> wrote in message news:d6v7jf$15om$1@digitaldaemon.com...
> Wow! A lot to think about, and a lot of very useful feedback that's
already got me thinking.
>
...

"Matthew" <admin.hat@stlsoft.dot.org> wrote in message news:d6v9va$19nt$1@digitaldaemon.com...
> I've updated the three C/C++ test files (C.c, Cpp.cpp and STL.cpp), and
also added two new test files (Cpp_minimal.cpp)
> and STL_minimal.cpp).

Can you put large files on the web site and post a link here instead? Lots of people access these newsgroups with dialups, and these large attachments make it difficult. Thanks!

For some reason, when it was in the send folder, it looked like 6 KB. Sorry.

"Walter" <newshound@digitalmars.com> wrote in message news:d6vspj$2236$1@digitaldaemon.com...
>
> "Matthew" <admin.hat@stlsoft.dot.org> wrote in message news:d6v9va$19nt$1@digitaldaemon.com...
>> I've updated the three C/C++ test files (C.c, Cpp.cpp and STL.cpp), and
> also added two new test files (Cpp_minimal.cpp)
>> and STL_minimal.cpp).
>
> Can you put large files on the web site and post a link here
> instead? Lots
> of people access these newsgroups with dialups, and these large
> attachments
> make it difficult. Thanks!
>
> 

> 1. Open source:
> The success of the open source model lies in (competent) others
> being able
> to (relatively) easily understand the code and hence develop
> confidence, use
> it and further be able to enhance it. Goes perfectly well with the
> idiom
> that programmers write code for other programmers (could be 'new'
> you, after
> 3 months!). Got to keep it simple, if can't, got to put comments
> next to it.
> I strictly follow that 95% of code be less than 20 line routines,
> but don't
> want to give others any advise on this issue. (Those who are sold
> on it,
> don't need the advice; others are going to call me impractical,
> anyway!).
> And, I have written and am maintaining reasonably big programs for
> a number
> of years (decades). Oops, giving away my age..
> So, others are going to look at your code first to see if they can
> understand it .. that decides whether they will use it. So,
> 'internal /
> worker' routines also need to be 'good'!

Agreed, and well taken.

I've updated Open-RJ - now 1.3.2 (http://openrj.org/) - in line with this good advice. I shall do the same with recls in a few days, and STLSoft for its next release.

Thanks for this

> 2. 'c' and 'stl'
> With practically every platform having a c++ compiler, today I use
> C only
> for the embedded projects, where it is not possible/often not
> required to
> have c++. Thus, the mention of 'c' usability gave me the hopes of
> using this
> with the smaller 8031 based 'c' projects. The small 'file system'
> would be a
> better standard to follow than a proprietary one for every
> project. From
> this perspective, the first look at the library is disappointing.

I don't agree here, as the aims for Open-RJ are not commensurate with that. Specifically:

- I wanted it to be as attractive to pure-C (particularly Linux)
folks, as C++; there are a lot of C programmers who aren't C++
programmers.
- I wanted it to be super small, and flexible, with as few
dependencies as possible. In its chopped down state - where you
bring your own allocator - it requires only stddef.h!
- A library that's to be mapped to many different languages must
have a C-API. Given that, and the fact that the impl is pretty
simple - the original was written in a couple of hours - it seems
pointless not to implement in C. This is informed, in part, by my
experience with recls, which _is_ implemented in C++ (though still
presents a C-API).

> The thick STL book convinced me that STL beats 'hand coding' in
> every
> situation. So, it should be used without reservation.

Agreed, for application code, and for library code that's only going to be used by C++ client code.

> What I am looking for,
> however, is 'minimum' use of STL to avoid problems with various
> 'differently' conforming compilers. Even here, as written, my
> 'desired'
> sample program effectively hides the use of STL, so it is largely
> usable
> as-is on the smaller configurations. So, no boost, no stlsoft. Not
> for such
> a small thing..

Can't respond to this, until I see the code.

Walter wrote a very small impl in D, which was great as far as it went, but it didn't go further than the parsing into fields and records. That's fine, if that's all that's wanted, but it's not a fully representative comparison. (FYI: May's instalment of "Positive Integration" http://www.cuj.com/documents/s=9784/cuj0505wilson/, discusses this very subject, and shows how the 100% D version is slightly slower. I expect the same from a pure C++ version, although I wouldn't bet the house on it. Hmmm, maybe I'll try it with string_view, that might be as fast.)

> Even then, aggressively using STL for every situation still eludes
> me. Hats
> off to guys like you who can understand the complex templates and
> 'meta'
> programming. Just keep on explaining and we will (hopefully) pick
> it up.

The only reason I write about it is because I find it confusing as well. :-)

> 3. Other design choices:
> This task of picking up records can be done easily (if the setup
> is right)
> using lex/yacc, boost spirit and any other dozens of similar
> parsing
> mechanisms, but all are overkills for separating 'name: value'.
> This looks
> like a simple C thing. What's the big deal about it?

There is no big deal. It's just a simple little library, with mappings to many languages. It's useful, small, portable. Just badly documented. (Or maybe 'was', as I'm hoping the 1.3.2 release answers a lot of your concerns. Please let me know if/where it doesn't.)

FYI: I read about the Record-JAR in Raymond's "The Art Of UNIX Programming" - a book I'd recommend to *everyone*. It is one of many formats he discusses, each of which has its own virtues and niches. I've found Open-RJ ideal for the configuration files of various programs where they need more than a few command-line args, and where XML would be overkill.

> Yes, this action needs
> to be center-stage. The table - record comments, count of records,
> selectively ignoring spaces are all peripheral. The present
> implementation
> needs to segregate these in obvious ways.
>
> Fields are smallest operable entities, each record has multiple
> fields
> (whether 2 records have same named fields is a design decision,
> OpenRj is
> attractive because it does not force us to), many records
> constitute a table
> (same size=rectangular table. Otherwise 'round'??), many tables
> make a
> database. A real database is a much bigger beast. This is a table
> handler,
> and it is useful as one. Pretend it to be anything else and it
> gets
> ridiculed. Anything slightly bigger and programmers would use
> Sqlite, MySql
> or other 'in-memory' databases capable of understanding SQL. Too
> much prior
> art there to ignore.

Well, I think anything more complex and one would use XML, but then I see it as a configuration format, rather than a database format. (I guess using the term Database was somewhat ill-chosen, then <g>)

> One class or two derived from a base? That's your choice. The
> possibility of
> testing using in-memory strings and then switching over to files
> is very
> real. Users want minimum change then. That's why the suggestion
> about a
> single class.

Because they inherit, there is no confusion/complexity here. The only difference is the constructors for the derived (file and memory) classes. I think it's very simple, orthogonal, discoverable.

> And that is where writing a real application before the
> library comes into the picture. So, please don't say
> 'vestigial/unfinished'
> !! Please start with the app, first.

Point taken.

> Also, I am talking about the use of 'foreach' in the example code.
> The
> library does not use it. This is a perfect way to illustrate a
> technique, in
> this case, the emphasis would be on the library classes 'capable'
> of being
> traversed in this way. In my illustrative design, the same ones
> may be
> traversed in the 'usual' begin-end way!! I needed your opinion on
> this as a
> researcher of iterations over containers. Are you aware of an
> implementation
> of 'foreach' for c++? I would find it hard to believe that what I
> have been
> using for quite some time (even before I started using STL) has
> not been in
> circulation...

Maybe I'll plug in "Ranges" into the Open-RJ/C++ and /STL mappings. A 1.4 feature, perhaps ...

> Already, we have discussed too much regarding this ;-) Only
> excusable, if we
> consider the principles in general, not for this project.

Sure

> The only reason I have still not posted any code is to make sure
> that this
> is read! I know I would rush to the code first and ignore
> everything ;-)

It is read. Now post! :-)

I have taken Walter's advice - instead of posting files here, I have posted
an article on CodeProject.
"Implementing 'foreach' for c++ as a Design Pattern"
http://www.codeproject.com/

Have attached the alternate OpenRj implementation as the 'sample project'. It has been tested (obviously) on DMC, as well as on VC6 & gcc under linux.

- Rajiv


"Matthew" <admin@stlsoft.dot.dot.dot.dot.org> wrote in message news:d73k78$dkh$1@digitaldaemon.com...
> > 1. Open source:
> > The success of the open source model lies in (competent) others
> > being able
> > to (relatively) easily understand the code and hence develop
> > confidence, use
> > it and further be able to enhance it. Goes perfectly well with the
> > idiom
> > that programmers write code for other programmers (could be 'new'
> > you, after
> > 3 months!). Got to keep it simple, if can't, got to put comments
> > next to it.
> > I strictly follow that 95% of code be less than 20 line routines,
> > but don't
> > want to give others any advise on this issue. (Those who are sold
> > on it,
> > don't need the advice; others are going to call me impractical,
> > anyway!).
> > And, I have written and am maintaining reasonably big programs for
> > a number
> > of years (decades). Oops, giving away my age..
> > So, others are going to look at your code first to see if they can
> > understand it .. that decides whether they will use it. So,
> > 'internal /
> > worker' routines also need to be 'good'!
>
> Agreed, and well taken.
>
> I've updated Open-RJ - now 1.3.2 (http://openrj.org/) - in line with this good advice. I shall do the same with recls in a few days, and STLSoft for its next release.
>
> Thanks for this
>
> > 2. 'c' and 'stl'
> > With practically every platform having a c++ compiler, today I use
> > C only
> > for the embedded projects, where it is not possible/often not
> > required to
> > have c++. Thus, the mention of 'c' usability gave me the hopes of
> > using this
> > with the smaller 8031 based 'c' projects. The small 'file system'
> > would be a
> > better standard to follow than a proprietary one for every
> > project. From
> > this perspective, the first look at the library is disappointing.
>
> I don't agree here, as the aims for Open-RJ are not commensurate with that. Specifically:
>
> - I wanted it to be as attractive to pure-C (particularly Linux)
> folks, as C++; there are a lot of C programmers who aren't C++
> programmers.
> - I wanted it to be super small, and flexible, with as few
> dependencies as possible. In its chopped down state - where you
> bring your own allocator - it requires only stddef.h!
> - A library that's to be mapped to many different languages must
> have a C-API. Given that, and the fact that the impl is pretty
> simple - the original was written in a couple of hours - it seems
> pointless not to implement in C. This is informed, in part, by my
> experience with recls, which _is_ implemented in C++ (though still
> presents a C-API).
>
> > The thick STL book convinced me that STL beats 'hand coding' in
> > every
> > situation. So, it should be used without reservation.
>
> Agreed, for application code, and for library code that's only going to be used by C++ client code.
>
> > What I am looking for,
> > however, is 'minimum' use of STL to avoid problems with various
> > 'differently' conforming compilers. Even here, as written, my
> > 'desired'
> > sample program effectively hides the use of STL, so it is largely
> > usable
> > as-is on the smaller configurations. So, no boost, no stlsoft. Not
> > for such
> > a small thing..
>
> Can't respond to this, until I see the code.
>
> Walter wrote a very small impl in D, which was great as far as it went, but it didn't go further than the parsing into fields and records. That's fine, if that's all that's wanted, but it's not a fully representative comparison. (FYI: May's instalment of "Positive Integration" http://www.cuj.com/documents/s=9784/cuj0505wilson/, discusses this very subject, and shows how the 100% D version is slightly slower. I expect the same from a pure C++ version, although I wouldn't bet the house on it. Hmmm, maybe I'll try it with string_view, that might be as fast.)
>
> > Even then, aggressively using STL for every situation still eludes
> > me. Hats
> > off to guys like you who can understand the complex templates and
> > 'meta'
> > programming. Just keep on explaining and we will (hopefully) pick
> > it up.
>
> The only reason I write about it is because I find it confusing as well. :-)
>
> > 3. Other design choices:
> > This task of picking up records can be done easily (if the setup
> > is right)
> > using lex/yacc, boost spirit and any other dozens of similar
> > parsing
> > mechanisms, but all are overkills for separating 'name: value'.
> > This looks
> > like a simple C thing. What's the big deal about it?
>
> There is no big deal. It's just a simple little library, with mappings to many languages. It's useful, small, portable. Just badly documented. (Or maybe 'was', as I'm hoping the 1.3.2 release answers a lot of your concerns. Please let me know if/where it doesn't.)
>
> FYI: I read about the Record-JAR in Raymond's "The Art Of UNIX Programming" - a book I'd recommend to *everyone*. It is one of many formats he discusses, each of which has its own virtues and niches. I've found Open-RJ ideal for the configuration files of various programs where they need more than a few command-line args, and where XML would be overkill.
>
> > Yes, this action needs
> > to be center-stage. The table - record comments, count of records,
> > selectively ignoring spaces are all peripheral. The present
> > implementation
> > needs to segregate these in obvious ways.
> >
> > Fields are smallest operable entities, each record has multiple
> > fields
> > (whether 2 records have same named fields is a design decision,
> > OpenRj is
> > attractive because it does not force us to), many records
> > constitute a table
> > (same size=rectangular table. Otherwise 'round'??), many tables
> > make a
> > database. A real database is a much bigger beast. This is a table
> > handler,
> > and it is useful as one. Pretend it to be anything else and it
> > gets
> > ridiculed. Anything slightly bigger and programmers would use
> > Sqlite, MySql
> > or other 'in-memory' databases capable of understanding SQL. Too
> > much prior
> > art there to ignore.
>
> Well, I think anything more complex and one would use XML, but then I see it as a configuration format, rather than a database format. (I guess using the term Database was somewhat ill-chosen, then <g>)
>
> > One class or two derived from a base? That's your choice. The
> > possibility of
> > testing using in-memory strings and then switching over to files
> > is very
> > real. Users want minimum change then. That's why the suggestion
> > about a
> > single class.
>
> Because they inherit, there is no confusion/complexity here. The only difference is the constructors for the derived (file and memory) classes. I think it's very simple, orthogonal, discoverable.
>
> > And that is where writing a real application before the
> > library comes into the picture. So, please don't say
> > 'vestigial/unfinished'
> > !! Please start with the app, first.
>
> Point taken.
>
> > Also, I am talking about the use of 'foreach' in the example code.
> > The
> > library does not use it. This is a perfect way to illustrate a
> > technique, in
> > this case, the emphasis would be on the library classes 'capable'
> > of being
> > traversed in this way. In my illustrative design, the same ones
> > may be
> > traversed in the 'usual' begin-end way!! I needed your opinion on
> > this as a
> > researcher of iterations over containers. Are you aware of an
> > implementation
> > of 'foreach' for c++? I would find it hard to believe that what I
> > have been
> > using for quite some time (even before I started using STL) has
> > not been in
> > circulation...
>
> Maybe I'll plug in "Ranges" into the Open-RJ/C++ and /STL mappings. A 1.4 feature, perhaps ...
>
> > Already, we have discussed too much regarding this ;-) Only
> > excusable, if we
> > consider the principles in general, not for this project.
>
> Sure
>
> > The only reason I have still not posted any code is to make sure
> > that this
> > is read! I know I would rush to the code first and ignore
> > everything ;-)
>
> It is read. Now post! :-)
>
>
>

"Matthew" <admin@stlsoft.dot.dot.dot.dot.org> wrote in message news:d73k78$dkh$1@digitaldaemon.com...
> Walter wrote a very small impl in D, which was great as far as it went, but it didn't go further than the parsing into fields and records.

Here it is, for comparison purposes:
------------------------------------------

// openrj.d
// placed into Public Domain

module std.openrj;

import std.string;

alias char[][] [char[]] [] openrj_t;

class OpenrjException : Exception
{
    uint linnum;

    this(uint linnum, char[] msg)
    {
 this.linnum = linnum;
 super(std.string.format("OpenrjException line %s: %s", linnum, msg));
    }
}

openrj_t parse(char[] db)
{
    openrj_t rj;
    char[][] lines;
    char[][] [char[]] record;

    lines = std.string.splitlines(db);

    for (uint linnum = 0; linnum < lines.length; linnum++)
    {
 char[] line = lines[linnum];

 // Splice lines ending with backslash
 while (line.length && line[length - 1] == '\\')
 {
     if (++linnum == lines.length)
  throw new OpenrjException(linnum, "no line after \\ line");
     line = line[0 .. length - 1] ~ lines[linnum];
 }

 if (line[0 .. 2] == "%%")
 {
     // Comment lines separate records
     if (record)
  rj ~= record;
     record = null;
     line = null;
     continue;
 }

 int colon = std.string.find(line, ':');
 if (colon == -1)
     throw new OpenrjException(linnum, "'key : value' expected");

 char[] key = std.string.strip(line[0 .. colon]);
 char[] value = std.string.strip(line[colon + 1 .. length]);

 char[][] fields = record[key];
 fields ~= value;
 record[key] = fields;
    }
    if (record)
 rj ~= record;
    return rj;
}
--------------------------------------------------------------------
And here's a simple driver for it:
--------------------------------------------------------------------
import std.stdio;
import std.file;
import std.openrj;

int main()
{
    char[] db = cast(char[])std.file.read("test.rj");
    openrj_t rj = std.openrj.parse(db);

    foreach (char[][] [char[]] record; rj)
    {
 foreach (char[] key, char[][] fields; record)
 {
     writefln(key, ":");
     foreach (char[] field; fields)
     {
  writefln("\t", field);
     }
 }
 writefln("---------------------");
    }

    return 0;
}
----------------------------------------------------

"Rajiv Bhagwat" <dataflow@vsnl.com> wrote in message news:d74nle$2j8j$1@digitaldaemon.com...
> I have taken Walter's advice - instead of posting files here, I have
posted
> an article on CodeProject.
> "Implementing 'foreach' for c++ as a Design Pattern"
> http://www.codeproject.com/
>
> Have attached the alternate OpenRj implementation as the 'sample project'. It has been tested (obviously) on DMC, as well as on VC6 & gcc under
linux.
>
> - Rajiv

The link is to the front page, which is awfully busy. Can we please have a link to the sub-page with the article? <g>

The new articles are placed in 'unedited user contributions'. They are moved later to appropriate sections later. Thats why I did not give the complete link. Pls look under 'C++ / MFC'.

-Rajiv


"Walter" <newshound@digitalmars.com> wrote in message news:d7570j$3085$2@digitaldaemon.com...
>
> "Rajiv Bhagwat" <dataflow@vsnl.com> wrote in message news:d74nle$2j8j$1@digitaldaemon.com...
> > I have taken Walter's advice - instead of posting files here, I have
> posted
> > an article on CodeProject.
> > "Implementing 'foreach' for c++ as a Design Pattern"
> > http://www.codeproject.com/
> >
> > Have attached the alternate OpenRj implementation as the 'sample
project'.
> > It has been tested (obviously) on DMC, as well as on VC6 & gcc under
> linux.
> >
> > - Rajiv
>
> The link is to the front page, which is awfully busy. Can we please have a link to the sub-page with the article? <g>
>
>

"Derek Parnell" <derek@psych.ward> wrote in message news:xz4xopc9utmh$.1ocjw4mimo341.dlg@40tude.net...
> On Tue, 24 May 2005 08:30:41 +1000, Matthew wrote:
> >    for(memory_database::const_iterator ri = db.begin(); ri !=
> > db.end(); ++ri) // Loop over the records
> >    {
> >        record    r(*ri);
> >
> >        std::cout << r.comment() << std::endl;
> >
> >        for(record::const_iterator fi = r.begin(); fi != r.end();
> > ++fi) // Loop over the fields
> >       {
> >            field f(*fi);
> >
> >            std::cout << f.name() << "= '" << f.value() << "'" <<
> > std::endl;
> >        }
> >        std::cout << std::endl;
> >    }
> >
> > And I'd respectfully suggest that it would be churlish to suggest that that's not easy to read.
>
>
> LOL!!! I be churlish then.

Perhaps it's my age, but I have never found any of the C++ iostreams' use of << and >> to be easy to read in any manner. The same goes for the < and > of C++ templates. My brain just parses them as shift and comparison operators, and refuses to change :-)