Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions.

First, and most importantly, what do we except out of a D XML library? I'd really like to have a discussion of the form, "Here is exactly the interface the structs/classes need to implement, go forth and implement." The general idea in my mind is "something SAX-like, with something a little DOM-like." I'm aware that std.xml has some issues support different encodings, so obvious that's included.

Second, is there an existing library that has gotten close to meeting whatever we need for the first point? If so, how far away is it from being able to meet all of the requirements and become the standard library version?

On Thursday, August 29, 2013 09:25:35 w0rp wrote:
> Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions.

Someone needs to step forward, write it, and get it through the review process. A while back, someone was working on a possible new version of std.xml, but they disappeared. No one has stepped up since. I'd love to do it if I had time, but I don't. There are probably several others around here in the same boat, but until someone who has the time and skill does do it, we won't have a new std.xml.

> First, and most importantly, what do we except out of a D XML library? I'd really like to have a discussion of the form, "Here is exactly the interface the structs/classes need to implement, go forth and implement."

Except that that's really the task of the person creating the new std.xml. Generally what happens is that the person writing the module comes up with an API and then presents it rather than asking others to come up with ideas to design it for them. Obviously, ideas can be discussed, but design-by-committee is arguably a bad idea. And it just works better to have a concrete design to discuss.

> The general idea in my mind is
> "something SAX-like, with something a little DOM-like."

What I personally think would be best is to have multiple parsers. First you have something STAX-like (or maybe even lower level - I don't recall exactly what STAX gives you at the moment) that basically tokenizes the XML and returns a range of that. Then SAX and DOM parsers can be built on top of that. That way, you get the fastest parser possible as well as higher level, more functional parsers.

But two of the biggest points of the design are that it's going to have to be range-based, and it's going to need to be able to take full advantage of slices (when used with any strings or random-access ranges) in order to avoid copying any of the data. That's the key design point which will allow a D parser to be extremely fast in comparison to parsers in most other languages.

> I'm aware
> that std.xml has some issues support different encodings, so
> obvious that's included.

Personally, I would have just said use ranges of dchar and be done with it without worrying about character encodings at all, but I don't remember what all the XML standard does with encodings.

> Second, is there an existing library that has gotten close to meeting whatever we need for the first point? If so, how far away is it from being able to meet all of the requirements and become the standard library version?

There are several D XML libraries floating around, but no one has taken the time to get any of the prepared for the Phobos review queue, and I suspect that very few of them are range-based like the Phobos XML solution needs to be, but I don't know.

- Jonathan M Davis

Am Thu, 29 Aug 2013 09:25:35 +0200
schrieb "w0rp" <devw0rp@gmail.com>:

> Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions.
> 
> First, and most importantly, what do we except out of a D XML library? I'd really like to have a discussion of the form, "Here is exactly the interface the structs/classes need to implement, go forth and implement." The general idea in my mind is "something SAX-like, with something a little DOM-like." I'm aware that std.xml has some issues support different encodings, so obvious that's included.

I most points here also apply to std.xml:
http://wiki.dlang.org/Wish_list/std.json
Those are not strict requirements though, I just summarized what I
remembered from old discussions.

> Second, is there an existing library that has gotten close to meeting whatever we need for the first point? If so, how far away is it from being able to meet all of the requirements and become the standard library version?

There's a std.xml2 in the review queue: http://wiki.dlang.org/Review_Queue

On Thursday, 29 August 2013 at 07:25:36 UTC, w0rp wrote:
> Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions.
>
> First, and most importantly, what do we except out of a D XML library? I'd really like to have a discussion of the form, "Here is exactly the interface the structs/classes need to implement, go forth and implement." The general idea in my mind is "something SAX-like, with something a little DOM-like." I'm aware that std.xml has some issues support different encodings, so obvious that's included.
>
> Second, is there an existing library that has gotten close to meeting whatever we need for the first point? If so, how far away is it from being able to meet all of the requirements and become the standard library version?

There is http://dsource.org/projects/xmlp, which at some point has been proposed for std.xml2. But that stalled for some time now.

On 08/29/2013 09:51 AM, Johannes Pfau wrote:
> I most points here also apply to std.xml: http://wiki.dlang.org/Wish_list/std.json Those are not strict requirements though, I just summarized what I remembered from old discussions.
I think, this even extends to access to all semi- and structured-data. Think csv, sql nosql, you name it. Something which deserves a name like Uniform Access. I don't want to care if data is laid out differently. I want to define my struct or class mark the members to fill a pass it to somebodies code and don't want to care if its xml, sql or whatever.

On 2013-08-29 09:47, Jonathan M Davis wrote:

> Personally, I would have just said use ranges of dchar and be done with it
> without worrying about character encodings at all, but I don't remember what
> all the XML standard does with encodings.

Won't that have the same problem as we talked about in of the threads about a D lexer? That is, doing unnecessary en/decoding.

-- 
/Jacob Carlborg

On 2013-08-29 10:15, Robert Schadek wrote:


> I think, this even extends to access to all semi- and structured-data.
> Think csv, sql nosql, you name it. Something which deserves a name like
> Uniform Access. I don't want to care if data is laid out differently. I
> want to define my struct or class mark the members to fill a pass it to
> somebodies code and don't want to care if its xml, sql or whatever.

So you want serialization :). Which we currently are reviewing. Unfortunately there might be too many changes needed to get it in Phobos this time.

-- 
/Jacob Carlborg

On Thursday, August 29, 2013 11:08:18 Jacob Carlborg wrote:
> On 2013-08-29 09:47, Jonathan M Davis wrote:
> > Personally, I would have just said use ranges of dchar and be done with it without worrying about character encodings at all, but I don't remember what all the XML standard does with encodings.
> 
> Won't that have the same problem as we talked about in of the threads about a D lexer? That is, doing unnecessary en/decoding.

Possibly, but then all you have to do is make it so that it treats strings as ranges of code units (and possibly support ranges of char and wchar), and you can avoid the unnecessary decoding. But aside from possibly support ranges of char or wchar, that would be completely internal to the parser, and the caller wouldn't care. An alternative would be to specifically support ranges of ubyte instead of strings, though given that XML is usually treated as a string, that would arguably be a bit odd. Regardless, as far as strings go, it's easy enough to avoid decoding in the implementation. IIRC, everything in XML is ASCII anyway, with stuff like HTML codes to indicate Unicode characters. And if that's the case, avoiding unnecessary decoding is trivial when operating on strings.

- Jonathan M Davis

On Thursday, 29 August 2013 at 07:47:35 UTC, Jonathan M Davis wrote:
> There are several D XML libraries floating around, but no one has taken the
> time to get any of the prepared for the Phobos review queue, and I suspect
> that very few of them are range-based like the Phobos XML solution needs to
> be, but I don't know.
I think it's great that there's no std.xml, as it implies that nobody using D would use a dumb tech like XML.  Let's keep it that way. :)

On 08/29/2013 11:09 AM, Jacob Carlborg wrote:
>
> So you want serialization :). Which we currently are reviewing. Unfortunately there might be too many changes needed to get it in Phobos this time.
>
well, sort of, but also with partial serialization (think sql update), more transparent interface and I want to define join results types at compile time and and and