dxml 0.2.0 has now been released.

I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.

As part of that, it became clear that dxml.parser.stax should be renamed to dxml.parser, since it's really the only parser (DOM support involves just providing a way to hold the results of the parser, not any actual parsing, and that's clear from the API rather than being an implementation detail), and it makes for a shorter import path. So, I figured that I should do a release sooner rather than later to reduce how many folks the rename ends up affecting.

For this release, dxml.parser.stax is now an empty, deprecated, module that publicly imports dxml.parser, but it will be removed in 0.3.0, whenever that is released. So, the few folks who grabbed the initial release won't end up with immediate code breakage if they upgrade.

One nice side effect of how I implemented DOM support is that it's trivial to get the DOM for a portion of an XML document rather than the entire thing, since it will produce a DOMEntity from any point in an EntityRange.

Documentation: http://jmdavisprog.com/docs/dxml/0.2.0/
Github: https://github.com/jmdavis/dxml/tree/v0.2.0
Dub: http://code.dlang.org/packages/dxml

- Jonathan M Davis

On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis wrote:
> dxml 0.2.0 has now been released.
>
> I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.
>
> As part of that, it became clear that dxml.parser.stax should be renamed to dxml.parser, since it's really the only parser (DOM support involves just providing a way to hold the results of the parser, not any actual parsing, and that's clear from the API rather than being an implementation detail), and it makes for a shorter import path. So, I figured that I should do a release sooner rather than later to reduce how many folks the rename ends up affecting.
>
> For this release, dxml.parser.stax is now an empty, deprecated, module that publicly imports dxml.parser, but it will be removed in 0.3.0, whenever that is released. So, the few folks who grabbed the initial release won't end up with immediate code breakage if they upgrade.
>
> One nice side effect of how I implemented DOM support is that it's trivial to get the DOM for a portion of an XML document rather than the entire thing, since it will produce a DOMEntity from any point in an EntityRange.
>
> Documentation: http://jmdavisprog.com/docs/dxml/0.2.0/
> Github: https://github.com/jmdavis/dxml/tree/v0.2.0
> Dub: http://code.dlang.org/packages/dxml
>
> - Jonathan M Davis

Awesome. Just tried it now as below and it works. Thanks for this library

import std.stdio;

import dxml.dom;

struct Record
{
    string name;
    string email;
}


Record[] parseRecords(string xml)
{
    Record[] records;
    auto d = parseDOM!simpleXML(xml);
    auto root = d.children[0];

    foreach(record; root.children)
    {
        auto rec = Record();
        foreach(ele; record.children)
        {
            if (ele.name == "name")
                rec.name = ele.children[0].text;
            if (ele.name == "email")
                rec.email = ele.children[0].text;
        }
        records ~= rec;
    }

    return records;
}

void main()
{
    auto xml = "<root>\n" ~
        "    <record>\n" ~
        "        <name>N1</name>\n" ~
        "        <email>E1</email>\n" ~
        "    </record>\n" ~
        "    <record>\n" ~
        "        <name>N2</name>\n" ~
        "        <email>E2</email>\n" ~
        "    </record>\n" ~
        "    <record>\n" ~
        "        <email>E3</email>\n" ~
        "        <name>N3</name>\n" ~
        "    </record>\n" ~
        "<!--no comment -->\n" ~
        "</root>";
    auto records = parseRecords(xml);
    writeln(records);
}

On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis wrote:
> dxml 0.2.0 has now been released.
>
> I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.
>
> [...]

Will this replace `std.xml` one day?

On 12/02/2018 12:38 PM, Chris wrote:
> On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis wrote:
>> dxml 0.2.0 has now been released.
>>
>> I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.
>>
>> [...]
> 
> Will this replace `std.xml` one day?

As long as DTD support is essentially non-existent, my vote will always be no.

On Monday, 12 February 2018 at 12:49:30 UTC, rikki cattermole wrote:
> On 12/02/2018 12:38 PM, Chris wrote:
>> On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis wrote:
>>> dxml 0.2.0 has now been released.
>>>
>>> I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.
>>>
>>> [...]
>> 
>> Will this replace `std.xml` one day?
>
> As long as DTD support is essentially non-existent, my vote will always be no.

How hard would it be to add DTD support? One could take dxml and extend it in order to include it in Phobos. I haven't used `std.xml` for years now. It is essentially dead and unusable atm.

On Monday, February 12, 2018 12:38:51 Chris via Digitalmars-d-announce wrote:
> On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis
>
> wrote:
> > dxml 0.2.0 has now been released.
> >
> > I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.
> >
> > [...]
>
> Will this replace `std.xml` one day?

Maybe. That depends on community feedback and ultimately on the Phobos review process. Assuming that there's support for putting it through the Phobos review process, then once I feel that it's complete enough and had enough use to make it clear that I didn't miss something critical, then I'll submit it for review.

What little feedback there has been thus far has been positive, but it would be nice to get it battle-tested a bit, and there is still functionality that I need to add.

Given that std.xml needs to be replaced, I think that it would be good if dxml were able to do that, but that depends heavily on what others think of what I've done and what they think Phobos' xml solution should look like. But the way things are going though, if dxml doesn't replace std.xml, I don't know that anything ever will. XML parsers are one of those things that everyone seems to want and no one seems to want to work on.

However, if folks as a whole think that Phobos' xml parser needs to support the DTD section to be acceptable, then dxml won't replace std.xml, because dxml is not going to implement DTD support. DTD support fundamentally does not fit in with dxml's design. Someone would basically have to write an entirely new parser to be able to handle it (some of dxml's internals could be reused, but they'd also have to be refactored a fair bit, and a ton of extra stuff would have to be added). Such a parser could theoretically coexist with dxml's parser, since each would provide its own advantages, but I have no plans to implement an XML parser to handle the DTD section. It's simply not worth my time or effort, and this project has already taken way more time and effort than I anticipated.

However, std.xml does not support the DTD section, and glancing over it, it doesn't look like it even handles skipping the DTD section properly (it doesn't handle the fact that '>' can appear within quoted sections within the DTD). So, dxml is not worse than std.xml in that regard, and we wouldn't lose any functionality by having dxml replace std.xml. It just wouldn't necessarily do as much as some folks might like.

My guess is that DTD support won't be a deal breaker given that std.xml doesn't support it, that std.xml has needed to be replaced for years now, and that no one else is working on replacing it, but I don't know. Disagreements over what should be done with std.json's replacement has meant that it has never been replaced even though significant work was done towards replacing it, so unfortunately, there's already precedence for a module not being replaced with something better due to disagreements over what the replacement would ideally be. So, I don't know.

- Jonathan M Davis

On 12/02/2018 1:51 PM, Chris wrote:
> On Monday, 12 February 2018 at 12:49:30 UTC, rikki cattermole wrote:
>> On 12/02/2018 12:38 PM, Chris wrote:
>>> On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis wrote:
>>>> dxml 0.2.0 has now been released.
>>>>
>>>> I really wasn't planning on releasing anything this quickly after announcing dxml, but when I went to start working on DOM support, it turned out to be surprisingly quick and easy to implement. So, dxml now has basic DOM support.
>>>>
>>>> [...]
>>>
>>> Will this replace `std.xml` one day?
>>
>> As long as DTD support is essentially non-existent, my vote will always be no.
> 
> How hard would it be to add DTD support? One could take dxml and extend it in order to include it in Phobos. I haven't used `std.xml` for years now. It is essentially dead and unusable atm.

From what I read in the other thread, it would require a complete redesign and a major performance hit.

I don't care what J.M.D. puts in his own library. We just can't advertise to having an 'XML' library when we out right ignore a large portion of (and fairly important to real world adoption IMO) the specification for no other reason than personal opinions of the author.

Now if you want a subset as the 'default' but have full support including DTD as an opt-in with the only difference is how you initialize the parser, I'd be happy and so will our end users in the future.

On Monday, 12 February 2018 at 14:04:38 UTC, Jonathan M Davis wrote:
> On Monday, February 12, 2018 12:38:51 Chris via Digitalmars-d-announce wrote:
>> On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis

>
> However, std.xml does not support the DTD section, and glancing over it, it doesn't look like it even handles skipping the DTD section properly (it doesn't handle the fact that '>' can appear within quoted sections within the DTD). So, dxml is not worse than std.xml in that regard, and we wouldn't lose any functionality by having dxml replace std.xml. It just wouldn't necessarily do as much as some folks might like.

I thought the same when I glanced over std.xml. There's no DTD support there either and I don't think it would be a deal breaker for most users.

> My guess is that DTD support won't be a deal breaker given that std.xml doesn't support it, that std.xml has needed to be replaced for years now, and that no one else is working on replacing it, but I don't know. Disagreements over what should be done with std.json's replacement has meant that it has never been replaced even though significant work was done towards replacing it, so unfortunately, there's already precedence for a module not being replaced with something better due to disagreements over what the replacement would ideally be. So, I don't know.
>
> - Jonathan M Davis

Wasn't there a replacement module that never got past the initial review steps? Some GSoC thing or so. But I wonder if that module would be up to the latest D standards.

While one may argue that DTD support is important, I would rather have something fast and simple like dxml that covers, say, 90% of the cases than nothing. It doesn't make sense to me that we should accept the current situation, only because of some bikeshedding that concerns 10% of the use cases. After all, it's only a module not a fundamental decision that concerns the direction D will take in the future. I think stuff like that can seriously turn off potential users. A lot of useful things begin with one person deciding to give it a go. vibe.d, dub, DScanner and DlangUI, for example. If the creators had started bikeshedding before writing the first line of code, there would still be a flamewar about the best way to go about it - and nothing would have happened.

On Monday, 12 February 2018 at 14:04:38 UTC, Jonathan M Davis wrote:
> XML parsers are one of those things that everyone seems to want and no one seems to want to work on.

I wrote one 8 years ago... though mine is more focused on HTML parsing, and the XML aspect is just a side effect!

On 12/02/2018 2:45 PM, Chris wrote:
> On Monday, 12 February 2018 at 14:04:38 UTC, Jonathan M Davis wrote:
>> On Monday, February 12, 2018 12:38:51 Chris via Digitalmars-d-announce wrote:
>>> On Monday, 12 February 2018 at 05:36:51 UTC, Jonathan M Davis
> 
>>
>> However, std.xml does not support the DTD section, and glancing over it, it doesn't look like it even handles skipping the DTD section properly (it doesn't handle the fact that '>' can appear within quoted sections within the DTD). So, dxml is not worse than std.xml in that regard, and we wouldn't lose any functionality by having dxml replace std.xml. It just wouldn't necessarily do as much as some folks might like.
> 
> I thought the same when I glanced over std.xml. There's no DTD support there either and I don't think it would be a deal breaker for most users.
> 
>> My guess is that DTD support won't be a deal breaker given that std.xml doesn't support it, that std.xml has needed to be replaced for years now, and that no one else is working on replacing it, but I don't know. Disagreements over what should be done with std.json's replacement has meant that it has never been replaced even though significant work was done towards replacing it, so unfortunately, there's already precedence for a module not being replaced with something better due to disagreements over what the replacement would ideally be. So, I don't know.
>>
>> - Jonathan M Davis
> 
> Wasn't there a replacement module that never got past the initial review steps? Some GSoC thing or so. But I wonder if that module would be up to the latest D standards.

https://github.com/dlang-community/experimental.xml

Code isn't great, and not complete yet.
Author has just disappeared sadly.

> While one may argue that DTD support is important, I would rather have something fast and simple like dxml that covers, say, 90% of the cases than nothing. It doesn't make sense to me that we should accept the current situation, only because of some bikeshedding that concerns 10% of the use cases. After all, it's only a module not a fundamental decision that concerns the direction D will take in the future. I think stuff like that can seriously turn off potential users. A lot of useful things begin with one person deciding to give it a go. vibe.d, dub, DScanner and DlangUI, for example. If the creators had started bikeshedding before writing the first line of code, there would still be a flamewar about the best way to go about it - and nothing would have happened.

Everything you have mentioned is not in Phobos. Just because something is 'good enough' does not make it 'good enough' for Phobos. In the words of Andrei "Good enough is not good enough", we need to aim higher to show what we actually can do.

Personally I find J.M.D. arguments quite reasonable for a third-party library, since yes it does cover 90% of the use cases.