The current std.xml needs a replacement (I think it has already been decided so), something with a familiar DOM API with facilities to manipulate XML documents.

I needed to do a quick XML file transform and std.xml was of little use. Then I found this Adam D. Ruppe's library:

https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff

Its "dom" module has that sort of Javascript-like DOM manipulation code. It has getElementsByTagName(), getElementById(), firstChild, nodeValue, innerText (read/write), toString, etc. Easy and performing. The XML processing I needed to make took minutes with that!

I thus suggest, if licensing allows (?) and no better exists, to base a newer std.xml module on that code as a starting point. Well, after cleaning up and fixing anything necessary. For instance, that module was designed for Web server programming and is targeted at HTML DOM mainly. It has stuff to deal with CSS styles, HTML specific elements and their special handling (table, a, form, audio, input, ...). A lot of that can be left out in XML.

On Tuesday, February 07, 2012 00:15:37 Alvaro wrote:
> The current std.xml needs a replacement (I think it has already been decided so), something with a familiar DOM API with facilities to manipulate XML documents.
> 
> I needed to do a quick XML file transform and std.xml was of little use. Then I found this Adam D. Ruppe's library:
> 
> https://github.com/adamdruppe/misc-stuff-including-D-programming-language-we b-stuff
> 
> Its "dom" module has that sort of Javascript-like DOM manipulation code.
> It has getElementsByTagName(), getElementById(), firstChild, nodeValue,
> innerText (read/write), toString, etc. Easy and performing. The XML
> processing I needed to make took minutes with that!
> 
> I thus suggest, if licensing allows (?) and no better exists, to base a newer std.xml module on that code as a starting point. Well, after cleaning up and fixing anything necessary. For instance, that module was designed for Web server programming and is targeted at HTML DOM mainly. It has stuff to deal with CSS styles, HTML specific elements and their special handling (table, a, form, audio, input, ...). A lot of that can be left out in XML.

Tomek Sowiński was working on a new std.xml which was intended to become the new std.xml, but he hasn't posted since June, so I have no idea where that stands. And I believe that xmlp is intended as a possible candidate as well.

The main issue is that we need someone to design it, implement it, and push it through the review process. Just suggesting it is not enough. Someone needs to champion the new module. And no one has gone all the way with that yet.

Also, two of the major requirements for an improved std.xml are that it needs to have a range-based API, and it needs to be fast. I don't know how Adam's stacks up against that. Tango's XML parser has pretty much set the bar on speed ( http://dotnot.org/blog/archives/2008/03/12/why-is-dtango-so-fast-at- parsing-xml/ ), and while we may not reach that (especially with the initial implementation), we want a design which is going to be fast and have the potential of reaching Tango-like speeds (whether that's currently possible or not with a range-based interface is probably highly dependent on dmd's current optimizition capabalities - inlining in particular).

So, if Adam wants to work on getting his XML module into Phobos (which I question, since if he really wanted to, I would have expected him to do it already) or if someone else wants to work on it (and the license allows it), then it may be possible to get it into Phobos. But someone needs to put all of that time and effort in.

- Jonathan M Davis

On Monday, 6 February 2012 at 23:47:08 UTC, Jonathan M Davis wrote:
> Also, two of the major requirements for an improved std.xml are that it needs to have a range-based API, and it needs to be fast.

What does range based API mean in this context? I do offer
a couple ranges over the tree, but it really isn't the main
thing there.

Check out Element.tree() for the main one.


But, if you mean taking a range for input, no, doesn't
do that. I've been thinking about rewriting the parse
function (if you look at it, you'll probably hate it
too!). But, what I have works and is tested on a variety
of input, including garbage that was a pain to get working
right, so I'm in no rush to change it.

> Tango's XML parser has pretty much set the bar on speed

Yeah, I'm pretty sure Tango whips me hard on speed. I spent
some time in the profiler a month or two ago and got a
significant speedup over the datasets I use (html files),
but I'm sure there's a whole lot more that could be done.



The biggest thing is I don't think you could use my parse
function as a stream.

On Monday, 6 February 2012 at 23:15:50 UTC, Alvaro wrote:
> Its "dom" module has that sort of Javascript-like DOM manipulation code

I'm biased (obviously), but I've never seen a more convenient
way to work with xml. I like my convenience functions a lot.


> I thus suggest, if licensing allows (?

It is Boost licensed, so you can use it (I put the license
at the bottom of my files).

I don't know if it is phobos material, but if there's demand,
maybe I can make that happen.

> A lot of that can be left out in XML.

Right, though you can ignore it too. I sometimes use it
to work with other kinds of xml (web apis sending and receiving),
RSS, and others.

It works well enough for me.

You know it, web stuff documentation is a weak point. web stuff looks very interesting ...  so a real world sample app would be nice..

I would like to see a sample RIA -  M- V-C  wise, using (say using dojo dijit as View layer ) in conjunction with the D web stuff . (Model- Controler)

I think atm your library is not made for the masses, but it would be nevertheless interesting to see how someone can glue backend (web stuff) and frontend stuff (dojo/dijit) together.
thanks for reading.

On Tuesday, 7 February 2012 at 19:27:46 UTC, bls wrote:
> You know it, web stuff documentation is a weak point.

Yeah, I know.

> looks very interesting ...  so a real world sample app would be nice..

The closest i have is a little blog like thing that I started
and haven't worked on since.

http://arsdnet.net/blog/my-source

> I would like to see a sample RIA -  M- V-C  wise, using (say using dojo dijit as View layer ) in conjunction with the D web stuff . (Model- Controler)

I haven't used one of those toolkits, but plain javascript
is easy to do like this.


I'm writing a little browser game on the weekends. When I finish
it, I might be able to show it to you. (I'll have to replace the
copyrighted images in there right now though.)



Basically you just call the server functions in your event handlers.


Want to replace a div with content from the other side?

D
===
import arsd.web;
class Server : ApiProvider {
  string hello(string name) { return "hello " ~ name; }
}
mixin FancyMain!Server;
===

HTML/Javascript
===
<script src="app/functions.js"></script>
<div id="holder"></div>
<button onclick="Server.hello(prompt('Name?')).useToReplace('holder');">Click me</button>
===



And when you click that button, the string D returns will be
put inside as text.

That's the basic of it.




I'm taking this to an extreme with this:

http://arsdnet.net:8080/

(that's my custom D web server, so if it doesn't respond, I probably
closed the program. Will leave it open for a while though.)


Click on the link and the body. You should get some text added.



Take a look at the javascript though:
http://arsdnet.net:8080/sse.js


	document.body.addEventListener("click", function(event) {
		var response = Test.cancelableEvent(
			event.type,
			event.target.getAttribute("data-node-index")
		).getSync();
 [snip]



It uses synchronous ajax on the click handler to forward the
event to the server, which then passes it through the server
side dom.


D:
===

/* the library implements EventResponse cancelableEvent(...),
which dispatches the event on the server, so you can write: */

        // body is a keyword in D, hence mainBody instead.
document.mainBody.addEventListener("click"), (Event ev) {
    ev.target.appendText("got click!");
    ev.preventDefault();
});

===


sync ajax requests suck because it blocks the whole web app
waiting for a response. But writing server side event handlers
is a fun toy.








I do write real applications with web.d, but they are almost
all proprietary, so little toys is all I really have to show
right now.

> On Tuesday, 7 February 2012 at 19:27:46 UTC, bls wrote:
>>
>> You know it, web stuff documentation is a weak point.
>
>
> Yeah, I know.
>
>
>> looks very interesting ...  so a real world sample app would be nice..
>
>
> The closest i have is a little blog like thing that I started and haven't worked on since.
>
> http://arsdnet.net/blog/my-source
>
>
>> I would like to see a sample RIA -  M- V-C  wise, using (say using dojo
>> dijit as View layer ) in conjunction with the D web stuff . (Model-
>> Controler)
>
>
> I haven't used one of those toolkits, but plain javascript is easy to do like this.
>
>
> I'm writing a little browser game on the weekends. When I finish it, I might be able to show it to you. (I'll have to replace the copyrighted images in there right now though.)
>
>
>
> Basically you just call the server functions in your event handlers.
>
>
> Want to replace a div with content from the other side?
>
> D
> ===
> import arsd.web;
> class Server : ApiProvider {
>  string hello(string name) { return "hello " ~ name; }
> }
> mixin FancyMain!Server;
> ===
>
> HTML/Javascript
> ===
> <script src="app/functions.js"></script>
> <div id="holder"></div>
> <button
> onclick="Server.hello(prompt('Name?')).useToReplace('holder');">Click
> me</button>
> ===
>
>
>
> And when you click that button, the string D returns will be put inside as text.
>
> That's the basic of it.
>
>
>
>
> I'm taking this to an extreme with this:
>
> http://arsdnet.net:8080/
>
> (that's my custom D web server, so if it doesn't respond, I probably closed the program. Will leave it open for a while though.)
>
>
> Click on the link and the body. You should get some text added.
>
>
>
> Take a look at the javascript though: http://arsdnet.net:8080/sse.js
>
>
>        document.body.addEventListener("click", function(event) {
>                var response = Test.cancelableEvent(
>                        event.type,
>                        event.target.getAttribute("data-node-index")
>                ).getSync();
>  [snip]
>
>
>
> It uses synchronous ajax on the click handler to forward the event to the server, which then passes it through the server side dom.
>
>
> D:
> ===
>
> /* the library implements EventResponse cancelableEvent(...), which dispatches the event on the server, so you can write: */
>
>        // body is a keyword in D, hence mainBody instead.
> document.mainBody.addEventListener("click"), (Event ev) {
>    ev.target.appendText("got click!");
>    ev.preventDefault();
> });
>
> ===
>
>
> sync ajax requests suck because it blocks the whole web app waiting for a response. But writing server side event handlers is a fun toy.
>
>
>
>
>
>
>
>
> I do write real applications with web.d, but they are almost all proprietary, so little toys is all I really have to show right now.

As somebody that frequently laments the lack of documentation in general (I use Silverstripe at work, in which the documentation is patchy at best) I work hard on my documentation.

Adam's stuff is very good, I plan to take a look at it and "borrow" some code for a web framework I'm working on. But I am also open-sourcing modules that I am using as I go along. It would be cool if we ended up with a set of independent modules that worked well together but had few dependencies.

I guess the point is whether Adam is ok with the community extending his work separately, since "misc-stuff-including-D-programming-language-web-stuff" isn't exactly a catchy name :P.

It would be unfortunate if Adam's work wasn't built upon, and that work was needlessly replicated, then tightly integrated into some other project, rather than being something that everybody could use.

On Tuesday, February 07, 2012 00:56:40 Adam D. Ruppe wrote:
> On Monday, 6 February 2012 at 23:47:08 UTC, Jonathan M Davis
> 
> wrote:
> > Also, two of the major requirements for an improved std.xml are that it needs to have a range-based API, and it needs to be fast.
> 
> What does range based API mean in this context? I do offer a couple ranges over the tree, but it really isn't the main thing there.
> 
> Check out Element.tree() for the main one.
> 
> 
> But, if you mean taking a range for input, no, doesn't
> do that. I've been thinking about rewriting the parse
> function (if you look at it, you'll probably hate it
> too!). But, what I have works and is tested on a variety
> of input, including garbage that was a pain to get working
> right, so I'm in no rush to change it.
> 
> > Tango's XML parser has pretty much set the bar on speed
> 
> Yeah, I'm pretty sure Tango whips me hard on speed. I spent some time in the profiler a month or two ago and got a significant speedup over the datasets I use (html files), but I'm sure there's a whole lot more that could be done.
> 
> 
> 
> The biggest thing is I don't think you could use my parse function as a stream.

Ideally, std.xml would operate of ranges of dchar (but obviously be optimized for strings, since there are lots of optimizations that can be done with string processing - at least as far as unicode goes) and it would return a range of some kind. The result would probably be a document type of some kind which provided a range of its top level nodes (or maybe just the root node) which each then provided ranges over their sub-nodes, etc. At least, that's the kind of thing that I would expect. Other calls on the document and nodes may not be range-based at all (e.g. xpaths should probably be supported, and that doesn't necessarily involve ranges). The best way to handle it all would probably depend on the implementation. I haven't implemented a full-blown XML parser, so I don't know what the best way to go about it would be, but ideally, you'd be able to process the nodes as a range.

- Jonathan M Davis

On Wednesday, 8 February 2012 at 01:33:51 UTC, James Miller wrote:
> As somebody that frequently laments the lack of documentation

I'm a bit of a hypocrite here; I'll complain until the cows
come home about /other/ people's crappy documentation...
then turn around and do a bad job at it myself.

I tried to do cgi.d
overview: http://arsdnet.net/web.d/cgi.d.html
   ddoc: http://arsdnet.net/web.d/cgi.html


But I still have a lot to do on web.d documentation.
The best source there is probably random newsgroup posts,
(better than the comments in the code!) and I've changed
things more than once too.

> Adam's stuff is very good

Thanks!


> It would be cool if we ended up with a set of independent
> modules that worked well together but had few dependencies.

Yeah, I think that would be great.


If you want to modify my files, you can always fork it,
but the best way is probably to do a pull request on the
github.

I try to reply to those quickly, and we can try to keep
one copy there to avoid duplication of effort.

Then, add on modules outside the scope of the code can be
done by anyone. I'd put links to stuff in the documentation,
so it is easy to find extended functionality.

Here's more ddocs.

http://arsdnet.net/web.d/web.html
http://arsdnet.net/web.d/dom.html


Not terribly useful, I'll admit. The Javascript
discussion at the bottom of the first link might be
good to read though.

The dom.html there is mostly just (incomplete) method
listing. I didn't write most of it up at all.

When I started doing dom.d, I was going to be strictly
a clone of the browser implementation, so some of the
comments still say things like "extension", but I went
my own direction a long time ago.

I don't know... I think most of the dom is self-explanatory
if you're already experienced with javascript anyway though.


BTW here's something I wrote up on aother site as an
example of how awesome D is:




examples:
[code="D ROX"]

// don't need a document to create elements
auto div = Element.make("div", "Hello <there>"); // convenience params do innerText
assert(div.innerText == "Hello <there>"); // text set
assert(div.innerHTML == "Hello &lt;there&gt;"); // html properly encoded mindlessly

// getting and setting attributes works w/ property syntax
// of course they are always properly encoded for charset and html stuffs
div.id = "whoa";
assert(div.id == "whoa");
div.customAtrribute = div.id ~ "works";
assert(div.customAttribute == "whoaworks");

// i also support the dataset sugar over attributes

div.dataset.myData = "cool";
assert(div.getAttribute("data-my-data") == "cool" == div.dataset.myData);

// as well as with the style..

div.style = "top: 10px;" // works with strings just like any other attribute

div.style.top = "15px"; // or with property syntax like in javascript

assert(div.style.top == "15px"); // reading works no matter how you set it
assert(div.style == "top: 15px;"); // or you can read it as a string


// i have convenience methods too

div.innerText = "sweet"; // worx, w/ proper encoding

// calls Element.make and appendChild in one go.
// can easily set text and/or a tag specific attribute
auto a = div.addChild("a", "Click me!", "link.html"); // tagName, value, value2, dependent on tag

a.addClass("cool").addClass("beans"); // convenience methods (chainable btw) for the class attribute
assert(a.hasClass("cool") && a.hasClass("beans") && !a.hasClass("coolbeans"));
assert(a.className == "cool beans");

// subclasses rock too, especially with automatic checked casts
Link link = div.requireSelector!Link("a"); // css selector syntax supported
// alternatively:
link = cast(Link) a; // but if a isn't actually a Link, this can be null

// easy working with the link url
a.setValue("cool", "param");
assert(a.href == "link.html?cool=param");

// arsd.html also includes functions to convert links into POST forms btw

// the Form subclass rox too

auto form = cast(Form) Element.make("form");
form.method = "POST";
// convenience functions from the browsers are here but who needs them when the dom rox?
form.innerHTML = "<select name=\"cool\"><option>Whoa</option><option>WTF</option></select>";

// similarly to the Link class, you can set values with ease
form.setValue("cool", "WTF"); // even works on non-text form elements
form.setValue("sweet", "whoa"); // also can implicitly create a hidden element to carry a value (can lead to mistakes but so useful)


// and the Table class is pretty sweet

auto table = cast(Table) Element.make("table");
table.caption = "Cool"; // sets the caption element, not an attribute

// variadic templates rok
table.appendRow("sweet", "whoa", 10, Element.make("a")); // adds to the <tbody> a <tr> with appropriate children
[/code]


some people luv jquery and think it is the best thing ever

those people have never used my dom library


Speaking of jquery, what about collections of elements?

Well D has this thing called foreach which does that. But, just
to prove I could, I wrote a couple dozen lines of D to do this:

==
document["p a"].addClass("mutated").appendText("all links in paragraphs get this text and that class!");
==



Operator overloading template craziness!

But i'm pretty meh on that since foreach is better anyway.



foreach rox

d rox