On Thursday, 12 April 2012 at 18:36:01 UTC, SomeDude wrote:
> Hi,
>
> I've noticed a couple of times (i.e very rarely, mind you), that the page generation of the forum had hiccups.
> Have other people experienced such hiccups ?
> If yes, I wonder if we saw the GC kicking in, or if it was SQLite's fault (or was it a network latency ?).
>
> Anyhow, the forum seems to me a good opportunity to measure the D runtime in action on the server side, and in particular the GC.
> A few questions :
> Is the server regularly restarted or is it running smoothly without having to be restarted at all ?
> Does it measure the time for the page generation ? Would it be interesting to see at the bottom of the page something like "Page generated in xxx ms", when I (or other people) come accross those hiccups, and to collect these data for statistics over a week/month ?

I'm pretty sure it is the NG itself. Every news reader I've used has shown some form of "hiccup" If you try making a post under a heavy load you'll get a nice page telling you the load and what it needed to be.

On Friday, 13 April 2012 at 02:25:19 UTC, Somedude wrote:
> Well, it's not really urgent. It's fast enough.
> Still, collecting statistics over time would have been nice, so that one could analyse the general behaviour of the GC, for instance, by getting an idea of the standard deviation and maximum response time over weeks of use.

Such statistics wouldn't be very meaningful due to the highly varying server load, caused by a large number of other websites and services running on the same machine.

> But I have experienced a response of several seconds once, I don't know where this comes from (although it's most likely one of your reasons).

Long page loads are caused by a combination of high server load +
RAM cache misses + SQLite queries that require a lot of seeks. I'm quite sure this is the case.

> BTW, could you repost the source code address ? Thx.

There is a link to the source code on the Help page.

> Oh, I just saw you wrote the max page generation was around 50 ms.

When there are no cache misses, yes.

On Friday, 13 April 2012 at 03:24:19 UTC, Jesse Phillips wrote:
> I'm pretty sure it is the NG itself. Every news reader I've used has shown some form of "hiccup" If you try making a post under a heavy load you'll get a nice page telling you the load and what it needed to be.

Long page loads are caused by a combination of high server load +
RAM cache misses + SQLite queries that require a lot of seeks.
Network operations, including fetching new posts from the NG and
posting to the NG, are asynchronous, and never block web requests.

On Friday, 13 April 2012 at 08:09:09 UTC, Vladimir Panteleev wrote:
> On Friday, 13 April 2012 at 03:24:19 UTC, Jesse Phillips wrote:
>> I'm pretty sure it is the NG itself. Every news reader I've used has shown some form of "hiccup" If you try making a post under a heavy load you'll get a nice page telling you the load and what it needed to be.
>
> Long page loads are caused by a combination of high server load +
> RAM cache misses + SQLite queries that require a lot of seeks.
> Network operations, including fetching new posts from the NG and
> posting to the NG, are asynchronous, and never block web requests.

I use the split-thread view, I haven't had long page loads so maybe I've miss understood what he means by a hiccup. But I have seen it take time to retrieve a message and display it to me. I would expect a page could be considered "loading" if it is still waiting on fetching the post from the NG.

On Saturday, 14 April 2012 at 03:31:29 UTC, Jesse Phillips wrote:
> I use the split-thread view, I haven't had long page loads so maybe I've miss understood what he means by a hiccup. But I have seen it take time to retrieve a message and display it to me. I would expect a page could be considered "loading" if it is still waiting on fetching the post from the NG.

All posts are stored in a local SQLite database. Posts are retrieved from the NG when the program starts, and as they are posted (a listening connection is kept open). If a message is listed in the forum, then it is already in the local database.