View mode: basic / threaded / horizontal-split · Log in · Help
April 13, 2012
Re: Measuring the page generation of the forum
On Thursday, 12 April 2012 at 18:36:01 UTC, SomeDude wrote:
> Hi,
>
> I've noticed a couple of times (i.e very rarely, mind you), 
> that the page generation of the forum had hiccups.
> Have other people experienced such hiccups ?
> If yes, I wonder if we saw the GC kicking in, or if it was 
> SQLite's fault (or was it a network latency ?).
>
> Anyhow, the forum seems to me a good opportunity to measure the 
> D runtime in action on the server side, and in particular the 
> GC.
> A few questions :
> Is the server regularly restarted or is it running smoothly 
> without having to be restarted at all ?
> Does it measure the time for the page generation ? Would it be 
> interesting to see at the bottom of the page something like 
> "Page generated in xxx ms", when I (or other people) come 
> accross those hiccups, and to collect these data for statistics 
> over a week/month ?

I'm pretty sure it is the NG itself. Every news reader I've used 
has shown some form of "hiccup" If you try making a post under a 
heavy load you'll get a nice page telling you the load and what 
it needed to be.
April 13, 2012
Re: Measuring the page generation of the forum
On Friday, 13 April 2012 at 02:25:19 UTC, Somedude wrote:
> Well, it's not really urgent. It's fast enough.
> Still, collecting statistics over time would have been nice, so 
> that one could analyse the general behaviour of the GC, for 
> instance, by getting an idea of the standard deviation and 
> maximum response time over weeks of use.

Such statistics wouldn't be very meaningful due to the highly 
varying server load, caused by a large number of other websites 
and services running on the same machine.

> But I have experienced a response of several seconds once, I 
> don't know where this comes from (although it's most likely one 
> of your reasons).

Long page loads are caused by a combination of high server load +
RAM cache misses + SQLite queries that require a lot of seeks. 
I'm quite sure this is the case.

> BTW, could you repost the source code address ? Thx.

There is a link to the source code on the Help page.

> Oh, I just saw you wrote the max page generation was around 50 
> ms.

When there are no cache misses, yes.
April 13, 2012
Re: Measuring the page generation of the forum
On Friday, 13 April 2012 at 03:24:19 UTC, Jesse Phillips wrote:
> I'm pretty sure it is the NG itself. Every news reader I've 
> used has shown some form of "hiccup" If you try making a post 
> under a heavy load you'll get a nice page telling you the load 
> and what it needed to be.

Long page loads are caused by a combination of high server load +
RAM cache misses + SQLite queries that require a lot of seeks.
Network operations, including fetching new posts from the NG and
posting to the NG, are asynchronous, and never block web requests.
April 14, 2012
Re: Measuring the page generation of the forum
On Friday, 13 April 2012 at 08:09:09 UTC, Vladimir Panteleev 
wrote:
> On Friday, 13 April 2012 at 03:24:19 UTC, Jesse Phillips wrote:
>> I'm pretty sure it is the NG itself. Every news reader I've 
>> used has shown some form of "hiccup" If you try making a post 
>> under a heavy load you'll get a nice page telling you the load 
>> and what it needed to be.
>
> Long page loads are caused by a combination of high server load 
> +
> RAM cache misses + SQLite queries that require a lot of seeks.
> Network operations, including fetching new posts from the NG and
> posting to the NG, are asynchronous, and never block web 
> requests.

I use the split-thread view, I haven't had long page loads so 
maybe I've miss understood what he means by a hiccup. But I have 
seen it take time to retrieve a message and display it to me. I 
would expect a page could be considered "loading" if it is still 
waiting on fetching the post from the NG.
April 14, 2012
Re: Measuring the page generation of the forum
On Saturday, 14 April 2012 at 03:31:29 UTC, Jesse Phillips wrote:
> I use the split-thread view, I haven't had long page loads so 
> maybe I've miss understood what he means by a hiccup. But I 
> have seen it take time to retrieve a message and display it to 
> me. I would expect a page could be considered "loading" if it 
> is still waiting on fetching the post from the NG.

All posts are stored in a local SQLite database. Posts are 
retrieved from the NG when the program starts, and as they are 
posted (a listening connection is kept open). If a message is 
listed in the forum, then it is already in the local database.
Next ›   Last »
1 2
Top | Discussion index | About this forum | D home