On Sunday, 16 December 2012 at 07:47:48 UTC, Rob T wrote:
> On Sunday, 16 December 2012 at 05:37:57 UTC, SomeDude wrote:
>>
>> Isn't the memory management completely negligible when compared to the database access here ?
>
> Here are the details ...
>
> My test run selects and returns 206,085 records with 14 fields per record.
>
> With all dynamic memory allocations disabled that are used to create the data structure containing the returned rows, a run takes 5 seconds. This does not return any data, but it runs exactly through all the records in the same way but returns to a temporary stack allocated value of appropriate type.
>
> If I disable the GC before the run and re-enable it immediately after, it takes 7 seconds. I presume a full 2 seconds are used to disable and re-enable the GC which seems like a lot of time.
>
> With all dynamic memory allocations enabled that are used to create the data structure containing the returned rows, a run takes 28 seconds. In this case, all 206K records are returned in a dynamically generate list.
>
> If I disable the GC before the run and re-enable it immediately after, it takes 11 seconds. Since a full 2 seconds are used to disable and re-enable the GC, then 9 seconds are used, and since 5 seconds are used without memory allocations, the allocations are using 4 seconds, but I'm doing a lot of allocations.
>
> In my case, the structure is dynamically generated by allocating each individual field for each record returned, so there's 206,085 records x 14 fields = 2,885,190 allocations being performed. I can cut the individual allocations down to 206,000 by allocating the full record in one shot, however this is a stress test designed to work D as hard as possible and compare it with an identically stressed C++ version.
>

You cannot expect the GC to perform like manual memory management. It's a completely unrealistic microbenchmark to allocate each individual field, even for manual MM. The least you can do to be a little bit realistic is indeed to allocate one row at a time. I hope that's what you intend to do. But usually, database drivers allow the user to tweak the queries and decide how many rows can be fetched at a time, and it's pretty common to fetch 50 or 100 rows at a time, meaning one allocation only each time. It would be interesting to compare the performance of the two languages in these situations, i.e one row at a time, and 50 rows at a time.