Thread overview | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
April 18, 2017 The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
I am writing app that extract data from DB to array of structures. void getSingleTrackInfo() { foreach(item; getTablesGPSSensorList) { ResultRange result = mysqlconnection.query(sqlquery); auto MySQLPointsLonLat = result.array; carGPSPoint cargpspoint; // create struct carGPSPoint [] cargpspoints; // create array of structures foreach(i, point;MySQLPointsLonLat) { cargpspoint.id = point[0].coerce!ulong; cargpspoint.recordDate = point[1].coerce!string; cargpspoint.velocity = point[2].coerce!double; cargpspoint.lat = point[3].coerce!string; cargpspoint.lon = point[4].coerce!string; cargpspoints ~=cargpspoint; } } } I expected that on every step: `foreach(item; getTablesGPSSensorList)` structures `carGPSPoint` will be recreated and memory will be free. But App after starting begin eat memory, and do it till reach 1750 of RAM. After it it's not crush, but simply stop other processing. 1. By the code on every iterate `carGPSPoint` should be recreated and memory should be free, why this does not happen? 2. How to free my memory? |
April 18, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Suliman | Also I can't understand why app take so much memory? I checked array of structures size with this code: auto mymem = cargpspoints.length * typeof(cargpspoints[0]).sizeof; writeln(mymem); And it's print: 16963440 it's about 16MB... What is takes all other memory? |
April 18, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Suliman | On Tuesday, 18 April 2017 at 13:28:57 UTC, Suliman wrote: > Also I can't understand why app take so much memory? I checked array of structures size with this code: > > auto mymem = cargpspoints.length * typeof(cargpspoints[0]).sizeof; > writeln(mymem); > > And it's print: 16963440 > it's about 16MB... > > What is takes all other memory? 1. You're measuring it wrong. Array length is already measured in terms of type size. But remember that every call result.array will allocate, and very call coerce!string will also allocate. 2. Since you're iterating over every result once, there's no point in converting it to an array first. 3. Consider using the Row.toStruct() method for conversion. 4. Consider using std.array.appender instead of ~=. void getSingleTrackInfo() { foreach(item; getTablesGPSSensorList) { ResultRange result = mysqlconnection.query(sqlquery); carGPSPoint cargpspoint; // create struct auto arr = appender!(carGPSPoint[]); // create array of structures foreach(row; result) { arr ~= row.toStruct(cargpspoint); } // arr.data should hold the array of structures } } |
April 18, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stanislav Blinov | On Tuesday, 18 April 2017 at 14:09:28 UTC, Stanislav Blinov wrote:
> foreach(row; result)
> {
> arr ~= row.toStruct(cargpspoint);
> }
Sorry, this should be
foreach(row; result)
{
row.toStruct(cargpspoint);
arr ~= cargpspoint;
}
|
April 18, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stanislav Blinov | On Tuesday, 18 April 2017 at 14:15:59 UTC, Stanislav Blinov wrote:
> On Tuesday, 18 April 2017 at 14:09:28 UTC, Stanislav Blinov wrote:
>
>> foreach(row; result)
>> {
>> arr ~= row.toStruct(cargpspoint);
>> }
>
> Sorry, this should be
>
> foreach(row; result)
> {
> row.toStruct(cargpspoint);
> arr ~= cargpspoint;
> }
Thanks I will try! But why in my code memory do not released? Do you have any idea?
|
April 19, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Stanislav Blinov | >> auto mymem = cargpspoints.length * typeof(cargpspoints[0]).sizeof; >> writeln(mymem); >> >> And it's print: 16963440 >> it's about 16MB... >> >> What is takes all other memory? > > 1. You're measuring it wrong. Array length is already measured in terms of type size. So should I do: cargpspoints.length * cargpspoints[0].sizeof ? Btw, `cargpspoints.length * typeof(cargpspoints[0]).sizeof` and `cargpspoints.length * cargpspoints[0].sizeof` show same result. |
April 19, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Suliman | I have added GC.stat https://dlang.org/library/core/memory/gc.stats.html here the result: freeSize: 49698640 | usedSize: 170502320 freeSize: 41174592 | usedSize: 217823680 freeSize: 53868576 | usedSize: 247072736 freeSize: 86494800 | usedSize: 307769776 freeSize: 58176640 | usedSize: 499665792 freeSize: 148233232 | usedSize: 534389744 freeSize: 148141376 | usedSize: 534481600 freeSize: 108467312 | usedSize: 641264528 freeSize: 55118432 | usedSize: 694613408 freeSize: 80579472 | usedSize: 803370096 freeSize: 78444480 | usedSize: 1006831680 freeSize: 291629360 | usedSize: 860755664 freeSize: 242912736 | usedSize: 976581152 freeSize: 241673232 | usedSize: 977820656 freeSize: 168092160 | usedSize: 1118510592 freeSize: 128405616 | usedSize: 1426632592 freeSize: 73146272 | usedSize: 1616109664 freeSize: 17962320 | usedSize: 1671293616 freeSize: 20342912 | usedSize: 1736021888 on the last value app is hanging. |
April 19, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Suliman | On Tuesday, 18 April 2017 at 11:43:24 UTC, Suliman wrote:
> I am writing app that extract data from DB to array of structures.
>
> void getSingleTrackInfo()
> {
>
> foreach(item; getTablesGPSSensorList)
> {
> ResultRange result = mysqlconnection.query(sqlquery);
> auto MySQLPointsLonLat = result.array;
Is ResultRange closing query when exhausted? Did you try to call .close() on it after work?
|
April 19, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to crimaniak | On Wednesday, 19 April 2017 at 15:18:32 UTC, crimaniak wrote: > On Tuesday, 18 April 2017 at 11:43:24 UTC, Suliman wrote: >> I am writing app that extract data from DB to array of structures. >> >> void getSingleTrackInfo() >> { >> >> foreach(item; getTablesGPSSensorList) >> { >> ResultRange result = mysqlconnection.query(sqlquery); >> auto MySQLPointsLonLat = result.array; > Is ResultRange closing query when exhausted? Did you try to call .close() on it after work? Yes I tried: ResultRange result = mysqlconnection.query(sqlquery); auto MySQLPointsLonLat = result.array; result.close(); Did not help :( |
April 19, 2017 Re: The app hanging after reach 1750MB of RAM | ||||
---|---|---|---|---|
| ||||
Posted in reply to Suliman | On Wednesday, 19 April 2017 at 07:28:32 UTC, Suliman wrote:
>> 1. You're measuring it wrong. Array length is already measured in terms of type size.
>
> So should I do:
> cargpspoints.length * cargpspoints[0].sizeof ?
No. .sizeof is the statically known size of a type, it can't take into account dynamically allocated memory.
cargpspoint.length * cargpspoints[0].sizeof will tell you the estimate size of the array, in bytes.
But each of the elements also has strings - dynamic arrays that are allocated elsewhere, their length is not included in that calculation.
So you could iterate over the array and sum up lengths of all strings to get an estimate.
Even then, that's just that: an estimate. Actual amount of memory allocated for a dynamic array T[] *may* be greater than length * T.sizeof. The only way to know that is to query the allocator used (in this case, GC).
|
Copyright © 1999-2021 by the D Language Foundation