Thread overview
GC and sensible data read by File
Oct 04, 2020
frame
Edit
Oct 05, 2020
frame
Oct 05, 2020
ryuukk_
Oct 24, 2020
frame
Oct 24, 2020
Imperatorn
October 04, 2020
Hello,

I'm new to D and try to find out a memory leak in my program. I inspected the private bytes with VmMap on Windows to see which data is still kept.

Besides the actual memory leak I was surprised to find out there are contents of my previoulsy read configuration INI file.

So my question is: Is this expected behaviour? Why does the GC then not overwrite the contents with 0? Or did I something wrong in my function or does File not clear the data afterwards? And if so, how can I read sensible data from a file with standard methods which should not kept in memory?

I also tried manually GC.free() on buf and other places but still same result.

Basically this (removed some encforce() statements) and do not show Converter method here because the string in memory is raw string like a INI-comment which is never passed to my converter, so it's not relevant and cannot cause the problem.

Variant[string][string] parse() {


        size_t lineNr;
        char[] buf;

        string sectionName;
        string keyName;
        Variant[string][string] sections;

        auto fp = File(_fileName);
        while (!fp.eof()) {
            buf.length = 0;
            fp.readln(buf);

            string line = to!string(buf).trim();

            ++lineNr;

            // filter out empty lines
            if (line.length == 0)
                continue;

            // filter out comments
            if (line.startsWith(";"))
                continue;

            if (line.startsWith("[")) {
                // section start / previous section complete
                sectionName = line.trim("[]");
                continue;
            }

            // regular entry
            Variant v;
            ptrdiff_t sepPos = line.indexOf("=");

            keyName = to!string(line[0 .. sepPos]).trim();
            bool isArrayEntry = keyName.indexOf("[]") > -1 && keyName.endsWith("]");

            string tmpVal = to!string(line[sepPos + 1 .. $]).trim();

            // trim text delimiter chars, if any
            tmpVal = tmpVal.trim("'\"");

            // if array, just append to existing value or create it
            if (isArrayEntry) {
                keyName = keyName.trim("[]");

                // array is always considered as string
                // other values makes no sense in this syntax
                string oldVal;

                if (auto sectionPtr = sectionName in sections) {
                    if (auto keyPtr = keyName in *sectionPtr) {
                        oldVal = (*keyPtr).get!(string);
                    }
                }

                v = Variant(oldVal ~ tmpVal);
            }
            else {
                v = Converter.autoConvertFromString(tmpVal);
            }

            sections[sectionName][keyName] = v;
        }

        return sections;
    }

October 05, 2020
So I found out that there is nothing wrong with the method as in a test environment the allocated memory block is removed after GC.minimize().

Still need to find out why other blocks are not released.

However, is there a way to debug currently allocated variables by the GC?
October 05, 2020
On Monday, 5 October 2020 at 11:14:47 UTC, frame wrote:
> So I found out that there is nothing wrong with the method as in a test environment the allocated memory block is removed after GC.minimize().
>
> Still need to find out why other blocks are not released.
>
> However, is there a way to debug currently allocated variables by the GC?


```
	"buildTypes": {
	
		"debug_gc": {
			"buildOptions": ["debugMode", "debugInfo"],
			"dflags": ["-profile=gc"]
		}

	},
```

and run using: ``dub run --compiler=dmd --build=debug_gc``

once program exit it'll create a log file: ``profilegc.log`` with all allocations
October 24, 2020
On Monday, 5 October 2020 at 11:28:56 UTC, ryuukk_ wrote:
> On Monday, 5 October 2020 at 11:14:47 UTC, frame wrote:
>> So I found out that there is nothing wrong with the method as in a test environment the allocated memory block is removed after GC.minimize().
>>
>> Still need to find out why other blocks are not released.
>>
>> However, is there a way to debug currently allocated variables by the GC?
>
>
> ```
> 	"buildTypes": {
> 	
> 		"debug_gc": {
> 			"buildOptions": ["debugMode", "debugInfo"],
> 			"dflags": ["-profile=gc"]
> 		}
>
> 	},
> ```
>
> and run using: ``dub run --compiler=dmd --build=debug_gc``
>
> once program exit it'll create a log file: ``profilegc.log`` with all allocations

I tried that but it gives me only a profile of used allocations over all time. What I really need is a debug feature to see which variable/pointer is currently known by the GC or at least which one was still referenced by end of the program.

October 24, 2020
On Sunday, 4 October 2020 at 12:10:39 UTC, frame wrote:
> Hello,
>
> I'm new to D and try to find out a memory leak in my program. I inspected the private bytes with VmMap on Windows to see which data is still kept.
>
> [...]

The GC doesn't zero it out, so that's expected.