August 04, 2022

Hi,
I want to use MmFile to manage memory mapped files. I want multiple processes to be able to read in many files using the virtual memory system to page them in and reuse the memory.

If I read the contents of the file, I can cast it to immutable and then I get segfaults when accessing the data.

Or I have to .dup the data, causing memory allocation in addition to the paging to get the data in the first place.

Is this special memory not managed by the GC ? Do I need to keep all the MmFile references around to avoid segfaults ?
Alex

August 03, 2022
On Thu, Aug 04, 2022 at 12:35:55AM +0000, Alex Burton via Digitalmars-d-learn wrote:
> Hi,
> I want to use MmFile to manage memory mapped files. I want multiple
> processes to be able to read in many files using the virtual memory
> system to page them in and reuse the memory.
> 
> If I read the contents of the file, I can cast it to immutable and then I get segfaults when accessing the data.

You need to keep the MmFile instance around as long as you're using the memory, since it will unmap the memory when the GC collects it.


> Or I have to .dup the data, causing memory allocation in addition to the paging to get the data in the first place.

That would defeat the purpose of using MmFile in the first place.


> Is this special memory not managed by the GC ? Do I need to keep all the MmFile references around to avoid segfaults ?
[...]

Memory-mapped memory is not managed by the GC; it's managed by the OS. The MmFile class, however, will unmap the memory upon destruction, so you need to keep at least one reference to it live throughout the course of your threads accessing the memory.  Otherwise you will get segfaults after the MmFile is collected, because then it will unmap the memory and all your references will become invalid.

Sharing the memory among threads can be done by just passing the memory reference to the threads. It will be up to you, however, to manage synchronization of access if you're planning to write to the memory. If not, you can probably just cast the memory reference to immutable and pass that around to threads.  Do NOT cast the MmFile object itself to immutable, though. I'm almost certain that'll break it because it probably needs to keep internal state in order to manage the memory mapping with the OS.  Only cast the slice returned by Mmfile.opSlice, not the MmFile object itself.


T

-- 
Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? -- Michael Beibl