Thread overview
Disk write in a "for" loop with RwMutex never happens
Aug 28, 2022
Gavin Ray
Aug 29, 2022
bauss
Aug 29, 2022
Gavin Ray
Aug 29, 2022
rikki cattermole
Aug 29, 2022
Gavin Ray
Aug 30, 2022
rikki cattermole
Aug 29, 2022
ag0aep6g
Aug 29, 2022
ag0aep6g
August 28, 2022

I've put the code, stripped to a minimal example here:

You can see that the single write + read version of the code works just fine:

pageData[0..4] = [1, 2, 3, 4]
readData[0..4] = [1, 2, 3, 4]

Where here, pageData is the data to be written to a file, and readData is the result of trying to read the freshly written file data.

But if the same code is placed inside of a for loop, suddenly no writes occur:

pageData[0..4] = [0, 0, 0, 0]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 1]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 2]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 3]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 4]
readData[0..4] = [0, 0, 0, 0]

// ...

Does anyone know what is happening here? It's really puzzling.

August 29, 2022

On Sunday, 28 August 2022 at 22:46:17 UTC, Gavin Ray wrote:

>

I've put the code, stripped to a minimal example here:

You can see that the single write + read version of the code works just fine:

pageData[0..4] = [1, 2, 3, 4]
readData[0..4] = [1, 2, 3, 4]

Where here, pageData is the data to be written to a file, and readData is the result of trying to read the freshly written file data.

But if the same code is placed inside of a for loop, suddenly no writes occur:

pageData[0..4] = [0, 0, 0, 0]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 1]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 2]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 3]
readData[0..4] = [0, 0, 0, 0]

pageData[0..4] = [0, 0, 0, 4]
readData[0..4] = [0, 0, 0, 0]

// ...

Does anyone know what is happening here? It's really puzzling.

You probably need to flush the output.

August 29, 2022

On Monday, 29 August 2022 at 07:04:49 UTC, bauss wrote:

> >

Does anyone know what is happening here? It's really puzzling.

You probably need to flush the output.

That's a good idea. I gave it a shot, and the following doesn't seem to change anything unfortunately:

void writePage(PageId pageId, ubyte[PAGE_SIZE] pageData)
{
    synchronized (dbIOMutex.writer)
    {
        dbFile.seek(pageId * PAGE_SIZE);
        dbFile.rawWrite(pageData);
        dbFile.flush();
        dbFile.sync();
    }
}
August 30, 2022
After a bunch of playing around I managed to determine that it is as simple as the mode.

exists(dbFileName) ? "r+" : "w+"



Will fix it.

Of course you shouldn't delete the file like that method is doing. It should probably reinitialize the FILE* descriptor.
August 29, 2022

On Sunday, 28 August 2022 at 22:46:17 UTC, Gavin Ray wrote:

>

I've put the code, stripped to a minimal example here:

But if the same code is placed inside of a for loop, suddenly no writes occur:
[...]

Does anyone know what is happening here? It's really puzzling.

Relevant pieces of the code:

class DiskManager
{
    void writePage(PageId pageId, ubyte[PAGE_SIZE] pageData)
    {
        synchronized (dbIOMutex.writer)
        {
            dbFile.seek(pageId * PAGE_SIZE);
            dbFile.rawWrite(pageData);
        }
    }
}

void singleReadWrite()
{
    PageId pageId = 0;

    diskManager.writePage(pageId, pageData);
}

void multiReadWrite()
{
    PageId pageId = 0;

    foreach (i; 0 .. 10)
    {
        diskManager.writePage(pageId, pageData);
    }
}

You never change pageId. So as far as I can tell, you're always seek-ing to the same position, and you just overwrite the same piece of the file again and again.

August 29, 2022

On Monday, 29 August 2022 at 16:21:53 UTC, ag0aep6g wrote:

>

You never change pageId. So as far as I can tell, you're always seek-ing to the same position, and you just overwrite the same piece of the file again and again.

Whoops. I guess I missed the point of the question there.

August 29, 2022

On Monday, 29 August 2022 at 15:52:31 UTC, rikki cattermole wrote:

>

After a bunch of playing around I managed to determine that it is as simple as the mode.

exists(dbFileName) ? "r+" : "w+"

Will fix it.

Of course you shouldn't delete the file like that method is doing. It should probably reinitialize the FILE* descriptor.

D'oh!

I didn't even think about the mode:

>

a+ or ab+ or a+b
"Append; open or create file for update, writing at end-of-file."

It must have been the "writing at end of file" bit?

Thanks Rikki. /embarrassed

August 30, 2022
On 30/08/2022 8:16 AM, Gavin Ray wrote:
> It must have been the "writing at end of file" bit?

I don't know.

It read like it should work.

The offsets were correct, it just didn't work *shrug*.