I've released D:GameVFS 0.1, a minimalist VFS library for game developers.

I'm trying to librarize various pieces of code I'm working on for my projects; this is a very simple virtual file system library. Can only read/write/create files and directories, not delete them.

Currently there is no archive support -  shouldn't be too difficult to implement, but I'll only implement it if I need it (contributions are welcome).

It can stack directories, which I need for easy moddability.

There are also no security features - if a directory you're working with gets deleted, D:GameVFS can't handle that.

The API is inspired by Tango VFS API, but made with ranges in mind, using RAII/refcounting and again, very minimal.

API is not stable, and will change e.g. when std.stream gets rewritten.


You can get D:GameVFS 0.1 here: https://github.com/kiith-sa/D-GameVFS/downloads For basic introduction, see the tutorial in the README at https://github.com/kiith-sa/D-GameVFS and the API documentation in the package.

Am Fri, 20 Jan 2012 00:36:11 +0100
schrieb Kiith-Sa <42@theanswer.com>:

> You can get D:GameVFS 0.1 here: https://github.com/kiith-sa/D-GameVFS/downloads For basic introduction, see the tutorial in the README at https://github.com/kiith-sa/D-GameVFS and the API documentation in the package.

I didn't check your code, but does it handle case-insensitive Windows OS as well as Unix like? Looking at the archive files of the game F.E.A.R. I noticed that at some point they started using lower case exclusively (and that game only runs on Windows). Maybe it was easier to compare directories and patches that way.
The stacking of directories can also be used to patch the game, of course. What striked me as strange in said game, was that they added the game server/client code (for single player as well) in archives, but the main executable, DB driver or video codec where plain files in the base directory. So the latter got overwritten with every update, while the former just added up with each patch level, so at patch 1.08, there were 9 version of them - irreversible yet, unless you archived the old main executables as well.
I'm not trying to say that an update should generally overwrite previous files, though. Just that their system didn't make sense in every bit. The separate archives for each update had one clear benefit: The first expansion was created at the time of 1.07, and they could just link in the base archives + updates up to 1.07 without running the risk that later updates to the main game would interfere with that expansion.

They used a config file in the main executable's directory that listed in ascending priority, the directories or archives that should be used. They didn't reuse the main executable for expansions though, so those went into sub directories with their own configuration file that linked in base game and patches using ../some_archive.

Also they split their archives, both to stay below 4GB and to have localizations (voices) in one easily replaceable file, which may be have been handy at release time. Did you think of ways to probably even switch the language from a menu?

From your example in the readme, it isn't immediately clear what the names 'user', 'main' and 'root' are used for. I assume that StackDir inherits from FSDir which is a nice design, but I'd probably expect things to work out as in F.E.A.R. or Quake where these things are implementation details. In other words, the game *never* attempts a write access into it's data files, and there is only one file-system root, not necessarily a singleton. This root object then offers methods to mount directories or archives.

  auto fs = new FileSystem();
  fs.mount("main"); // can be directory "main" or "main.zip"
  fs.mount("user"); // overrides main now

Something like this would be nice to 'seamlessly' extract an archive to a directory with the same name. But you may have your own ideas there. Writing archive files can be difficult as some archives may require recompression or check sum creation over the whole archive. I guess most developers just release their game with read-only archives and development happens with full write access on the extracted directories through external tools (unless the game is the level editor).

-- 
Marco

On Monday, 16 July 2012 at 17:27:32 UTC, Marco Leise wrote:
> Am Fri, 20 Jan 2012 00:36:11 +0100

> I didn't check your code, but does it handle case-insensitive Windows OS as well as Unix like?

I didn't think consider this when working on it, so it's probably
always case-sensitive.

> Did you think of ways to probably even switch the language from a menu?

Outside of scope of D:GameVFS. Of course, you could just
mount different directories with language files based on user input.


> From your example in the readme, it isn't immediately clear what the names 'user', 'main' and 'root' are used for.

No particular purpose, they could be used for anything. In my game project,
I have "root" for read-only (what goes to somewhere in /usr on Linux),
and "user" for read-write e.g. config, logs, screenshots, etc.
(what goes in /home/username/.gamedirectory).

> I assume that StackDir inherits from FSDir which is a nice design.

No. As you can see in the API docs, StackDir and FSDir both inherit from
VFSDir. StackDir wraps one or more VFSDirs to wrap them. So you could even
stack a zip dir (if/when a ZipDir is implemented) on top of an FSDir.
I actually stack StackDirs on top of each other in my project.

> I'd probably expect things to work out as in F.E.A.R. or Quake where these things are implementation details. In other words, the game *never* attempts a write access into it's data files.

Directories can be read-only. You also need write support for stuff like
saves, config, screenshots, etc.

>   auto fs = new FileSystem();
>   fs.mount("main"); // can be directory "main" or "main.zip"
>   fs.mount("user"); // overrides main now

You can use a StackDir, and mount a ZipDir (if/when implemented) _or_ an FSDir based on whether e.g. a zip exists or not, but you't have to detect that yourself. I don't plan on adding such API to D:GameVFS, but it would only
take a simple function on top of it.



Note that D:GameVFS is just 0.1 now. I just implemented what I needed at the moment so far, stuff like a ZipDir will be only added once I need it.
Of course, any contributions are welcome - it shouldn't be too difficult.

For the game project, see https://github.com/kiith-sa/ICE, but it doesn't
really use D:GameVFS that heavily - only to separate read-only and read-write.
I'm planning to use it for mods later.

I'm currently outside the D world (working on a non-D related GSoC project),
but I'm planning on revisiting D:GameVFS and D:YAML in September.
That will only be to update compatibility to DMD 2.060 (if it's released by then) and to fix bugs, though.

Tango (which should now have a D2 port) also has a VFS API, which has more features (e.g. zip support) - the reason I started D:GameVFS was that I couldn't do some things (related to stacking) I wanted with the Tango API.
Don't remember what exactly was the problem, though.