| |
 | Posted by Ernesto Castellotti |  Permalink Reply |
|
Ernesto Castellotti 
| I am happy to announce the first preliminary version of neomimalloc!
neomimalloc is a wrapper to easily use the mimalloc allocator developed by Microsoft Research, this allocator is having a great success and it seems that the Rust community is thinking of using it by default instead of jemalloc (https://github.com/rust-lang/rust/pull/62340) so I think it is very useful for the DLang community to have a working wrapper for this modern allocator.
I created neomimalloc based on two principles that I consider fundamental for this kind of wrapper:
1) Provide a complete solution easy to use, I think it is particularly stupid to force the user to install a shared library to use a third party allocator, doing so would be very difficult to use in all DLang software.
So I realized a solution: the neomimalloc dub package compiles the allocator and includes the symbols directly in its static library, so the user only needs to add neomimalloc as a dependency to his project in order to have mimalloc included in his executable.
2) Being able to use the allocator with the high level API present in phobos (https://dlang.org/phobos/std_experimental_allocator.html), this is really very useful because it allows users to use the allocator as if it were included in the library D standard!
The usage is really the same, to do this I implemented the wrapper following the instructions in the phobos documentation (https://dlang.org/phobos/std_experimental_allocator_building_blocks.html)
3) In addition to the high-level interface, provide the bindings with C to the user to allow the use directly and to provide those APIs specific to mimalloc that could not be used by the high-level interface
Currently there are some issues in this wrapper, mainly:
1) It only supports POSIX operating systems, this is not linked to a lack of mimalloc but it is a problem of my package because currently it is not able to compile mimalloc itself from dub.json for Windows operating systems
This is mainly related to my total non knowledge of Windows operating systems so if someone knows them better than me, please help me solve the problem, pull requests are welcome!
2) Need for git during the compilation phase of the dub package, this because dub does not support repository cloning recursively (https://github.com/dlang/dub/issues/1044) so I have to do it manually with a bad hack (clone recursively the neomimalloc repository again in a temporary directory, very bad to do)
3) mimalloc provides many other implementations to allocate memory in addition to the classic malloc (eg zalloc, calloc etc.), it would be nice to support all allocators with high-level interfaces, of course low-level bindings support all mimalllc functions
The dub package allows 5 types of configuration:
1) standard: is the default configuration of neomimalloc, builda mimalloc without any particular additional feature
2) secure: builds mimalloc in secure mode (adding guard pages, randomized allocation, encrypted free lists, etc) to protect against various heap vulnerabilities with a slight performance penalty
3) override-malloc: like the standard configuration but overriding the libc allocator symbols, for example the core.stdc.stdlib.malloc function instead of calling the classic libc allocator will use mimalloc (obviously this will also happen with calloc, realloc etc)
4) override-malloc-secure: like the override-malloc but enable secure mode
5) manual-link: link to mimalloc as shared library without builds mimalloc, currently it is not implemented yet so it should not be used
dub package: http://neomimalloc.dub.pm
neomimalloc git repository: https://github.com/ErnyTech/neomimalloc.git
neomimalloc docs: https://neomimalloc.dpldocs.info
mimalloc git repository: https://github.com/microsoft/mimalloc.git
mimalloc docs: https://microsoft.github.io/mimalloc
mimalloc publication: https://www.microsoft.com/en-us/research/uploads/prod/2019/06/mimalloc-tr-v1.pdf
|
