I use malloc to allocate some memory, then free it later. For monitoring purposes, I would like to know how much is free'ed by free by just knowing the object. Or, rather, given a ptr allocated by malloc, bet the block size it allocated from the ptr alone.

Some C compilers have special intrinsics and such for this, does D have any ability? If not, any hacks? I just need something that works.

On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:
> I use malloc to allocate some memory, then free it later. For monitoring purposes, I would like to know how much is free'ed by free by just knowing the object. Or, rather, given a ptr allocated by malloc, bet the block size it allocated from the ptr alone.
>
> Some C compilers have special intrinsics and such for this, does D have any ability? If not, any hacks? I just need something that works.

You can wrap the C memory allocations functions with a version identifier, e.g


version(stat)
{
    __gshared size_t[size_t] sizes;
}

version(stat)
{
    auto malloc(size_t size)
    {
        auto result = std.c.stdlib.malloc;
        sizes[result] = size;
        return result;
    }
}
else alias malloc = std.c.stdlib.malloc;

version(stat)
{
    void free(void* ptr)
    {
        std.c.stdlib.free(ptr);
        sizes.remove(ptr);
    }
}
else alias free = std.c.stdlib.free;


that a bit a DIY but this would work.

On Friday, 5 August 2016 at 20:43:12 UTC, H.Loom wrote:
> On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:
>> [...]
>
> You can wrap the C memory allocations functions with a version identifier, e.g
>
>
> version(stat)
> {
>     __gshared size_t[size_t] sizes;
> }
>
> version(stat)
> {
>     auto malloc(size_t size)
>     {
>         auto result = std.c.stdlib.malloc;
>         sizes[result] = size;
>         return result;
>     }
> }
> else alias malloc = std.c.stdlib.malloc;
>
> version(stat)
> {
>     void free(void* ptr)
>     {
>         std.c.stdlib.free(ptr);
>         sizes.remove(ptr);
>     }
> }
> else alias free = std.c.stdlib.free;
>
>
> that a bit a DIY but this would work.

Yeah, I ended up just storing the sizes in the ptr on malloc. Essentially the same but I don't have to keep a global around and deal with that.

On Friday, 5 August 2016 at 20:54:59 UTC, Mark "J" Twain wrote:
> On Friday, 5 August 2016 at 20:43:12 UTC, H.Loom wrote:
>> On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:
>>> [...]
>>
>> You can wrap the C memory allocations functions with a version identifier, e.g
>>
>>
>> version(stat)
>> {
>>     __gshared size_t[size_t] sizes;
>> }
>>
>> version(stat)
>> {
>>     auto malloc(size_t size)
>>     {
>>         auto result = std.c.stdlib.malloc(size);
>>         sizes[result] = size;
>>         return result;
>>     }
>> }
>> else alias malloc = std.c.stdlib.malloc;
>>
>> version(stat)
>> {
>>     void free(void* ptr)
>>     {
>>         std.c.stdlib.free(ptr);
>>         sizes.remove(ptr);
>>     }
>> }
>> else alias free = std.c.stdlib.free;
>>
>>
>> that a bit a DIY but this would work.
>
> Yeah, I ended up just storing the sizes in the ptr on malloc. Essentially the same but I don't have to keep a global around and deal with that.

you cannot get rid of a global variable to store the sizes. Even if it's hidden in an API you didn't write, in which case it's just hidden.

Have you looked at experimental allocators ?

On Friday, 5 August 2016 at 21:07:02 UTC, H.Loom wrote:
> On Friday, 5 August 2016 at 20:54:59 UTC, Mark "J" Twain wrote:
>> On Friday, 5 August 2016 at 20:43:12 UTC, H.Loom wrote:
>>> On Friday, 5 August 2016 at 19:55:22 UTC, Mark "J" Twain wrote:
>>>> [...]
>>>
>>> You can wrap the C memory allocations functions with a version identifier, e.g
>>>
>>>
>>> version(stat)
>>> {
>>>     __gshared size_t[size_t] sizes;
>>> }
>>>
>>> version(stat)
>>> {
>>>     auto malloc(size_t size)
>>>     {
>>>         auto result = std.c.stdlib.malloc(size);
>>>         sizes[result] = size;
>>>         return result;
>>>     }
>>> }
>>> else alias malloc = std.c.stdlib.malloc;
>>>
>>> version(stat)
>>> {
>>>     void free(void* ptr)
>>>     {
>>>         std.c.stdlib.free(ptr);
>>>         sizes.remove(ptr);
>>>     }
>>> }
>>> else alias free = std.c.stdlib.free;
>>>
>>>
>>> that a bit a DIY but this would work.
>>
>> Yeah, I ended up just storing the sizes in the ptr on malloc. Essentially the same but I don't have to keep a global around and deal with that.
>
> you cannot get rid of a global variable to store the sizes. Even if it's hidden in an API you didn't write, in which case it's just hidden.
>
> Have you looked at experimental allocators ?

More especially https://github.com/dlang/phobos/blob/master/std/experimental/allocator/building_blocks/stats_collector.d

instead of using malloc/free you can use Mallocator (this is the same).

Maybe that StatsCollector!Mallocator == win for you.

On 08/05/2016 12:55 PM, Mark J Twain wrote:
> I use malloc to allocate some memory, then free it later. For monitoring
> purposes, I would like to know how much is free'ed by free by just
> knowing the object. Or, rather, given a ptr allocated by malloc, bet the
> block size it allocated from the ptr alone.
>
> Some C compilers have special intrinsics and such for this, does D have
> any ability? If not, any hacks? I just need something that works.
>

malloc stores the allocation size and other information right before the pointer that it returns. You can do better than I do below if you know the exact objects that it stores there. And of course you don't need a struct. I just liked the property idea there. :)

import std.stdio;
import core.stdc.stdlib;

struct AllocatedMemory {
    void *p;

    alias SizeType = size_t;    // perhaps ulong?

    @property SizeType size() {
        return *cast(SizeType*)(p - SizeType.sizeof);
    }
}

void myFree(void * p) {
    auto a = AllocatedMemory(p);
    writefln("Freeing something like 0x%x bytes", a.size);
}

void main() {
    auto p = malloc(0x12345);
    myFree(p);
}

Prints:

Freeing something like 0x12351 bytes

Ali

On Friday, 5 August 2016 at 21:25:41 UTC, Ali Çehreli wrote:
> malloc stores the allocation size and other information right before the pointer that it returns.

TIL. Is it guaranteed for any malloc implementation ? I'm a bit surpised to read that. I thought these kind of trick was only used for aligned allocations !