I need to allocate a large number (>10K) of small objects.

class Small {
some field ...
}

Right now I call "new Small()" for each allocation, the overall performance of the program is slow.  I'm wondering if I can use the C++ idiom in D, e.g.

char *pool = new char[ N * Small.size_of ];
char *next = pool;

Small myNew() {
Small obj = cast(Small)next;
next += Small.size_of;
}

is there any problem with it?  What's the correct way to do this in D?

Thanks.

newbie@d.com wrote:
> I need to allocate a large number (>10K) of small objects.  
> 
> class Small {
> some field ...
> }
> 
> Right now I call "new Small()" for each allocation, the overall performance of
> the program is slow.  I'm wondering if I can use the C++ idiom in D, e.g.
> 
> char *pool = new char[ N * Small.size_of ];
> char *next = pool;
> 
> Small myNew() {
> Small obj = cast(Small)next;
> next += Small.size_of;
> }
> 
> is there any problem with it?  What's the correct way to do this in D?
> 
> Thanks.

You can write a custom allocator without any "hacks":
http://www.digitalmars.com/d/memory.html#newdelete

BTW, I think it should be Small.init.sizeof, Small.sizeof returns the size of a reference, not an instance..


xs0

In article <dhvr8g$vg$1@digitaldaemon.com>, newbie@d.com says...
>
>I need to allocate a large number (>10K) of small objects.
>
>class Small {
>some field ...
>}
>
>Right now I call "new Small()" for each allocation, the overall performance of the program is slow.  I'm wondering if I can use the C++ idiom in D, e.g.
>
>char *pool = new char[ N * Small.size_of ];
>char *next = pool;
>
>Small myNew() {
>Small obj = cast(Small)next;
>next += Small.size_of;
>}
>
>is there any problem with it?  What's the correct way to do this in D?

Another solution apart from what has already been suggested:

> const uint POOL_SIZE = 1024*1024*5; // 5MB
> ubyte[POOL_SIZE] pool = void;
> ubyte* next = pool.ptr;
> Small obj = cast(Small)next;
> next += Small.size_of;

The above will get you a 5MB pool of memory, allocated to a ubyte array (which is slightly better form than char[]).  The 'void' initalizer instructs the compiler to not initalize the entire array to zero, which can cost you time for big chunks of memory.

- EricAnderton at yahoo

<newbie@d.com> wrote in message news:dhvr8g$vg$1@digitaldaemon.com...
>I need to allocate a large number (>10K) of small objects.
>
> class Small {
> some field ...
> }
>
> Right now I call "new Small()" for each allocation, the overall
> performance of
> the program is slow.  I'm wondering if I can use the C++ idiom in D, e.g.
>
> char *pool = new char[ N * Small.size_of ];
> char *next = pool;
>
> Small myNew() {
> Small obj = cast(Small)next;
> next += Small.size_of;
> }
>
> is there any problem with it?  What's the correct way to do this in D?
>
> Thanks.
>
>

Be careful about casting a random pointer to a Small. The resulting "object" won't have a vtable or monitor or be initialized so you won't be able to call any methods or synchronize on the object. If you don't care about either of those I suggesting making Small a struct.

In article <di05kl$a40$1@digitaldaemon.com>, xs0 says...
>
>BTW, I think it should be Small.init.sizeof, Small.sizeof returns the size of a reference, not an instance..

Really?  But Small is the type name.  Classes are just a special case all around it seems.


Sean

In article <di0m9b$p23$1@digitaldaemon.com>, Ben Hinkle says...
>
>
><newbie@d.com> wrote in message news:dhvr8g$vg$1@digitaldaemon.com...
>>I need to allocate a large number (>10K) of small objects.
>>
>> class Small {
>> some field ...
>> }
>>
>> Right now I call "new Small()" for each allocation, the overall
>> performance of
>> the program is slow.  I'm wondering if I can use the C++ idiom in D, e.g.
>>
>> char *pool = new char[ N * Small.size_of ];
>> char *next = pool;
>>
>> Small myNew() {
>> Small obj = cast(Small)next;
>> next += Small.size_of;
>> }
>>
>> is there any problem with it?  What's the correct way to do this in D?
>>
>> Thanks.
>>
>>
>
>Be careful about casting a random pointer to a Small. The resulting "object" won't have a vtable or monitor or be initialized so you won't be able to call any methods or synchronize on the object. If you don't care about either of those I suggesting making Small a struct.

It should be simple enough to do a placement new on the memory block though...


Sean

In article <di0m9b$p23$1@digitaldaemon.com>, Ben Hinkle says...
>
>Be careful about casting a random pointer to a Small. The resulting "object" won't have a vtable or monitor or be initialized so you won't be able to call any methods or synchronize on the object. If you don't care about either of those I suggesting making Small a struct.
>

I cannot use struct.  So under this constraint, can I do something as I showed earlier?  And what's the best way to make sure the *object* I got from the pool is properly initialized?

In article <di05kl$a40$1@digitaldaemon.com>, xs0 says...
>
>You can write a custom allocator without any "hacks": http://www.digitalmars.com/d/memory.html#newdelete

I checked that page, same question: what's the magic to make sure the *object* is properly initialize?

new() {
p = std.c.stdlib.malloc(sz);
gc.addRange(p, p + sz);
return p;
}

Thanks.

> BTW, I think it should be Small.init.sizeof, Small.sizeof returns the size of a reference, not an instance..

I think you mean Small.classinfo.init.length Small.init.sizeof == Small.sizeof

> I checked that page, same question: what's the magic to make sure the
> *object*
> is properly initialize?
>
> new() {
> p = std.c.stdlib.malloc(sz);
> gc.addRange(p, p + sz);
> return p;
> }

I haven't had to ever use custom allocators so I haven't tried it myself but presumably from the description on http://www.digitalmars.com/d/class.html#allocators the result of new() is initialized.