On 10/24/13 4:01 PM, bearophile wrote:
> Andrei Alexandrescu:
>
>>> - Can you create an hierarchical allocator with those?
>>
>> Not sure what that means (I recall the phrase has been discussed but
>> forgot what it means).
>
> I meant something like this:
>
> http://swapped.cc/?_escaped_fragment_=/halloc#!/halloc

I see. These are akin to regions to some extent, just a bit more dynamic. The design doesn't explicitly address the pattern, so I expect there would be some hiccups.

>> (Including awareness of virtual memory management from the OS).
>>
>> Definitely!
>>
>> * http://stackoverflow.com/questions/3839922/aligned-malloc-in-gcc
>> comes to mind.
>>
>> * http://en.wikipedia.org/wiki/Sbrk
>>
>> * http://en.wikipedia.org/wiki/Mmap
>
> By that "awareness" I meant something like what the Azul JavaVM does,
> here a panoramic view of similar ideas:
>
> http://web.eece.maine.edu/~jyue/papers/iccci.pdf

Once we get to types and garbage, such stuff will become more relevant.


Andrei

On 2013-10-24 19:54:41 +0000, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> Please destroy! I've literally sweat as I'm sending this :o).

Seems good. Although I'm no expert on allocators I find it very easy to grasp. One remark though.

While it generally makes sense that you know the size of an allocation when you want to deallocate, in some case it might be suboptimal to have to provide the size. For instance, you could use an allocator to allocate an object (with a virtual table and all). When comes the time to deallocate, to get the size you might have to dereference two pointers to extract the value from the classinfo, then create the proper void[] range to feed deallocate(). If your allocator is a wrapper for malloc/free, the size bit is ignored when calling free and all that work for retrieving the actual size of the object is wasted.

I don't know if this is something worth addressing.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On 10/24/13 5:13 PM, Michel Fortin wrote:
> On 2013-10-24 19:54:41 +0000, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> said:
>
>> Please destroy! I've literally sweat as I'm sending this :o).
>
> Seems good. Although I'm no expert on allocators I find it very easy to
> grasp. One remark though.
>
> While it generally makes sense that you know the size of an allocation
> when you want to deallocate, in some case it might be suboptimal to have
> to provide the size. For instance, you could use an allocator to
> allocate an object (with a virtual table and all). When comes the time
> to deallocate, to get the size you might have to dereference two
> pointers to extract the value from the classinfo, then create the proper
> void[] range to feed deallocate(). If your allocator is a wrapper for
> malloc/free, the size bit is ignored when calling free and all that work
> for retrieving the actual size of the object is wasted.
>
> I don't know if this is something worth addressing.

If that happens, perhaps AffixAllocator!(A, size_t) could be of use by sticking the allocated size just before the allocation. But then you have a different problem - tapping into possibly cold memory when deallocating.

I have plans to define a type that allows user-defined extra data grouped together (e.g. in an array or a list). Due to packing, subword per-allocation data can be stored efficiently. Such a design would allow storing all object sizes together.


Andrei

On 2013-10-25 00:20:41 +0000, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> If that happens, perhaps AffixAllocator!(A, size_t) could be of use by sticking the allocated size just before the allocation. But then you have a different problem - tapping into possibly cold memory when deallocating.

The size is already available in the classinfo, but the underlying allocator doesn't need it when deallocating (the size part will get ignored). The goal is to not have to give the size to deallocate when it doesn't need it (to save you from retrieving it). All you need is to know that deallocate() ignores the size part of the given array telling you whether or not it's safe to call deallocate(pointer[0..0]). You could add an optional "deallocateIgnoresSize" property for that.

By the way, I think the idea of adding a boolean property for this offers less room for misuse than adding an optional deallocate(void*b) overload. With an overload you're duplicating the API and it won't be immediately clear whether or not it's best to provide the size.

But I'm still not sure it's worth the trouble. I'll leave others be the judge of that.

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On 10/24/2013 12:54 PM, Andrei Alexandrescu wrote:
> I know it's been a long wait. Hopefully it was worth it. The alpha release of
> untyped allocators is ready for tire-kicking and a test drive.

This is not a comment on the allocator design, but the module layout.

Would it be possible that this use the "package" idea with one allocator per file instead of the all-in-one-file setup?

On Friday, 25 October 2013 at 01:00:04 UTC, Michel Fortin wrote:
> On 2013-10-25 00:20:41 +0000, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:
>
>> If that happens, perhaps AffixAllocator!(A, size_t) could be of use by sticking the allocated size just before the allocation. But then you have a different problem - tapping into possibly cold memory when deallocating.
>
> The size is already available in the classinfo, but the underlying allocator doesn't need it when deallocating (the size part will get ignored). The goal is to not have to give the size to deallocate when it doesn't need it (to save you from retrieving it). All you need is to know that deallocate() ignores the size part of the given array telling you whether or not it's safe to call deallocate(pointer[0..0]). You could add an optional "deallocateIgnoresSize" property for that.
>
> By the way, I think the idea of adding a boolean property for this offers less room for misuse than adding an optional deallocate(void*b) overload. With an overload you're duplicating the API and it won't be immediately clear whether or not it's best to provide the size.
>
> But I'm still not sure it's worth the trouble. I'll leave others be the judge of that.

Many allocators have different policies for different size. So, if you don't pass the size, you ends up have to recover it, which can be a tedious constraint.

If the length is not used, I hope that The compiler may be able to optimize it way if the allocator is simple (so the deallocate function is inlined). If the deallocate function is quite complex, then I guess this extra calculation is required anyway, and won't matter that much.

On Thursday, 24 October 2013 at 23:22:19 UTC, deadalnix wrote:
> The first thing I notice is that this module is dead unsafe. I'm not sure it belong to std. We should probably reserve that for a "guru" part of the standard lib. Putting some system function in the standard lib is OK, as they are packaged around concepts (range, array, algorithms, etc . . .) but an entire module dedicated to @system stuff seems like it belong to another place.

std.unsafe.*?

On Friday, 25 October 2013 at 01:10:24 UTC, Meta wrote:
> On Thursday, 24 October 2013 at 23:22:19 UTC, deadalnix wrote:
>> The first thing I notice is that this module is dead unsafe. I'm not sure it belong to std. We should probably reserve that for a "guru" part of the standard lib. Putting some system function in the standard lib is OK, as they are packaged around concepts (range, array, algorithms, etc . . .) but an entire module dedicated to @system stuff seems like it belong to another place.
>
> std.unsafe.*?

uac.*

Many bonus points for who get this.

On Thursday, 24 October 2013 at 19:53:56 UTC, Andrei Alexandrescu wrote:
> Hello,
>
>
> I know it's been a long wait. Hopefully it was worth it. The alpha release of untyped allocators is ready for tire-kicking and a test drive.
>
> Code: https://github.com/andralex/phobos/blob/allocator/std/allocator.d
>
> Dox: http://erdani.com/d/phobos-prerelease/std_allocator.html
>
> Warning: this is alpha quality. Unit tests are thin, and there are no benchmarks. Both would be appreciated, particularly benchmarks to validate the gains (which I speculate can be very sizable) of custom-built, special-purpose allocators compared to traditional allocators.
>
> I acknowledge I'm clearly in no position to evaluate this design. I have been knocking around it for long enough to have no idea how easy it is to get into it from the outside, or how good it is. By all signs I could gather this feels like good design, and one of the best I've ever put together. The allocators defined have an archetypal feeling, are flexible both statically and dynamically, and morph and combine in infinite ways.
>
> CAllocator and CAllocatorImpl make the link between the static and dynamic worlds. Once an allocator is assembled out of pieces and finely tuned, wrapping it in a dynamic API is a snap.
>
> Please destroy! I've literally sweat as I'm sending this :o).
>
>
> Andrei

Looks good at first glance. A few notes:

- GC.realloc() is nothrow, but you are still catching OOM exception in GCAllocator.reallocate(). You don't in other methods (e.g. allocate)
- methods aren't marked as nothrow, they probably should be
- I don't understand how UntypedAllocator.collect() works - since it is not allowed to store any pointers in it, calling collect() on it should just free all allocated memory, right? In this case, it might be best to rename it. It will also be beneficial to pass proper bitmask attribute to GC.allocate() (e.g. BlkAttr.NO_SCAN).
- Many constants are global lowercase (i.e. platformAlignment, unbounded etc). It confused me for a bit when I was reading code ("where the hell is this variable coming from?").

Will read the rest later.

On 2013-10-24 21:54, Andrei Alexandrescu wrote:
> Hello,
>
>
> I know it's been a long wait. Hopefully it was worth it. The alpha
> release of untyped allocators is ready for tire-kicking and a test drive.
>
> Code: https://github.com/andralex/phobos/blob/allocator/std/allocator.d
>
> Dox: http://erdani.com/d/phobos-prerelease/std_allocator.html

As someone else mentioned. Shouldn't this possibly be place in druntime instead. It might take advantage of some allocator where it cannot use the GC.

I think we can come up with better names for "it" and "goodAllocSize".

Typo:

In the table, the description of goodAllocSize:

"This module defines a default mplementation"

Missing an "i" in "implementation".

-- 
/Jacob Carlborg