Am Thu, 24 Oct 2013 12:54:41 -0700
schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:

> Hello,
> 
> 
> I know it's been a long wait. Hopefully it was worth it. The alpha release of untyped allocators is ready for tire-kicking and a test drive.
> 
> Code: https://github.com/andralex/phobos/blob/allocator/std/allocator.d
> 
> Dox: http://erdani.com/d/phobos-prerelease/std_allocator.html
> 
> Warning: this is alpha quality. Unit tests are thin, and there are no benchmarks. Both would be appreciated, particularly benchmarks to validate the gains (which I speculate can be very sizable) of custom-built, special-purpose allocators compared to traditional allocators.
> 
> I acknowledge I'm clearly in no position to evaluate this design. I have been knocking around it for long enough to have no idea how easy it is to get into it from the outside, or how good it is. By all signs I could gather this feels like good design, and one of the best I've ever put together. The allocators defined have an archetypal feeling, are flexible both statically and dynamically, and morph and combine in infinite ways.
> 
> CAllocator and CAllocatorImpl make the link between the static and dynamic worlds. Once an allocator is assembled out of pieces and finely tuned, wrapping it in a dynamic API is a snap.
> 
> Please destroy! I've literally sweat as I'm sending this :o).
> 
> 
> Andrei

The overall design looks great!


Some small nitpicks:
* byAllocation is not accessible from outside std.allocator
* Is it intentional that AllocatorWithStats can't log the
  __FUNCTION__ / __PRETTY_FUNCTION__ ?
* The documentation for byAllocation should probably state that it only
  lists 'alive' allocations and that this is especially useful for
  finding memory leaks.


And one bigger problem: CAllocatorImpl doesn't work well with AllocatorWithStats.

AllocatorWithStats special members (byAllocation, etc) are not directly accessible. Although we can access them by using .impl it'd be nice to have a special AllocatorWithStats class which directly exposes those members.

The second problem is that the logged file/line is always in std.allocator.d. It's probably not easy to get this working in all cases (especially regarding polymorphism).

On 10/25/13 7:23 AM, Manu wrote:
> 1. I'm still sad there are no allocAligned() functions or something of
> that type to request explicit alignment with allocations. I'm not sure
> there is sufficient support for requesting alignment with allocations.
> The set-able alignment property approach seems a little weird (and only
> seemed to be supported on one allocator?).

Woke up this morning with the following idea.

1. We add two optional API functions:

void[] alignedAllocate(size_t, uint);
bool alignedReallocate(ref void[], size_t, uint);

These would satisfy aligned allocation requests. The pointer thus allocated should be deallocated the usual way (there is no alignedDeallocate).

2. We make alignment a statically-known enum and delete all code that allows it to be get and set at runtime.

Consequences:

1. The API gets larger due to the two new APIs. However, that is offset (probably more than enough) by the removal the option to set the alignment at runtime, which currently is causing severe ripple effects. I think the result will be a simpler overall design and implementation.

2. HeapBlock should be able to implement the two functions. All regions should be able to implement the first.

3. Mallocator could also implement these on Posix: http://man7.org/linux/man-pages/man3/posix_memalign.3.html. However, Windows requires a specific call for deallocating aligned memory. To accommodate both portably, we leave Mallocator as is and create AlignedMallocator that uses the _aligned_* family on Windows and the respective functions on Posix. On Windows, allocate() requests would pass a default of platformSize (i.e. 16 I suspect) to _aligned_malloc.

Is this satisfactory for everyone?


Andrei

On 10/26/13 4:02 AM, Jacob Carlborg wrote:
> On 2013-10-26 11:03, Marco Leise wrote:
>
>> I'm actually fine with that name, probably because I tend to
>> prefix my D reference types in that manner, too. 'I' for
>> interfaces, 'C' for classes and maybe 'A' for abstract classes.
>
> That's just horrible.

I think it's a commonly-used convention.

Andrei

On 24/10/13 21:54, Andrei Alexandrescu wrote:
> I know it's been a long wait. Hopefully it was worth it. The alpha release of
> untyped allocators is ready for tire-kicking and a test drive.
>
>  ...
>
> Please destroy! I've literally sweat as I'm sending this :o).

Hmm, seems like I'm coming late to quite a party ... :-)

Like John Colvin, I feel that much of this is over my head -- I don't have the experience/expertise to judge requirements or solutions.  But it looks like very nice code indeed and I'm encouraged by the fact that most "Can it do ...?" questions seem to have a ready answer.

My own interests have always been less in allocators per se and more in the functionality they unlock -- stuff like containers, for example.  It may be unfamiliarity -- my own code has either used new/delete (the latter only in C++, not D...) or malloc/dealloc -- but looking at this module as it stands I don't really see how to practically make use of it; which makes me wonder whether the people calling for it to be in core rather than std have a point.

Maybe it would help to have code examples that really show these allocators really being used to do something -- the existing examples are comprehensive and show what to do, but not really what to do _with_ the constructs thus created.

Or, maybe I should just educate myself more ... ;-)

Suffice to say that my overwhelming feeling is of gratitude and admiration for all the obvious thought and hard work put into creating this module.  Thanks and congratulations, Andrei! :-)

Best wishes,

    -- Joe

On 25/10/13 11:37, Namespace wrote:
> We would have then the possibility to manage our memory by ourself. One of D's
> promises is, that the GC can be disabled. Yes, it can, but then we have many
> many things which do not work. For example built-in arrays. With the ability of
> allocators the promise could come true.

That's something I'd really like to know more about.

My D code almost invariably works with the "natural" way to handle memory in D, which is to use "new" where needed, plus stuff like array appending ~ and alterations to array lengths, with all allocations handled behind the scenes by the GC.  I've always felt bad about the fact that this therefore imposes use of the GC on anyone who uses my code.  It would be great if one could just write idiomatic D code and know that others using it could dictate different memory-management strategies and have them "just work".

On Saturday, 26 October 2013 at 15:42:48 UTC, Joseph Rushton Wakeling wrote:
> On 25/10/13 11:37, Namespace wrote:
>> We would have then the possibility to manage our memory by ourself. One of D's
>> promises is, that the GC can be disabled. Yes, it can, but then we have many
>> many things which do not work. For example built-in arrays. With the ability of
>> allocators the promise could come true.
>
> That's something I'd really like to know more about.
>
> My D code almost invariably works with the "natural" way to handle memory in D, which is to use "new" where needed, plus stuff like array appending ~ and alterations to array lengths, with all allocations handled behind the scenes by the GC.  I've always felt bad about the fact that this therefore imposes use of the GC on anyone who uses my code.  It would be great if one could just write idiomatic D code and know that others using it could dictate different memory-management strategies and have them "just work".

Read on from here to find out more:
http://forum.dlang.org/thread/l4btsk$5u8$1@digitalmars.com?page=5#post-uqolhuqqygquxnaxahkz:40forum.dlang.org

:)

On 26/10/13 17:46, Namespace wrote:
> Read on from here to find out more:
> http://forum.dlang.org/thread/l4btsk$5u8$1@digitalmars.com?page=5#post-uqolhuqqygquxnaxahkz:40forum.dlang.org

I already did, and if I understand right, things are looking good ... but I wanted to be sure whether or not I understand :-)

On 25/10/13 20:41, Namespace wrote:
> Did you mean to get rid of built-in arrays / kill int[] and replace it with
> Array!T?

Array!T is problematic as things stand -- e.g. you can't foreach over one.  So, forgetting syntax preferences, there needs to be some work on containers before they can "just work" like the builtins.

If it's possible, I'd rather see the converse -- that code that assumes the GC will "just work" with other allocation strategies, so one can use the builtins without worrying.  But am I being naively hopeful in seeking that? :-)

26-Oct-2013 20:01, Joseph Rushton Wakeling пишет:
> On 25/10/13 20:41, Namespace wrote:
>> Did you mean to get rid of built-in arrays / kill int[] and replace it
>> with
>> Array!T?
>
> Array!T is problematic as things stand -- e.g. you can't foreach over
> one.

Sure you can. Try it and rejoice:

void main()
{
    import std.container, std.stdio;
    Array!int a = make!(Array!int)(1,2,3,4);
    //the rule is: if a can be sliced then slice it and use that slice
    foreach(v; a)
    {
        writeln(v);
    }
}

> So, forgetting syntax preferences, there needs to be some work on
> containers before they can "just work" like the builtins.

Depends on what's required for you to consider it "just works".
But in general they can't be complete replica of built-ins for many reasons, built-ins being designed with GC in mind is one.
Other problems include having no user-land analog of implicit tail-const of arrays.

> If it's possible, I'd rather see the converse -- that code that assumes
> the GC will "just work" with other allocation strategies, so one can use
> the builtins without worrying.

Only if you are switching to from one GC kind to another. There is no way out of automatic memory management.

> But am I being naively hopeful in
> seeking that? :-)

Yes.

-- 
Dmitry Olshansky

On Saturday, 26 October 2013 at 16:10:46 UTC, Dmitry Olshansky wrote:
>> If it's possible, I'd rather see the converse -- that code that assumes
>> the GC will "just work" with other allocation strategies, so one can use
>> the builtins without worrying.
>
> Only if you are switching to from one GC kind to another. There is no way out of automatic memory management.

Waiting for ARC :)