January 07, 2013
Hello, folks!

I'm on to a project, which requires manual memory management using custom allocators, but I can't seem to get dynamic arrays and associative arrays to work.

The std.conv.emplace only allocates the pointer and the size of the dynamic array and pointer to the associative array, which is half the issue.

I can work around the dynamic array by manually allocating the elements of the array and returning a slice to the result (although I'd be really glad if I could directly use arrays with my custom allocators).

The biggest problem is the associative array, the storage of which is completely hidden and implemenentation-specific, so I can't work around it the way I can with dynamic arrays.

How can I have an associative array, which uses a custom allocator?

-- 
Bye,
Gor Gyolchanyan.


January 07, 2013
On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
> How can I have an associative array, which uses a custom allocator?

I'm afraid the only viable solution right now is to implement your own AA type as a struct with overloaded operators (which is in fact what the built-in AAs are lowered to as well).

There are two downside to this, though - besides, of course, the fact that you need a custom implementation:
 - You cannot pass your type to library functions expecting a built-in associative array.
 - You lose the convenient literal syntax. This could be fixed in the language, though, by providing a rewrite to a variadic constructor of user types for array/AA literals, thus eliminating the need for GC allocations (gah, another thing I just need to find the time to write up a DIP for…).

David
January 07, 2013
On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger <see@klickverbot.at> wrote:

> On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
>
>> How can I have an associative array, which uses a custom allocator?
>>
>
> I'm afraid the only viable solution right now is to implement your own AA type as a struct with overloaded operators (which is in fact what the built-in AAs are lowered to as well).
>
> There are two downside to this, though - besides, of course, the fact that
> you need a custom implementation:
>  - You cannot pass your type to library functions expecting a built-in
> associative array.
>  - You lose the convenient literal syntax. This could be fixed in the
> language, though, by providing a rewrite to a variadic constructor of user
> types for array/AA literals, thus eliminating the need for GC allocations
> (gah, another thing I just need to find the time to write up a DIP for…).
>
> David
>

This means, that dlang.org is lying. D doesn't provide both a garbage collector and manual memory management. It provides a garbage collector and a lousy excuse for manual memory management. As much as I love D for it's metaprogramming and generative programming, it's not even remotely fit for system-level programming the way it claims it is.

I don't mean to be trolling, but it's not the first time I got grossly disappointed in D.

-- 
Bye,
Gor Gyolchanyan.


January 07, 2013
How is D manual memory management any worse than plain C one?
Plenty of language features depend on GC but stuff that is left can hardly be named "a lousy excuse". It lacks some convenience and guidelines based on practical experience but it is already as capable as some of wide-spread solutions for systems programming (C). In fact I'd be much more afraid of runtime issues when doing system stuff than GC ones.

On Monday, 7 January 2013 at 15:49:50 UTC, Gor Gyolchanyan wrote:
> On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger <see@klickverbot.at> wrote:
>
>> On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
>>
>>> How can I have an associative array, which uses a custom allocator?
>>>
>>
>> I'm afraid the only viable solution right now is to implement your own AA
>> type as a struct with overloaded operators (which is in fact what the
>> built-in AAs are lowered to as well).
>>
>> There are two downside to this, though - besides, of course, the fact that
>> you need a custom implementation:
>>  - You cannot pass your type to library functions expecting a built-in
>> associative array.
>>  - You lose the convenient literal syntax. This could be fixed in the
>> language, though, by providing a rewrite to a variadic constructor of user
>> types for array/AA literals, thus eliminating the need for GC allocations
>> (gah, another thing I just need to find the time to write up a DIP for…).
>>
>> David
>>
>
> This means, that dlang.org is lying. D doesn't provide both a garbage
> collector and manual memory management. It provides a garbage collector and
> a lousy excuse for manual memory management. As much as I love D for it's
> metaprogramming and generative programming, it's not even remotely fit for
> system-level programming the way it claims it is.
>
> I don't mean to be trolling, but it's not the first time I got grossly
> disappointed in D.
January 07, 2013
On Monday, 7 January 2013 at 16:12:22 UTC, mist wrote:
> How is D manual memory management any worse than plain C one?
> Plenty of language features depend on GC but stuff that is left can hardly be named "a lousy excuse". It lacks some convenience and guidelines based on practical experience but it is already as capable as some of wide-spread solutions for systems programming (C). In fact I'd be much more afraid of runtime issues when doing system stuff than GC ones.

I think the point being made was that built in language features should not be dependent on the need for a GC because it means that you cannot fully use the language without a GC present and active. We can perhaps excuse the std library, but certainly not the language itself, because the claim is made that D's GC is fully optional.

--rt
January 07, 2013
On Monday, January 07, 2013 17:55:35 Rob T wrote:
> On Monday, 7 January 2013 at 16:12:22 UTC, mist wrote:
> > How is D manual memory management any worse than plain C one? Plenty of language features depend on GC but stuff that is left can hardly be named "a lousy excuse". It lacks some convenience and guidelines based on practical experience but it is already as capable as some of wide-spread solutions for systems programming (C). In fact I'd be much more afraid of runtime issues when doing system stuff than GC ones.
> 
> I think the point being made was that built in language features should not be dependent on the need for a GC because it means that you cannot fully use the language without a GC present and active. We can perhaps excuse the std library, but certainly not the language itself, because the claim is made that D's GC is fully optional.

I don't think that any of the documentation or D's developers have ever claimed that you could use the full language without the GC. Quite the opposite in fact. There are a number of language features that require the GC - including AAs, array concatenation, and closures. You _can_ program in D without the GC, but you lose features, and there's no way around that. It may be the case that some features currently require the GC when they shouldn't, but there are definitely features that _must_ have the GC and _cannot_ be implemented otherwise (e.g. array concatenation and closures). So, if you want to ditch the GC completely, it comes at a cost, and AFAIK no one around here is saying otherwise. You _can_ do it though if you really want to.

In general however, the best approach if you want to minimize GC involvement is to generally use manual memory management and minimize your usage of features that require the GC rather than try and get rid of it entirely, because going the extra mile to remove its use completely generally just isn't worth it. Kith-Sa posted some good advice on this just the other day, and he's written a game engine in D:

http://forum.dlang.org/post/vbsajlgotanuhmmpnspf@forum.dlang.org

- Jonathan M Davis
January 07, 2013
On Mon, Jan 7, 2013 at 8:55 PM, Rob T <rob@ucora.com> wrote:

> On Monday, 7 January 2013 at 16:12:22 UTC, mist wrote:
>
>> How is D manual memory management any worse than plain C one?
>> Plenty of language features depend on GC but stuff that is left can
>> hardly be named "a lousy excuse". It lacks some convenience and guidelines
>> based on practical experience but it is already as capable as some of
>> wide-spread solutions for systems programming (C). In fact I'd be much more
>> afraid of runtime issues when doing system stuff than GC ones.
>>
>
> I think the point being made was that built in language features should not be dependent on the need for a GC because it means that you cannot fully use the language without a GC present and active. We can perhaps excuse the std library, but certainly not the language itself, because the claim is made that D's GC is fully optional.
>
> --rt
>


You're absolutely right. D would be far better if there was a way to specify  custom allocators for built-in data structures. Perhaps another magical property:

int[int] a;
a.allocator = new MyCustomAllocator;
a[5] = 5;

That's the least code-breaking way I can think of.

-- 
Bye,
Gor Gyolchanyan.


January 07, 2013
Am 07.01.2013 16:49, schrieb Gor Gyolchanyan:
> On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger <see@klickverbot.at
> <mailto:see@klickverbot.at>> wrote:
>
>     On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
>
>         How can I have an associative array, which uses a custom allocator?
>
>
>     I'm afraid the only viable solution right now is to implement your
>     own AA type as a struct with overloaded operators (which is in fact
>     what the built-in AAs are lowered to as well).
>
>     There are two downside to this, though - besides, of course, the
>     fact that you need a custom implementation:
>       - You cannot pass your type to library functions expecting a
>     built-in associative array.
>       - You lose the convenient literal syntax. This could be fixed in
>     the language, though, by providing a rewrite to a variadic
>     constructor of user types for array/AA literals, thus eliminating
>     the need for GC allocations (gah, another thing I just need to find
>     the time to write up a DIP for…).
>
>     David
>
>
> This means, that dlang.org <http://dlang.org> is lying. D doesn't
> provide both a garbage collector and manual memory management. It
> provides a garbage collector and a lousy excuse for manual memory
> management. As much as I love D for it's metaprogramming and generative
> programming, it's not even remotely fit for system-level programming the
> way it claims it is.
>
> I don't mean to be trolling, but it's not the first time I got grossly
> disappointed in D.
>
> --
> Bye,
> Gor Gyolchanyan.

You can use my GC free hashmap if you want:
https://github.com/Ingrater/druntime/blob/master/src/core/hashmap.d

Kind Regards
Benjamin Thaut

January 07, 2013
On Monday, 7 January 2013 at 17:57:49 UTC, Benjamin Thaut wrote:
> Am 07.01.2013 16:49, schrieb Gor Gyolchanyan:
>> On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger <see@klickverbot.at
>> <mailto:see@klickverbot.at>> wrote:
>>
>>    On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
>>
>>        How can I have an associative array, which uses a custom allocator?
>>
>>
>>    I'm afraid the only viable solution right now is to implement your
>>    own AA type as a struct with overloaded operators (which is in fact
>>    what the built-in AAs are lowered to as well).
>>
>>    There are two downside to this, though - besides, of course, the
>>    fact that you need a custom implementation:
>>      - You cannot pass your type to library functions expecting a
>>    built-in associative array.
>>      - You lose the convenient literal syntax. This could be fixed in
>>    the language, though, by providing a rewrite to a variadic
>>    constructor of user types for array/AA literals, thus eliminating
>>    the need for GC allocations (gah, another thing I just need to find
>>    the time to write up a DIP for…).
>>
>>    David
>>
>>
>> This means, that dlang.org <http://dlang.org> is lying. D doesn't
>> provide both a garbage collector and manual memory management. It
>> provides a garbage collector and a lousy excuse for manual memory
>> management. As much as I love D for it's metaprogramming and generative
>> programming, it's not even remotely fit for system-level programming the
>> way it claims it is.
>>
>> I don't mean to be trolling, but it's not the first time I got grossly
>> disappointed in D.
>>
>> --
>> Bye,
>> Gor Gyolchanyan.
>
> You can use my GC free hashmap if you want:
> https://github.com/Ingrater/druntime/blob/master/src/core/hashmap.d
>
> Kind Regards
> Benjamin Thaut

Benjamin, maybe you could in your spare time draw a DIP for Allocators?
This is really very important thing we need for yesterday and putting all the work on Andrei won't help with it.

Your solution seems to work very well in practice, maybe it would be possible to adapt it for Druntime/Phobos needs? Or maybe its already fully designed and only needs polishing and pull request?

Thanks!
January 07, 2013
Am 07.01.2013 20:10, schrieb nazriel:
> Benjamin, maybe you could in your spare time draw a DIP for Allocators?
> This is really very important thing we need for yesterday and putting
> all the work on Andrei won't help with it.
>
> Your solution seems to work very well in practice, maybe it would be
> possible to adapt it for Druntime/Phobos needs? Or maybe its already
> fully designed and only needs polishing and pull request?
>
> Thanks!

I'm very busy with university atm. But if I have some free time again I could think about doing this. But I actually would want someone who can do a decision on this to actually look at my fork and confirm that this is actually wanted. I'm not going to do all the work for pull request without actually having some kind of confirmation.

Kind Regards
Benjamin Thaut
« First   ‹ Prev
1 2 3 4 5 6 7 8 9 10 11
Top | Discussion index | About this forum | D home