View mode: basic / threaded / horizontal-split · Log in · Help
January 07, 2013
manual memory management
Hello, folks!

I'm on to a project, which requires manual memory management using custom
allocators, but I can't seem to get dynamic arrays and associative arrays
to work.

The std.conv.emplace only allocates the pointer and the size of the dynamic
array and pointer to the associative array, which is half the issue.

I can work around the dynamic array by manually allocating the elements of
the array and returning a slice to the result (although I'd be really glad
if I could directly use arrays with my custom allocators).

The biggest problem is the associative array, the storage of which is
completely hidden and implemenentation-specific, so I can't work around it
the way I can with dynamic arrays.

How can I have an associative array, which uses a custom allocator?

-- 
Bye,
Gor Gyolchanyan.
January 07, 2013
Re: manual memory management
On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
> How can I have an associative array, which uses a custom 
> allocator?

I'm afraid the only viable solution right now is to implement 
your own AA type as a struct with overloaded operators (which is 
in fact what the built-in AAs are lowered to as well).

There are two downside to this, though - besides, of course, the 
fact that you need a custom implementation:
 - You cannot pass your type to library functions expecting a 
built-in associative array.
 - You lose the convenient literal syntax. This could be fixed in 
the language, though, by providing a rewrite to a variadic 
constructor of user types for array/AA literals, thus eliminating 
the need for GC allocations (gah, another thing I just need to 
find the time to write up a DIP for…).

David
January 07, 2013
Re: manual memory management
On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger <see@klickverbot.at> wrote:

> On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
>
>> How can I have an associative array, which uses a custom allocator?
>>
>
> I'm afraid the only viable solution right now is to implement your own AA
> type as a struct with overloaded operators (which is in fact what the
> built-in AAs are lowered to as well).
>
> There are two downside to this, though - besides, of course, the fact that
> you need a custom implementation:
>  - You cannot pass your type to library functions expecting a built-in
> associative array.
>  - You lose the convenient literal syntax. This could be fixed in the
> language, though, by providing a rewrite to a variadic constructor of user
> types for array/AA literals, thus eliminating the need for GC allocations
> (gah, another thing I just need to find the time to write up a DIP for…).
>
> David
>

This means, that dlang.org is lying. D doesn't provide both a garbage
collector and manual memory management. It provides a garbage collector and
a lousy excuse for manual memory management. As much as I love D for it's
metaprogramming and generative programming, it's not even remotely fit for
system-level programming the way it claims it is.

I don't mean to be trolling, but it's not the first time I got grossly
disappointed in D.

-- 
Bye,
Gor Gyolchanyan.
January 07, 2013
Re: manual memory management
How is D manual memory management any worse than plain C one?
Plenty of language features depend on GC but stuff that is left 
can hardly be named "a lousy excuse". It lacks some convenience 
and guidelines based on practical experience but it is already as 
capable as some of wide-spread solutions for systems programming 
(C). In fact I'd be much more afraid of runtime issues when doing 
system stuff than GC ones.

On Monday, 7 January 2013 at 15:49:50 UTC, Gor Gyolchanyan wrote:
> On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger 
> <see@klickverbot.at> wrote:
>
>> On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan 
>> wrote:
>>
>>> How can I have an associative array, which uses a custom 
>>> allocator?
>>>
>>
>> I'm afraid the only viable solution right now is to implement 
>> your own AA
>> type as a struct with overloaded operators (which is in fact 
>> what the
>> built-in AAs are lowered to as well).
>>
>> There are two downside to this, though - besides, of course, 
>> the fact that
>> you need a custom implementation:
>>  - You cannot pass your type to library functions expecting a 
>> built-in
>> associative array.
>>  - You lose the convenient literal syntax. This could be fixed 
>> in the
>> language, though, by providing a rewrite to a variadic 
>> constructor of user
>> types for array/AA literals, thus eliminating the need for GC 
>> allocations
>> (gah, another thing I just need to find the time to write up a 
>> DIP for…).
>>
>> David
>>
>
> This means, that dlang.org is lying. D doesn't provide both a 
> garbage
> collector and manual memory management. It provides a garbage 
> collector and
> a lousy excuse for manual memory management. As much as I love 
> D for it's
> metaprogramming and generative programming, it's not even 
> remotely fit for
> system-level programming the way it claims it is.
>
> I don't mean to be trolling, but it's not the first time I got 
> grossly
> disappointed in D.
January 07, 2013
Re: manual memory management
On Monday, 7 January 2013 at 16:12:22 UTC, mist wrote:
> How is D manual memory management any worse than plain C one?
> Plenty of language features depend on GC but stuff that is left 
> can hardly be named "a lousy excuse". It lacks some convenience 
> and guidelines based on practical experience but it is already 
> as capable as some of wide-spread solutions for systems 
> programming (C). In fact I'd be much more afraid of runtime 
> issues when doing system stuff than GC ones.

I think the point being made was that built in language features 
should not be dependent on the need for a GC because it means 
that you cannot fully use the language without a GC present and 
active. We can perhaps excuse the std library, but certainly not 
the language itself, because the claim is made that D's GC is 
fully optional.

--rt
January 07, 2013
Re: manual memory management
On Monday, January 07, 2013 17:55:35 Rob T wrote:
> On Monday, 7 January 2013 at 16:12:22 UTC, mist wrote:
> > How is D manual memory management any worse than plain C one?
> > Plenty of language features depend on GC but stuff that is left
> > can hardly be named "a lousy excuse". It lacks some convenience
> > and guidelines based on practical experience but it is already
> > as capable as some of wide-spread solutions for systems
> > programming (C). In fact I'd be much more afraid of runtime
> > issues when doing system stuff than GC ones.
> 
> I think the point being made was that built in language features
> should not be dependent on the need for a GC because it means
> that you cannot fully use the language without a GC present and
> active. We can perhaps excuse the std library, but certainly not
> the language itself, because the claim is made that D's GC is
> fully optional.

I don't think that any of the documentation or D's developers have ever 
claimed that you could use the full language without the GC. Quite the 
opposite in fact. There are a number of language features that require the GC 
- including AAs, array concatenation, and closures. You _can_ program in D 
without the GC, but you lose features, and there's no way around that. It may 
be the case that some features currently require the GC when they shouldn't, 
but there are definitely features that _must_ have the GC and _cannot_ be 
implemented otherwise (e.g. array concatenation and closures). So, if you want 
to ditch the GC completely, it comes at a cost, and AFAIK no one around here 
is saying otherwise. You _can_ do it though if you really want to.

In general however, the best approach if you want to minimize GC involvement 
is to generally use manual memory management and minimize your usage of 
features that require the GC rather than try and get rid of it entirely, 
because going the extra mile to remove its use completely generally just isn't 
worth it. Kith-Sa posted some good advice on this just the other day, and he's 
written a game engine in D:

http://forum.dlang.org/post/vbsajlgotanuhmmpnspf@forum.dlang.org

- Jonathan M Davis
January 07, 2013
Re: manual memory management
On Mon, Jan 7, 2013 at 8:55 PM, Rob T <rob@ucora.com> wrote:

> On Monday, 7 January 2013 at 16:12:22 UTC, mist wrote:
>
>> How is D manual memory management any worse than plain C one?
>> Plenty of language features depend on GC but stuff that is left can
>> hardly be named "a lousy excuse". It lacks some convenience and guidelines
>> based on practical experience but it is already as capable as some of
>> wide-spread solutions for systems programming (C). In fact I'd be much more
>> afraid of runtime issues when doing system stuff than GC ones.
>>
>
> I think the point being made was that built in language features should
> not be dependent on the need for a GC because it means that you cannot
> fully use the language without a GC present and active. We can perhaps
> excuse the std library, but certainly not the language itself, because the
> claim is made that D's GC is fully optional.
>
> --rt
>


You're absolutely right. D would be far better if there was a way to
specify  custom allocators for built-in data structures. Perhaps another
magical property:

int[int] a;
a.allocator = new MyCustomAllocator;
a[5] = 5;

That's the least code-breaking way I can think of.

-- 
Bye,
Gor Gyolchanyan.
January 07, 2013
Re: manual memory management
Am 07.01.2013 16:49, schrieb Gor Gyolchanyan:
> On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger <see@klickverbot.at
> <mailto:see@klickverbot.at>> wrote:
>
>     On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan wrote:
>
>         How can I have an associative array, which uses a custom allocator?
>
>
>     I'm afraid the only viable solution right now is to implement your
>     own AA type as a struct with overloaded operators (which is in fact
>     what the built-in AAs are lowered to as well).
>
>     There are two downside to this, though - besides, of course, the
>     fact that you need a custom implementation:
>       - You cannot pass your type to library functions expecting a
>     built-in associative array.
>       - You lose the convenient literal syntax. This could be fixed in
>     the language, though, by providing a rewrite to a variadic
>     constructor of user types for array/AA literals, thus eliminating
>     the need for GC allocations (gah, another thing I just need to find
>     the time to write up a DIP for…).
>
>     David
>
>
> This means, that dlang.org <http://dlang.org> is lying. D doesn't
> provide both a garbage collector and manual memory management. It
> provides a garbage collector and a lousy excuse for manual memory
> management. As much as I love D for it's metaprogramming and generative
> programming, it's not even remotely fit for system-level programming the
> way it claims it is.
>
> I don't mean to be trolling, but it's not the first time I got grossly
> disappointed in D.
>
> --
> Bye,
> Gor Gyolchanyan.

You can use my GC free hashmap if you want:
https://github.com/Ingrater/druntime/blob/master/src/core/hashmap.d

Kind Regards
Benjamin Thaut
January 07, 2013
Re: manual memory management
On Monday, 7 January 2013 at 17:57:49 UTC, Benjamin Thaut wrote:
> Am 07.01.2013 16:49, schrieb Gor Gyolchanyan:
>> On Mon, Jan 7, 2013 at 7:25 PM, David Nadlinger 
>> <see@klickverbot.at
>> <mailto:see@klickverbot.at>> wrote:
>>
>>    On Monday, 7 January 2013 at 15:01:27 UTC, Gor Gyolchanyan 
>> wrote:
>>
>>        How can I have an associative array, which uses a 
>> custom allocator?
>>
>>
>>    I'm afraid the only viable solution right now is to 
>> implement your
>>    own AA type as a struct with overloaded operators (which is 
>> in fact
>>    what the built-in AAs are lowered to as well).
>>
>>    There are two downside to this, though - besides, of 
>> course, the
>>    fact that you need a custom implementation:
>>      - You cannot pass your type to library functions 
>> expecting a
>>    built-in associative array.
>>      - You lose the convenient literal syntax. This could be 
>> fixed in
>>    the language, though, by providing a rewrite to a variadic
>>    constructor of user types for array/AA literals, thus 
>> eliminating
>>    the need for GC allocations (gah, another thing I just need 
>> to find
>>    the time to write up a DIP for…).
>>
>>    David
>>
>>
>> This means, that dlang.org <http://dlang.org> is lying. D 
>> doesn't
>> provide both a garbage collector and manual memory management. 
>> It
>> provides a garbage collector and a lousy excuse for manual 
>> memory
>> management. As much as I love D for it's metaprogramming and 
>> generative
>> programming, it's not even remotely fit for system-level 
>> programming the
>> way it claims it is.
>>
>> I don't mean to be trolling, but it's not the first time I got 
>> grossly
>> disappointed in D.
>>
>> --
>> Bye,
>> Gor Gyolchanyan.
>
> You can use my GC free hashmap if you want:
> https://github.com/Ingrater/druntime/blob/master/src/core/hashmap.d
>
> Kind Regards
> Benjamin Thaut

Benjamin, maybe you could in your spare time draw a DIP for 
Allocators?
This is really very important thing we need for yesterday and 
putting all the work on Andrei won't help with it.

Your solution seems to work very well in practice, maybe it would 
be possible to adapt it for Druntime/Phobos needs? Or maybe its 
already fully designed and only needs polishing and pull request?

Thanks!
January 07, 2013
Re: manual memory management
Am 07.01.2013 20:10, schrieb nazriel:
> Benjamin, maybe you could in your spare time draw a DIP for Allocators?
> This is really very important thing we need for yesterday and putting
> all the work on Andrei won't help with it.
>
> Your solution seems to work very well in practice, maybe it would be
> possible to adapt it for Druntime/Phobos needs? Or maybe its already
> fully designed and only needs polishing and pull request?
>
> Thanks!

I'm very busy with university atm. But if I have some free time again I 
could think about doing this. But I actually would want someone who can 
do a decision on this to actually look at my fork and confirm that this 
is actually wanted. I'm not going to do all the work for pull request 
without actually having some kind of confirmation.

Kind Regards
Benjamin Thaut
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home