I hesitated to post this on the topic of druntime because probably I will learn something new if I am totally/stupidly wrong.

In druntime/blob/master/src/rt/aaA.d Lines 553-562:

...
    auto p = aa.findSlotInsert(hash);
    if (p.deleted)
        --aa.deleted;
    // check load factor and possibly grow
    else if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
    {
        aa.grow(ti.key);
        p = aa.findSlotInsert(hash);
        assert(p.empty);
    }
...

findSlotInsert are called two times. Why not:

    if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
        aa.grow(ti.key);

    auto p = aa.findSlotInsert(hash); // only one call is enough?

    if (p.deleted)
        --aa.deleted;
...

If I am not wrong this modification will not corrupt the current state of the hash table?

On Friday, 14 February 2020 at 22:57:31 UTC, Ferhat Kurtulmuş wrote:
> findSlotInsert are called two times. Why not:
>
>     if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
>         aa.grow(ti.key);
>
>     auto p = aa.findSlotInsert(hash); // only one call is enough?
>
>     if (p.deleted)
>         --aa.deleted;
> ...
>
> If I am not wrong this modification will not corrupt the current state of the hash table?

`used` counts both filled and deleted buckets, so it shouldn't be incremented when changing a deleted bucket into a filled bucket.

On 2/14/20 5:57 PM, Ferhat Kurtulmuş wrote:
> I hesitated to post this on the topic of druntime because probably I will learn something new if I am totally/stupidly wrong.

There are no stupid questions, and this is the learn forum. So you are in the right place!

> 
> In druntime/blob/master/src/rt/aaA.d Lines 553-562:

One thing to learn, you can select a section of lines from github (click on first line, then shift-click on last line), then press the 'y' key, and it will generate a permanent link to those lines.

https://github.com/dlang/druntime/blob/a581dc6d1d3bf796fcebd982221d0b3d6bbae437/src/rt/aaA.d#L553-L562

> 
> ...
>      auto p = aa.findSlotInsert(hash);
>      if (p.deleted)
>          --aa.deleted;
>      // check load factor and possibly grow
>      else if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
>      {
>          aa.grow(ti.key);
>          p = aa.findSlotInsert(hash);
>          assert(p.empty);
>      }
> ...
> 
> findSlotInsert are called two times. Why not:
> 
>      if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
>          aa.grow(ti.key);
> 
>      auto p = aa.findSlotInsert(hash); // only one call is enough?
> 
>      if (p.deleted)
>          --aa.deleted;
> ...
> 
> If I am not wrong this modification will not corrupt the current state of the hash table?

A cursory reading:

I think the case where you find the insert slot and the p.deleted is true, you can avoid the grow function.

The grow function is much more expensive than findInsertSlot, so calling it twice is preferable.

The reason we have the deleted status is because we want to reuse memory, but we can't free the memory if it had a dangling reference to it.

The only time those deleted nodes turn into garbage is on a resize.

HOWEVER, one case where we can avoid the double search is if there are no deleted nodes (we keep a count of them). This should be reasonably often in most cases because you insert nodes and don't remove them, or you grew the AA and all deleted nodes are purged.

So maybe change to:

Bucket *p;

if(aa.deleted > 0 && (p = aa.findSlotInsert(hash)).deleted)
   --aa.deleted;
else // everything else the same

-Steve

On Friday, 14 February 2020 at 23:19:31 UTC, Paul Backus wrote:
> On Friday, 14 February 2020 at 22:57:31 UTC, Ferhat Kurtulmuş wrote:
>> findSlotInsert are called two times. Why not:
>>
>>     if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
>>         aa.grow(ti.key);
>>
>>     auto p = aa.findSlotInsert(hash); // only one call is enough?
>>
>>     if (p.deleted)
>>         --aa.deleted;
>> ...
>>
>> If I am not wrong this modification will not corrupt the current state of the hash table?
>
> `used` counts both filled and deleted buckets, so it shouldn't be incremented when changing a deleted bucket into a filled bucket.

    İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM)
        aa.grow(ti.key);

    auto p = aa.findSlotInsert(hash); // only one call is enough?

    if (p.deleted)
        --aa.deleted;
    else
        ++aa.used;

On 2/14/20 6:36 PM, Ferhat Kurtulmuş wrote:
>      İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM)
>          aa.grow(ti.key);

This call is expensive (it reallocates all the buckets and reinserts all the existing data into the new bucket), it should be avoided if in the end we will not be incrementing used.

-Steve

On Friday, 14 February 2020 at 23:28:39 UTC, Steven Schveighoffer wrote:
> On 2/14/20 5:57 PM, Ferhat Kurtulmuş wrote:

> One thing to learn, you can select a section of lines from github (click on first line, then shift-click on last line), then press the 'y' key, and it will generate a permanent link to those lines.
>
> https://github.com/dlang/druntime/blob/a581dc6d1d3bf796fcebd982221d0b3d6bbae437/src/rt/aaA.d#L553-L562

Thank you, I didn't know that.

On Friday, 14 February 2020 at 23:41:45 UTC, Steven Schveighoffer wrote:
> On 2/14/20 6:36 PM, Ferhat Kurtulmuş wrote:
>>      İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM)
>>          aa.grow(ti.key);
>
> This call is expensive (it reallocates all the buckets and reinserts all the existing data into the new bucket), it should be avoided if in the end we will not be incrementing used.
>
> -Steve

I am trying to write a gc-free AA based on the original runtime code (mem with malloc and free). My question is that why my version is slower (about 1.5 times slower) than the runtime version?

https://controlc.com/2e58c305

Those are some differences from runtime version:

- nogc and no typeid of course.
- does not care about postblit of key types (don't need it, assume only basic types are allowed for key type)
- runtime version uses typeid to do some alignments which are not implemented in my version. Is that the reason why my code is slower?

On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote:
> On Friday, 14 February 2020 at 23:41:45 UTC, Steven Schveighoffer wrote:
>> On 2/14/20 6:36 PM, Ferhat Kurtulmuş wrote:
>>>      İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM)
>>>          aa.grow(ti.key);
>>
>> This call is expensive (it reallocates all the buckets and reinserts all the existing data into the new bucket), it should be avoided if in the end we will not be incrementing used.
>>
> 
> I am trying to write a gc-free AA based on the original runtime code (mem with malloc and free). My question is that why my version is slower (about 1.5 times slower) than the runtime version?

Have you ensured you are using -inline, -O, and -release? This is how druntime is compiled.

> https://controlc.com/2e58c305
> 
> Those are some differences from runtime version:
> 
> - nogc and no typeid of course.
> - does not care about postblit of key types (don't need it, assume only basic types are allowed for key type)

Postblit should be automatic for your code because you are doing the types directly.

> - runtime version uses typeid to do some alignments which are not implemented in my version. Is that the reason why my code is slower?

No, alignments are important only if you don't have the type, which you do. The runtime uses void pointers and opaque structs, so it has to duplicate what the compiler is already doing for you.

Your code looks like a direct port, except for the allocations, so it should be as fast if compiled the same. Your code might even be faster, because it's going to be able to inline more aggressively. There may be issues with cache coherency, but I'm not sure how to find or diagnoce those.

I'll note that you are going to leak some memory because you are not freeing deleted buckets when you resize. In the GC version, the GC takes care of those.

-Steve

On Saturday, 15 February 2020 at 14:30:20 UTC, Steven Schveighoffer wrote:
> On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote:
>> On Friday, 14 February 2020 at 23:41:45 UTC, Steven

> I'll note that you are going to leak some memory because you are not freeing deleted buckets when you resize. In the GC version, the GC takes care of those.
>
> -Steve

I appreciate it for reviewing the code and your comments. Speed is good now. I put it on the dub db. I hope I am not violating any copyright. I included name of the original author (Martin Nowak) in the code, and explicitly stated that "betterC port of druntime/blob/master/src/rt/aaA.d".

What do you think about this one? I am not free-ing deleted entry in remove method:
https://github.com/aferust/bcaa/blob/a37b4ee4455477abc82425f32e9cf45394f4c4a1/source/bcaa.d#L228-L230

but here in resize:
https://github.com/aferust/bcaa/blob/a37b4ee4455477abc82425f32e9cf45394f4c4a1/source/bcaa.d#L190-L194

Thus, deleted buckets will wait until a resize call to free them. I think this is better for speed.

On Saturday, 15 February 2020 at 15:21:08 UTC, Ferhat Kurtulmuş wrote:
> On Saturday, 15 February 2020 at 14:30:20 UTC, Steven Schveighoffer wrote:
>> On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote:
>>> On Friday, 14 February 2020 at 23:41:45 UTC, Steven

Ops I ve made commits. Here are the links up to date
https://github.com/aferust/bcaa/blob/3f0f4eaf550a28b80cd95a4f2589b1f3a53fe6c0/source/bcaa.d#L193-L195

https://github.com/aferust/bcaa/blob/3f0f4eaf550a28b80cd95a4f2589b1f3a53fe6c0/source/bcaa.d#L231-L233