Hi everyone, just wanting some help with optimisation if anyone is kind enough :-)

I have a loop that iterates potentially millions of times, and inside that loop I have code that appends some strings together, e.g.:

string key = s1 ~ "_" ~ s2;

I discovered that due to the memory allocation required, this slows the execution significantly.

s1 is always a two character string, e.g "AA", and s2 is always a single character.

What I want to do is something like this:

Outside the loop:

char[4] buffer;
buffer[2] = '_';

Then inside the loop

buffer[0] = s1[0];
buffer[1] = s1[1];
buffer[3] = s2[0];

This works OK, however, I then need to use the buffer value to check for an existing value in a hashmap / associative array.

Code such as:

if(buffer in myHash) {

}

throws an access violation.  A string value works without error.  Is there a way for me to use a buffer AND use it in functions expecting strings?

I can use idup() on the char[] to make a string, but again we're allocating memory which I'd rather avoid.

Thanks sincerely in advance,
Cheers,
Andrew.

On 06/22/2016 02:57 PM, Andrew Chapman wrote:

> Code such as:
>
> if(buffer in myHash) {
>
> }
>
> throws an access violation.  A string value works without error.

Does it throw an exception? Can you reproduce the issue with a short program?

Ali

On Wed, Jun 22, 2016 at 09:57:04PM +0000, Andrew Chapman via Digitalmars-d-learn wrote:
> Hi everyone, just wanting some help with optimisation if anyone is kind enough :-)
> 
> I have a loop that iterates potentially millions of times, and inside that loop I have code that appends some strings together, e.g.:
> 
> string key = s1 ~ "_" ~ s2;
> 
> I discovered that due to the memory allocation required, this slows the execution significantly.
> 
> s1 is always a two character string, e.g "AA", and s2 is always a single character.

Yes, frequent allocation of small strings is a performance killer. Using a static array (as you've done below) is much better.


> What I want to do is something like this:
> 
> Outside the loop:
> 
> char[4] buffer;
> buffer[2] = '_';
> 
> Then inside the loop
> 
> buffer[0] = s1[0];
> buffer[1] = s1[1];
> buffer[3] = s2[0];
> 
> This works OK, however, I then need to use the buffer value to check for an existing value in a hashmap / associative array.
> 
> Code such as:
> 
> if(buffer in myHash) {
> 
> }
> 
> throws an access violation.  A string value works without error.  Is there a way for me to use a buffer AND use it in functions expecting strings?

Maybe try:

	if (buffer[] in myHash) { ... }

?  Does that make a difference?


T

-- 
This sentence is false.

Perfect, thank you! :-) Works like a charm.

On Wednesday, 22 June 2016 at 22:41:24 UTC, H. S. Teoh wrote:
> On Wed, Jun 22, 2016 at 09:57:04PM +0000, Andrew Chapman via Digitalmars-d-learn wrote:

> Maybe try:
>
> 	if (buffer[] in myHash) { ... }
>
> ?  Does that make a difference?
>
>
> T

On Thursday, 23 June 2016 at 05:59:10 UTC, Andrew Chapman wrote:
> Perfect, thank you! :-) Works like a charm.
>
> On Wednesday, 22 June 2016 at 22:41:24 UTC, H. S. Teoh wrote:
>> On Wed, Jun 22, 2016 at 09:57:04PM +0000, Andrew Chapman via Digitalmars-d-learn wrote:
>
>> Maybe try:
>>
>> 	if (buffer[] in myHash) { ... }
>>
>> ?  Does that make a difference?
>>
>>
>> T

If the keys are fixed-length, you may want to consider using immutable(char[4]) as key. That way there is no GC allocation for the strings at all, the only allocations are done by the AA.

On 6/23/16 1:59 AM, Andrew Chapman wrote:
> Perfect, thank you! :-) Works like a charm.
>
> On Wednesday, 22 June 2016 at 22:41:24 UTC, H. S. Teoh wrote:
>> On Wed, Jun 22, 2016 at 09:57:04PM +0000, Andrew Chapman via
>> Digitalmars-d-learn wrote:
>
>> Maybe try:
>>
>>     if (buffer[] in myHash) { ... }
>>
>> ?  Does that make a difference?
>>
>>
>> T
>
>

This seems like a bug in the runtime, or am I missing something?

-Steve

On Thu, Jun 23, 2016 at 10:00:33AM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:
> On 6/23/16 1:59 AM, Andrew Chapman wrote:
> > Perfect, thank you! :-) Works like a charm.
> > 
> > On Wednesday, 22 June 2016 at 22:41:24 UTC, H. S. Teoh wrote:
> > > On Wed, Jun 22, 2016 at 09:57:04PM +0000, Andrew Chapman via Digitalmars-d-learn wrote:
> > 
> > > Maybe try:
> > > 
> > >     if (buffer[] in myHash) { ... }
> > > 
> > > ?  Does that make a difference?
> > > 
> > > 
> > > T
> > 
> > 
> 
> This seems like a bug in the runtime, or am I missing something?
[...]

It might well be a bug. I'll take a look.


T

-- 
Genius may have its limitations, but stupidity is not thus handicapped. -- Elbert Hubbard

On Thu, Jun 23, 2016 at 08:36:57AM -0700, H. S. Teoh via Digitalmars-d-learn wrote:
> On Thu, Jun 23, 2016 at 10:00:33AM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote:
[...]
> > > On Wednesday, 22 June 2016 at 22:41:24 UTC, H. S. Teoh wrote:
[...]
> > > > Maybe try:
> > > > 
> > > >     if (buffer[] in myHash) { ... }
> > > > 
> > > > ?  Does that make a difference?
> > > > 
> > > > 
> > > > T
> > > 
> > > 
> > 
> > This seems like a bug in the runtime, or am I missing something?
> [...]
> 
> It might well be a bug. I'll take a look.
[...]

It is indeed a bug. It's caused by the AA implementation receiving a
static char array when it's expecting a string (immutable(char)[])
according to the declared key type of the AA.

Filed a bug for this:

	https://issues.dlang.org/show_bug.cgi?id=16199

The compiler should either reject indexing an AA with a char[n], or it should automatically slice the char[n] so that the AA implementation gets the correct type in the key argument.


T

-- 
Those who don't understand D are condemned to reinvent it, poorly. -- Daniel N