In the talk Andrei seems to mentions that D's associative arrays are lacking in performance somehow. I'm very new to D, but it's not obvious to me what the shortcoming is. I assume it's that for some reason it's hard to specialize associative arrays to specfic types to give increased performance in specfic cases, but I'm unclear why that would be difficult. Could someone please elaborate?

On 6/30/2013 2:50 AM, Joakim wrote:
> I wondered if you have any opinion on such code reuse, if someone takes your
> code and closes it, even if you wouldn't try to block it because you have
> already released it under a permissive license.

No, I don't have an opinion on it, other than that I'd rather they didn't try to create an incompatible language and still call it "D".

On Sunday, 30 June 2013 at 19:24:54 UTC, Walter Bright wrote:
> On 6/30/2013 2:50 AM, Joakim wrote:
>> I wondered if you have any opinion on such code reuse, if someone takes your
>> code and closes it, even if you wouldn't try to block it because you have
>> already released it under a permissive license.
>
> No, I don't have an opinion on it, other than that I'd rather they didn't try to create an incompatible language and still call it "D".
OK, glad to hear that you wouldn't be against it.  You'd be surprised how many who use permissive licenses still go nuts when you propose to do exactly what the license allows, ie close up parts of the source.

Since you have been so gracious to use such permissive licenses for almost all of D, I'm sure someone will try the closed/paid experiment someday and see if which of us is right. :)

On Sunday, June 30, 2013 21:05:41 CJS wrote:
> In the talk Andrei seems to mentions that D's associative arrays are lacking in performance somehow. I'm very new to D, but it's not obvious to me what the shortcoming is. I assume it's that for some reason it's hard to specialize associative arrays to specfic types to give increased performance in specfic cases, but I'm unclear why that would be difficult. Could someone please elaborate?

There's one implementation, and you can't swap it out, whereas different use cases may perform better with different implementations. On top of that, the current implementation is rather buggy and fragile, but that's an implementation issue rather than an inherent one.

- Jonathan M Davis

On Sun, 30 Jun 2013 15:51:32 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Sunday, June 30, 2013 21:05:41 CJS wrote:
>> In the talk Andrei seems to mentions that D's associative arrays
>> are lacking in performance somehow. I'm very new to D, but it's
>> not obvious to me what the shortcoming is. I assume it's that for
>> some reason it's hard to specialize associative arrays to specfic
>> types to give increased performance in specfic cases, but I'm
>> unclear why that would be difficult. Could someone please
>> elaborate?
>
> There's one implementation, and you can't swap it out, whereas different use
> cases may perform better with different implementations. On top of that, the
> current implementation is rather buggy and fragile, but that's an
> implementation issue rather than an inherent one.

No, the main issue is the current one is runtime-only, and so simple function calls such as toHash and opCmp cannot be inlined.

You absolutely can change implementations (Walter did a few years ago from tree-based collision resolution to linked-list based).  What you can't do is switch to a fully generated AA, or change the compiler-expected API.

-Steve

On Sunday, June 30, 2013 19:20:47 Steven Schveighoffer wrote:
> On Sun, 30 Jun 2013 15:51:32 -0400, Jonathan M Davis <jmdavisProg@gmx.com>
> 
> wrote:
> > On Sunday, June 30, 2013 21:05:41 CJS wrote:
> >> In the talk Andrei seems to mentions that D's associative arrays are lacking in performance somehow. I'm very new to D, but it's not obvious to me what the shortcoming is. I assume it's that for some reason it's hard to specialize associative arrays to specfic types to give increased performance in specfic cases, but I'm unclear why that would be difficult. Could someone please elaborate?
> > 
> > There's one implementation, and you can't swap it out, whereas different
> > use
> > cases may perform better with different implementations. On top of that,
> > the
> > current implementation is rather buggy and fragile, but that's an
> > implementation issue rather than an inherent one.
> 
> No, the main issue is the current one is runtime-only, and so simple function calls such as toHash and opCmp cannot be inlined.

Yeah. That's a big problem. We really need to templatize all that - though the current implementation is enough of a mess to make that difficult.

> You absolutely can change implementations (Walter did a few years ago from tree-based collision resolution to linked-list based).  What you can't do is switch to a fully generated AA, or change the compiler-expected API.

Okay. I didn't know that. But I think that they key issue with swapping out the implementation is not whether you can swap out the implementation for your whole program but rather being able to choose different implementations for different parts of your program. If you really care about your containers enough to worry about optimizing them for your particular use cases, then unless you only have one use case within your program, there's a good chance that you're going to want different implementations for different parts of your program. With library containers, that's as simple as swapping which one you use. With the bulit-in stuff like AAs, you can't do that. You only get one (even if you can make it different across programs).

Now, if you just use library types, you don't have that problem, so in the long run, folks who really want to optimize their containers will probably do that. And if they _really_ want to optimize their containers, they're probably writing them themselves anyway.

Regardless, while having a built-in AA simplifies the common case, it _is_ more limiting, and if you really care about how your AAs function, you're going to have to use a library solution (even if the bult-in AAs have a solid implementation).

- Jonathan M Davis

On Sun, 30 Jun 2013 21:43:53 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> But I think that they key issue with swapping out
> the implementation is not whether you can swap out the implementation for your
> whole program but rather being able to choose different implementations for
> different parts of your program.

This would never happen.  AAs are only ever going to be one implementation.  If you want to use another map type, you will have to use a struct/class.  I suppose AA's could simply be polymorphic, but I don't see the benefit.

-Steve

On Sun, 30 Jun 2013 21:43:53 -0400, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Sunday, June 30, 2013 19:20:47 Steven Schveighoffer wrote:

>> No, the main issue is the current one is runtime-only, and so simple
>> function calls such as toHash and opCmp cannot be inlined.
>
> Yeah. That's a big problem. We really need to templatize all that - though the
> current implementation is enough of a mess to make that difficult.

The current implementation suffers from two problems:

1. The compiler doesn't treat T[U] as a direct instantiation of AssocArray!(T, U) in all cases.
2. The compiler has bugs in terms of pure/@safe/ctfe/etc that it "overlooks" when using built-in AAs.  I think those who have tried to make a complete library-replacement for AAs have found this out.

The best path to resolution I think is:

1. make a complete replacement for AAs that can be instantiated and operate without mapping to the AA syntax.  It may not build, but it should be written and bugs against the compiler filed.
2. Fix all bugs to make 1. compile/possible
3. Switch compiler to use type in 1. whenever AA's are used.

I believe some have made a very good attempt to do 1 (H.S. Teoh I think? maybe someone else)

-Steve

On Sunday, June 30, 2013 21:54:08 Steven Schveighoffer wrote:
> On Sun, 30 Jun 2013 21:43:53 -0400, Jonathan M Davis <jmdavisProg@gmx.com>
> 
> wrote:
> > But I think that they key issue with swapping out
> > the implementation is not whether you can swap out the implementation
> > for your
> > whole program but rather being able to choose different implementations
> > for
> > different parts of your program.
> 
> This would never happen.  AAs are only ever going to be one implementation.  If you want to use another map type, you will have to use a struct/class.  I suppose AA's could simply be polymorphic, but I don't see the benefit.

I know. My point was that that's an inherent problem with built-in AAs that can't be overcome (regardless of how well they're implemented). If you want that level of control, you _have_ to use a library solution.

- Jonathan M Davis

On Sunday, June 30, 2013 21:59:45 Steven Schveighoffer wrote:
> On Sun, 30 Jun 2013 21:43:53 -0400, Jonathan M Davis <jmdavisProg@gmx.com>
> 
> wrote:
> > On Sunday, June 30, 2013 19:20:47 Steven Schveighoffer wrote:
> >> No, the main issue is the current one is runtime-only, and so simple function calls such as toHash and opCmp cannot be inlined.
> > 
> > Yeah. That's a big problem. We really need to templatize all that -
> > though the
> > current implementation is enough of a mess to make that difficult.
> 
> The current implementation suffers from two problems:
> 
> 1. The compiler doesn't treat T[U] as a direct instantiation of
> AssocArray!(T, U) in all cases.
> 2. The compiler has bugs in terms of pure/@safe/ctfe/etc that it
> "overlooks" when using built-in AAs.  I think those who have tried to make
> a complete library-replacement for AAs have found this out.
> 
> The best path to resolution I think is:
> 
> 1. make a complete replacement for AAs that can be instantiated and
> operate without mapping to the AA syntax.  It may not build, but it should
> be written and bugs against the compiler filed.
> 2. Fix all bugs to make 1. compile/possible
> 3. Switch compiler to use type in 1. whenever AA's are used.
> 
> I believe some have made a very good attempt to do 1 (H.S. Teoh I think?
> maybe someone else)

Yeah. He was working on it and seems to have pretty much decided that the job is too big for one man (and/or that he doesn't have enough time). I believe that there was a post on D.Learn a few months back where you pointed to where he had his changes thus far (so that others could look at it and potentially continue his work), but AFAIK, he's given up on the whole thing for now.

- Jonathan M Davis