View mode: basic / threaded / horizontal-split · Log in · Help
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On 3/15/12 11:02 AM, Don Clugston wrote:
> This is good, and very, very important. Do *not* make any attempt at
> compiler integration until it is *completely* ready.
>
> This includes AA literals. They need to be accepted somehow. The
> compiler will give you syntax sugar and *nothing* more.
> One possibility might be to accept a pair of array literals,
> representing keys and values.
> Possibly it should call a CTFE function to convert them into some other
> form?

Offhand, I think this rewrite should be sufficient:

[e11:e12, e21:e22]

--->

.object.associativeArrayLiteral(e11, e12, e21, e22)

Then type manipulation and template constraints can take care of the 
rest. Am I missing something?


Andrei
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Thu, 15 Mar 2012 12:05:46 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail@erdani.org> wrote:

> On 3/15/12 11:02 AM, Don Clugston wrote:
>> This is good, and very, very important. Do *not* make any attempt at
>> compiler integration until it is *completely* ready.
>>
>> This includes AA literals. They need to be accepted somehow. The
>> compiler will give you syntax sugar and *nothing* more.
>> One possibility might be to accept a pair of array literals,
>> representing keys and values.
>> Possibly it should call a CTFE function to convert them into some other
>> form?
>
> Offhand, I think this rewrite should be sufficient:
>
> [e11:e12, e21:e22]
>
> --->
>
> .object.associativeArrayLiteral(e11, e12, e21, e22)
>
> Then type manipulation and template constraints can take care of the  
> rest. Am I missing something?

Wouldn't this require a new template instantiation for each sized AA per  
key/value type?  Or were you planning to use varargs?

So long as the data that is actually passed to the AA is on the stack (and  
simply a slice is passed), I like Don's idea better.

-Steve
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Thu, Mar 15, 2012 at 05:02:06PM +0100, Don Clugston wrote:
> On 15/03/12 00:16, H. S. Teoh wrote:
[...]
> This is good, and very, very important. Do *not* make any attempt at
> compiler integration until it is *completely* ready.
> 
> This includes AA literals. They need to be accepted somehow. The
> compiler will give you syntax sugar and *nothing* more.

How does the compiler currently work with AA literals? I see two
functions in aaA.d for constructing AA's from literals. Which one is the
one actually being used?


> One possibility might be to accept a pair of array literals,
> representing keys and values.

OK. This shouldn't be hard to do. I'll take a stab at it.


> Possibly it should call a CTFE function to convert them into some
> other form?

This is one major area that I forgot to mention, and that is, making AA
literals work at compile-time. Currently things like this don't work:

	enum myAA = ["abc":123, "def":456];

I'd like to make that work. That would require compile-time construction
of the AA and storing it in some form that the compiler can put into the
object file. I'm thinking of something along the lines of using mixins
to generate explicit instances of Slot structs, so the above would
translate to something like:

	Impl __myAA_literal_impl = Impl(__myAA_literal_slots[0..$], 2);
	Slot[4] __myAA_literal_slots = [
		// The exact order in here will depend on precise hash
		// values, which will need to be somehow computed by
		// CTFE. Is that even remotely possible right now??
		null,
		&__myAA_literal_slot1,
		null,
		&__myAA_literal_slot2
	];
	Slot __myAA_literal_slot1 = Slot(null, /*hash value*/, "abc", 123);
	Slot __myAA_literal_slot2 = Slot(null, /*hash value*/, "def", 456);
	enum myAA = AssociativeArray!(string,int)(&__myAA_literal_impl);

Would something like this be workable? Is it even possible to compute
the necessary hashes at compile-time?


T

-- 
Never step over a puddle, always step around it. Chances are that whatever made it is still dripping.
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On 3/15/12 12:12 PM, Steven Schveighoffer wrote:
> On Thu, 15 Mar 2012 12:05:46 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>
>> On 3/15/12 11:02 AM, Don Clugston wrote:
>>> This is good, and very, very important. Do *not* make any attempt at
>>> compiler integration until it is *completely* ready.
>>>
>>> This includes AA literals. They need to be accepted somehow. The
>>> compiler will give you syntax sugar and *nothing* more.
>>> One possibility might be to accept a pair of array literals,
>>> representing keys and values.
>>> Possibly it should call a CTFE function to convert them into some other
>>> form?
>>
>> Offhand, I think this rewrite should be sufficient:
>>
>> [e11:e12, e21:e22]
>>
>> --->
>>
>> .object.associativeArrayLiteral(e11, e12, e21, e22)
>>
>> Then type manipulation and template constraints can take care of the
>> rest. Am I missing something?
>
> Wouldn't this require a new template instantiation for each sized AA per
> key/value type? Or were you planning to use varargs?

Template function takes over, does whatever is necessary, such as 
possibly conversion to varargs.

> So long as the data that is actually passed to the AA is on the stack
> (and simply a slice is passed), I like Don's idea better.

What would that look like?



Andrei
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Thu, 15 Mar 2012 13:24:24 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail@erdani.org> wrote:

> On 3/15/12 12:12 PM, Steven Schveighoffer wrote:
>> On Thu, 15 Mar 2012 12:05:46 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> On 3/15/12 11:02 AM, Don Clugston wrote:
>>>> This is good, and very, very important. Do *not* make any attempt at
>>>> compiler integration until it is *completely* ready.
>>>>
>>>> This includes AA literals. They need to be accepted somehow. The
>>>> compiler will give you syntax sugar and *nothing* more.
>>>> One possibility might be to accept a pair of array literals,
>>>> representing keys and values.
>>>> Possibly it should call a CTFE function to convert them into some  
>>>> other
>>>> form?
>>>
>>> Offhand, I think this rewrite should be sufficient:
>>>
>>> [e11:e12, e21:e22]
>>>
>>> --->
>>>
>>> .object.associativeArrayLiteral(e11, e12, e21, e22)
>>>
>>> Then type manipulation and template constraints can take care of the
>>> rest. Am I missing something?
>>
>> Wouldn't this require a new template instantiation for each sized AA per
>> key/value type? Or were you planning to use varargs?
>
> Template function takes over, does whatever is necessary, such as  
> possibly conversion to varargs.

Right, but with a template:

[1:1]
[1:1, 2:2]

become two separate template instantiations, but with something that just  
takes two arrays, it's only one template, no matter how many elements you  
are initializing with.

>> So long as the data that is actually passed to the AA is on the stack
>> (and simply a slice is passed), I like Don's idea better.
>
> What would that look like?

auto aa = [1:1];

 becomes:

int[1] __k = [1]; // obviously, no heap allocation should happen here,  
that needs fixing in the compiler
int[1] __v = [1];
auto aa = AssociativeArray!(int, int)(__k, __v); // same instantiation, no  
matter how many elements are in literal.

-Steve
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Thu, 15 Mar 2012 13:39:19 -0400, Steven Schveighoffer  
<schveiguy@yahoo.com> wrote:

> On Thu, 15 Mar 2012 13:24:24 -0400, Andrei Alexandrescu  
> <SeeWebsiteForEmail@erdani.org> wrote:
>> What would that look like?
>
> auto aa = [1:1];
>
>   becomes:
>
> int[1] __k = [1]; // obviously, no heap allocation should happen here,  
> that needs fixing in the compiler
> int[1] __v = [1];
> auto aa = AssociativeArray!(int, int)(__k, __v); // same instantiation,  
> no matter how many elements are in literal.

Sorry, that should really be:

auto aa = AssociativeArray!(int, int)(__k[], __v[]);

-Steve
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On 3/15/12 12:39 PM, Steven Schveighoffer wrote:
> On Thu, 15 Mar 2012 13:24:24 -0400, Andrei Alexandrescu
> <SeeWebsiteForEmail@erdani.org> wrote:
>> Template function takes over, does whatever is necessary, such as
>> possibly conversion to varargs.
>
> Right, but with a template:
>
> [1:1]
> [1:1, 2:2]
>
> become two separate template instantiations, but with something that
> just takes two arrays, it's only one template, no matter how many
> elements you are initializing with.

I guess I should ask this then: why is instantiating a template function 
a problem?

>>> So long as the data that is actually passed to the AA is on the stack
>>> (and simply a slice is passed), I like Don's idea better.
>>
>> What would that look like?
>
> auto aa = [1:1];
>
> becomes:
>
> int[1] __k = [1]; // obviously, no heap allocation should happen here,
> that needs fixing in the compiler
> int[1] __v = [1];
> auto aa = AssociativeArray!(int, int)(__k, __v); // same instantiation,
> no matter how many elements are in literal.

That should work, too. Use braces to not require a fix:

int[1] __k = {1};
int[1] __v = {1};
auto aa = AssociativeArray!(int, int)(__k, __v);

Off the top of my head it changes the order of evaluation in the general 
case (or complicates code generation if left-to-right preservation is 
needed). Also the constructor needs to be @trusted because references to 
static arrays can't escape in safe code. All workable matters, but 
generally I prefer migrating cleverness from code generation into 
library code.


Andrei
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Thu, Mar 15, 2012 at 10:22:06AM -0700, H. S. Teoh wrote:
[...]
> This is one major area that I forgot to mention, and that is, making AA
> literals work at compile-time. Currently things like this don't work:
> 
> 	enum myAA = ["abc":123, "def":456];
> 
> I'd like to make that work. That would require compile-time construction
> of the AA and storing it in some form that the compiler can put into the
> object file. I'm thinking of something along the lines of using mixins
> to generate explicit instances of Slot structs, so the above would
> translate to something like:
[...]

Hmm. I just did a little test to see how feasible this is, and I create
the AA Slots and array of pointers to slots fine, but hashOf() doesn't
work in CTFE:

/usr/src/d/druntime/src/rt/util/hash.d(38): Error: reinterpreting cast from const(ubyte)* to ushort* is not supported in CTFE
/usr/src/d/druntime/src/rt/util/hash.d(72):        called from here: get16bits(data)

Should I submit a pull request for a __ctfe variant of hashOf so that it
can be used for this purpose? Any gotchas I should be aware of?


T

-- 
"I'm running Windows '98."
"Yes."
"My computer isn't working now."
"Yes, you already said that."
-- User-Friendly
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Thu, 15 Mar 2012 14:11:11 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail@erdani.org> wrote:

> On 3/15/12 12:39 PM, Steven Schveighoffer wrote:
>> On Thu, 15 Mar 2012 13:24:24 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>> Template function takes over, does whatever is necessary, such as
>>> possibly conversion to varargs.
>>
>> Right, but with a template:
>>
>> [1:1]
>> [1:1, 2:2]
>>
>> become two separate template instantiations, but with something that
>> just takes two arrays, it's only one template, no matter how many
>> elements you are initializing with.
>
> I guess I should ask this then: why is instantiating a template function  
> a problem?

1. non-inlined functions (dmd doesn't inline by default) result in extra  
calls.
2. I think even if it's inlined, dmd generates a function and sticks it in  
the object file, which will never be called. (bloat).  I'm not sure if  
anything else is put into the binary, or if the function is optimized out  
on linking.

> Off the top of my head it changes the order of evaluation in the general  
> case (or complicates code generation if left-to-right preservation is  
> needed). Also the constructor needs to be @trusted because references to  
> static arrays can't escape in safe code. All workable matters, but  
> generally I prefer migrating cleverness from code generation into  
> library code.

All good points.  One thing to be sure, given the fact that this is one  
hook implemented in one location, it should be easy to fix later if we use  
a bad approach.

Perhaps the best way to address my concerns are to fix how unused  
templates are included in the binary.  I could also be wrong about that.

-Steve
March 15, 2012
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On 15.03.2012 21:22, H. S. Teoh wrote:
> On Thu, Mar 15, 2012 at 05:02:06PM +0100, Don Clugston wrote:
>> On 15/03/12 00:16, H. S. Teoh wrote:
> [...]
>> This is good, and very, very important. Do *not* make any attempt at
>> compiler integration until it is *completely* ready.
>>
>> This includes AA literals. They need to be accepted somehow. The
>> compiler will give you syntax sugar and *nothing* more.
>
> How does the compiler currently work with AA literals? I see two
> functions in aaA.d for constructing AA's from literals. Which one is the
> one actually being used?
>
>
>> One possibility might be to accept a pair of array literals,
>> representing keys and values.
>
> OK. This shouldn't be hard to do. I'll take a stab at it.
>
>
>> Possibly it should call a CTFE function to convert them into some
>> other form?
>
> This is one major area that I forgot to mention, and that is, making AA
> literals work at compile-time. Currently things like this don't work:
>
> 	enum myAA = ["abc":123, "def":456];
>
> I'd like to make that work. That would require compile-time construction
> of the AA and storing it in some form that the compiler can put into the
> object file.

What's wrong with AA structure itself, it's link-pointer based?
At any rate, I stored complex structs as immutable, though links were 
index-based.

I'm thinking of something along the lines of using mixins
> to generate explicit instances of Slot structs, so the above would
> translate to something like:
>
> 	Impl __myAA_literal_impl = Impl(__myAA_literal_slots[0..$], 2);
> 	Slot[4] __myAA_literal_slots = [
> 		// The exact order in here will depend on precise hash
> 		// values, which will need to be somehow computed by
> 		// CTFE. Is that even remotely possible right now??
> 		null,
> 		&__myAA_literal_slot1,
> 		null,
> 		&__myAA_literal_slot2
> 	];
> 	Slot __myAA_literal_slot1 = Slot(null, /*hash value*/, "abc", 123);
> 	Slot __myAA_literal_slot2 = Slot(null, /*hash value*/, "def", 456);
> 	enum myAA = AssociativeArray!(string,int)(&__myAA_literal_impl);
>
> Would something like this be workable? Is it even possible to compute
> the necessary hashes at compile-time?
>
>
> T
>


-- 
Dmitry Olshansky
1 2 3 4
Top | Discussion index | About this forum | D home