Hello all,

I'm trying to make this work:

---
pure string[string] parseTwoColumnCsv(string inputCsv)
{
	import std.csv;
	import std.typecons;
	
	string[string] result;
	
	foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
		result[record[0]] = record[1];
	
	return result;
}

immutable string[string] dataLookup = parseTwoColumnCsv(import("some_data.csv"));

void main()
{
	import std.stdio;
	writefln("dataLookup = %s", dataLookup);
}
---

But (with DMD 2.073.1) I am getting this error:
main.d(14): Error: non-constant expression [['a', 'b', 'c']:['x', 'y', 'z'], ['1', '2', '3']:['4', '5', '6']]


The case with normal (non-associative) arrays seems to work fine, but is not what I needed:

---
pure string[][] parseTwoColumnCsv(string inputCsv)
{
	import std.csv;
	import std.typecons;
	
	string[][] result;
	
	foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
		result ~= [record[0],record[1]];
	
	return result;
}

immutable string[][] dataLookup = parseTwoColumnCsv(import("some_data.csv"));

void main()
{
	import std.stdio;
	writefln("dataLookup = %s", dataLookup);
}
---

Any idea how I can get this working?

I have tried a couple other things, like having the parseTwoColumnCsv function return an immutable string[string] and casting 'result' to that in the return statement, and I also tried just casting the value returned from parseTwoColumnCsv as it appears in the declaration of 'dataLookup'.  I tried std.exception's assumeUnique, but the function/template signature doesn't seem to support associative arrays.

Thanks.

On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote:
> Hello all,
>
> I'm trying to make this work:
>
> [...]

You cannot create AA's at ctfe and carry them over to runtime use.
You'd have to use a costum dictionary-type.
I think the vibe.d project has one you could use.

On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote:
> On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote:
>> Hello all,
>>
>> I'm trying to make this work:
>>
>> [...]
>
> You cannot create AA's at ctfe and carry them over to runtime use.
> You'd have to use a costum dictionary-type.
> I think the vibe.d project has one you could use.

I wondered if this might be the case.

Well, I won't worry about it then.  I'll have to find another way around.

Thanks a bunch!

On Tuesday, 21 February 2017 at 22:26:01 UTC, Chad Joan wrote:
> On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote:
>> On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote:
>>> Hello all,
>>>
>>> I'm trying to make this work:
>>>
>>> [...]
>>
>> You cannot create AA's at ctfe and carry them over to runtime use.
>> You'd have to use a costum dictionary-type.
>> I think the vibe.d project has one you could use.
>
> I wondered if this might be the case.
>
> Well, I won't worry about it then.  I'll have to find another way around.
>
> Thanks a bunch!

For the sake of the rest of the internet, here is what I'm probably going to stick with:

---
pure string[string] parseTwoColumnCsv(string inputCsv)
{
	import std.csv;
	import std.typecons;
	
	string[string] result;
	
	foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
		result[record[0]] = record[1];
	
	return result;
}

immutable string[string] dataLookup;

static this()
{
	dataLookup = parseTwoColumnCsv(import("some_data.csv"));
}

void main()
{
	import std.stdio;
	writefln("dataLookup = %s", dataLookup);
}
---

In this case the AA isn't actually coded into the executable; but at least the configuration from some_data.csv will be in the executable as a string.  The program will construct the AA at startup.  It's not as "cool", but it should get the job done.

HTH.

On Tue, Feb 21, 2017 at 10:34:57PM +0000, Chad Joan via Digitalmars-d-learn wrote:
> On Tuesday, 21 February 2017 at 22:26:01 UTC, Chad Joan wrote:
> > On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote:
> > > On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote:
> > > > Hello all,
> > > > 
> > > > I'm trying to make this work:
> > > > 
> > > > [...]
> > > 
> > > You cannot create AA's at ctfe and carry them over to runtime use.
> > > You'd have to use a costum dictionary-type.
> > > I think the vibe.d project has one you could use.
> > 
> > I wondered if this might be the case.
> > 
> > Well, I won't worry about it then.  I'll have to find another way around.
> > 
> > Thanks a bunch!
> 
> For the sake of the rest of the internet, here is what I'm probably going to stick with:
> 
> ---
> pure string[string] parseTwoColumnCsv(string inputCsv)
> {
> 	import std.csv;
> 	import std.typecons;
> 
> 	string[string] result;
> 
> 	foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
> 		result[record[0]] = record[1];
> 
> 	return result;
> }
> 
> immutable string[string] dataLookup;
> 
> static this()
> {
> 	dataLookup = parseTwoColumnCsv(import("some_data.csv"));
> }
> 
> void main()
> {
> 	import std.stdio;
> 	writefln("dataLookup = %s", dataLookup);
> }
> ---
> 
> In this case the AA isn't actually coded into the executable; but at least the configuration from some_data.csv will be in the executable as a string. The program will construct the AA at startup.  It's not as "cool", but it should get the job done.
[...]

Parsing strings at program startup is ugly and slow.  What about parsing at compile-time with CTFE into an array literal and transforming that into an AA at startup, which should be a lot faster?

	// Warning: untested code
	pure string[2][] parseTwoColumnCsv(string inputCsv)
	{
		import std.csv;
		import std.typecons;

		string[2][] result;
		foreach (record; csvReader!(Tuple!(string,string))(inputCsv)) {
			result ~= [record[0], record[1]];
		}
		return result;
	}

	immutable string[string] dataLookup;
	static this()
	{
		enum halfCookedData = parseTwoColumnCsv(import("some_data.csv"));
		foreach (p; halfCookedData) {
			dataLookup[p[0]] = p[1];
		}
	}


T

-- 
Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at it. -- Pete Bleackley

On Tue, Feb 21, 2017 at 11:50:02PM +0100, Daniel Kozak via Digitalmars-d-learn wrote:
> I have similar issue and I beleive I was able to workaround that somehow, but it is so many years now :(. Have you tried enum dataLookup instead of immutable string[string] dataLookup

That may appear to work, but I would *strongly* recommend against it, because what happens when you use enum with an AA, is that the AA will be created *at runtime*, *every single time* it is referenced.  (It is as if you copy-n-pasted the entire AA into the code each time you reference the enum.)  Which will introduce ridiculous amounts of redundant work at runtime and cause a big performance penalty.


T

-- 
What did the alien say to Schubert? "Take me to your lieder."

I have similar issue and I beleive I was able to workaround that somehow, but it is so many years now :(. Have you tried enum dataLookup instead of immutable string[string] dataLookup


Dne 21.2.2017 v 22:53 Chad Joan via Digitalmars-d-learn napsal(a):
> Hello all,
>
> I'm trying to make this work:
>
> ---
> pure string[string] parseTwoColumnCsv(string inputCsv)
> {
>     import std.csv;
>     import std.typecons;
>
>     string[string] result;
>
>     foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
>         result[record[0]] = record[1];
>
>     return result;
> }
>
> immutable string[string] dataLookup = parseTwoColumnCsv(import("some_data.csv"));
>
> void main()
> {
>     import std.stdio;
>     writefln("dataLookup = %s", dataLookup);
> }
> ---
>
> But (with DMD 2.073.1) I am getting this error:
> main.d(14): Error: non-constant expression [['a', 'b', 'c']:['x', 'y', 'z'], ['1', '2', '3']:['4', '5', '6']]
>
>
> The case with normal (non-associative) arrays seems to work fine, but is not what I needed:
>
> ---
> pure string[][] parseTwoColumnCsv(string inputCsv)
> {
>     import std.csv;
>     import std.typecons;
>
>     string[][] result;
>
>     foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
>         result ~= [record[0],record[1]];
>
>     return result;
> }
>
> immutable string[][] dataLookup = parseTwoColumnCsv(import("some_data.csv"));
>
> void main()
> {
>     import std.stdio;
>     writefln("dataLookup = %s", dataLookup);
> }
> ---
>
> Any idea how I can get this working?
>
> I have tried a couple other things, like having the parseTwoColumnCsv function return an immutable string[string] and casting 'result' to that in the return statement, and I also tried just casting the value returned from parseTwoColumnCsv as it appears in the declaration of 'dataLookup'.  I tried std.exception's assumeUnique, but the function/template signature doesn't seem to support associative arrays.
>
> Thanks.

On Tuesday, 21 February 2017 at 22:43:15 UTC, H. S. Teoh wrote:
>
> Parsing strings at program startup is ugly and slow.  What about parsing at compile-time with CTFE into an array literal and transforming that into an AA at startup, which should be a lot faster?
>
> 	// Warning: untested code
> 	pure string[2][] parseTwoColumnCsv(string inputCsv)
> 	{
> 		import std.csv;
> 		import std.typecons;
>
> 		string[2][] result;
> 		foreach (record; csvReader!(Tuple!(string,string))(inputCsv)) {
> 			result ~= [record[0], record[1]];
> 		}
> 		return result;
> 	}
>
> 	immutable string[string] dataLookup;
> 	static this()
> 	{
> 		enum halfCookedData = parseTwoColumnCsv(import("some_data.csv"));
> 		foreach (p; halfCookedData) {
> 			dataLookup[p[0]] = p[1];
> 		}
> 	}
>
>
> T

Hi,

That makes a lot of sense and it had crossed my mind.

In my case the data sets are super small, so I'm probably going to be lazy/productive and leave it the way it is.

Anyone else from the internet copying these examples: try to just do it H.S. Teoh's way from the start ;)

Thanks for the suggestion.
- Chad

On Tuesday, 21 February 2017 at 23:30:52 UTC, Chad Joan wrote:
> On Tuesday, 21 February 2017 at 22:43:15 UTC, H. S. Teoh wrote:
>>
>> Parsing strings at program startup is ugly and slow.  What about parsing at compile-time with CTFE into an array literal and transforming that into an AA at startup, which should be a lot faster?
>>
>> 	// Warning: untested code
>> 	pure string[2][] parseTwoColumnCsv(string inputCsv)
>> 	{
>> 		import std.csv;
>> 		import std.typecons;
>>
>> 		string[2][] result;
>> 		foreach (record; csvReader!(Tuple!(string,string))(inputCsv)) {
>> 			result ~= [record[0], record[1]];
>> 		}
>> 		return result;
>> 	}
>>
>> 	immutable string[string] dataLookup;
>> 	static this()
>> 	{
>> 		enum halfCookedData = parseTwoColumnCsv(import("some_data.csv"));
>> 		foreach (p; halfCookedData) {
>> 			dataLookup[p[0]] = p[1];
>> 		}
>> 	}
>>
>>
>> T
>
> Hi,
>
> That makes a lot of sense and it had crossed my mind.
>
> In my case the data sets are super small, so I'm probably going to be lazy/productive and leave it the way it is.
>
> Anyone else from the internet copying these examples: try to just do it H.S. Teoh's way from the start ;)
>
> Thanks for the suggestion.
> - Chad

OK I couldn't help it:

---
pure private string[][] parseTwoColumnCsv(string inputCsv)
{
	import std.csv;
	import std.typecons;
	
	string[][] result;
	
	foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
		result ~= [record[0],record[1]];
	
	return result;
}

pure private string[string] twoColumnArrayToAA(const string[][] arr)
{
	string[string] result;
	foreach ( pair; arr )
		result[pair[0]] = pair[1];
	return result;
}

pure private string[string] importTwoColumnCsv(string csvFile)()
{
	// Force the parse to happen at compile time.
	immutable string[][] tempArray = import(csvFile).parseTwoColumnCsv();
	
	// Convert the parsed array into a runtime Associative Array and return it.
	return tempArray.twoColumnArrayToAA();
}

immutable string[string] dataLookup;
immutable string[string] dataLookup1;
immutable string[string] dataLookup2;

static this()
{
	import std.range;
	dataLookup1 = importTwoColumnCsv!"some_data1.csv";
	dataLookup2 = importTwoColumnCsv!"some_data2.csv";
	foreach( pair; chain(dataLookup1.byKeyValue, dataLookup2.byKeyValue) )
		dataLookup[pair.key] = pair.value;
}

void main()
{
	import std.stdio;
	writefln("dataLookup = %s", dataLookup);
}
---

This example also shows joining associative arrays.

On Wed, Feb 22, 2017 at 12:38:47AM +0000, Chad Joan via Digitalmars-d-learn wrote: [...]

Hmm. It's actually not necessary to manually write a foreach loop to
convert an array to an AA. We could, instead, change parseTwoColumnCsv()
to return an array of std.typecons.Tuple instead (which, incidentally,
means you can elide that foreach loop too!), then there's this handy
function std.array.assocArray that will do the transcription for us,
as follows.

(In fact, you can merge AA's without ever needing to write `foreach` at
all! See below.)

---
pure private auto parseTwoColumnCsv(string inputCsv)
{
	import std.csv;
	import std.typecons;
	return csvReader!(Tuple!(string,string))(inputCsv);
}

immutable string[string] dataLookup;
immutable string[string] dataLookup1;
immutable string[string] dataLookup2;

static this()
{
	import std.array : assocArray;
	import std.range : chain;

	// Force CTFE
	immutable tuples1 = parseTwoColumnCsv("some_data1.csv");
	immutable tuples2 = parseTwoColumnCsv("some_data2.csv");

	dataLookup1 = tuples1.assocArray;	// Bam! :-P
	dataLookup2 = tuples2.assocArray;	// Bam! :-P

	// Bam, bam! - merge AA's in a single step!
	dataLookup3 = chain(tuples1, tuples2).assocArray;
}
---


T

-- 
You have to expect the unexpected. -- RL