On Sunday, 21 February 2016 at 18:08:30 UTC, Chris Wright wrote:
> That means never using an index.
How so ?

On Sun, 21 Feb 2016 18:12:27 +0000, Stefan Koch wrote:

> On Sunday, 21 February 2016 at 18:08:30 UTC, Chris Wright wrote:
>> That means never using an index.
> How so ?

Or rather, parsing a query to use an index becomes exponential unless you parse X86 instructions.

D doesn't let you access the contents of a lambda expression. (C# does.) So in order to evaluate the lambda, you have to invoke it.

You can't override <= separately from <. You only have an opCmp invocation and return value. So with a query filter like:

query.where!("age", (age) => age >= 18);

You know it's being compared to 10, but you don't know what comparison. So the first time, you have opCmp return -1, and the lambda yields false. Try again, have opCmp return 0, and the lambda yields true. Try a third time, have opCmp return 1, and it yields true.

That's three invocations for one variable. What about two variables?

query.where!("age,parental_supervision",
  (age, parentalSupervision) => age >= 18 || parentalSupervision == true);

Now I have to try three different values for the 'age' expression and two for the 'parentalSupervision' variable, and there's a combinatorial expansion.

That yields a giant truth table, and that's probably good enough. You look at which columns have an index and, of those, which have the best correlation with the lambda's result. That even lets you reduce the amount of memory you use -- you don't need to hold the whole truth table, just a series of counters. But you're still dealing with O(2**n) evaluations of the lambda.

On Sunday, 21 February 2016 at 18:32:29 UTC, Chris Wright wrote:
> Or rather, parsing a query to use an index becomes exponential unless you parse X86 instructions.
>
> D doesn't let you access the contents of a lambda expression. (C# does.) So in order to evaluate the lambda, you have to invoke it.
>
> You can't override <= separately from <. You only have an opCmp invocation and return value. So with a query filter like:
>
> query.where!("age", (age) => age >= 18);
>
> You know it's being compared to 10, but you don't know what comparison. So the first time, you have opCmp return -1, and the lambda yields false. Try again, have opCmp return 0, and the lambda yields true. Try a third time, have opCmp return 1, and it yields true.
>
> That's three invocations for one variable. What about two variables?
>
> query.where!("age,parental_supervision",
>   (age, parentalSupervision) => age >= 18 || parentalSupervision == true);
>
> Now I have to try three different values for the 'age' expression and two for the 'parentalSupervision' variable, and there's a combinatorial expansion.
>
> That yields a giant truth table, and that's probably good enough. You look at which columns have an index and, of those, which have the best correlation with the lambda's result. That even lets you reduce the amount of memory you use -- you don't need to hold the whole truth table, just a series of counters. But you're still dealing with O(2**n) evaluations of the lambda.

I don't parse anything.
the string "age" gets evaluated once so I know which columns your lambda is referring to
then the lambda gets evaluated exactly n times where n is the number of rows.
I your example above you should have written :
query.where!("age","parentalSupervision", (age,pa) => age.as!uint >= 18 || pa == true)

On Thursday, 18 February 2016 at 21:09:15 UTC, Stefan Koch wrote:
> Hello,
>
> It is not quite ready to post in Announce,
> but I would like to inform you that I am planing to open-source my native-sqlite database driver. (well currently it just reads them).
>
> However it works fully at CTFE.
> so if you want to you can extract sourceCode out of a sql-database and make a mixin with it :D
> given you string imported the database file.

Awesome! Really looking forward to it. It will probably change the way I store data forever. I currently do it in json files, which suck for simply storing data :D

On Sunday, 21 February 2016 at 19:21:31 UTC, Stefan Koch wrote:
> where n is the number of rows.

That means your doing a full table scan. When the table gets large enough, this gets problematic.

On Sun, 21 Feb 2016 19:21:31 +0000, Stefan Koch wrote:
> I don't parse anything.

Right, which prevents you from using indexes.

On Sunday, 21 February 2016 at 19:55:38 UTC, Any wrote:
> On Sunday, 21 February 2016 at 19:21:31 UTC, Stefan Koch wrote:
>> where n is the number of rows.
>
> That means your doing a full table scan. When the table gets large enough, this gets problematic.

When the table get's large enough you probably don't worry about CTFE'ing it anymore.

On Sunday, 21 February 2016 at 16:05:49 UTC, Stefan Koch wrote:
> just a small update on the API
> It could look something like this
>
> auto table = db.tables("personal");
> auto results = table.select("name","surname").where!("age","sex", (age, sex) => sex.as!Sex == Sex.female, age.as!uint < 40));
> auto names = results[0].as!string;
> auto surnames = results[0].as!string;

 Looks good! Although took me a little bit to notice the Lambda :P

 The result from names, surnames, once you access the entry it consumes it? That seems wrong.

 Hmmm is there a short example of how the left/right/natural joins would be done? I almost worry if there will be heavy uses for lambdas for that as well.

 Also the heavy template/explicit type use of _as_ makes me wonder if those can be avoided somehow. Then again SQLite might not care enough from what i recall so it may be unavoidable.

On Monday, 22 February 2016 at 02:52:41 UTC, Era Scarecrow wrote:
>  Looks good! Although took me a little bit to notice the Lambda :P
>
>  The result from names, surnames, once you access the entry it consumes it? That seems wrong.
>
>  Hmmm is there a short example of how the left/right/natural joins would be done? I almost worry if there will be heavy uses for lambdas for that as well.
>
>  Also the heavy template/explicit type use of _as_ makes me wonder if those can be avoided somehow. Then again SQLite might not care enough from what i recall so it may be unavoidable.

The Intention is not to be another SQLite but rather to provide a compatible D-style solution.
As for the conversions they could be avoided. However SQLite does not gurantee a type for a colum... And you cannot use costum types when you don't convert explicitly.

On Sun, 21 Feb 2016 21:15:01 +0000, Stefan Koch wrote:

> On Sunday, 21 February 2016 at 19:55:38 UTC, Any wrote:
>> On Sunday, 21 February 2016 at 19:21:31 UTC, Stefan Koch wrote:
>>> where n is the number of rows.
>>
>> That means your doing a full table scan. When the table gets large enough, this gets problematic.
> 
> When the table get's large enough you probably don't worry about CTFE'ing it anymore.

So you intend this to work *only* at compile time? Or would you supply a different API for querying at runtime?