May 25, 2016
On Wednesday, 25 May 2016 at 03:48:48 UTC, Stefan Koch wrote:
> On Saturday, 27 February 2016 at 16:08:21 UTC, Stefan Koch wrote:
>> Hello,
>>
>> I am happy to announce the official alpha version of sqlite-d!
>>
>> sqlite-d is a reader for the SQLite File Format 3.
>>
>> In the future I will implement a SQL-like API on top of it.
>>
>> Top-notch performance is one of the explicit goals of this project!
>>
>> please note that currently only the ctfe branch is populated.
>>
>> I welcome contributions!
>>
>> Repo-URL : https://github.com/UplinkCoder/sqlite-d
>
> Well not much has changed since I made this announcement.
> By fixing a really jarring bug I had a slight performance regression.
> But this is still the fastest SQLite reader I know of.
>
> This project is currently on the back burner.
> However in the next days there will be another significant performance improvement :)

Even faster then official version?

What about futures, would it possible to make it's 100% compatibility with C version?
May 25, 2016
On Wednesday, 25 May 2016 at 06:52:22 UTC, Suliman wrote:
>
> Even faster then official version?
>
> What about futures, would it possible to make it's 100% compatibility with C version?

Not really.
The reason why it is faster is because there is no indirection in working with the data.
If I had full SQL support like there is in the C version, and you take that route rather then the direct access.
It will be as slow as the C version, and probably slower.
May 25, 2016
On Wednesday, 25 May 2016 at 03:48:48 UTC, Stefan Koch wrote:
>
> Well not much has changed since I made this announcement.
> By fixing a really jarring bug I had a slight performance regression.
> But this is still the fastest SQLite reader I know of.
>
> This project is currently on the back burner.
> However in the next days there will be another significant performance improvement :)

You might consider writing up some kind of comparison.

May 25, 2016
On Wednesday, 25 May 2016 at 03:48:48 UTC, Stefan Koch wrote:
>
> This project is currently on the back burner.
> However in the next days there will be another significant performance improvement :)

A 20% performance improvement has landed in master!
It is possible that there are more places in the code-base where unrolling can buy you performance.

Never forget.
PERFMATTERS!


May 26, 2016
On Wednesday, 25 May 2016 at 14:10:41 UTC, Stefan Koch wrote:
> On Wednesday, 25 May 2016 at 06:52:22 UTC, Suliman wrote:
>>
>> Even faster then official version?
>>
>> What about futures, would it possible to make it's 100% compatibility with C version?
>
> Not really.
> The reason why it is faster is because there is no indirection in working with the data.
> If I had full SQL support like there is in the C version, and you take that route rather then the direct access.
> It will be as slow as the C version, and probably slower.

Could you explain more details? What do you mean by indirection work with data?
May 26, 2016
On Thursday, 26 May 2016 at 14:45:58 UTC, Suliman wrote:
>
> Could you explain more details? What do you mean by indirection work with data?

Sure, I can explain.
So, all that sqlite-d does is reading the sqlite-db files.
However the proper sqlite does much more:
It implements a whole ByteCode-Compiler and Interpreter to be able to precompile and execute SQL-statements.

Thus even for simple querys you have a significant overhead.
If you want to use sqlite-d you basically write all your queries as D code which can be directly executed, rather then having to be compiled from SQL to ByteCode.

May 26, 2016
On Thursday, 26 May 2016 at 19:11:50 UTC, Stefan Koch wrote:
> On Thursday, 26 May 2016 at 14:45:58 UTC, Suliman wrote:
>>
>> Could you explain more details? What do you mean by indirection work with data?
>
> Sure, I can explain.
> So, all that sqlite-d does is reading the sqlite-db files.
> However the proper sqlite does much more:
> It implements a whole ByteCode-Compiler and Interpreter to be able to precompile and execute SQL-statements.
>
> Thus even for simple querys you have a significant overhead.
> If you want to use sqlite-d you basically write all your queries as D code which can be directly executed, rather then having to be compiled from SQL to ByteCode.

Oh! Look like for all time I misunderstood the purpose of your project. Do you mean that your tool is created not for being SQL compatible driver, but make possible to use D code for iteration with DB?

And does it's mean that is not possible to write SQL-query with this driver?
May 26, 2016
On Thursday, 26 May 2016 at 19:35:22 UTC, Suliman wrote:

> Oh! Look like for all time I misunderstood the purpose of your project. Do you mean that your tool is created not for being SQL compatible driver, but make possible to use D code for iteration with DB?

Yes and no, currently it does only provide primitives to iterate over sqlite-tables.
And originally it was only intended for this purpose.
However per-compiling SQL at compile-time is a great way for D's features to shine.

> And does it's mean that is not possible to write SQL-query with this driver?

I want to add the capability to precompile and execute SQL, however that is still a long way away.

May 27, 2016
On Saturday, 27 February 2016 at 16:08:21 UTC, Stefan Koch wrote:
> Hello,
>
> I am happy to announce the official alpha version of sqlite-d!
>
> sqlite-d is a reader for the SQLite File Format 3.

On Thursday, 26 May 2016 at 19:11:50 UTC, Stefan Koch wrote:
> So, all that sqlite-d does is reading the sqlite-db files.
> However the proper sqlite does much more:
> It implements a whole ByteCode-Compiler and Interpreter to be able to precompile and execute SQL-statements.
>
> Thus even for simple querys you have a significant overhead.
> If you want to use sqlite-d you basically write all your queries as D code which can be directly executed, rather then having to be compiled from SQL to ByteCode.

How does that play with transactions and ACID in general ?

For example, I have one thread with traditional (slow) SQLite client, which seldom updates data. And another thread which reads data with sqlite-d. Will not program crash or read trash/inconsistent data ?
May 27, 2016
On Friday, 27 May 2016 at 05:03:43 UTC, xenon325 wrote:
>
> For example, I have one thread with traditional (slow) SQLite client, which seldom updates data. And another thread which reads data with sqlite-d. Will not program crash or read trash/inconsistent data ?

sqlite-d provides no safety whatsoever.
You will the database read in whatever state the operation system see the underlying file.

It is unlikely that it will crash.
But of course it may read stale or incomplete data.

There is a locking mechanism in SQLite-proper.
However SQLite-D currently makes no attempt in looking for the lock-page.

As a side note:
I would like to make clear that SQLite is very fast for what it does.
It is among the fastest of SQL-databases, and has a high quality of implementation.