On Monday, 30 September 2013 at 16:27:16 UTC, Brad Anderson wrote:
> On Saturday, 28 September 2013 at 16:39:27 UTC, simendsjo wrote:
>> I've been working on a more or less complete rewrite of the mysql-native module.
>
> I don't have any specific comments.  Just want to say that Steve Teale originally wrote the module as part of his std.database efforts.  I think it would be nice if any redesign done would aim for eventual phobos inclusion.

I don't agree that this library should aim for any inclusion in phobos. I plan on exposing the MySQL API as closely as possible and a couple of higher level convenience functions. The std.database would have to be database agnostic, and should be a library interfacing to mysql-native, postgres etc.

We should contact the author of ddbc for input: https://github.com/buggins/ddbc

On Monday, 30 September 2013 at 17:19:44 UTC, Jacob Carlborg wrote:
> On 2013-09-30 16:08, Dicebot wrote:
>
>> I don't think there is a common consensus here. Don't even think it is
>> possible. Doom of proper multi-paradigm language :)
>
> At least a few point can be made, like:
>
> * For reference types use classes, or use structs with reference counting
> * For sub typing use classes, or use structs with alias this
> * For polymorphic types use classes
> * For value types use structs
> * For deterministic destruction use structs

That's a good point to start. The points above could be fleshed out in a tutorial or article. Another good section would be the pitfalls involved in using structs. I think it was H. S. Teoh who mentioned a subtle bug due to a missing ref or something like that.

On 2013-09-30 20:14, simendsjo wrote:

> I don't agree that this library should aim for any inclusion in phobos.
> I plan on exposing the MySQL API as closely as possible and a couple of
> higher level convenience functions. The std.database would have to be
> database agnostic, and should be a library interfacing to mysql-native,
> postgres etc.

Yes, std.database should be database agnostic. But that doesn't mean we can't include one or a couple of database adapters. For example, Ruby on Rails contains a couple of database adapters, but not for all databases. It has no adapter for SQL Server, for that you need a separate gem/plugin.

An adapter for MySQL would probably be good to include. Even better, an adapter for SQLite. Perhaps we can include the whole SQLite source code, since it's public domain.

-- 
/Jacob Carlborg

On Monday, 30 September 2013 at 18:14:13 UTC, simendsjo wrote:
> On Monday, 30 September 2013 at 16:27:16 UTC, Brad Anderson wrote:
>> On Saturday, 28 September 2013 at 16:39:27 UTC, simendsjo wrote:
>>> I've been working on a more or less complete rewrite of the mysql-native module.
>>
>> I don't have any specific comments.  Just want to say that Steve Teale originally wrote the module as part of his std.database efforts.  I think it would be nice if any redesign done would aim for eventual phobos inclusion.
>
> I don't agree that this library should aim for any inclusion in phobos. I plan on exposing the MySQL API as closely as possible and a couple of higher level convenience functions. The std.database would have to be database agnostic, and should be a library interfacing to mysql-native, postgres etc.
>
> We should contact the author of ddbc for input: https://github.com/buggins/ddbc

No, you are wrong. We need both low-level and high-level API,
Low-level API like mysql-native driver should present a specific database as much as possible and provide access to all database features.
High-level API like ORM HibernateD must provide common API for all databases, but it can't present all database specific features due database differences.
So, I can use low-level API if I need access to all database specific features. I can use high-level API if I need common API for all databases.

On Tuesday, 1 October 2013 at 08:05:29 UTC, ilya-stromberg wrote:
> On Monday, 30 September 2013 at 18:14:13 UTC, simendsjo wrote:
>> On Monday, 30 September 2013 at 16:27:16 UTC, Brad Anderson wrote:
>>> On Saturday, 28 September 2013 at 16:39:27 UTC, simendsjo wrote:
>>>> I've been working on a more or less complete rewrite of the mysql-native module.
>>>
>>> I don't have any specific comments.  Just want to say that Steve Teale originally wrote the module as part of his std.database efforts.  I think it would be nice if any redesign done would aim for eventual phobos inclusion.
>>
>> I don't agree that this library should aim for any inclusion in phobos. I plan on exposing the MySQL API as closely as possible and a couple of higher level convenience functions. The std.database would have to be database agnostic, and should be a library interfacing to mysql-native, postgres etc.
>>
>> We should contact the author of ddbc for input: https://github.com/buggins/ddbc
>
> No, you are wrong. We need both low-level and high-level API,
> Low-level API like mysql-native driver should present a specific database as much as possible and provide access to all database features.
> High-level API like ORM HibernateD must provide common API for all databases, but it can't present all database specific features due database differences.
> So, I can use low-level API if I need access to all database specific features. I can use high-level API if I need common API for all databases.

That was my point.

On Mon, 30 Sep 2013 20:14:12 +0200
"simendsjo" <simendsjo@gmail.com> wrote:

> On Monday, 30 September 2013 at 16:27:16 UTC, Brad Anderson wrote:
> > On Saturday, 28 September 2013 at 16:39:27 UTC, simendsjo wrote:
> >> I've been working on a more or less complete rewrite of the mysql-native module.
> >
> > I don't have any specific comments.  Just want to say that Steve Teale originally wrote the module as part of his std.database efforts.  I think it would be nice if any redesign done would aim for eventual phobos inclusion.
> 
> I don't agree that this library should aim for any inclusion in phobos.

While certainly I have no objection to database stuff in Phobos, I
don't think it's particularly important now that we have dub.
Steve Teale's intension of getting it into Phobos predated anything
like that, so Phobos needed to be "kitchen sick". Not sure that's quite
as applicable anymore.

On Sat, 28 Sep 2013 18:38:47 +0200, simendsjo wrote:

> I've been working on a more or less complete rewrite of the mysql-native module.
> 

I think this is a great first step. Code is more readable, easier to follow (compared to the pointer stuff in the original).

Code has some distinctly 64bit parts because Length Encoded Integer
can be as large as 48bits and consume functions use a mix of size_t
and (u)long. Code doesn't compile as is on 32bit machine (windows).
I have changed some (u)longs to size_t and put in some cast(size_t) on
the assumption that on a 32bit machine those values won't overflow a
uint and on 64bit machine the will be (u)long as written. It compiles
and works for some basic queries that I use. This is a hack, not a
solution to the problem.

Also, the socket reads are asserting because windows reads partial packets. I change it to read until a full packet is read or an error or disconnect is returned.

simendsjo, you have done a great job pushing this forward. Would you consider putting it on your github account to receive some patches?

ollie

On Tuesday, 8 October 2013 at 17:47:02 UTC, ollie wrote:
> On Sat, 28 Sep 2013 18:38:47 +0200, simendsjo wrote:
>
>> I've been working on a more or less complete rewrite of the mysql-native module.
>> 
>
> I think this is a great first step. Code is more readable, easier
> to follow (compared to the pointer stuff in the original).


I removed most of the pointer stuff in my initial cleanup, but there has still
been plenty of room for improvements.


> Code has some distinctly 64bit parts because Length Encoded Integer
> can be as large as 48bits and consume functions use a mix of size_t
> and (u)long. Code doesn't compile as is on 32bit machine (windows).
> I have changed some (u)longs to size_t and put in some cast(size_t) on
> the assumption that on a 32bit machine those values won't overflow a
> uint and on 64bit machine the will be (u)long as written. It compiles
> and works for some basic queries that I use. This is a hack, not a
> solution to the problem.


I might have mixed up some of these I guess.. Using size_t in the protocol
specific code is probably wrong. I'll have to go through this in detail.


> Also, the socket reads are asserting because windows reads partial
> packets. I change it to read until a full packet is read or an error
> or disconnect is returned.


Auch.. Hope this fix solves it once and for all - otherwise there will probably
be a lot of subtle breakage in the code.

> simendsjo, you have done a great job pushing this forward. Would you
> consider putting it on your github account to receive some patches?
>
> ollie


https://github.com/simendsjo/mysqln

I think the only thing I changed from the upload is bugfixes in parameter
binding and added a couple of integration tests. I would have hoped to solve
a lot of these things before uploading the code, but if you wish to contribute,
I'm not going to stop you :)

There is still plenty to do though (some has been raised in this thread):
o 32-bit support
o Windows support
  (and OSX and FreeBSD)
o Documentation
o Examples
o Unittests
o More integration tests
o Implement stored procedures
  (and replication..?)
o SSL
o Remove type special handling (TINYINT(1) -> bool and perhaps others)
o Range interface for COM_QUERY and COM_STMT_EXECUTE that returns results.
o .. Which means ResultSet also needs a range interface and some way to
  construct it by taking all raws eagerly. I'm thinking "queryLazy" vs "query".
o COM_QUERY and COM_STMT_EXECUTE should be classes with close() methods to stop
  the execution. This will probably mean we need some data purging or
  something, and it has to be an error to start a new command before a previous
  has finished.
o Rewrite parameter binding code - it is very ugly and hard to understand.
  The old way was to store Variants and use typeid.toString to find out how to
  bind the parameter. I've rewritten this to store the parameter in a packet
  right away so it works with literals, const etc. The code is very ugly
  though. See packets.ComStmtPrepare
o There are stuff I haven't figured out in the prepared statements yet.
  Cursors and new_params_bound comes to mind.
o There might be unused functions in packet_helpers, but I'd like full
  integration-tests before removing anything.
o .. And probably many other stuff