There is need for mysql-native <https://github.com/rejectedsoftware/mysql-native> to support both Vibe.d's sockets and Phobos sockets.

First, a little background (feel free to skip ahead if you wish):
------------------------------------
Steve Teale's original version of mysql-native only supported Phobos's sockets (there was no Vibe.d at the time). But all activity and maintenance since then has been on a fork that *replaces* use of Phobos's sockets with Vibe.d's sockets. Vibe.d uses its own socket implementation instead of Phobos's because (AIUI) Phobos sockets don't support the particular Async I/O model that's a cornerstone of Vibe.d.

There are people, including myself, who have need to use mysql-native together with Vibe.d. But naturally, there's also very valid reasons why mysql-native should also be usable with nothing more than Phobos, in projects that choose not to rely on Vibe.d.
------------ (End of background) ------------

So, there's a need for mysql-native to support *both* Vibe.d and Phobos-only, and I've already created a branch which does exactly that:

https://github.com/Abscissa/mysql-native/blob/optional-phobos-sockets/source/mysql/connection.d

You can use that with either type of socket, even switch between them at runtime (although not within the same connection, obviously). And you can also use an optional "-verson=" to completely eliminate any compile-time dependency on Vibe.d (which naturally disables the ability to use Vibe.d sockets, of course).

I've tested it on both types of sockets with mysql-native's included test app ('source/app.d'), and it all works for me.

I *could* push that to the main mysql-native master repo right now <https://github.com/rejectedsoftware/mysql-native>. But I'm hesitant for one reason:

Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc.

That is the only breaking change, but depending how your code uses mysql-native, it can be a very significant breaking change.

So before merging that, I wanted to get input from anyone else who has used or contributed to either mysql-native or HibernateD <http://sourceforge.net/p/hibernated/> which currently uses a Phobos-only fork of  mysql-native for its MySQL capability (which is actually my main original motivating factor for making these changes).

Couple alternatives I've considered:

- I considered templating the types on an actual socket type instead of on "enum MySQLSocketType { phobos, vibed }", but I can't imagine a need for any other socket type being likely to arise. And even if it did, mysql-native would likely still need special-handling for it anyway.

- The *ideal* solution would be for Phobos and Vibe.d to agree on a common interface, defined in Phobos, and inherit their socket classes from that interface. The reasons I didn't go this route are because 1) I don't know enough about the background/internals/motiovations of either of the socket types to be the right person to spearhead this. and 2) Such a change in Phobos would likely require a lot of time and effort whereas my primary interest is to take mysql-native and just *make it work* on both Phobos and Vibe.d.

If the appropriate people on both the Phobos and Vibe.d sides wanted to spearhead an effort to make the respective sockets compatible, and fast-track it all, then I'd be happy to make mysql-native just use that. But I realize priorities might be elsewhere right now, and that's perfectly fine. I figure we can just "make it work" with both types of sockets right now (as I've already done, albeit in an unofficial branch), and if/when a merged socket design emerges, then mysql-native can just keep one option and deprecate the other.

Thoughts? Opinions? Etc?

On 2013-05-12 23:16, Nick Sabalausky wrote:

> I *could* push that to the main mysql-native master repo right now
> <https://github.com/rejectedsoftware/mysql-native>. But I'm hesitant
> for one reason:
>
> Since Phobos's and Vibe.d's sockets have incompatible APIs, my design
> converts most of the types in mysql-native into templates. For example,
> 'Connection' becomes either 'Connection!mySQLSocketVibeD' or
> 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either
> 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the
> same goes for most other types, such as Row, ResultSet, etc.
>
> That is the only breaking change, but depending how your code uses
> mysql-native, it can be a very significant breaking change.

Why don't you just relay on a version flag? Is it really necessary to change this at runtime?

-- 
/Jacob Carlborg

On Mon, 13 May 2013 09:03:08 +0200
Jacob Carlborg <doob@me.com> wrote:

> On 2013-05-12 23:16, Nick Sabalausky wrote:
> 
> > I *could* push that to the main mysql-native master repo right now <https://github.com/rejectedsoftware/mysql-native>. But I'm hesitant for one reason:
> >
> > Since Phobos's and Vibe.d's sockets have incompatible APIs, my design converts most of the types in mysql-native into templates. For example, 'Connection' becomes either 'Connection!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the same goes for most other types, such as Row, ResultSet, etc.
> >
> > That is the only breaking change, but depending how your code uses mysql-native, it can be a very significant breaking change.
> 
> Why don't you just relay on a version flag? Is it really necessary to change this at runtime?
> 

As this is intended as a "core-ish" low-level lib for any MySQL-client needs, I didn't want to unduly preclude any potential use-cases, even if it's not a typical use-case.

However, since it isn't a typical use-case, maybe what I'll do is this:

- Use new names for all of the newly templated types: Ex: 'Connection' becomes 'ConnectionX!mySQLSocketVibeD' and 'ConnectionX!mySQLSocketPhobos'.

- Require a -version to specify which socket type is the default, and then set up backwards-compatible aliases. So for example, if '-version=MySQLN_DefaultSocketVibeD' is used, then 'Connection' (for example) is defined as an alias to 'ConnectionX!mySQLSocketVibeD'.

- If a unified socket interface ever emerges, and obviates the needs for the templates, then these -version idents can just simply be ignored and output "that -version is no longer needed" pragma messages.

On Sunday, 12 May 2013 at 21:16:33 UTC, Nick Sabalausky wrote:
> There is need for mysql-native
> <https://github.com/rejectedsoftware/mysql-native> to support both
> Vibe.d's sockets and Phobos sockets.
>
> Since Phobos's and Vibe.d's sockets have incompatible APIs, my design
> converts most of the types in mysql-native into templates. For example,
> 'Connection' becomes either 'Connection!mySQLSocketVibeD' or
> 'Connection!mySQLSocketPhobos'. The type 'Command' becomes either
> 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'. And the
> same goes for most other types, such as Row, ResultSet, etc.
>

Does "incompatible APIs" preclude a design like this?

-----
interface MysqlSocket {
    // ...
}

class PhobosSocket : MysqlSocket {
    this(std.socket.Socket _backend) {
        // ...
    }
    // ...
}

class VibeSocket : MysqlSocket {
    this(vibe.core.net.TCPConnection _backend) {
        // ...
    }
}
-----

Sorry if I've overlooked anything obvious,
NMS

On 2013-05-14 01:53, Nick Sabalausky wrote:

> As this is intended as a "core-ish" low-level lib for any MySQL-client needs, I didn't want to unduly preclude any potential use-cases, even if it's not a typical use-case.
>
> However, since it isn't a typical use-case, maybe what I'll do is this:
>
> - Use new names for all of the newly templated types: Ex: 'Connection'
> becomes 'ConnectionX!mySQLSocketVibeD' and
> 'ConnectionX!mySQLSocketPhobos'.

That sounds reasonable.

-- 
/Jacob Carlborg

On Tue, 14 May 2013 04:35:20 +0200
"Nathan M. Swan" <nathanmswan@gmail.com> wrote:
> On Sunday, 12 May 2013 at 21:16:33 UTC, Nick Sabalausky wrote:
> > There is need for mysql-native
> > <https://github.com/rejectedsoftware/mysql-native> to support
> > both
> > Vibe.d's sockets and Phobos sockets.
> >
> > Since Phobos's and Vibe.d's sockets have incompatible APIs, my
> > design
> > converts most of the types in mysql-native into templates. For
> > example,
> > 'Connection' becomes either 'Connection!mySQLSocketVibeD' or
> > 'Connection!mySQLSocketPhobos'. The type 'Command' becomes
> > either
> > 'Command!mySQLSocketVibeD' or 'Connection!mySQLSocketPhobos'.
> > And the
> > same goes for most other types, such as Row, ResultSet, etc.
> >
> 
> Does "incompatible APIs"

By "incompatible APIs" I just meant that I can't use structural/duck typing on them (at least not directly), and they don't have a common base type aside from Object.

> preclude a design like this?
> 
> -----
> interface MysqlSocket {
>      // ...
> }
> 
> class PhobosSocket : MysqlSocket {
>      this(std.socket.Socket _backend) {
>          // ...
>      }
>      // ...
> }
> 
> class VibeSocket : MysqlSocket {
>      this(vibe.core.net.TCPConnection _backend) {
>          // ...
>      }
> }
> -----
> 

That's actually somewhat similar to what I ended up doing internally: I created a minimal wrapper for std.socket.Socket to expose a vibe-like socket API (but only for what's actually necessary for mysqln, nothing more, so it isn't a general, full-fledged phobos->vibed socket adapter). Then, based on how the templates are instantiated, the socket will be typed as either vibe.core.net.TCPConnection or the internally-defined phobos socket wrapper. I had some reasons for doing it that way, however...

However, now that you mention it, I think you're right, I should be able to just add a trivial pass-through wrapper for Vibe.d's sockets, define a minimal common interface for that and the Phobos socket wrapper, and just pass that interface into Connection.this().

There were some reasons I didn't go down that route before, but now that I think about more, I don't think those reasons were valid. And this definitely leads to a much cleaner design and API, plus it's more powerful (can use subclasses of either socket type). And while virtual method invocation isn't free, it should be completely negligible compared to actually performing network I/O, even if it's just localhost.

I'm going to try that, see how it plays out.

> Sorry if I've overlooked anything obvious,
> NMS

I think I'm the one who was overlooking things ;) Thanks!

On Tue, 14 May 2013 15:37:50 -0400
Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:

> On Tue, 14 May 2013 04:35:20 +0200
> "Nathan M. Swan" <nathanmswan@gmail.com> wrote:
> 
> > preclude a design like this?
> > 
> > -----
> > interface MysqlSocket {
> >      // ...
> > }
> > 
> > class PhobosSocket : MysqlSocket {
> >      this(std.socket.Socket _backend) {
> >          // ...
> >      }
> >      // ...
> > }
> > 
> > class VibeSocket : MysqlSocket {
> >      this(vibe.core.net.TCPConnection _backend) {
> >          // ...
> >      }
> > }
> > -----
> > 
> 
> That's actually somewhat similar to what I ended up doing internally: I created a minimal wrapper for std.socket.Socket to expose a vibe-like socket API (but only for what's actually necessary for mysqln, nothing more, so it isn't a general, full-fledged phobos->vibed socket adapter). Then, based on how the templates are instantiated, the socket will be typed as either vibe.core.net.TCPConnection or the internally-defined phobos socket wrapper. I had some reasons for doing it that way, however...
> 
> However, now that you mention it, I think you're right, I should be able to just add a trivial pass-through wrapper for Vibe.d's sockets, define a minimal common interface for that and the Phobos socket wrapper, and just pass that interface into Connection.this().
> 
> There were some reasons I didn't go down that route before, but now that I think about more, I don't think those reasons were valid. And this definitely leads to a much cleaner design and API, plus it's more powerful (can use subclasses of either socket type). And while virtual method invocation isn't free, it should be completely negligible compared to actually performing network I/O, even if it's just localhost.
> 
> I'm going to try that, see how it plays out.
> 

Hmm, there is a problem: For Vibe.d, mysql-native's Connection class is expected to derive from EventedObject. But EventedObject is part of Vibe.d. I'm not sure offhand how to handle that without making mysql-native's Connection templated.

On Thu, 16 May 2013 18:50:57 -0400
Nick Sabalausky <SeeWebsiteToContactMe@semitwist.com> wrote:
> 
> Hmm, there is a problem: For Vibe.d, mysql-native's Connection class is expected to derive from EventedObject. But EventedObject is part of Vibe.d. I'm not sure offhand how to handle that without making mysql-native's Connection templated.
> 

Actually, I think I figured out how to handle it:

There's no reason Connection's base type can't *always* be EventedObject *WHEN* Vibe.d is actually available, regardless of whether the user chooses to use Vibe's sockets or Phobos sockets. The *only* time Connection actually *needs* to not use EventedObject is when the user specifies -version=MySQLN_NoVibeD to disable mysqln's static dependency on Vibe.d (a feature I was planning to add anyway).

So I just statically choose to inherit from EventedObject or plain-old Object depending on whether MySQLN_NoVibeD exists, and that should work fine.

Sorry for the noise!

Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code.

MySql schema:

CREATE TABLE tblUser (
  id int(11) NOT NULL AUTO_INCREMENT,
  name varchar(255) NOT NULL,
  age varchar(45) NOT NULL,
  date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (id),
  UNIQUE KEY (name)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

D code:

import mysql.connection;
import std.stdio;

void main(string[] arguments)
{
	auto connection = new Connection(MySQLSocketType.phobos, "127.0.0.1", "root", "r00t", "Project");
	auto command    = new Command(connection, "SELECT * FROM tblUser");
	auto results    = command.execSQLResult();

	foreach (row; results)
	{
		writeln(row);
	}

	connection.close();
}

Error:

core.exception.AssertError@mysql.connection(1046): Assertion failure
----------------
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(_d_assertm+0x26) [0x4a6a06]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main() [0x4a38ce]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.SQLValue mysql.connection.consumeNonBinaryValueIfComplete!(std.datetime.DateTime).consumeNo
nBinaryValueIfComplete(ref ubyte[], bool)+0x126) [0x49e8ee]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.SQLValue mysql.connection.consumeIfComplete!(std.datetime.DateTime, 8).consumeIfComplete(re
f ubyte[], bool, bool)+0x36) [0x49ded2]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.SQLValue mysql.connection.consumeIfComplete!().consumeIfComplete(ref ubyte[], mysql.connect
ion.SQLType, bool, bool)+0x308) [0x494efc]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.Row mysql.connection.Row.__ctor(mysql.connection.Connection, ref ubyte[], mysql.connection.
ResultSetHeaders, bool)+0x221) [0x48be95]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(mysql.connection.ResultSet mysql.connection.Command.execSQLResult(mysql.connection.ColumnSpecialization[])+0
x1a3) [0x490f13]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(_Dmain+0xc9) [0x469a19]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void runMain()+0x18) [
0x4a71c8]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void tryExec(scope voi
d delegate())+0x2a) [0x4a6d06]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void runAll()+0x3b) [0
x4a720f]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(extern (C) int rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).void tryExec(scope voi
d delegate())+0x2a) [0x4a6d06]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(_d_run_main+0x1a8) [0x4a6cc0]
/tmp/.rdmd-1000/rdmd-main.d-EFA691A0BC2EFA572D0F077AB8E17D5C/main(main+0x17) [0x4a6b13]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f391473476d]
----------------
core.exception.InvalidMemoryOperationError

If i comment out the assert on line 1046 in connection.d everything seems to work fine. Any thoughts?

On Sat, 18 May 2013 17:42:24 +0200
"Gary Willoughby" <dev@kalekold.net> wrote:

> Hi, i've tried using this library for a simple query and i run into an error. I wondered if this is a bug in the connection code.
> 

Hmm, I'll look into it...