Thread overview
ReQL: pluses and minuses of pipeline-style queries
Apr 25, 2015
w0rp
Apr 25, 2015
monty
Apr 26, 2015
Rikki Cattermole
Apr 26, 2015
Laeeth Isharc
Apr 26, 2015
Rikki Cattermole
Apr 26, 2015
Idan Arye
Apr 26, 2015
Rikki Cattermole
Apr 26, 2015
Idan Arye
April 25, 2015
Found this on reddit a few days ago: http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/

A good discussion of the pros and cons of pipeline-style queries (the ReQL query language reminiscent of D's algorithms/ranges) vs. classic SQL.


Andrei
April 25, 2015
On Saturday, 25 April 2015 at 05:16:21 UTC, Andrei Alexandrescu wrote:
> Found this on reddit a few days ago: http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/
>
> A good discussion of the pros and cons of pipeline-style queries (the ReQL query language reminiscent of D's algorithms/ranges) vs. classic SQL.
>
>
> Andrei

One thing *kind of* related that I have really enjoyed is Django's querysets for building SQL queries. In Django, a QuerySet has methods query yields new QuerySet objects, so you can build a set of parameters and SQL is eventually generated and then executed to yield the results.

andrei_queryset = (
    People.objects
    .filter(first_name="Andrei", last_name__startswith="Al")
    .order_by("-affinity_for_template_metaprogramming")
    .select_related("organisation")
    [0:5]
)

The above would, when evaluated, generate something like the following in order to build the objects with.

SELECT p.*, o.* FROM people AS p
INNER JOIN organisation AS o
ON o.id = p.organisation_id
WHERE first_name = "Andrei"
AND last_name LIKE "Al%"
ORDER BY affinity_for_template_metaprogramming DESC
LIMIT 5;

I've been trying to think of a way to create something similar in D, maybe for something like HiberateD.
April 25, 2015
On Saturday, 25 April 2015 at 13:59:41 UTC, w0rp wrote:
> One thing *kind of* related that I have really enjoyed is Django's querysets for building SQL queries. In Django, a QuerySet has methods query yields new QuerySet objects, so you can build a set of parameters and SQL is eventually generated and then executed to yield the results.

another excellent example is ruby's sequel:
http://sequel.jeremyevans.net/
i think its even cleaner and much nicer to work with.

April 26, 2015
On 26/04/2015 1:59 a.m., w0rp wrote:
> On Saturday, 25 April 2015 at 05:16:21 UTC, Andrei Alexandrescu wrote:
>> Found this on reddit a few days ago:
>> http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/
>>
>> A good discussion of the pros and cons of pipeline-style queries (the
>> ReQL query language reminiscent of D's algorithms/ranges) vs. classic
>> SQL.
>>
>>
>> Andrei
>
> One thing *kind of* related that I have really enjoyed is Django's
> querysets for building SQL queries. In Django, a QuerySet has methods
> query yields new QuerySet objects, so you can build a set of parameters
> and SQL is eventually generated and then executed to yield the results.
>
> andrei_queryset = (
>      People.objects
>      .filter(first_name="Andrei", last_name__startswith="Al")
>      .order_by("-affinity_for_template_metaprogramming")
>      .select_related("organisation")
>      [0:5]
> )
>
> The above would, when evaluated, generate something like the following
> in order to build the objects with.
>
> SELECT p.*, o.* FROM people AS p
> INNER JOIN organisation AS o
> ON o.id = p.organisation_id
> WHERE first_name = "Andrei"
> AND last_name LIKE "Al%"
> ORDER BY affinity_for_template_metaprogramming DESC
> LIMIT 5;
>
> I've been trying to think of a way to create something similar in D,
> maybe for something like HiberateD.

I'm personally moving towards a DSL.

unittest {
	auto myQuery = """
using webdev.base.orm.query.parser.defs # allow for D class name instead of table name
; # end of \"sentence\"

from MyModel
where key == $0
as simple
# as index # but only one can be defined, two in total internal
; # end of \"sentence\"

from MyModel
where value contains $0
as complex
# as index # but only one can be defined, two in total internal
; # end of \"sentence\"

""".query;

	MyModel[] gotValues = myQuery.simple("test");
	gotValues = myQuery.perform("complex", "another");
}

Pros:
- Runtime reloadability
- Runtime composability
- More flexible
Cons:
- It's a string
April 26, 2015
On Saturday, 25 April 2015 at 05:16:21 UTC, Andrei Alexandrescu wrote:
> Found this on reddit a few days ago: http://rob.conery.io/2015/04/17/rethinkdb-2-0-is-amazing/
>
> A good discussion of the pros and cons of pipeline-style queries (the ReQL query language reminiscent of D's algorithms/ranges) vs. classic SQL.
>
>
> Andrei

I think his example of composability is targeting the exact examples where ReQL is composable and SQL isn't. It's easy enough to create an opposite example: let's say that our initial query is this:

select vendor.name from vendor
inner join catalog on vendor.id = catalog.vendor_id
inner join details on details.catalog_id = catalog.id

r.db("music").table("catalog").map(function(album){
  return {
    artist : album("vendor")("name")
  }
})

If we want to add the `type_id = 2` filter, in SQL it's a matter of appending a line:

select vendor.name from vendor
inner join catalog on vendor.id = catalog.vendor_id
inner join details on details.catalog_id = catalog.id
where details.media_type_id = 2;

But in ReQL you'd have to insert the filter in the middle:

r.db("music").table("catalog").filter(function(album){
  return album("details").filter(function(track){
    return track("media_type_id").eq(2)
  })
}).map(function(album){
  return {
    artist : album("vendor")("name")
  }
})

So, composablity seems to be just a matter of where the part you want to add happens to be placed in the query's structure...


At this point one might claim that in the SQL case we had to modify a string while in the ReQL case we could simply use methods of the existing query object - but that's because the ReQL query is wrapped for us(the actual format sent to the server is not the one written in the source code) while the SQL isn't(OK, that's a lie - the SQL query is compiled to some more efficient format. But that happens in the driver/server, so the bindings don't have to do it). In reality SQL is also wrapped, either by entirely new languages(ORMs like Hibernate) or by query builders that let you use SQL almost directly, and simply make it more convenient(I had good experience with
PetaPoco(http://www.toptensoftware.com/petapoco/), for example). If you use these wrappers you get the same composability as ReQL(modify query by invoking methods of existing query objects).
April 26, 2015
On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:
>
> I'm personally moving towards a DSL.
>
> unittest {
> 	auto myQuery = """
> using webdev.base.orm.query.parser.defs # allow for D class name instead of table name
> ; # end of \"sentence\"
>
> from MyModel
> where key == $0
> as simple
> # as index # but only one can be defined, two in total internal
> ; # end of \"sentence\"
>
> from MyModel
> where value contains $0
> as complex
> # as index # but only one can be defined, two in total internal
> ; # end of \"sentence\"
>
> """.query;
>
> 	MyModel[] gotValues = myQuery.simple("test");
> 	gotValues = myQuery.perform("complex", "another");
> }
>
> Pros:
> - Runtime reloadability
> - Runtime composability
> - More flexible
> Cons:
> - It's a string

Can Pegged be usefully applied to implement this?

Incidentally, it seems that although Hibernated is very nice, there is still work to be done.  Eg I would like to insert 10 million rows, and it seems like a transaction would be the best way of doing so, but it's not yet supported.  No big deal since the schema is very simple and I can do it by hand, but it would be nice to have at some point.  (I looked at your own ORM, but keeping it in memory won't work for me).
April 26, 2015
On 27/04/2015 12:10 a.m., Laeeth Isharc wrote:
> On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:
>>
>> I'm personally moving towards a DSL.
>>
>> unittest {
>>     auto myQuery = """
>> using webdev.base.orm.query.parser.defs # allow for D class name
>> instead of table name
>> ; # end of \"sentence\"
>>
>> from MyModel
>> where key == $0
>> as simple
>> # as index # but only one can be defined, two in total internal
>> ; # end of \"sentence\"
>>
>> from MyModel
>> where value contains $0
>> as complex
>> # as index # but only one can be defined, two in total internal
>> ; # end of \"sentence\"
>>
>> """.query;
>>
>>     MyModel[] gotValues = myQuery.simple("test");
>>     gotValues = myQuery.perform("complex", "another");
>> }
>>
>> Pros:
>> - Runtime reloadability
>> - Runtime composability
>> - More flexible
>> Cons:
>> - It's a string
>
> Can Pegged be usefully applied to implement this?
>
> Incidentally, it seems that although Hibernated is very nice, there is
> still work to be done.  Eg I would like to insert 10 million rows, and
> it seems like a transaction would be the best way of doing so, but it's
> not yet supported.  No big deal since the schema is very simple and I
> can do it by hand, but it would be nice to have at some point.  (I
> looked at your own ORM, but keeping it in memory won't work for me).

Dvorm like Cmsed is going bye byes.
My next web service framework is going to have a very different approach to ORM's and routing!

The given code example, is an actual unittest for the parser. It'll be using a reflection API to wrap ORM models up. So the ORM will no longer do serialization. Instead the backend will to the appropriate types.

Just so you get a small taste of the reflection capabilities. I know it may not seem like much, but imagine e.g. lua to have wrapper code around this and to act as if the data models were written natively in it or to pass a template (lets say lua based) a bunch of data models and have it call D methods on it!
The only problem is that the web server that it is meant to compliment isn't ready yet.
https://github.com/DNetDev/webserver

Until I or somebody else gets round to implementing this https://github.com/rejectedsoftware/vibe.d/issues/1074 mindset wise, I can't continue on.

Also you are welcome to join https://gitter.im/DNetDev/Public if you want to talk more.

module webdev.base.reflection.model;
import webdev.base.traits.are : isADataModel, isADataModelProperty, isDataModelMemberId, isADataModelQueryMethod, isADataModelQueryStaticMethod;
import webdev.base.traits.have : getDataModelName, getDataModelDescription, getDataModelPropertyDescription, getDataModelPropertyHints, getDataModelPropertyName;
import webdev.base.udas : OrmPropertyTypes;

private __gshared {
	import std.variant : Algebraic;
	import std.traits : fullyQualifiedName, isArray, ReturnType, ParameterTypeTuple;

	AReflectedModel*[string] models;
	AReflectedModel*[string] modelsByTableName;
}

/*
 * Basic interactions of the different kinds of models
 */

/**
 * Gets all names of data models registered
 * Uses the fully qualified name (package + module + class/struct name)
 *
 * Returns:
 * 		The names to all data models
 */
string[] reflectedModelNames() {
	return models.keys;
}

/**
 * Lazily registers and gets a reflected model given the data model type
 *
 * Returns:
 * 		The reflected model
 */
AReflectedModel* getReflectModel(T)() if(isADataModel!T) {
	return getReflectedModel(fullyQualifiedName!T);
}


/**
 * Gets a reflected model based upon its fully qualified name
 *
 * Params:
 * 		name	=	Name of the model
 *
 * Returns:
 * 		The reflected model
 */
AReflectedModel* getReflectedModel(string name) {
	if (name !in models) return null;
	return models[name].dup();
}

/**
 * Gets a reflected model based upon its table name
 *
 * Params:
 * 		name	=	Name of the model
 *
 * Returns:
 * 		The reflected model
 */
AReflectedModel* getReflectedModelByTableName(string name) {
	if (name !in modelsByTableName) return null;
	return modelsByTableName[name].dup();
}

/*
 * General reflection based types
 */

///
enum OrmActualPropertyTypes {
	Unknown,

	UByte,
	Byte,
	UShort,
	Short,
	UInt,
	Int,
	ULong,
	Long,
	Float,
	Double,
	String,
	WString,
	DString,
	
	Array,
	DataModel
}

///
alias ModelValidTypes = Algebraic!(AReflectedModelInstance*, ubyte, byte, ushort, short, uint, int, ulong, long, float, double, string, wstring, dstring);

///
struct PropertyHint {
	///
	string name;

	///
	OrmActualPropertyTypes actualType;

	///
	OrmActualPropertyTypes arrayActualType;

	///
	AReflectedModel* objectActualType;

	///
	OrmPropertyTypes hintType;

	///
	size_t size;

	///
	string description;

	///
	bool isId;
}

///
struct QueryDescriptor {
	///
	QueryTypeDescriptor[] arguments;

	///
	QueryTypeDescriptor returnType;

	struct QueryTypeDescriptor {
		///
		OrmActualPropertyTypes actualType;
		
		///
		OrmActualPropertyTypes arrayActualType;
		
		///
		AReflectedModel* objectActualType;
	}
}

/*
 * The actual reflection mechanism
 */

struct AReflectedModel {
	static AReflectedModel* reflect(T)() if(isADataModel!T) {
		AReflectedModel* ret = new AReflectedModel;

		ret.dup = () { return AReflectedModel.reflect!T(); };

		/// constructs function delegates for a model instance aware of the type
		void funcCalls(AReflectedModelInstance* retm) {
			static if (__traits(hasMember, T, "isValid")) {
				retm.isValid = () { return (cast(T*)retm.instance_).isValid(); };
			} else {
				retm.isValid = () { return true; };
			}

			retm.get = (string name) {
				foreach(member; __traits(allMembers, T)) {
					static if (isADataModelProperty!(T, member)) {
						if (member == name) {
							return new ModelValidTypes(mixin("(cast(T*)retm.instance_)." ~ member));
						}
					}
				}

				return null;
			};

			retm.set = (string name, ModelValidTypes* value) {
				foreach(member; __traits(allMembers, T)) {
					mixin("alias MTYPE = typeof(T." ~ member ~ ");");

					static if (isADataModelProperty!(T, member)) {
						if (member == name) {
							static if (__traits(compiles, {MTYPE t = null;})) {
								if (value is null) {
									mixin("(cast(T*)retm.instance_)." ~ member ~ " = null;");
								} else if (value.convertsTo!MTYPE) {
									mixin("(cast(T*)retm.instance_)." ~ member ~ " = value.get!MTYPE;");
								} else {
									assert(0, value.type.toString ~ " is not convertable to " ~ MTYPE.stringof);
								}
							} else {
								if (value !is null && value.convertsTo!MTYPE) {
									mixin("(cast(T*)retm.instance_)." ~ member ~ " = value.get!MTYPE;");
								} else {
									assert(0, value.type.toString ~ " is not convertable to " ~ MTYPE.stringof);
								}
							}
						}
					}
				}
			};

			retm.query = (string name, ModelValidTypes[] values...) {
				foreach(member; __traits(allMembers, T)) {
					mixin("alias MTYPE = typeof(T." ~ member ~ ");");

					static if (isADataModelQueryMethod!(T, member)) {
						alias MRET = ReturnType!MTYPE;

						if (member == name) {
							alias ARGU = ParameterTypeTuple!MTYPE;

							if (values.length != ARGU.length)
								assert(0, "Not enough arguments");
								
							foreach(i, ARG; ARGU) {
								if (!values[i].convertsTo!ARG)
									assert(0, "Wrong types for arguments");
							}

							static if (is(MRET == void)) {
								// return call
								mixin("(cast(T*)retm.instance)." ~ member ~ "(" ~ getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");
								return cast(ModelValidTypes[])null;
							} else {
								// call
								mixin("auto ret = (cast(T*)retm.instance_)." ~ member ~ "(" ~ getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");

								static if (isArray!MRET) {
									ModelValidTypes[] ret2;

									foreach(v; ret) {
										ret2 ~= ModelValidTypes(v);
									}

									return ret2;
								} else {
									return cast(ModelValidTypes[])[ModelValidTypes(v)];
								}
							}
						}
					}
				}

				assert(0);
			};
		}

		ret.create = () {
			AReflectedModelInstance* retm = new AReflectedModelInstance;
			retm.model_ = ret;

			static if (is(T == class))
				retm.instance_ = &(new T);
			else static if (is(T == struct))
				retm.instance_ = new T;
			else static assert(0);
			
			funcCalls(retm);
			return retm;
		};

		ret.fromInstance = (void* value) {
			/// in
	
			assert(value !is null);

			if (T* ttv = cast(T*)value){}
			else assert(0, "Argument is not of type " ~ T.stringof);

			/// body

			AReflectedModelInstance* retm = new AReflectedModelInstance;
			retm.model_ = ret;

			retm.instance_ = value;

			funcCalls(retm);
			return retm;
		};

		ret.tableName = () { return getDataModelName!T; };
		ret.fullName = () { return fullyQualifiedName!T; };
		ret.description = () { return getDataModelDescription!T; };

		ret.propertyNames = () {
			string[] retm;

			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelProperty!(T, member)) {
					retm ~= member;
				}
			}

			return retm;
		};

		ret.propertyHints = (string name) {
			PropertyHint retm;

			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelProperty!(T, member)) {
					if (member == name) {
						mixin("alias MTYPE = typeof(T." ~ member ~ ");");

						retm.name = getDataModelPropertyName!(T, member);

						// actualType
						enum MTYPEA = actualTypeFromType!MTYPE;
						retm.actualType = MTYPEA;

						// arrayActualType
						static if (MTYPEA == OrmActualPropertyTypes.Array)
							retm.arrayActualType = actualTypeFromType!(typeof(MTYPE.init)[0]);

						// objectActualType
						static if (MTYPEA == OrmActualPropertyTypes.DataModel)
							retm.objectActualType = getReflectModel!MTYPE;

						auto hint = getDataModelPropertyHints!(T, member);

						// hintType
						retm.hintType = hint.type;

						// size
						if (hint.size == 0) {
							static if (isArray!MTYPE) {
							} else
								hint.size = MTYPE.sizeof;
						} else
							retm.size = hint.size;

						// description
						retm.description = getDataModelPropertyDescription!(T, member);

						// isId
						retm.isId = isDataModelMemberId!(T, member);
					}
				}
			}

			return retm;
		};

		ret.query = (string name, ModelValidTypes[] values...) {
			foreach(member; __traits(allMembers, T)) {
				mixin("alias MTYPE = typeof(T." ~ member ~ ");");
				
				static if (isADataModelQueryStaticMethod!(T, member)) {
					alias MRET = ReturnType!MTYPE;
					
					if (member == name) {
						alias ARGU = ParameterTypeTuple!MTYPE;
						
						if (values.length != ARGU.length)
							assert(0, "Not enough arguments");
						
						foreach(i, ARG; ARGU) {
							if (!values[i].convertsTo!ARG)
								assert(0, "Wrong types for arguments");
						}
						
						static if (is(MRET == void)) {
							// return call
							mixin("T." ~ member ~ "(" ~ getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");
							return cast(ModelValidTypes[])null;
						} else {
							// call
							mixin("auto ret = T." ~ member ~ "(" ~ getCallToMethodSyntaxVarient!("values", "ARGU", ARGU) ~ ");");
							
							ModelValidTypes[] ret2;

							static if (isArray!MRET) {
								foreach(v; ret) {
									static if (is(typeof(v) == class) || is(typeof(v) == struct)) {
										ret2 ~= ModelValidTypes(getReflectModel!(typeof(v))().fromInstance(&v));
									} else {
										ret2 ~= ModelValidTypes(v);
									}
								}
							} else {
								static if (is(MRET == class) || is(MRET == struct)) {
									ret2 ~= ModelValidTypes(getReflectModel!(MRET)().fromInstance(&ret));
								} else {
									ret2 ~= ModelValidTypes(ret);
								}
							}

							return ret2;
						}
					}
				}
			}
			
			assert(0);
		};

		// queryNames
		ret.queryNames = () {
			string[] ret;

			foreach(member; __traits(allMembers, T)) {
				mixin("alias MTYPE = typeof(T." ~ member ~ ");");
				
				static if (isADataModelQueryStaticMethod!(T, member)) {
					ret ~= member;
				}
			}

			return ret;
		};

		// queryParameters
		ret.queryParameters = (string name) {
			QueryDescriptor ret;
			
			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelQueryStaticMethod!(T, member)) {
					if (member == name) {

						// arguments
						foreach(ARG; ParameterTypeTuple!(mixin("T." ~ name))) {
							QueryDescriptor.QueryTypeDescriptor rett;

							// actualType
							enum MTYPEA = actualTypeFromType!ARG;
							rett.actualType = MTYPEA;

							// arrayActualType
							static if (MTYPEA == OrmActualPropertyTypes.Array)
								rett.arrayActualType = actualTypeFromType!(typeof(ARG.init)[0]);

							// objectActualType
							static if (MTYPEA == OrmActualPropertyTypes.DataModel)
								rett.objectActualType = getReflectModel!ARG;

							ret.arguments ~= rett;
						}

						// return type

						alias RETM = ReturnType!(mixin("T." ~ m));

						// actualType
						enum MTYPEA = actualTypeFromType!RETM;
						ret.returnType.actualType = MTYPEA;
						
						// arrayActualType
						static if (MTYPEA == OrmActualPropertyTypes.Array)
							ret.returnType.arrayActualType = actualTypeFromType!(typeof(ARG.init)[0]);
						
						// objectActualType
						static if (MTYPEA == OrmActualPropertyTypes.DataModel)
							ret.returnType.objectActualType = getReflectModel!ARG;
					}
				}
			}
			
			return ret;
		};

		// queryInstanceNames
		ret.queryInstanceNames = () {
			string[] ret;
			
			foreach(member; __traits(allMembers, T)) {
				mixin("alias MTYPE = typeof(T." ~ member ~ ");");
				
				static if (isADataModelQueryMethod!(T, member)) {
					ret ~= member;
				}
			}
			
			return ret;
		};

		// queryInstanceParameters
		ret.queryInstanceParameters = (string name) {
			QueryDescriptor ret;
			
			foreach(member; __traits(allMembers, T)) {
				static if (isADataModelQueryMethod!(T, member)) {
					if (member == name) {
						
						// arguments
						foreach(ARG; ParameterTypeTuple!(mixin("T." ~ name))) {
							QueryDescriptor.QueryTypeDescriptor rett;
							
							// actualType
							enum MTYPEA = actualTypeFromType!ARG;
							rett.actualType = MTYPEA;
							
							// arrayActualType
							static if (MTYPEA == OrmActualPropertyTypes.Array)
								rett.arrayActualType = actualTypeFromType!(typeof(ARG.init)[0]);
							
							// objectActualType
							static if (MTYPEA == OrmActualPropertyTypes.DataModel)
								rett.objectActualType = getReflectModel!ARG;
							
							ret.arguments ~= rett;
						}
						
						// return type
						
						alias RETM = ReturnType!(mixin("T." ~ m));
						
						// actualType
						enum MTYPEA = actualTypeFromType!RETM;
						ret.returnType.actualType = MTYPEA;
						
						// arrayActualType
						static if (MTYPEA == OrmActualPropertyTypes.Array)
							ret.returnType.arrayActualType = actualTypeFromType!(typeof(ARG.init)[0]);
						
						// objectActualType
						static if (MTYPEA == OrmActualPropertyTypes.DataModel)
							ret.returnType.objectActualType = getReflectModel!ARG;
					}
				}
			}
			
			return ret;
		};

		models[fullyQualifiedName!T] = ret;
		models[ret.tableName()] = ret;
		return ret;
	}

	AReflectedModel* delegate() dup;
	AReflectedModelInstance* delegate() create;
	AReflectedModelInstance* delegate(void*) fromInstance;

	string delegate() tableName;
	string delegate() fullName;
	string delegate() description;

	const(string[]) delegate() propertyNames;
	PropertyHint delegate(string name) propertyHints;

	ModelValidTypes[] delegate(string func, ModelValidTypes[] values...) query;
	const(string[]) delegate() queryNames;
	QueryDescriptor delegate(string name) queryParameters; //FIXME: should not be void*

	const(string[]) delegate() queryInstanceNames;
	QueryDescriptor delegate(string name) queryInstanceParameters; //FIXME: should not be void*
}

struct AReflectedModelInstance {
	private {
		AReflectedModel* model_;

		void* instance_;
	}

	AReflectedModel* model() { return model; }
	void* instance() { return instance; }

	/*
	 *
	 * Shouldn't everything below this, stored in the model instead of the state?
	 *
	 */

	bool delegate() isValid;

	/*
	 * Get/Set for all approved properties given a name and the value for a model instance
	 */
	ModelValidTypes* delegate(string name) get;
	void delegate(string name, ModelValidTypes* value) set;

	ModelValidTypes[] delegate(string func, ModelValidTypes[] values...) query;
}

private {
	OrmActualPropertyTypes actualTypeFromType(T)() {
		static if (is(T == ubyte))
			return OrmActualPropertyTypes.UByte;
		else static if (is(T == byte))
			return OrmActualPropertyTypes.Byte;
		else static if (is(T == ushort))
			return OrmActualPropertyTypes.UShort;
		else static if (is(T == short))
			return OrmActualPropertyTypes.Short;
		else static if (is(T == uint))
			return OrmActualPropertyTypes.UInt;
		else static if (is(T == int))
			return OrmActualPropertyTypes.Int;
		else static if (is(T == ulong))
			return OrmActualPropertyTypes.ULong;
		else static if (is(T == long))
			return OrmActualPropertyTypes.Long;
		else static if (is(T == float))
			return OrmActualPropertyTypes.Float;
		else static if (is(T == double))
			return OrmActualPropertyTypes.Double;
		else static if (is(T == string))
			return OrmActualPropertyTypes.String;
		else static if (is(T == wstring))
			return OrmActualPropertyTypes.WString;
		else static if (is(T == dstring))
			return OrmActualPropertyTypes.DString;
		else static if (is(T == class) || is(T == struct))
			return OrmActualPropertyTypes.DataModel;
		else static if (isArray!T)
			return OrmActualPropertyTypes.Array;
		else
			return OrmActualPropertyTypes.Unknown;
	}

	string getCallToMethodSyntaxVarient(string valuesName, string argName, ARGS...)() pure {
		import std.conv : text;
		string ret;

		foreach(i, ARG; ARGS) {
			string TI = text(i);
			ret ~= valuesName ~ "[" ~ TI ~ "].get!(" ~ argName ~ "[" ~ TI ~ "]), ";
		}

		if (ARGS.length > 0)
			ret.length -= 2;

		return ret;
	}
}
April 26, 2015
On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:
> I'm personally moving towards a DSL.
>
> unittest {
> 	auto myQuery = """
> using webdev.base.orm.query.parser.defs # allow for D class name instead of table name
> ; # end of \"sentence\"
>
> from MyModel
> where key == $0
> as simple
> # as index # but only one can be defined, two in total internal
> ; # end of \"sentence\"
>
> from MyModel
> where value contains $0
> as complex
> # as index # but only one can be defined, two in total internal
> ; # end of \"sentence\"
>
> """.query;
>
> 	MyModel[] gotValues = myQuery.simple("test");
> 	gotValues = myQuery.perform("complex", "another");
> }
>
> Pros:
> - Runtime reloadability
> - Runtime composability
> - More flexible
> Cons:
> - It's a string

I get why people want to use a DSL based on the host language's constructs. The host language is built to deal with it's own constructs, so composing your queries from them allows you to easily do stuff like store query parts in variables, create functions that modify queries, use reflection on the queries, send variables and expressions from the host process directly to the query etc.

What do you gain from a string-based DSL? It doesn't allow you to do any of these things - at least not any more conveniently/efficiently/safely than you could with SQL strings - so essentially it's just a language to replace SQL.

Now, Some languages are so messed up that it's worthwhile to create a language the compiles to them(*cough* Javascript *cough*). SQL is not one of them. Even if we assume this language is better than SQL - is the difference enough to justify leanring this new language, to translate SQL errors caused by misuse of that langauge(or propagate and let the user deal with it), and deal with errors caused by bad implementation of the translation?
April 26, 2015
On 27/04/2015 1:12 a.m., Idan Arye wrote:
> On Sunday, 26 April 2015 at 01:03:12 UTC, Rikki Cattermole wrote:
>> I'm personally moving towards a DSL.
>>
>> unittest {
>>     auto myQuery = """
>> using webdev.base.orm.query.parser.defs # allow for D class name
>> instead of table name
>> ; # end of \"sentence\"
>>
>> from MyModel
>> where key == $0
>> as simple
>> # as index # but only one can be defined, two in total internal
>> ; # end of \"sentence\"
>>
>> from MyModel
>> where value contains $0
>> as complex
>> # as index # but only one can be defined, two in total internal
>> ; # end of \"sentence\"
>>
>> """.query;
>>
>>     MyModel[] gotValues = myQuery.simple("test");
>>     gotValues = myQuery.perform("complex", "another");
>> }
>>
>> Pros:
>> - Runtime reloadability
>> - Runtime composability
>> - More flexible
>> Cons:
>> - It's a string
>
> I get why people want to use a DSL based on the host language's
> constructs. The host language is built to deal with it's own constructs,
> so composing your queries from them allows you to easily do stuff like
> store query parts in variables, create functions that modify queries,
> use reflection on the queries, send variables and expressions from the
> host process directly to the query etc.
>
> What do you gain from a string-based DSL? It doesn't allow you to do any
> of these things - at least not any more conveniently/efficiently/safely
> than you could with SQL strings - so essentially it's just a language to
> replace SQL.
>
> Now, Some languages are so messed up that it's worthwhile to create a
> language the compiles to them(*cough* Javascript *cough*). SQL is not
> one of them. Even if we assume this language is better than SQL - is the
> difference enough to justify leanring this new language, to translate
> SQL errors caused by misuse of that langauge(or propagate and let the
> user deal with it), and deal with errors caused by bad implementation of
> the translation?

Worse case scenario, you are free to use the raw sql function for a query. But it will only work if its for an SQL database engine.
Also this won't compile to D.