I find the reasons for turining down my ER a bit moot.

[Issue 14593 – operator overloading can't be used with expression templates](https://issues.dlang.org/show_bug.cgi?id=14593)

AFAIK expression templates are the primary choice tom implement SIMD and
matrix libraries.
And I still have [this idea](http://dpaste.dzfl.pl/cd375ac594cf) of
implementing a nice query language for ORMs.

D currently doesn't allow to override some operators like < <= > >= && || !=.

At least the comparison operators are really limiting, e.g. it's not possible to efficiently implement logical indexing.

vec[vec < 15], vec[vec == 15], vec[(vec != 15) & (vec > 10)]

Also opCmp is less efficient to implement than opBinary!"<" for many types. Generally any good implementation of an algorithm should only require a less operator, not a full ordering opCmp.

The short circuit operators && and || have a special semantic and can't be easily, but there is & and | so it's not really required.

Now expression templates make an awful DSL when being used to create a HTML formatter, but when the original semantic of the operators is preserved they are really powerful b/c the compiler already handles typechecking, operator precedence, and captures variables.

Does anyone have a different idea how to make a nice query language? db.get!Person.where!(p => p.age > 21 && p.name == "Peter")

On Friday, 11 September 2015 at 19:41:41 UTC, Martin Nowak wrote:
> Does anyone have a different idea how to make a nice query language? db.get!Person.where!(p => p.age > 21 && p.name == "Peter")

Don't the suggestions for DSLs generally revolve around using string mixins to use whatever syntax you want?

- Jonathan M Davis

On 09/11/2015 09:40 PM, Martin Nowak wrote:
> I find the reasons for turining down my ER a bit moot.
> ...

+1.

> [Issue 14593 – operator overloading can't be used with expression
> templates](https://issues.dlang.org/show_bug.cgi?id=14593)
>
> AFAIK expression templates are the primary choice tom implement SIMD and
> matrix libraries.

OTOH, they are a hack.

> And I still have [this idea](http://dpaste.dzfl.pl/cd375ac594cf) of
> implementing a nice query language for ORMs.
>
> D currently doesn't allow to override some operators like < <= > >= &&
> || !=.
>
> At least the comparison operators are really limiting, e.g. it's not
> possible to efficiently implement logical indexing.
>
> vec[vec < 15], vec[vec == 15], vec[(vec != 15) & (vec > 10)]
>
> Also opCmp is less efficient to implement than opBinary!"<" for many
> types. Generally any good implementation of an algorithm should only
> require a less operator, not a full ordering opCmp.
> ...

The rewrite a < b => a.opCmp(b)<0 is usually wasteful, but both methods can be the most efficient choice depending on the application. Two calls to opBinary!"<" will usually not be more efficient than one call to "opCmp".

I.e. the precedence should be the other way: try a.opBinary!"<"(b) first and then fall back to a.opCmp(b)<0. (Maybe it would also make sense to automatically provide opCmp if missing when opBinary!"<" has been implemented, such that generic code can use opCmp for all comparable types. Built-in comparable types should also provide opCmp.)

> The short circuit operators && and || have a special semantic and can't
> be easily, but there is & and | so it's not really required.
> ...

There's the 'lazy' keyword.

> ...
>
> Does anyone have a different idea how to make a nice query language?
> db.get!Person.where!(p => p.age > 21 && p.name == "Peter")
>

You could give up on operator syntax.

On 09/11/2015 03:40 PM, Martin Nowak wrote:
> I find the reasons for turining down my ER a bit moot.
>
> [Issue 14593 – operator overloading can't be used with expression
> templates](https://issues.dlang.org/show_bug.cgi?id=14593)
>
> AFAIK expression templates are the primary choice tom implement SIMD and
> matrix libraries.
> And I still have [this idea](http://dpaste.dzfl.pl/cd375ac594cf) of
> implementing a nice query language for ORMs.

Expression templates are interesting, but from experience with them in C++ they're more trouble than they're worth. They haven't made much inroads in C++ outside exotic libraries because (a) they have odd and random limitations and corner cases and (b) they have really byzantine failure modes. Just look at any non-toy-example C++ use of ETs - it's completely bizarre.

I'd say if a language wants to support expression templates properly, there's a lot of careful design to get into it, way beyond intercepting operators. Like e.g. C#/LINQ. I don't think D is prepared for that.

> D currently doesn't allow to override some operators like < <= > >= &&
> || !=.
>
> At least the comparison operators are really limiting, e.g. it's not
> possible to efficiently implement logical indexing.
>
> vec[vec < 15], vec[vec == 15], vec[(vec != 15) & (vec > 10)]
>
> Also opCmp is less efficient to implement than opBinary!"<" for many
> types. Generally any good implementation of an algorithm should only
> require a less operator, not a full ordering opCmp.
>
> The short circuit operators && and || have a special semantic and can't
> be easily, but there is & and | so it's not really required.
>
> Now expression templates make an awful DSL when being used to create a
> HTML formatter, but when the original semantic of the operators is
> preserved they are really powerful b/c the compiler already handles
> typechecking, operator precedence, and captures variables.
>
> Does anyone have a different idea how to make a nice query language?
> db.get!Person.where!(p => p.age > 21 && p.name == "Peter")

There's two canonical ways to do that.

1. Use lambdas, which seem to already do what you want:

db.get!Person.filter!(p => p.age > 21 && p.name == "Peter")

The way this'd go, the db.get!Person() call returns an input range of Person. Presumably introspection is being used to bind fields in the query such as "age" and "name" to static field names in struct Person. Then good old std.algorithm.filter takes care of the rest.

2. If you want to embed real SQL into D, use string-based DSLs.


Andrei

On Fri, Sep 11, 2015 at 07:47:42PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> On 09/11/2015 03:40 PM, Martin Nowak wrote:
> >I find the reasons for turining down my ER a bit moot.
> >
> >[Issue 14593 – operator overloading can't be used with expression templates](https://issues.dlang.org/show_bug.cgi?id=14593)
> >
> >AFAIK expression templates are the primary choice tom implement SIMD
> >and matrix libraries.
> >And I still have [this idea](http://dpaste.dzfl.pl/cd375ac594cf) of
> >implementing a nice query language for ORMs.
> 
> Expression templates are interesting, but from experience with them in C++ they're more trouble than they're worth. They haven't made much inroads in C++ outside exotic libraries because (a) they have odd and random limitations and corner cases and (b) they have really byzantine failure modes. Just look at any non-toy-example C++ use of ETs - it's completely bizarre.

A number of years ago I wrote a computational application using Blitz++, an ET-based matrix / multidimensional array C++ library, and I have to say that the experience was mostly pleasant.  However, it did push the limits of operator overloading abuse, such as:

	Array<2,int> matrix = Array<2,int>(4, 4);
	matrix = 1, 0, 0, 0,
	         0, 1, 0, 0,
		 0, 0, 1, 0,
		 0, 0, 0, 1;

It also had unexpected reference/copying semantics (operator=() copies by reference, but `=` in a variable declarationi copies by value), as well as the occasional wrinkle when assigning the result of an expression template to a variable (sometimes you have to explicitly call a function to turn it into something storable in a variable).

The error messages however, were completely inscrutable, just as you said. They generally begin anywhere from 15-16 lines of compiler output per error, and only grows from there. Needless to say, I did not have the patience (nor persistence!) to decipher those error messages; most of my efforts lay in copying textbook examples from the documentation and modifying them piece by piece, checking their compilability at every step, until they matched what I ultimately wanted. Writing anything complex directly was an invitation to be faced with an incomprehensible screen-filling error message (often more than one), and endless hours of randomly modifying random bits of syntax in hopes that the error will somehow, magically, go away.


> I'd say if a language wants to support expression templates properly, there's a lot of careful design to get into it, way beyond intercepting operators. Like e.g. C#/LINQ. I don't think D is prepared for that.

Pluggable syntax modules seem like an attractive idea, though. But probably outside the scope of D2 at present.


[...]
> >Does anyone have a different idea how to make a nice query language? db.get!Person.where!(p => p.age > 21 && p.name == "Peter")
> 
> There's two canonical ways to do that.
> 
> 1. Use lambdas, which seem to already do what you want:
> 
> db.get!Person.filter!(p => p.age > 21 && p.name == "Peter")
> 
> The way this'd go, the db.get!Person() call returns an input range of Person. Presumably introspection is being used to bind fields in the query such as "age" and "name" to static field names in struct Person. Then good old std.algorithm.filter takes care of the rest.
> 
> 2. If you want to embed real SQL into D, use string-based DSLs.
[...]

Yeah, string-based DSLs make more sense in D: avoid complicating the core language, thorny issues surrounding operator overloading (and the abuse thereof), and also allow fully-free syntax of your choice.  With CTFE, you can parse just about any DSL with any syntax at compile-time, thus incur none of the performance issues that runtime string-based DSLs would have.


T

-- 
The only difference between male factor and malefactor is just a little emptiness inside.

On 09/11/2015 08:03 PM, H. S. Teoh via Digitalmars-d wrote:
> Needless to say, I did not have
> the patience (nor persistence!) to decipher those error messages; most
> of my efforts lay in copying textbook examples from the documentation
> and modifying them piece by piece, checking their compilability at every
> step, until they matched what I ultimately wanted. Writing anything
> complex directly was an invitation to be faced with an incomprehensible
> screen-filling error message (often more than one), and endless hours of
> randomly modifying random bits of syntax in hopes that the error will
> somehow, magically, go away.

All the more ironic considering Blitz++ was intended for scientists who were not supposed to be programmer experts. -- Andrei

On Saturday, 12 September 2015 at 00:27:45 UTC, Andrei Alexandrescu wrote:
> On 09/11/2015 08:03 PM, H. S. Teoh via Digitalmars-d wrote:
>> Needless to say, I did not have
>> the patience (nor persistence!) to decipher those error messages; most
>> of my efforts lay in copying textbook examples from the documentation
>> and modifying them piece by piece, checking their compilability at every
>> step, until they matched what I ultimately wanted. Writing anything
>> complex directly was an invitation to be faced with an incomprehensible
>> screen-filling error message (often more than one), and endless hours of
>> randomly modifying random bits of syntax in hopes that the error will
>> somehow, magically, go away.
>
> All the more ironic considering Blitz++ was intended for scientists who were not supposed to be programmer experts. -- Andrei

I do think it says more about C++ than it does about expression templates.

On 2015-09-12 01:47, Andrei Alexandrescu wrote:

> There's two canonical ways to do that.
>
> 1. Use lambdas, which seem to already do what you want:
>
> db.get!Person.filter!(p => p.age > 21 && p.name == "Peter")
>
> The way this'd go, the db.get!Person() call returns an input range of
> Person. Presumably introspection is being used to bind fields in the
> query such as "age" and "name" to static field names in struct Person.
> Then good old std.algorithm.filter takes care of the rest.

A database is really good at what it does. Therefore one wants the filter/where statement to be executed by the database, not the application code.

It would also be quite wasteful to have to create an instance of Person to make the comparison then realizing it doesn't fulfill the condition and throw it away.

I'm not sure how database protocols work but one definitely don't want to query the database for each row or return all the rows from the database.

> 2. If you want to embed real SQL into D, use string-based DSLs.

The point is to _not_ use raw SQL.

-- 
/Jacob Carlborg

On 2015-09-11 21:40, Martin Nowak wrote:
> I find the reasons for turining down my ER a bit moot.
>
> [Issue 14593 – operator overloading can't be used with expression
> templates](https://issues.dlang.org/show_bug.cgi?id=14593)
>
> AFAIK expression templates are the primary choice tom implement SIMD and
> matrix libraries.
> And I still have [this idea](http://dpaste.dzfl.pl/cd375ac594cf) of
> implementing a nice query language for ORMs.

I would really like this.

> The short circuit operators && and || have a special semantic and can't
> be easily, but there is & and | so it's not really required.

Not sure if this would be a problem in D but there's a library in Ruby that does exactly this. Due to the operator precedence you end up needing to wrap everything in parentheses:

Person.where{ |p| (p.age > 21) & (p.name == "Peter") | (p.name == "Foo") }

> Does anyone have a different idea how to make a nice query language?
> db.get!Person.where!(p => p.age > 21 && p.name == "Peter")

AST macros ;)

-- 
/Jacob Carlborg

On Friday, 11 September 2015 at 19:41:41 UTC, Martin Nowak wrote:
> Does anyone have a different idea how to make a nice query language? db.get!Person.where!(p => p.age > 21 && p.name == "Peter")

Do it at runtime?