I'm working on a DSL for generating SQL queries, based loosely on Python's SQLAlchemy and Ruby's Sequel. One nice thing about the DSL is the compact syntax for specifying WHERE clauses. With some fiddling, I got it working for opEquals, a simplified example:

  foreach(network; db["networks"].each) {
    writefln("network: %s", network.name);
    foreach(host; db["hosts"].where(db.c.id == network.id)) {
      writefln("\thost: %s", host.address);
    }
  }

This works because db.c.id returns a struct which defines an opEquals which returns a "Filter" struct, rather than an int. I'm not positive that it should really be allowed by the compiler, but it works:

struct Filter { ... }
struct Column {
    Filter opEquals(T)(T rhs) { ... }
}

Then the .where call takes a filter, and uses it to output a snippet of sql like "id = 5"

However, this won't work for comparison operators like < and >, which all map to opCmp, or for != (since that's rewritten to !(a == b))

I guess I have two questions, one, am I going to shoot myself in the foot by going down this path, because it only happens to work due to the compiler being too lax? And is there interest in extending D to allow the rest of the operators to return non-boolean results? I'm thinking something like falling back to opBinary!("<"), etc, if opCmp isn't defined for a struct.

On Mon, 19 Mar 2012 01:54:27 +0100, Brian Palmer <brian+d@codekitchen.net> wrote:

> I'm working on a DSL for generating SQL queries, based loosely on Python's SQLAlchemy and Ruby's Sequel. One nice thing about the DSL is the compact syntax for specifying WHERE clauses. With some fiddling, I got it working for opEquals, a simplified example:
>
>    foreach(network; db["networks"].each) {
>      writefln("network: %s", network.name);
>      foreach(host; db["hosts"].where(db.c.id == network.id)) {
>        writefln("\thost: %s", host.address);
>      }
>    }
>
> This works because db.c.id returns a struct which defines an opEquals which returns a "Filter" struct, rather than an int. I'm not positive that it should really be allowed by the compiler, but it works:
>
> struct Filter { ... }
> struct Column {
>      Filter opEquals(T)(T rhs) { ... }
> }
>
> Then the .where call takes a filter, and uses it to output a snippet of sql like "id = 5"
>
> However, this won't work for comparison operators like < and >, which all map to opCmp, or for != (since that's rewritten to !(a == b))
>
> I guess I have two questions, one, am I going to shoot myself in the foot by going down this path, because it only happens to work due to the compiler being too lax? And is there interest in extending D to allow the rest of the operators to return non-boolean results? I'm thinking something like falling back to opBinary!("<"), etc, if opCmp isn't defined for a struct.

I for one, agree that having opEquals and opCmp return irregular types
is awesome.

Now, I see there are very good reasons to leave the behavior of all
those 'derived' operators to the compiler. Except, of course, when you
want to do something weird, like you're doing now. I also think that
the 'D creed' states that 'the simple should be easy, the complex
possible'.

I like the idea of opBinary!"<", but would perhaps argue for
opBinaryComparison!"<", to distinguish it from the 'normal' operators.

On Mon, Mar 19, 2012 at 01:54:27AM +0100, Brian Palmer wrote: [...]
> However, this won't work for comparison operators like < and >, which
> all map to opCmp, or for != (since that's rewritten to !(a == b))
> 
> I guess I have two questions, one, am I going to shoot myself in the foot by going down this path, because it only happens to work due to the compiler being too lax? And is there interest in extending D to allow the rest of the operators to return non-boolean results? I'm thinking something like falling back to opBinary!("<"), etc, if opCmp isn't defined for a struct.

IIRC, the reason D chose to go with opCmp and opEquals rather than the C++ route of operator...() for each comparison operator is so that the language can give some guarantees about the consistency of a<b, a<=b, a==b, etc..

In C++, for example, you can define operator<() and operator>() in completely arbitrary ways, which means they can be totally unrelated to each other, and return results that have nothing to do with each other. This causes inconsistency in that a<b does not necessarily imply b>a, and vice versa. Which makes for inconsistent code.

By using opCmp in D, such inconsistency is avoided, and the user is
spared the tedium of having to define operator<(), operator<=(),
operator>(), operator>=(), (and in D, you also have to add !<=, !>=, !<,
!>, etc.), ad nauseum; a single operator opCmp() takes care of all these
cases.


T

-- 
Prosperity breeds contempt, and poverty breeds consent. -- Suck.com

On 03/19/2012 02:31 AM, H. S. Teoh wrote:
> On Mon, Mar 19, 2012 at 01:54:27AM +0100, Brian Palmer wrote:
> [...]
>> However, this won't work for comparison operators like<  and>, which
>> all map to opCmp, or for != (since that's rewritten to !(a == b))
>>
>> I guess I have two questions, one, am I going to shoot myself in the
>> foot by going down this path, because it only happens to work due to
>> the compiler being too lax? And is there interest in extending D to
>> allow the rest of the operators to return non-boolean results? I'm
>> thinking something like falling back to opBinary!("<"), etc, if opCmp
>> isn't defined for a struct.
>
> IIRC, the reason D chose to go with opCmp and opEquals rather than the
> C++ route of operator...() for each comparison operator is so that the
> language can give some guarantees about the consistency of a<b, a<=b,
> a==b, etc..
>

That is fine. Iff you want it.

> In C++, for example, you can define operator<() and operator>() in
> completely arbitrary ways, which means they can be totally unrelated to
> each other, and return results that have nothing to do with each other.

I can also write a program that behaves in an arbitrary way. I usually want it to do something useful though.

> This causes inconsistency in that a<b does not necessarily imply b>a,
> and vice versa.

I suppose you meant b>=a. But even then, that is already sometimes the case for built-in types.

> Which makes for inconsistent code.
>

This is dependent on the context.


  foreach(network; db["networks"].each) {
    writefln("network: %s", network.name);
    foreach(host; db["hosts"].where(db.c.id == network.id)) {
      writefln("\thost: %s", host.address);
    }
  }

vs.


  foreach(network; db["networks"].each) {
    writefln("network: %s", network.name);
    foreach(host; db["hosts"].where(db.c.id.less(network.id))) {
      writefln("\thost: %s", host.address);
    }
  }


This inconsistency is merely syntactical though and could be fixed using an 'equals' method instead of opEquals.


> By using opCmp in D, such inconsistency is avoided, and the user is
> spared the tedium of having to define operator<(), operator<=(),
> operator>(), operator>=(), (and in D, you also have to add !<=, !>=, !<,
> !>, etc.), ad nauseum; a single operator opCmp() takes care of all these
> cases.
>
>
> T
>

The current limitations make it impossible to define (for example) a floating point type with NaN that behaves like built-in float/double/real.

On 2012-03-19 01:54, Brian Palmer wrote:
> I'm working on a DSL for generating SQL queries, based loosely on
> Python's SQLAlchemy and Ruby's Sequel. One nice thing about the DSL is
> the compact syntax for specifying WHERE clauses. With some fiddling, I
> got it working for opEquals, a simplified example:
>
> foreach(network; db["networks"].each) {
> writefln("network: %s", network.name);
> foreach(host; db["hosts"].where(db.c.id == network.id)) {
> writefln("\thost: %s", host.address);
> }
> }

I've been playing with the exact same idea:

Foo.where(x => x.name == "foo");

Generates this SQL:

select foos.* from foos where foos.name = 'foo'

I was hoping this would work for the other operators as well.

-- 
/Jacob Carlborg

On Monday, 19 March 2012 at 01:29:50 UTC, H. S. Teoh wrote:

> In C++, for example, you can define operator<() and operator>() in
> completely arbitrary ways, which means they can be totally unrelated to
> each other, and return results that have nothing to do with each other.
> This causes inconsistency in that a<b does not necessarily imply b>a,
> and vice versa. Which makes for inconsistent code.

While I totally get that concern, I've never really seen it become a real issue in any of the large C++ systems I've worked on. Maybe I've just been lucky? Ruby also allows these arbitrary operator redefinitions, and it's never been an issue in the large Ruby systems I've worked on, either. Python also allows them, though I don't have much Python experience.

In fact, a lot of the most useful DSLs in Ruby rely heavily on being able to do these overrides. I think D today is missing out on a lot of those possibilities.

I think the only people who have issues with operator definitions don't
think of them as function names, in the abstract mathematics sense.

+, < and so on are just simbolic names for some operation.

If I want to know what + does, I have to do the same as when I see Add, and
look to the definition being called.

--
Paulo

"Brian Palmer"  wrote in message news:bafsgxezdurhidyvveaz@forum.dlang.org...

On Monday, 19 March 2012 at 01:29:50 UTC, H. S. Teoh wrote:

> In C++, for example, you can define operator<() and operator>() in
> completely arbitrary ways, which means they can be totally unrelated to
> each other, and return results that have nothing to do with each other.
> This causes inconsistency in that a<b does not necessarily imply b>a,
> and vice versa. Which makes for inconsistent code.

While I totally get that concern, I've never really seen it
become a real issue in any of the large C++ systems I've worked
on. Maybe I've just been lucky? Ruby also allows these arbitrary
operator redefinitions, and it's never been an issue in the large
Ruby systems I've worked on, either. Python also allows them,
though I don't have much Python experience.

In fact, a lot of the most useful DSLs in Ruby rely heavily on
being able to do these overrides. I think D today is missing out
on a lot of those possibilities. 

On 3/19/12 10:17 AM, Brian Palmer wrote:
> On Monday, 19 March 2012 at 01:29:50 UTC, H. S. Teoh wrote:
>
>> In C++, for example, you can define operator<() and operator>() in
>> completely arbitrary ways, which means they can be totally unrelated to
>> each other, and return results that have nothing to do with each other.
>> This causes inconsistency in that a<b does not necessarily imply b>a,
>> and vice versa. Which makes for inconsistent code.
>
> While I totally get that concern, I've never really seen it become a
> real issue in any of the large C++ systems I've worked on. Maybe I've
> just been lucky? Ruby also allows these arbitrary operator
> redefinitions, and it's never been an issue in the large Ruby systems
> I've worked on, either. Python also allows them, though I don't have
> much Python experience.

I think the concern is not bugs in the implementation, but instead a lot of boilerplate text (e.g. in the C++ standard) and code (all of those mind-numbing one-liners that swap parameter order etc).

> In fact, a lot of the most useful DSLs in Ruby rely heavily on being
> able to do these overrides. I think D today is missing out on a lot of
> those possibilities.

For D, hooking built-in operators is a poor way to implement DSLs. Best is to use strings, CTFE, and mixin.


Andrei

On Mon, 19 Mar 2012 09:29:34 +0100, Timon Gehr <timon.gehr@gmx.ch> wrote:

> The current limitations make it impossible to define (for example) a floating point type with NaN that behaves like built-in float/double/real.

As it turns out, this is possible. opCmp can return a float, and
things work just fine (!<, !>=, etc). The problem appears with
opEquals.

This is the generated assembly:

fldz
fucompp
fnstsw   ax
sahf
jne      <somewhere>

So it compares the result to 0.0, copies status flags to the CPU,
then checks if any of the flags are set. If the returned value *is*
equal to 0.0, the C3 flag is set. The result is that the jump is
taken only when the returned value is *less* than 0.0. I have a
feeling this is wrong. Should I file this in BugZilla?

Anyways. With this newly-won knowledge, we can design an opEquals
that returns a float, and behaves correctly (until the above bug
[if I'm right] is fixed):

struct MyInt {
    int n;
    float opEquals(MyInt other) const {
        if (n == int.min || other.n == int.min) {
            return float.nan;
        }
        return n == other.n ? -1.0 : 1.0; // Note workaround.
        //return n - other.n; // Should work.
    }

    float opCmp(MyInt other) const {
        if (n == int.min || other.n == int.min) {
            return float.nan;
        }
        return n - other.n;
    }
}

unittest {
    assert( MyInt(int.min) != MyInt(int.min) );
    assert( MyInt(0) == MyInt(0) );
    assert( MyInt(int.min) !< MyInt(0) );
    assert( MyInt(int.min) !> MyInt(0) );
    assert( MyInt(int.min) !<= MyInt(0) );
    assert( MyInt(int.min) !>= MyInt(0) );
    assert( MyInt(0) !< MyInt(int.min) );
    assert( MyInt(0) !> MyInt(int.min) );
    assert( MyInt(0) !<= MyInt(int.min) );
    assert( MyInt(0) !>= MyInt(int.min) );
    assert( MyInt(1) > MyInt(0) );
    assert( MyInt(1) >= MyInt(0) );
    assert( MyInt(0) < MyInt(1) );
    assert( MyInt(0) <= MyInt(1) );
}

On Monday, 19 March 2012 at 15:33:33 UTC, Andrei Alexandrescu
wrote:

> For D, hooking built-in operators is a poor way to implement DSLs. Best is to use strings, CTFE, and mixin.
>
>
> Andrei

Hmm I have to say I'm not (yet) convinced. But I'll try to drink
the kool-aid and see if it converts me. I'd love some help, as
I'm still no D expert. My initial gut reaction, since I already
plan to support arbitrary SQL snippets in where clauses, is to
modify the where() method to something like:


   foreach(network; db["networks"].each) {
     writefln("network: %s", network.name);
     foreach(host; db["hosts"].where("hosts.id = ?", network.id)) {
       writefln("\thost: %s", host.address);
     }
   }

Which, in a lot of ways, it's very cool that I can
parse that at compile time. However, I see a few problems
already. First of all, where can a reasonable line be drawn
between arbitrary SQL, and snippets where we parse and actually
understand the snippet?

On one extreme, I could implement a full SQL parser and parse any
given snippet to pull out identifiers and such. Sounds painful,
but potentially awesome.

The other extreme is to just treat *all* where clauses as SQL
snippets with no extra "smarts", but if I do that, suddenly I've
lost a whole lot of functionality and my library is less useful
(it can't auto-quote identifiers, intelligently combine
conditionals and add aliases or scopes to identifiers, a whole
slew of useful functionality is gone).

A middle ground would be to just parse a simple subset of
possible snippets, and treat any we don't understand as raw SQL.
But that seems like a surprising, frustrating experience for
users of the library.

Anyway, I'm going to play around with the idea, see if I can
convince myself. Any advice on how to convince myself would be appreciated. :)