Hello,

What are the default semantics for '==' on structs?

I ask this because I was forced to write opEquals on a struct to get expected behaviour. This struct is basically:

struct Lexeme {
    string tag;
    string slice;
    Ordinal index;
}

Equal Lexeme's compare unequal using default '=='. When I add:

    const bool opEquals (ref const(Lexeme) l) {
        return (
               this.tag   == l.tag
            && this.slice == l.slice
            && this.index == l.index
        );
    }

then all works fine. What do I miss?

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On Wed, 02 Feb 2011 15:55:53 +0100, spir wrote:

> Hello,
> 
> What are the default semantics for '==' on structs?
> 
> I ask this because I was forced to write opEquals on a struct to get expected behaviour. This struct is basically:
> 
> struct Lexeme {
>      string tag;
>      string slice;
>      Ordinal index;
> }
> 
> Equal Lexeme's compare unequal using default '=='. When I add:
> 
>      const bool opEquals (ref const(Lexeme) l) {
>          return (
>                 this.tag   == l.tag
>              && this.slice == l.slice
>              && this.index == l.index
>          );
>      }
> 
> then all works fine. What do I miss?

I think the compiler does a bitwise comparison in this case, meaning that it compares the arrays' pointers instead of their data.  Related bug report:

  http://d.puremagic.com/issues/show_bug.cgi?id=3433

-Lars

On 02/02/2011 04:20 PM, Lars T. Kyllingstad wrote:
> On Wed, 02 Feb 2011 15:55:53 +0100, spir wrote:
>
>> Hello,
>>
>> What are the default semantics for '==' on structs?
>>
>> I ask this because I was forced to write opEquals on a struct to get
>> expected behaviour. This struct is basically:
>>
>> struct Lexeme {
>>       string tag;
>>       string slice;
>>       Ordinal index;
>> }
>>
>> Equal Lexeme's compare unequal using default '=='. When I add:
>>
>>       const bool opEquals (ref const(Lexeme) l) {
>>           return (
>>                  this.tag   == l.tag
>>               && this.slice == l.slice
>>               && this.index == l.index
>>           );
>>       }
>>
>> then all works fine. What do I miss?
>
> I think the compiler does a bitwise comparison in this case, meaning that
> it compares the arrays' pointers instead of their data.  Related bug
> report:
>
>    http://d.puremagic.com/issues/show_bug.cgi?id=3433
>
> -Lars

Thank you, Lars.
In fact, I do not really understand what you mean. But it helped me think further :-)
Two points:

* The issue reported is about '==' on structs not using member opEquals when defined, instead performing bitwise comparison. This is not my case: Lexeme members are plain strings and an uint. They should just be compared as is. Bitwise comparison should just work fine.
Also, this issue is marked solved for dmd 2.037 (I use 2.051).

* The following works as expected:

struct Floats {float f1, f2;}
struct Strings {string s1, s2;}
struct Lexeme {
    string tag;
    string slice;
    uint index;
}

unittest {
    assert ( Floats(1.1,2.2)  == Floats(1.1,2.2) );
    assert ( Strings("a","b") == Strings("a","b") );
    assert ( Lexeme("a","b",1) == Lexeme("a","b",1) );
}

This shows, if I'm right:
1. Array (string) members are compared by value, not by ref/pointer.
2. Comparing Lexeme's works in this test case.

* Why does my app then need opEquals, just to compare member per member (see code above)?
The issue happens in a unittest. Lexemes are generated by a typical use of the module's features, then assert() compares them to expected result:
	assert ( lexeme == Lexeme(expected_data) );
I'll try to reduce the issue to isolate the key point.

Thank you for your help,
denis
-- 
_________________
vita es estrany
spir.wikidot.com

What is Ordinal defined as? If it's a uint, I get the expected results:

alias uint Ordinal;

struct Lexeme {
   string tag;
   string slice;
   Ordinal index;
}

void main()
{
    auto lex1 = Lexeme("a","b",1);
    auto lex2 = Lexeme("a","b",1);

    assert(lex1 == lex2);
    assert(lex1 == Lexeme("a","b",1));
}

Can't say much more without knowing what your app does though.

spir:

> * The issue reported is about '==' on structs not using member opEquals when
> defined, instead performing bitwise comparison. This is not my case: Lexeme
> members are plain strings and an uint. They should just be compared as is.
> Bitwise comparison should just work fine.
> Also, this issue is marked solved for dmd 2.037 (I use 2.051).

Lars is right, the == among structs is broken still:

struct Foo { string s; }
void main() {
    string s1 = "he";
    string s2 = "llo";
    string s3 = "hel";
    string s4 = "lo";
    auto f1 = Foo(s1 ~ s2);
    auto f2 = Foo(s3 ~ s4);
    assert((s1 ~ s2) == (s3 ~ s4));
    assert(f1 == f2);
}

Bye,
bearophile

> Lars is right, the == among structs is broken still:

If necessary please open a new bug report, this is an important bug.

Bye,
bearophile

On 02/02/2011 05:49 PM, Andrej Mitrovic wrote:
> What is Ordinal defined as? If it's a uint, I get the expected results:
>
> alias uint Ordinal;
>
> struct Lexeme {
>     string tag;
>     string slice;
>     Ordinal index;
> }
>
> void main()
> {
>      auto lex1 = Lexeme("a","b",1);
>      auto lex2 = Lexeme("a","b",1);
>
>      assert(lex1 == lex2);
>      assert(lex1 == Lexeme("a","b",1));
> }
>
> Can't say much more without knowing what your app does though.

Actually, its size_t. But I also have everything working fine in a test case exactly similar to yours (see other post). Dunno yet why I need to add an opEquals just comparing members individually for my unittests to pass.

I take the opportunity to say a few words about the module; case (1) it helps debugging (2) some people are interested in it.
The module is a lexing toolkit. It allows creating a lexer from a language's "morphology", then use it to scan source. Example for simple arithmetics:

        Morphology morphology = [
            [ "SPACING" ,        `[\ \t]*` ],
            [ "OPEN_GROUP" ,     `(` ],
            [ "CLOSE_GROUP" ,    `)` ],
            [ "operator" ,       `[+*-/]` ],
            [ "symbol" ,         `[a-zA-A][a-zA-A0-9]*` ],
            [ "number" ,         `[+-]?[0-9]+(\.[0-9]+)?` ],
        ];
        auto lexer = new Lexer(morphology);
        auto lexemes = lexer.lexemes(source);

As you see, each lexeme kind is defined by a string tag and a regex format.
The output is an array of lexemes holding the matched slice, wrapped in a class LexemeStream. This class mainly provides a match method:
	Lexeme* match (tag)
Match returns a pointer to the current lexeme if it is of the right kind, else null (same principle as D's builtin 'in' operator). So, one can either ignore the lexeme if all what is needed is testing the match (case of punctuation), or use the lexeme's slice (case of values).

The issue I get happens when checking that a result stream of lexemes is as expected: '==' failed. I then checked its first/last lexemes only: ditto. Thus, I started to wonder about the default semantics of '==' for structs, so that I wrote my own opEquals ==> pass for individual lexemes, pass for whole lexeme streams. Why? dunno.

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On 02/02/2011 07:05 PM, bearophile wrote:
> spir:
>
>> * The issue reported is about '==' on structs not using member opEquals when
>> defined, instead performing bitwise comparison. This is not my case: Lexeme
>> members are plain strings and an uint. They should just be compared as is.
>> Bitwise comparison should just work fine.
>> Also, this issue is marked solved for dmd 2.037 (I use 2.051).
>
> Lars is right, the == among structs is broken still:
>
> struct Foo { string s; }
> void main() {
>      string s1 = "he";
>      string s2 = "llo";
>      string s3 = "hel";
>      string s4 = "lo";
>      auto f1 = Foo(s1 ~ s2);
>      auto f2 = Foo(s3 ~ s4);
>      assert((s1 ~ s2) == (s3 ~ s4));
>      assert(f1 == f2);
> }

Thank you, this helps much. I don't get the details yet, but think some similar issue is playing a role in my case. String members of the compared Lexeme structs are not concatenated, but one of them is sliced from the scanned source.
If I dup'ed instead of slicing, this would create brand new strings; thus '==' performing bitwise comp should run fine, don't you think? I'll try in a short while.

Do you know more about why/how the above fails?

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On 02/02/2011 07:09 PM, bearophile wrote:
>> Lars is right, the == among structs is broken still:
>
> If necessary please open a new bug report, this is an important bug.

Right, i'll do it when (hopefully) I understand more about the details of why/how '==' fails in my case.

Denis
-- 
_________________
vita es estrany
spir.wikidot.com

On 02/02/2011 07:09 PM, bearophile wrote:
>> Lars is right, the == among structs is broken still:
>
> If necessary please open a new bug report, this is an important bug.
>
> Bye,
> bearophile

Right, reduced the bug cases I found to:

struct S {string s;}
unittest {
    // concat
    string s1 = "he"; string s2 = "llo";
    string s3 = "hel"; string s4 = "lo";
    assert ( S(s1 ~ s2) != S(s3 ~ s4) );
    // slice
    string s = "hello";
    assert ( S(s[1..$-1]) != S("ell") );
}

Same for array members (indeed):

struct A {int[] a;}
unittest {
    // concat
    int[] a1 = [1,2]; int[] a2 = [3];
    int[] a3 = [1]; int[] a4 = [2,3];
    assert ( A(a1 ~ a2) != A(a3 ~ a4) );
    // slice
    int[] a = [1,2,3];
    assert ( A(a[1..$-1]) != A([2]) );
}

But this is not very relevant, because plain arrays /members/ (unlike strings) seem to be compared by ref (exactly by "array struct"):

unittest {
    // string
    string s1 = "hello"; string s2 = "hello";
    assert ( S(s1) == S(s2) );
    // array (note '!=' assert)
    int[] a1 = [1,2,3]; int[] a2 = [1,2,3];
    assert ( A(a1) != A(a2) );
}

I think at opening a new bug report in a short while, with reference to issue #3433 (http://d.puremagic.com/issues/show_bug.cgi?id=3433) which was (unduly?) marked as fixed for dmd 2.037. In the meanwhile, if anyone knows about related cases of bug, or has more info, please tell.

On the other hand, the example of arrays let me doubt about correct / desirable semantics.
1. Indeed, I think string members should be compared by value.
2. But arrays are not, so should strings be compared by ref as well, if only to avoid inconsistency?
3. But then, why the already existing difference between strings & arrays?
4. Or should arrays be compared by value like string?
5. But strings are not /really/ compared by value as of now...
The current behaviour is weird. I don't how it can only happen.

Denis
-- 
_________________
vita es estrany
spir.wikidot.com