On Thursday, May 16, 2019 8:41:09 AM MDT Steven Schveighoffer via Digitalmars-d wrote:
> On 5/16/19 3:15 PM, Jonathan M Davis wrote:
> > On Thursday, May 16, 2019 7:53:45 AM MDT Steven Schveighoffer via
> >
> > Digitalmars-d wrote:
> >> On 5/15/19 11:51 PM, Andrei Alexandrescu wrote:
> >>> OK! I finally remembered why we went with the design "any two
> >>> ProtoObjects must be comparable". We should have a way to document
> >>> this
> >>> stuff, I'd internalized it so much I'd forgotten the reason.
> >>>
> >>> It has to do with the way D implements built-in [hash]tables. They use
> >>> indirect calls via TypeInfo classes that essentially require the
> >>> existence of equality and hashing in class objects as virtual
> >>> functions
> >>> with fixed signatures.
> >>
> >> Yes, but that can be redone. Basically the TypeInfo for an object uses Object.toHash expecting the base to be object (and BTW is calling this on a specifically non-CONST object, even though keys are supposed to be const!)
> >
> > Really, they need to be immtable if you want to guarantee that they
> > don't
> > change. Having them be const doesn't really buy you anything other than
> > ensuring that the AA implementation doesn't modify it (and that sort of
> > thing in druntime has a nasty habit of casting to get around such
> > things).
> That's beside the point. You shouldn't call a non-const function on something that is const *or* immutable. Which is what druntime is currently doing.

Well, as I said, druntime has a nasty habit of using casts to work around problems. The type safety of druntime code in general is likely suspect as a result. And that really should be fixed.

> For most intents and purposes, I think it should be fine to use non-const keys, as long as the keys are not modified while they are used as keys. You could, for instance, have a key that has some mutable data that doesn't affect the hash/equality. The language shouldn't impose restrictions that don't make sense for some cases, it just reduces the code you can write, even if that code would be perfectly valid.

I don't really care whether we require that the keys be immutable. My point was that requiring const makes no sense, because it doesn't guarantee anything that matters to the AA implementation. Either we require immutable, or we don't care and let the user shoot themselves in the foot if they use a mutable object and then mutate it, because requiring immutable is too restrictive. Either way, I completely agree that druntime shouldn't be casting away const.

- Jonathan M Davis

On Thu, May 16, 2019 at 09:11:48AM -0600, Jonathan M Davis via Digitalmars-d wrote:
> On Thursday, May 16, 2019 8:41:09 AM MDT Steven Schveighoffer via Digitalmars-d wrote:
[...]
> > For most intents and purposes, I think it should be fine to use non-const keys, as long as the keys are not modified while they are used as keys. You could, for instance, have a key that has some mutable data that doesn't affect the hash/equality. The language shouldn't impose restrictions that don't make sense for some cases, it just reduces the code you can write, even if that code would be perfectly valid.

That's a good point.  So really, what matters is that once a key is inserted into the AA, there must be no changes visible through .opEquals (and possibly .opCmp) and .toHash.

Although TBH, that smells like bending over backwards to do something that's ill-advised to begin with. If you're writing a custom type as an AA key, why would you deliberately write it so that it has external mutators and extra state?  If only part of the object is actually the effective key, then it should be factored out into an immutable object and *that* should be used as key instead of the entire object.


> I don't really care whether we require that the keys be immutable. My point was that requiring const makes no sense, because it doesn't guarantee anything that matters to the AA implementation. Either we require immutable, or we don't care and let the user shoot themselves in the foot if they use a mutable object and then mutate it, because requiring immutable is too restrictive.

At the very least I'd appreciate a warning that a non-immutable key is being used to create AA entries.  There have been a number of bugs filed in the past, caused by "ghost" entries in the AA that exist when you iterate but otherwise are inaccessible, because their keys have mutated and no longer match the stored hash value in the AA.


> Either way, I completely agree that druntime shouldn't be casting away const.
[...]

Yeah, that has the potential of blowing up if we ever implement compile-time AA literals that get put into ROM. (Very distant possibility, though, granted.)


T

-- 
Latin's a dead language, as dead as can be; it killed off all the Romans, and now it's killing me! -- Schoolboy

On 2019-05-16 00:51, Andrei Alexandrescu wrote:

> So if we go with a statically-driven solution we lose hashtables keyed on any class not derived from Object. That would be awkward but hopefully acceptable until we have good containers (which are becoming possible only now with copy ctors and the upcoming __RefCounted type in druntime).

Can we use both?

-- 
/Jacob Carlborg

Any thoughts about the spaceship operator design as accepted for C++20?

Not invented here, but there seems to be room for improvement:

https://stackoverflow.com/q/47485803

On Wednesday, 15 May 2019 at 00:32:32 UTC, Andrei Alexandrescu wrote:
> ---
> import std.stdio;
>
> struct OverengineeredCmpResult {
>     enum R { lt, eq, gt, ionno }
>     private R payload;
>     int opCmp(int alwaysZero) {
>         writeln("b");
>         return 0;
>     }
> }
>
> struct A {
>     OverengineeredCmpResult opCmp(A rhs) {
>         writeln("a");
>         return OverengineeredCmpResult(OverengineeredCmpResult.R.ionno);
>     }
> }
>
> void main() {
>     A a, b;
>     if (a < b) {}
> }
> ---

On Thu, May 16, 2019 at 05:31:19PM +0000, Noob via Digitalmars-d wrote:
> Any thoughts about the spaceship operator design as accepted for C++20?
> 
> Not invented here, but there seems to be room for improvement:
> 
> https://stackoverflow.com/q/47485803
[...]

Actually, D used to *have* the spaceship operator, and even several of its friends like !<=, !>=, !<>=, and several others.

We deprecated them, and about a year or three ago finally got rid of them.  Good riddance, I say.  Nobody uses them anyway, and nobody cares. They are just a dead weight in the language, beautiful in theory but never actually used in practice.

Don't bring those darned aliens back. ;-)


T

-- 
Trying to define yourself is like trying to bite your own teeth. -- Alan Watts

On 5/16/19 9:53 AM, Steven Schveighoffer wrote:
> On 5/15/19 11:51 PM, Andrei Alexandrescu wrote:
>> OK! I finally remembered why we went with the design "any two ProtoObjects must be comparable". We should have a way to document this stuff, I'd internalized it so much I'd forgotten the reason.
>>
>> It has to do with the way D implements built-in [hash]tables. They use indirect calls via TypeInfo classes that essentially require the existence of equality and hashing in class objects as virtual functions with fixed signatures.
> 
> Yes, but that can be redone. Basically the TypeInfo for an object uses Object.toHash expecting the base to be object (and BTW is calling this on a specifically non-CONST object, even though keys are supposed to be const!)
> 
> The TypeInfo_ProtoObject (which would be what we are talking about here), can use the same technique as structs -- the compiler writes an xtoHash function for it, and puts it in the TypeInfo as a function pointer.
> 
>> My thinking is, built-in hashtables are overdue for improvement anyway so keeping compatibility with them isn't awfully motivating.
> 
> We need to redo builtin hash tables. As I understand it, the biggest hurdle is this feature of builtin hashtables which isn't exactly easy to do with our current operator overloading scheme:
> 
> int[int[int]] nestedAA;
> 
> nestedAA[1][2] = 3; // magically knows to create the intermediate AA if it doesn't exist.

Good reminder. Wasn't also the magical rebinding of immutable keys?

string[immutable int[]] t;
foreach (k, v; t) { ... }

Or/also something having to do with fixed-size arrays as keys?

string[immutable int[4]] t;
foreach (k, v; t) { ... }

We should document this institutional memory in a wiki somewhere.

On 5/16/19 5:24 PM, H. S. Teoh wrote:
> On Thu, May 16, 2019 at 09:11:48AM -0600, Jonathan M Davis via Digitalmars-d wrote:
>> On Thursday, May 16, 2019 8:41:09 AM MDT Steven Schveighoffer via
>> Digitalmars-d wrote:
> [...]
>>> For most intents and purposes, I think it should be fine to use
>>> non-const keys, as long as the keys are not modified while they are
>>> used as keys. You could, for instance, have a key that has some
>>> mutable data that doesn't affect the hash/equality. The language
>>> shouldn't impose restrictions that don't make sense for some cases,
>>> it just reduces the code you can write, even if that code would be
>>> perfectly valid.
> 
> That's a good point.  So really, what matters is that once a key is
> inserted into the AA, there must be no changes visible through .opEquals
> (and possibly .opCmp) and .toHash.

Since we went to non-tree-based buckets, we have not required opCmp anymore. That requirement is not going to come back.

> Although TBH, that smells like bending over backwards to do something
> that's ill-advised to begin with. If you're writing a custom type as an
> AA key, why would you deliberately write it so that it has external
> mutators and extra state?  If only part of the object is actually the
> effective key, then it should be factored out into an immutable object
> and *that* should be used as key instead of the entire object.

Sometimes the custom type isn't really meant ONLY to be a hash key. Sometimes you use an object as one thing somewhere, and a key somewhere else. Having to painstakingly separate the immutable and mutable parts makes things unpleasant. Especially if the immutable parts are semantically immutable, and not actually marked immutable (for obvious reasons). In other words, we live in the real world ;)

I'm pretty sure in the past I had a good use case for this, but it was way in the past (like over 10 years ago), and it was with Tango. And dcollections.

In any case, I agree requiring const does zero to guarantee anything. My solution just differs from yours in that I think you should not require any modifiers to make it work. Don't restrict the developer, you don't know what he is doing, or why he needs to do it that way. Sometimes the contract between the library and user code cannot be expressed in the type system.

-Steve

On Thursday, 16 May 2019 at 14:41:09 UTC, Steven Schveighoffer wrote:

> That's beside the point. You shouldn't call a non-const function on something that is const *or* immutable. Which is what druntime is currently doing.

I'd be interested in knowing more about this.  Would you be willing to show me the specific function where this is occurring?

I've been working over the past few years to learn how to convert many runtime hooks to templates, and in that process I've discovered that the runtime hooks are not abiding by their contracts.

The runtime hooks are added to the binary in e2ir.d which is after the semantic phase, so purity, safety, throwability, etc... is not enforced. For example, code in a `pure` function is lowered to an impure runtime hook, and the compiler doesn't catch it. Is that what's going on here as well?

Mike