On Friday, 20 May 2016 at 18:42:44 UTC, Ali Çehreli wrote:
> On 05/20/2016 10:28 AM, Namespace wrote:
> > On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis
> wrote:
> >> On Thursday, May 19, 2016 20:44:54 ciechowoj via
> Digitalmars-d-learn
> >> wrote:
> >>> Is there D equivalent of C++'s mutable keyword? Like the
> one that
> >>> allows to modify a field of struct from constant method. Or
> some
> >>> alternative solution?
> >>
> >> No. D's const and immutable provide no backdoors.
> >
> > But you can cheat:
> > ----
> > int* _id;
> >
> > struct A
> > {
> >      int id = 0;
> >
> >      this(int id)
> >      {
> >          this.id = id;
> >          _id = &this.id;
>
> Point taken but considering that D structs are freely movable value types, I don't think that's a valid D program. The spec does ban self-referencing structs but I think it also bans the code above in spirit. :)
>
> >      }
> >
> >      void change() const
> >      {
> >          (*_id)++;
> >      }
> > }
> >
> > void main() {
> >      import std.stdio;
> >
> >      A a = A(42);
> >      a.change();
> >
> >      writeln(a.id);
> > }
> > ----
>
> Ali

Also works for classes. ;) It's valid as far as I can tell.

On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:
>
> No. D's const and immutable provide no backdoors. Rather, they provide strong guarantees. So, if a variable is const, then it cannot be mutated (even internally) except via a mutable reference to the same data.

The "even internally" is incorrect, and I think needs fixing: currently you can synchronize on an immutable/const object, but the synchronize implementation will write to __monitor inside the object (the field after vptr) and gone is the guarantee. This means that you cannot put immutable objects in readonly data, for example. We had to make TypeInfo objects mutable in LDC exactly because of optimization errors due to this const guarantee breakage.
I still have to write up a detailed bug report about it...

- Johan

On Friday, May 20, 2016 18:41:04 ciechowoj via Digitalmars-d-learn wrote:
> On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:
> > On Thursday, May 19, 2016 20:44:54 ciechowoj via
> >
> > Digitalmars-d-learn wrote:
> >> Is there D equivalent of C++'s mutable keyword? Like the one that allows to modify a field of struct from constant method. Or some alternative solution?
> >
> > Now, if your functions aren't pure, you can put state outside of the object itself and have a const member function access and mutate that external state, but that's not exactly great for encapsulation, and then you can't use that function in pure code. But it's the closest thing to a backdoor from const that exists in D, because const is set up so that it's actually const and not just const until the implementation decides to mutate it anyway. Whether that's better or worse than C++'s const depends on what you're trying to do, but the reality of the matter is that D's const is ultimately very different from C++'s const because of how restrictive it is. You get better guarantees but can't use it anywhere near as much precisely because of the restrictions that are required to provide those guarantees.
> >
> > - Jonathan M Davis
>
> Thanks for explanation. It makes implementing things like shared_ptr somewhat troublesome when they are supposed to work in const environment. Isn't there a way to escape a pure environment (like trusted for safe)? Or/and would modifying an external state from pure function be an undefined behavior?

There is no backdoor for pure. You can cast it way via function pointer just like you can cast away const, but it's then undefined behavior if you then do anything in that function which wouldn't be legal in a pure function.

If you want something that's ref-counted and works in pure code, const will _not_ work, because you can't legally alter the ref-count.

Now, Walter is currently working on language enhancements to add ref-counting capabilities to the language, and presumably, that will have a way around the problem for this particular use case, but when dealing with const, you need to write your types such that they can work without being able to mutate anything as long as they're const. And if your design requires something like the mutable keyword, then you pretty much have to give up on const. If you abandon pure, you can get around it to some extent, but you're almost certainly better off if you just abandon trying to use const if you can't refactor your code such that it doesn't need the mutable keyword while still playing nicely with pure.

The net result is that while const can be use quite heavily in some types of D code, it's often the case that it can't be used at all. This is particularly true if you use templates heavily, since if they're templated on type, you can't guarantee that the types being used play nicely with const (at least not without having the template constraints require it) and therefore usually can't use it. And ranges pretty much don't work with const at all right now, since they can't be iterated over if they're const, and there's not currently a way to get a tail-const slice of a range (even though it works for arrays), so range-heavy code will definitely abandon const.

So, while const is useful, its use is ultimately fairly limited.

- Jonathan M Davis

On Friday, May 20, 2016 18:23:26 Jack Applegame via Digitalmars-d-learn wrote:
> On Friday, 20 May 2016 at 17:28:55 UTC, Namespace wrote:
> > But you can cheat:
> You can just cast const away:
> struct A {
>   int id = 0;
>
>   this(int id) {
>       this.id = id;
>   }
>
>   void change() const {
>       (cast() id)++;
>   }
> }
>
> void main() {
>   import std.stdio;
>
>   A a = A(42);
>   a.change();
>
>   writeln(a.id);
> }

Casting away const and mutating is undefined behavior in D. No D program should ever do it.

- Jonathan M Davis

On Friday, May 20, 2016 19:33:49 Johan Engelen via Digitalmars-d-learn wrote:
> On Thursday, 19 May 2016 at 23:21:14 UTC, Jonathan M Davis wrote:
> > No. D's const and immutable provide no backdoors. Rather, they provide strong guarantees. So, if a variable is const, then it cannot be mutated (even internally) except via a mutable reference to the same data.
>
> The "even internally" is incorrect, and I think needs fixing:
> currently you can synchronize on an immutable/const object, but
> the synchronize implementation will write to __monitor inside the
> object (the field after vptr) and gone is the guarantee. This
> means that you cannot put immutable objects in readonly data, for
> example. We had to make TypeInfo objects mutable in LDC exactly
> because of optimization errors due to this const guarantee
> breakage.
> I still have to write up a detailed bug report about it...

The whole monitor thing is was a mistake, and I believe that it's pretty much been agreed upon that we should rip it out, but no one seems to have actually finished the job yet. IIRC, the proposal was to replace it with an attribute such that if you used the attribute, then you get more or less what we have now, but if you don't, then the class in question has no monitor.

But whenever you have a monitor, you're in pretty much the same boat as with something like having ref-counted in the language like Andrei wants added, and you use it with immutable. If you want it to work, then the ref-count needs to be outside of the object rather than in it, since the memory of the object has to be treated as immutable. But if that almost certainly means putting the ref-count (or monitor) next to the object in memory, which would likely still prevent putting the object in ROM even though the object itself would then be properly treated as immutable. But at least that would be opt-in, whereas the monitor mess as it is now is not.

- Jonathan M Davis

On Friday, 20 May 2016 at 20:45:05 UTC, Jonathan M Davis wrote:
> If you want something that's ref-counted and works in pure code, const will _not_ work, because you can't legally alter the ref-count.

What about something like this (ignoring multi-threading issues):

struct RefCountPool {
   size_t acquireIndex();
   void releaseIndex(size_t index);

   size_t* accessRefCounter(size_t index);
}

RefCountPool refCountPool;

struct SharedPtr
{
   size_t index;
   void* ptr;

   SharedPtr(void* ptr) {
       this.ptr = ptr;
       index = refCountPool.acquireIndex();
   }

   // more methods, counter manipulation through accessRefCounter
}

Is is still legal? Would it breach @pure requirements (I believe so...)? After all it doesn't differs much from having a pointer instead of index.

On Friday, May 20, 2016 21:35:27 ciechowoj via Digitalmars-d-learn wrote:
> On Friday, 20 May 2016 at 20:45:05 UTC, Jonathan M Davis wrote:
> > If you want something that's ref-counted and works in pure code, const will _not_ work, because you can't legally alter the ref-count.
>
> What about something like this (ignoring multi-threading issues):
>
> struct RefCountPool {
>     size_t acquireIndex();
>     void releaseIndex(size_t index);
>
>     size_t* accessRefCounter(size_t index);
> }
>
> RefCountPool refCountPool;
>
> struct SharedPtr
> {
>     size_t index;
>     void* ptr;
>
>     SharedPtr(void* ptr) {
>         this.ptr = ptr;
>         index = refCountPool.acquireIndex();
>     }
>
>     // more methods, counter manipulation through accessRefCounter
> }
>
> Is is still legal? Would it breach @pure requirements (I believe so...)? After all it doesn't differs much from having a pointer instead of index.

Well, if you actually tried marking functions with pure, you'd see pretty fast that this won't work with pure. A function that's marked with pure cannot access any global, mutable state. It can only access what's passed to it (though in the case of a member function, that includes the this pointer/reference). So, your refCountPool will not be accessible from any pure functions.

You can think of pure as @noglobal, because it can't access global variables (unless they're constants). That's it's only restriction, but it's enough to make it so that the only way that you'd have a backdoor out of const in a pure, const member function is if you passed a mutable reference to the object as one of the function arguments.

At this point, if you want ref-counting, you give up on const. They simply do not go together. The same goes for stuff like caching or lazy initialization.

Sure, you can get around const to some extent by giving up on pure, but that only works because you're putting the state of the object outside of the object itself, which is usally a bad idea. It also makes it so that const seems like a lie, since the state of the object isn't really const, since it's not actually in the object.

The standard library already has std.typecons.RefCounted, if you want to ref-count anything other than classes, but it really doesn't work with const and fundamentally can't. In order to have const ref-counting, we're going to need language support. D does not and likely will never have any form of "logical" const. If it's const, it's const. Either that fits with what you're doing, and you can use const, or it doesn't, and you can't.

- Jonathan M Davis

On Saturday, 21 May 2016 at 00:39:21 UTC, Jonathan M Davis wrote:
> Well, if you actually tried marking functions with pure, you'd see pretty fast that this won't work with pure. A function that's marked with pure cannot access any global, mutable state. It can only access what's passed to it (though in the case of a member function, that includes the this pointer/reference). So, your refCountPool will not be accessible from any pure functions.

Well, I do not have much experience with @pure in D. But I believe that as far as I'm not using refCounter-modifying methods of sharedPtr (constructors, assignement, destructor) it should work. Maybe I'll try it in some future.

> You can think of pure as @noglobal, because it can't access global variables (unless they're constants). That's it's only restriction, but it's enough to make it so that the only way that you'd have a backdoor out of const in a pure, const member function is if you passed a mutable reference to the object as one of the function arguments.
>
> At this point, if you want ref-counting, you give up on const. They simply do not go together. The same goes for stuff like caching or lazy initialization.
>
> Sure, you can get around const to some extent by giving up on pure, but that only works because you're putting the state of the object outside of the object itself, which is usally a bad idea. It also makes it so that const seems like a lie, since the state of the object isn't really const, since it's not actually in the object.

I didn't tried the proposed solution, but if this is only ideological problem and not a technical one, I would be good with such a solution. On one side the memory reachable from object isn't modified on the other side the object feels like a const for the end-used. I mean I miss a logical const from C++ : ).

> The standard library already has std.typecons.RefCounted, if you want to ref-count anything other than classes, but it really doesn't work with const and fundamentally can't. In order to have const ref-counting, we're going to need language support. D does not and likely will never have any form of "logical" const. If it's const, it's const. Either that fits with what you're doing, and you can use const, or it doesn't, and you can't.

I'm currently doing that, but std.typecons.RefCounted is uncomfortable to use. Probably for reasons mentioned above.

On Saturday, May 21, 2016 07:00:43 ciechowoj via Digitalmars-d-learn wrote:
> On Saturday, 21 May 2016 at 00:39:21 UTC, Jonathan M Davis wrote:
> > Well, if you actually tried marking functions with pure, you'd see pretty fast that this won't work with pure. A function that's marked with pure cannot access any global, mutable state. It can only access what's passed to it (though in the case of a member function, that includes the this pointer/reference). So, your refCountPool will not be accessible from any pure functions.
>
> Well, I do not have much experience with @pure in D. But I believe that as far as I'm not using refCounter-modifying methods of sharedPtr (constructors, assignement, destructor) it should work. Maybe I'll try it in some future.

The problem is the global variable you were using. If the ref-counting is completely internal, then it can be pure, but it can't access global variables. I'd suggest that you read this article:

http://klickverbot.at/blog/2012/05/purity-in-d/

> > You can think of pure as @noglobal, because it can't access global variables (unless they're constants). That's it's only restriction, but it's enough to make it so that the only way that you'd have a backdoor out of const in a pure, const member function is if you passed a mutable reference to the object as one of the function arguments.
> >
> > At this point, if you want ref-counting, you give up on const. They simply do not go together. The same goes for stuff like caching or lazy initialization.
> >
> > Sure, you can get around const to some extent by giving up on pure, but that only works because you're putting the state of the object outside of the object itself, which is usally a bad idea. It also makes it so that const seems like a lie, since the state of the object isn't really const, since it's not actually in the object.
>
> I didn't tried the proposed solution, but if this is only ideological problem and not a technical one, I would be good with such a solution. On one side the memory reachable from object isn't modified on the other side the object feels like a const for the end-used. I mean I miss a logical const from C++ : ).

Well, any function that isn't marked with pure is completely unusable in pure code, and it's generally best practice in D to use pure as much as possible. It makes it clear that the functions in question can't access anything that you don't give them, it allows for compiler optimizations in some cases, and it also makes it easier to do stuff like construct immutable objects, since if the compiler can guarantee that the return value of a pure function is unique, it can be implicitly converted to immutable (it might also be implicitly convertible to shared - I don't recall for sure). And of course, if you're interacting with D code that requires pure, then your code is going to need to work with pure.

And since const is already borderline useless for heavily templated code (which anything range-based tends to be), contorting your code to favor const over pure is ill-advised.

>From what I've seen, pretty much everyone who wants to do stuff like
ref-counting or lazy initialization abandons trying to use const. So, if you need "mutable," I'd strongly encourage you to do the same rather than trying to put state in global variables just to have "logical" const, but it's up to you.

> > The standard library already has std.typecons.RefCounted, if you want to ref-count anything other than classes, but it really doesn't work with const and fundamentally can't. In order to have const ref-counting, we're going to need language support. D does not and likely will never have any form of "logical" const. If it's const, it's const. Either that fits with what you're doing, and you can use const, or it doesn't, and you can't.
>
> I'm currently doing that, but std.typecons.RefCounted is uncomfortable to use. Probably for reasons mentioned above.

Well, fundamentally, ref-counting and const don't mix. C++ allows it by basically making it so that C++'s const guarantees nothing. All it really does is prevent accidental mutation and function as documentation that a member function isn't supposed to modify its object's state. But nothing is really guaranteed. D's const provides actual guarantees, but the result is that it's usuable in far fewer cases, because a lot of code simply doesn't work if it's forced to actually be const.

So, RefCounted will work on some level, but it really isn't going to work with const, and no library solution even _can_ play nicely with const - not without throwing pure out the window anyway, and the really isn't worth it, especially since we keep getting more stuff added to the language that's able to take advantage of pure.

But Walter is working on a language solution for ref-counting, and with that, it should become possible for ref-counting to play nicely with const.

Still, I get the impression that most D programmers use const pretty limitedly, because trying to use it like you would in C++ simply does not play nicely with D - especially idiomatic D, which is generally range-based.

- Jonathan M Davis

On Friday, 20 May 2016 at 20:46:18 UTC, Jonathan M Davis wrote:
> Casting away const and mutating is undefined behavior in D. No D program should ever do it.
Really? Mutating immutable is UB too, but look at std.typecons.Rebindable.