February 25, 2019
On Sunday, 24 February 2019 at 21:22:33 UTC, Walter Bright wrote:
> On 2/24/2019 1:02 PM, Manu wrote:
>> I mean like, my DIP was almost violently rejected,
>
> I thought it was clear what was needed to be done with it, and I thought you were going to rewrite it. Was I mistaken?

Absolutely no, no. Yes!

"What needs to be done" is scattered throughout forum discussions filled with incorrect assumptions, most of which were resolved needing little change, absolutely nothing that justifies a rewrite over minor amendments (the text will be 95% the same, the sentiment will be 100% the same), neither I nor Manu plan to rewrite it. Note that none of that is reflected in the official review, most (if not all) of which is wrong (and you should update it!).

What is _actually_ going to happen is this will be an item on the agenda for the Foundation meeting at dconf: you will clarify the problems you have with the DIP; those problems will be discussed, evaluated and acted upon; and the DIP process will be amended to avoid the root cause of those problems.
February 25, 2019
On 25/02/2019 10:22 AM, Walter Bright wrote:
>> I think we need to throw in the towel, C++'s const is right, and D's
>> const is just wrong, and no amount of pretending that's not true will
>> resolve the endless stream of issues.
>> Where's the DIP to migrate to C++-style const? That is the predicate
>> for basically every important development I've seen lately...
>> including this one.
> 
> If you want a DIP, write one! But I suggest the rvalue ref one first.

We have discussed this on IRC that const needs a redesign.
The problem is, without building a list of examples of where it fails we cannot really start the redesign process.

But once we do, I'm in.
February 24, 2019
On Sun, Feb 24, 2019 at 4:40 PM Walter Bright via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>
> The problem with C++ const is it only goes one level, i.e. what I call "head-const". If you pass a T to a const parameter, anything T references remains mutable. It's more of a suggestion than anything reliable or enforceable. It only works if your data structures are simple aggregates, not graphs.
>
> D's const has teeth. Nothing can be modified through T. If you're used to writing code that tweaks const data under the hood, D's const will never work for you. Yes, it means rethinking how the data and code is organized, and that can be painful. But it is how FP works. FP offers a number of advantages, and D's const offers a path into that.
>
> For example, most of DMD is written in the C++ style where functions frequently are written to both return some information *and* tweak the data structure. This does not work with const. It needs to be reorganized so that getting information about a data structure is separated from modifying the data structure. I've made such changes in a few places in DMD, and have been very pleased with the results - the code is much easier to understand.
>
> To sum up, you're quite right that you cannot write C++ style code using D const. It hast to be in a much more FP style. If you're not accustomed with FP style, this can be a difficult learning process. I know this from firsthand experience :-)

I agree with these facts, but your case-study is narrow, and you have
to stop saying "C++ style", which it really isn't.
It's very much D-style... almost all D code is written this way.
It's in conflict with too many other patterns, and they're not "C++
patterns", they're very legitimate D patterns.

Function pointers and delegates are often incompatible with const; practically any code with some sort of call-back behaviour, and anything that forms *any form* of traversible network where you'd like any part of it to const fails. I've never written a program that was a perfect tree. A small feature library maybe, but not a program that does anything interesting.

It's great that we can write FP-ish code in D, it's particularly useful for narrow, self-contained tasks; it helps me intellectually factor some potentially complex leaf-level call-trees out of the program structure, and I appreciate when libraries take that form; it helps them have a smaller footprint in the larger complex suite. But const doesn't play into that much, and if that can't interact with normal D code, which is most code, then it's just not a useful piece of language.

The proposition that everyone start writing straight-up FP code in D is unrealistic, and if they wanted that, they'd use Rust every time. People are here because they don't want to write Rust.

> For me the only real annoyance with const is I often cannot use "single assignment" style declarations with pointers:
>
> I.e.:
>
>      const int* p = &x;
>      p = &y; // error, good
>      *p = 4; // also error, not what I wished
>
> This C++ const does provide, and it's good, but it's not really worth that much.

Are you serious? You can't honestly say C++ const is worthless?
Especially in comparison to D's const, which is _actually_ almost
completely worthless.
It really doesn't make anything better, and there's a whole class of
troublesome language issues that emerge from it being defined this
way.
The way C++ defines const is such that const can be used, and you can
integrate that code with other code.

I mean it seriously where I say I've tried to defend D's const for as
long as I've used D, but I can't escape the plain and honest reality
that D's const is not useful for almost anything practical.
Even the way you describe it above is like indulging in a little bit
of fetish, and I understand that, I try that every time thinking "I'm
gonna get it right... this time for sure! What a cool guy I am!", but
that never works out beyond a very small scope. const with a narrow
scope is where it's least impactful.

Then to make matters worse, `const` is a combinatorial testing
nightmare; you write your code mostly without const (because
conventional wisdom), and then you try and call into your lib from
various contexts and it just doesn't work. You need to set-up heaps of
tests to try and prove out that your code is const-robust that are
very easy to miss otherwise.
Then someone else tries to use your code with their code which is
using const (attempting at least); I've seen lots of libraries where
it would have been possible to support const, at least to some extent,
but they just didn't because "don't use const", but the result is that
the client of that library can't use const in their own code because
the lib undermines their effort in some way.

I don't like this concept that a piece of library code 'supports' const, but that's where we are.

None of this is issue with C++ const, because it's defined in a way that's useful, but also sufficiently localised that it doesn't invade users code outwards.

const undermines its own value far too often, and with the effect I
describe above where one author's dismissing of const cascaded
outwards, it seals its own fate.
D's const rarely helps anything, it's honestly just a shiny thing that
you waste time chasing, and then you lose, and then the community
reminds you not to use const. It's incompatible with D in a lot of
fundamental ways, and if you argue that everyone should stop writing D
code and start writing something a lot more like Rust code instead,
you gotta admit that's a self-defeating argument?

If you're honest with yourself, we recognise that after 15-20 years of the sorts of recommendations you make above not taking effect in real-world D code, you must admit it's an objective failure. I haven't wanted to admit it for as long as I've defended it... but I think it's indefensible.

Also, the scope of your experiment above is not based in practical reality.
You said something to this effect "I reworked a little bit of code in
DMD, and was happy with the result. It feels simpler". I can write a
little bit of const-correct code too, but you need to get back to us
when you've covered all of DMD. This is a compiler; almost a perfect
tree, one of the most structurally simple and uniform programs
imaginable. But people don't write compilers, they write gigantic
metropolis's of libraries and glue, by 10s or 100's of authors
collaborating against a ticking clock.
If there were reasonable patterns to make D const work, we would have
discovered them by now, and there would be articles on the matter.
Apparently they don't exist though, and as such, it's usefulness is
constraints to narrow call-trees, very close to the leaf of your
application, and remains incompatible with a lot of other core
language and patterns.

On Sun, Feb 24, 2019 at 4:40 PM Walter Bright via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>
> The problem with C++ const is it only goes one level, i.e. what I call "head-const". If you pass a T to a const parameter, anything T references remains mutable. It's more of a suggestion than anything reliable or enforceable. It only works if your data structures are simple aggregates, not graphs.
>
> D's const has teeth. Nothing can be modified through T. If you're used to writing code that tweaks const data under the hood, D's const will never work for you. Yes, it means rethinking how the data and code is organized, and that can be painful. But it is how FP works. FP offers a number of advantages, and D's const offers a path into that.
>
> For example, most of DMD is written in the C++ style where functions frequently are written to both return some information *and* tweak the data structure. This does not work with const. It needs to be reorganized so that getting information about a data structure is separated from modifying the data structure. I've made such changes in a few places in DMD, and have been very pleased with the results - the code is much easier to understand.
>
> To sum up, you're quite right that you cannot write C++ style code using D const. It hast to be in a much more FP style. If you're not accustomed with FP style, this can be a difficult learning process. I know this from firsthand experience :-)
>
> For me the only real annoyance with const is I often cannot use "single assignment" style declarations with pointers:
>
> I.e.:
>
>      const int* p = &x;
>      p = &y; // error, good
>      *p = 4; // also error, not what I wished
>
> This C++ const does provide, and it's good, but it's not really worth that much.
February 24, 2019
On Sun, Feb 24, 2019 at 4:25 PM Walter Bright via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>
> Thanks for letting me know you're abandoning the rvalue ref DIP.

It's not an "rvalue ref" DIP (which I think has confused a lot of
people), it's an rvalue *by-ref* DIP.
In my head, an "rvalue ref" DIP is something completely different,
useful for implementing move semantics of mismatching types.

Are you talking about my DIP or that other thing?

> I had held off
> working on it because I didn't want to duplicate efforts; we're short-staffed
> enough as it is.

'abandoning's a strong word, but I don't have motivation to work on it right now. Please, be my guest!
February 24, 2019
I agree with your point that C++ const can be used in a lot more places than D const. Absolutely true.

Missing from the post, however, is an explanation of what value C++ const semantics have. How does it:

1. make code easier to understand?
2. prevent common programming bugs?
3. help with multithreaded coding problems?
4. improve code generation?

I know technically what it does (after all, I implemented it), but its value escapes me.
February 24, 2019
Your DIP, and nobody else is going to do it, so it falls to me.

BTW, everyone should expect the DIP process to be brutal. It has to be, we're long past the stage where we can allow misshapen stuff to get into the language. Nobody was very kind with my bottom type DIP :-)

You should read the dismissal of mine and Andrei's "static if" proposal for C++! (Ironically, it was later adopted after someone else redid it, after removing one of its keystone features.)
February 24, 2019
On Sun, Feb 24, 2019 at 6:35 PM Walter Bright via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>
> I agree with your point that C++ const can be used in a lot more places than D const. Absolutely true.
>
> Missing from the post, however, is an explanation of what value C++ const semantics have. How does it:
>
> 1. make code easier to understand?

const code is self documenting and protective against modification by issuing the user helpful error messages.


> 2. prevent common programming bugs?

You can't modify const data, for instance, a copy constructor can't freely modify the source value...


> 3. help with multithreaded coding problems?

This is a different conversation about `immutable` and `shared`. `const` doesn't say anything about D's decisions relating to thread-locality by default, which obviously still applies.

Maybe you're trying to argue that a const object which contains an escape-pointer to mutable data may lead to races? But that's not the case, because all data is thread-local in D, so there's no races on the mutable data either way unless it's `shared`... and then we need to refer back to the thread I created months ago where `shared` is useless and broken, and we REALLY need to fix that. (that is; `shared` must have NO READ OR WRITE ACCESS to data members, only shared methods, otherwise it's completely hollow)


> 4. improve code generation?

Not a lot. But this is a red-herring; D's const won't improve code
generation where it's not present in the code.
Contrary to C++, D has a much higher probability of seeing the whole
AST and not encountering opaque extern barriers, which means it would
be relatively easy for D to recognise that the const object contains
no pointers to mutable data (assessed recursively), and then enable
any such optimisations that const offers to D today.


> I know technically what it does (after all, I implemented it), but its value
> escapes me.

I mean, you speak as if `const` is a synonym for `mutable` in C++... const things are const. It is however possible that they contain a pointer that leads out of the const data back into the mutable world, and that's *desirable* in a whole lot of circumstances. Take that away, and we arrive where we are in D.

It's also easy to NOT have pointers to mutable data escaping const
objects; make them const too!
If you want to implement a semantic where the const-ness of a member
tracks the const-ness of the owner, maybe we can apply `inout` to
behave that way.
Assuming we apply rules similar to C++, it looks like:

  const(S) const_s; // const instance of S
  struct S
  {
    int* a; // becomes `int const(*)`
    const(int)* b; // const(int*)
    inout(int)* c; // becomes const(int*) (or immtable(int*), etc)
  }

Alternatively, if const were spec-ed similar to C++ const, it would be very easy to implement TransitiveConst!T as a tool. By any of these means, could deploy it deliberately instead of unwillingly.
February 24, 2019
On 2/24/2019 8:04 PM, Manu wrote:
>> 1. make code easier to understand?
> 
> const code is self documenting and protective against modification by
> issuing the user helpful error messages.

I've had many people tell me they mean transitive const when they use const in C++. That is not self documentation. Nor is it very helpful. The very fact that const code compiles in C++ and surprises people when it doesn't compile in D shows the misunderstanding it engenders in C++ code. People infer that it is transitive when it is not.

Really, of what value is it to know that only the head can't be changed, with no information about the rest?

An interesting manifestation of this uselessness in C++ is the notion of "logical const", where a supposedly "const" value is lazily set to a value upon first use. I.e. it isn't const, it's just pretend const.


>> 2. prevent common programming bugs?
> 
> You can't modify const data, for instance,

I know technically what it does. But that wasn't my question.


> a copy constructor can't freely modify the source value...

C++ copy constructors are not required to declare the source as const:

"A non-template constructor for class X is a copy constructor if its first parameter is of type X&, const X&, volatile X& or const volatile X&, and either there are no other parameters or else all other parameters have default arguments"
 -- CPP98.12.8

Again, I know technically what it does. Not the value of it. I used C for a decade before const was introduced. I tried using it, and discovered that head const added nothing much of any value and I stopped bothering with it.


>> 3. help with multithreaded coding problems?
> 
> This is a different conversation about `immutable` and `shared`.
> `const` doesn't say anything about D's decisions relating to
> thread-locality by default, which obviously still applies.

'immutable' data is implicitly shareable. Const allows the same functions to work on both shared immutable and non-shared mutable data. C++ has no way to say a function can work on both kinds of data, or even how to describe such data.


>> 4. improve code generation?
> Not a lot.

Actually, not at all. C++ const is useless for code generation because:

1. people cast away const (yes, they do!). I learned this the hard way and had to back out those optimizations.

2. other references may mutate it.

Because of (2) it is similarly useless in D, however, it allows the same functions to work on immutable and mutable data, and that does have value. (The optimizer can make use of immutable.)


> I mean, you speak as if `const` is a synonym for `mutable` in C++...
> const things are const.

Not only are they not (legal to cast it away!), they can be mutated by other, non-const pointers. (There's no way to specify "pointer to immutable" in C++.)
Furthermore, const tells you nothing about the rest of the data structure. C++ const is all fine if your data structure fits entirely in one struct. It's rather meaningless for more complex objects. You cannot write generic code:

    void doSomething(T)(const T x) {...}

and rely on it to not modify T if T is a non-trivial object.


> Alternatively, if const were spec-ed similar to C++ const, it would be
> very easy to implement TransitiveConst!T as a tool.

You could specify it, but you couldn't use it. (There'd be all kinds of implicit conversion problems.) There's good reason why const is treated specially for overload resolution and implicit conversion. I.e.:

    void foo(const T const * const * const * p);
             const T       * const * const * q;
    foo(q); // oops!

----

One thing I quite agree with you on, though, if you really want to modify non-trivial const objects, then D's const isn't for you :-) If you don't believe the rewards of carefully designing the data structures and functions that operate on them so they can be const is worth it, then D's const isn't for you. For me, I think it is worth it. And yes, it ain't easy, as old habits die hard.
February 25, 2019
On Monday, 25 February 2019 at 02:56:13 UTC, Walter Bright wrote:
> Your DIP, and nobody else is going to do it, so it falls to me.

It will be reviewed at Dconf, please make sure you have an _accurate_ summary of your criticisms of the DIP ready for then.

> BTW, everyone should expect the DIP process to be brutal.

No! It should be thorough, that is a critical distinction.

> It has to be, we're long past the stage where we can allow misshapen stuff to get into the language. Nobody was very kind with my bottom type DIP :-)

Perhaps because you responded to _precisely none_ of the feedback and wasted everybody's time?

> You should read the dismissal of mine and Andrei's "static if" proposal for C++! (Ironically, it was later adopted after someone else redid it, after removing one of its keystone features.)

I have, it is hysterical in all senses to the word. I don't see how that relates to any of this.
February 25, 2019
On Monday, 25 February 2019 at 00:38:02 UTC, Walter Bright wrote:
> The problem with C++ const is it only goes one level, i.e. what I call "head-const". If you pass a T to a const parameter, anything T references remains mutable. It's more of a suggestion than anything reliable or enforceable. It only works if your data structures are simple aggregates, not graphs.
>
> D's const has teeth. Nothing can be modified through T. If you're used to writing code that tweaks const data under the hood, D's const will never work for you. Yes, it means rethinking how the data and code is organized, and that can be painful. But it is how FP works. FP offers a number of advantages, and D's const offers a path into that.

I have no idea what people are talking about when they mention on this forum that D's const is useless. Nearly every function parameter in my code is `in`. Nearly every variable declaration uses `const` instead of `auto`, the main exception being when a function is constructing a value to return. Most member functions I write are const or inout. I use const in D as I do in C++: pretty much everywhere. Nearly every single time a function parameter in my code isn't const is when it's an input range.

I happened to be working on dpp at the time I wrote this:

$ ag 'auto ' source | wc -l
67
$ ag 'const ' source | wc -l
204

The *only* problem I have with const in D is that const values can't be copied, which is silly. I'd expect DIP1018 to fix that.