September 12, 2007
BTW, Janice, I appreciate your thoughts on this. You're obviously an experienced programmer.

Janice Caron wrote:
> But you could guarantee that /non-private/ data will not change. And since non-private data is never seen outside the file in which it is declared, I don't understand why this is a problem.

Because multithreaded programming doesn't start working if the interface hides the changing variables. If it did, parallel programming would be easy. Logical constness is *still* changing state, and since it is, all the boogeymen of synchronization, race conditions, deadlocks, etc., are all there.

Logical constness hides the state change from the awareness of the programmer, but it's still changing state.

>     and even worse, I can't even tell this is happening. "I" here meaning
>     the compiler, and our hapless code auditor.
> Providing you restrict "logical constness" to private variables, is that really true? Inside the module, the compiler knows everything. Outside the module, the private variables are irrelevant anyway as they can never be accessed in any way.

Being in a module doesn't prevent another thread from using the module's interface and thereby changing the state.


> Functional programming isn't possible anyway unless the function can also guarantee that it won't modify /global/ variables, and I see no way of specifying that in the prototype.

The upcoming 'pure' storage class for a function will resolve that.


> Even if you could, it's more complicated than that. A function might modify global variables and yet still be threadsafe - providing it uses mutexes to ensure it has exclusive access to the shared variables. A good example of this is writef(). The writef() function writes to standard output, and so /must/ modify some global state - but it's thread-safe because file access is all mutex locked.

Yes, but that requires programmer discipline, which is unreliable and scales poorly. Very few programmers are able to successfully write such code. With FP programming, most programmers will be able to.


> Moreover, multithreaded programming might /require/ you to lock a mutex, do something, then unlock said mutex. The mutex itself needs to be modifiable - that is, "logically const".
> 
> More thought needs to be put into that one.

It has come up before. The answer is to not declare logically const things as being const, because they aren't const. Logical constness belongs as a comment because it is not compiler checkable.



>     3) having a tightly specified interface
> 
> Interfaces are my concern. If an Interface specifies that a function be const (in the sense of non modifying member variables), then it would make a lot of sense that classes which implement that interface be allowed to assume the interface means "logical constness", for the reasons of all the use-cases given so far.

I'll reiterate that const-that-isn't-checkably-constant has no value beyond being a comment, so it might as well just be a comment.


>     It goes back to painting a stripe across your hips and calling it a
>     seatbelt.
> 
>     Given this, it isn't any surprise that C++ is disastrously difficult to
>     write functional & multithreaded programs in, and C++ mutability is one
>     of the reasons why. In C++, you can write const this and const that and
>     it doesn't mean jack squat. Many experienced C++ programmers will tell
>     you that const is little more than a documentation aid.
> 
> 
> I /am/ an experienced C++ programmer,

I can tell <g>. The logical constness pattern comes from C++.

> Outside the module, that's all you need to know. Inside the module, the compiler knows all anyway.

The D compiler does not necessarily know all there is to know about a type:
1) If you're writing generic code, with things like const(T), you don't know what T is. Therefore, as a programmer, you will not be able to write generically pure code.
2) D allows for so-called abstract types, where the compiler does not know the internals of the type. This is used in the Phobos to hide implementation details. Therefore, the compiler cannot tell if a const T has mutability or not.


> And I'd end up with something that was much nastier than the very thing you are trying to avoid.

The error I see in the example is declaring rand() to be const. Why do that if it IS NOT constant? You can already hide members the user shouldn't be modifying with 'private', why layer 'const' over that, too?
September 12, 2007
Janice Caron wrote:
> But then, I don't understand Walter's objections to "logical
> constness".
> 
> Seems pretty simple to me. If a function is logically const but not truly const, then don't declare it pure.
> 
> Conversely, if it's not declared pure, then it can have logical
> constness.
> 
> Isn't that problem solved?

No. Pass a reference to a const through a non-pure function to a pure one.

September 12, 2007
Bruce Adams wrote:
> You were totally clear but using a C++ concept of const methods which I believe (and may be wrong) has no analogue in D. I believe the intention was to keep it out of the language as it is an ambomination. That said, I'm not clear how you can get the proper semantics for e.g. overloads of [] in D without it.
> 
> Bruce.

I believe the issue of overloading opIndex and similiar is going to be taken care of using the 'return' storage-class (discussed in WalterAndrei.pdf).  Basically it makes the return type of the function const or not, according to whether a particular parameter (e.g. 'this') is const or not.

Thanks,
Nathan Reed
September 12, 2007
Janice Caron wrote:
>> And why is this nonsense?? Because x is public? I don't see the problem
>> here.
> 
> What's the use-case for mutable non-private members?
> 
> I would say that it's nonsense if there's no need for them.
> 
> "Nonsense" may be the wrong word. "Should not be part of D" is more
> what I really mean.

I can't name a use case right now, but I do think significant use-cases for mutable non-private members exist.

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
September 12, 2007
On 9/12/07, Janice Caron <caron800@googlemail.com> wrote:
> See, I was basically arguing with you,

That's "with", as in, "on the same side as", not "arguing" as in "disagreeing". Guess I could have worded that better! Ahem. I'll try again...


See, I was basically agreeing with you and supporting your claim...

> that transitivity is a good thing


The rest of the post should now make sense!

> -- /provided/ we can have mutable members. Now, if we can't have that, then suddenly transitivity no longer seems so welcoming. If const-transitivity becomes a strait-jacket, then folk are going to end up just not declaring anything const at all, because they can't get their code to compile otherwise.
September 12, 2007
On 9/12/07, Walter Bright <newshound1@digitalmars.com> wrote:

> Because multithreaded programming doesn't start working if the interface hides the changing variables. If it did, parallel programming would be easy. Logical constness is *still* changing state, and since it is, all the boogeymen of synchronization, race conditions, deadlocks, etc., are all there.

I've written a lot of multithreaded programming. A lot.

...which is perhaps a bad thing, because sometimes I forget that D does things differently. I don't need to use a separate mutex class if you've got synchronized.

(By the way - could you also allow the spelling "synchronised" for the
benefit of speakers of British English?) :-)

synchronised is a blunt tool though. In my C++ code, I arrange it so that multiple threads can simultaneously obtain read-access, or exactly one thread can obtain write-access. Correct me if I'm wrong, but the built-in synchronized feature isn't that smart?

So anyway, as I mentioned further up this thread, I had this loopup class member function that got stuff from a file and cached it. I assure you it was thread-safe. It even allowed multiple readers (if the key/value pair was already cached), but if a lookup was not cached, one thread and one thread only got to open the file, read a chunk of data, and stick it in the file. This was a class that knew what it was doing.

You're probably going to tell me that I can still do all that in D, providing I don't declare it const. The problem is, if I don't declare it const, /and/ all consts in D are transitive, then I can't store a reference to that object in any const structure. Basically I'd end up declaring just about nothing const, and I'd see no benefit.


> Logical constness hides the state change from the awareness of the programmer, but it's still changing state.

By definition, yes.


> Being in a module doesn't prevent another thread from using the module's interface and thereby changing the state.

Is that why you object?

I would have thought it the programmer's problem to ensure thread safety, not the compiler's. If I have a variable that's going to be accessed by multiple threads, then I'm going to make sure it's properly synchronised, and I if screw up, then it's my bug.



> Yes, but that requires programmer discipline, which is unreliable and scales poorly.

I think it could scale very well with the right language support. But that's another subject.


> Very few programmers are able to successfully write such code. With FP programming, most programmers will be able to.

That's obviously true. I'm not sure why it's relevant though. You said you had a "pure" keyword in mind for that. But if I don't use the pure keyword, then thread safety should be down to me, right?


> It has come up before. The answer is to not declare logically const things as being const, because they aren't const. Logical constness belongs as a comment because it is not compiler checkable.

But you might want to declare logically const things as being const for efficiency reasons. And to argue that "they aren't const" is a matter of definition. "const" in C++ is /defined/ to mean logical const, so const they are! You use the word to mean "physically const". We don't all agree on the definition.

Suppose there exists a class String with member function uppercase(),
declared const (or invariant, as you would have it in D). So far, so
good.

But then, along comes a better (or at least, different) string class with the same interface, and it happens to be faster, and I want to use the new FastString class in place of the old String class. All I have to do is search and replace, right? Or just make an alias. But there's a problem. Turns out, the reason it's faster is because it caches some results internally so it doesn't have to keep recomputing them.

From the point of view of "implementation should be independent of interface" this internal detail is something I shouldn't need to know or care about.

But now suddenly, I do need to know about it. And care. Because now that uppercase() function I mentioned earlier is no longer physically const. That means, if I use it as a drop-in replacement, my code now won't compile.



> I'll reiterate that const-that-isn't-checkably-constant has no value beyond being a comment, so it might as well just be a comment.

Oh contraire. It /is/ meaningful. It is a cast iron guarantee that no non mutable (which I argue should imply non-private) variables will be modified. Without the const declaration, you do not have that guarantee. It's a guarantee *which the compiler can use* to help the programmer catch errors! For example, this C++

class C
{
    mutable int x;
    int y;

    void f() const;
    {
        x = 1;
        y = 2; /* The compiler catches this error */
     }
}

See - this is useful to me. If I had not declared f const (which is basically what you're suggesting), then the compiler would have been no help to me whatsoever. It would not have flagged y=2 as an error.

I /want/ the compiler to assist me by flagging errors like this. I want it to be a compile-time error for f to modify y. But as soon as remove the words "const" and "mutable" from that example, I lose that compiler-assistance.



> The D compiler does not necessarily know all there is to know about a type:

OK. Good point. I hadn't thought that one through.

The difficulty is, if you don't allow mutable member variables, then we're back to what started this thread - transitivity sucks.

See, I was basically arguing with you, that transitivity is a good thing -- /provided/ we can have mutable members. Now, if we can't have that, then suddenly transitivity no longer seems so welcoming. If const-transitivity becomes a strait-jacket, then folk are going to end up just not declaring anything const at all, because they can't get their code to compile otherwise.
September 12, 2007
Janice Caron wrote:
> On 9/12/07, Walter Bright <newshound1@digitalmars.com> wrote:
> 
>> Because multithreaded programming doesn't start working if the interface
>> hides the changing variables. If it did, parallel programming would be
>> easy. Logical constness is *still* changing state, and since it is, all
>> the boogeymen of synchronization, race conditions, deadlocks, etc., are
>> all there.
> 
> I've written a lot of multithreaded programming. A lot.
> 
> ...which is perhaps a bad thing, because sometimes I forget that D
> does things differently. I don't need to use a separate mutex class if
> you've got synchronized.

In general that's true, though you might want a separate mutex class for specialized needs: inter-process synchronization, etc.

> (By the way - could you also allow the spelling "synchronised" for the
> benefit of speakers of British English?) :-)
> 
> synchronised is a blunt tool though. In my C++ code, I arrange it so
> that multiple threads can simultaneously obtain read-access, or
> exactly one thread can obtain write-access. Correct me if I'm wrong,
> but the built-in synchronized feature isn't that smart?

Yes and no :-)  The default implementation isn't that smart, but it can be extended.  Tango has a ReadWriteMutex that works like so:

    auto readWriteLock = new ReadWriteMutex;

    synchronized( readWriteLock.reader )
    {
        // any number of threads can be here simultaneously
    }
    synchronized( readWriteLock.writer )
    {
        // only one thread may enter this block simultaneously
        // and no threads may be in the reader block above either
    }

> So anyway, as I mentioned further up this thread, I had this loopup
> class member function that got stuff from a file and cached it. I
> assure you it was thread-safe. It even allowed multiple readers (if
> the key/value pair was already cached), but if a lookup was not
> cached, one thread and one thread only got to open the file, read a
> chunk of data, and stick it in the file. This was a class that knew
> what it was doing.
> 
> You're probably going to tell me that I can still do all that in D,
> providing I don't declare it const. The problem is, if I don't declare
> it const, /and/ all consts in D are transitive, then I can't store a
> reference to that object in any const structure. Basically I'd end up
> declaring just about nothing const, and I'd see no benefit.

I've been wondering about this.  So far, I've only been able to come up with two options:

1) Cast away const when dealing with 'mutable' data.
2) Store mutable data externally, perhaps in an AA.

I haven't yet decided whether there are any fundamental problems with option 1 other than it being generally bad practice.  Option 2 seems impractical in most cases however, because any common store of such mutable data must itself be protected by a mutex.

>> Very few programmers are able to successfully write such
>> code. With FP programming, most programmers will be able to.
> 
> That's obviously true. I'm not sure why it's relevant though. You said
> you had a "pure" keyword in mind for that. But if I don't use the pure
> keyword, then thread safety should be down to me, right?

I think Walter is suggesting that the language should still lend itself to code that isn't error-prone.  What remains unclear to me is whether the lack of 'mutable' will actually be an issue or if workarounds and alternate programming techniques (encouraged by the FP model) will be sufficient as a replacement.

>> It has come up before. The answer is to not declare logically const
>> things as being const, because they aren't const. Logical constness
>> belongs as a comment because it is not compiler checkable.
> 
> But you might want to declare logically const things as being const
> for efficiency reasons. And to argue that "they aren't const" is a
> matter of definition. "const" in C++ is /defined/ to mean logical
> const, so const they are! You use the word to mean "physically const".
> We don't all agree on the definition.
> 
> Suppose there exists a class String with member function uppercase(),
> declared const (or invariant, as you would have it in D). So far, so
> good.
> 
> But then, along comes a better (or at least, different) string class
> with the same interface, and it happens to be faster, and I want to
> use the new FastString class in place of the old String class. All I
> have to do is search and replace, right? Or just make an alias. But
> there's a problem. Turns out, the reason it's faster is because it
> caches some results internally so it doesn't have to keep recomputing
> them.
> 
> From the point of view of "implementation should be independent of
> interface" this internal detail is something I shouldn't need to know
> or care about.
> 
> But now suddenly, I do need to know about it. And care. Because now
> that uppercase() function I mentioned earlier is no longer physically
> const. That means, if I use it as a drop-in replacement, my code now
> won't compile.

That's a good example.  I do wonder, however, whether such an object would be a viable drop-in replacement in all cases.  In C++ my answer would be a confident "yes," but in D...

>> I'll reiterate that const-that-isn't-checkably-constant has no value
>> beyond being a comment, so it might as well just be a comment.
> 
> Oh contraire. It /is/ meaningful. It is a cast iron guarantee that no
> non mutable (which I argue should imply non-private) variables will be
> modified. Without the const declaration, you do not have that
> guarantee. It's a guarantee *which the compiler can use* to help the
> programmer catch errors!

Agreed.  Though I do think it's worth experimenting a bit more with the D approach to see if it is a reasonable replacement.  I do think that the traditional OO design that logical const supports in C++ may end up not being the general approach to program design in D.  But by the same token, perhaps that isn't sufficient reason to not support the approach.


Sean
September 13, 2007
Janice Caron wrote:
> I was trying to simplifly. Sigh. The gist of this particular example
> is that have what /starts off/ as perfectly good, non-member-modifying
> code, and all is well. But then you later put in some diagnostics to
> make it write stuff to a file (coz you're debugging) and suddenly it
> no longer compiles and you have keep taking const out of your code all
> over the place until it does.

Or you can cast away const-ness. But you are on your own if you do that.
September 13, 2007
Bruce Adams wrote:

> I not sure whether pure functions can modify class variables.

I hope not!

> The
> presentation states that they can't modify anything reachable through
> their arguments.

They are like mappings (e.g y = f(x) ) in math.

If you write sth. like this:
b = foo;
for(i = 0; i < rand_range(1,1000); i++) {
  a = pure_func(b);
}
writefln("%s", a);

b will still contain foo and a will always have the same value in every program run. pure_func will only be called once here. This is the possible compiler optimisation, if pure_func() is a pure function.


Best Regards

Ingo Oeser
September 13, 2007
Walter Bright wrote:
> Janice Caron wrote:
>> So the function was declared const, because /conceptually/ it was.
>>
>> But the class had a mutable cache, declared with the C++ keyword "mutable"
>>
>> Transitivity would wreck that.
> 
> You're right in that transitive const does not support the notion of "logical const" (which is the usual term for what you are referring to).
> 
> The problem with logical const, however, is it offers no semantic guarantees. I pass a logical const reference around, and the underlying data may or may not change. I have no guarantees one way or the other, and even worse, I can't even tell this is happening. "I" here meaning the compiler, and our hapless code auditor.
> 
> This just pulls the rug out from under:
> 
> 1) functional programming
> 2) multithreaded programming
> 3) having a tightly specified interface
> 
> It goes back to painting a stripe across your hips and calling it a seatbelt.
> 
> Given this, it isn't any surprise that C++ is disastrously difficult to write functional & multithreaded programs in, and C++ mutability is one of the reasons why. In C++, you can write const this and const that and it doesn't mean jack squat. Many experienced C++ programmers will tell you that const is little more than a documentation aid.
> 
> In other words, you're quite right that transitive const totally wrecks using const to specify logical constness. And that's a good thing <g>.

In the interest of this discussion, here's a very nice article regarding the topic of logical constness: http://www.ddj.com/cpp/184403892
(the fact that it mentions Walter is coincidence, the article is simply the first result I got when I googled for "logical constness")

Anyways, we now see that 'mutable' and logical constness is dangerous when one wants to do multi-threaded optimizations. However, since D is going to have several flavors of const (such as 'const', 'invariant' and also the 'pure' functions), why not allowing logical constness on some of these flavors, but disallow it in others.
Namely, we could allow logical constness in 'const' and 'invariant' data, that is, changing 'mutable' members of 'const' and 'invariant' data. Thus D would allow for the the use cases of logical constness. But on 'pure' functions, no changes whatsover would be allowed, even of 'mutable' members, and so we could still be able to use the 'pure' construct to safely create multi-threaded an parallelization optimizations.
Would this be ok?

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D