Kirk McDonald wrote:
> Bill Baxter wrote:
>> Andrei Alexandrescu (See Website For Email) wrote:
>>
>>> Bill Baxter wrote:
>>>
>>>> Derek Parnell wrote:
>>>>
>>>>> On Mon, 19 Mar 2007 15:05:29 -0700, Andrei Alexandrescu (See Website For
>>>>> Email) wrote:
>>>>>
>>>>>
>>>>> I wish it wasn't so hard to get a simple straight answer from the experts.
>>>>
>>>>
>>>> I think discussion would much improved if big changes like this actually came with a detailed proposal on a website somewhere that people could refer to and which would actually be updated to reflect vague points clarified during discussion.  As it is, there are a lot of points that have been clarified, and questions answered, but they're strewn across 100 different posts.  I'm pretty lost.  I gave up trying to follow this myself.  I'm hoping you'll figure it out and post something coherent. :-)
>>>>
>>>> On the bright side, thanks to Andrei, at least there *is* an open discussion going on about the feature before it just shows up in the change log as a done deal.
>>>
>>>
>>> I agree. (How couldn't I? Flattery goes a long way :o).)
>>>
>>> Maybe we could imitate what other languages do, e.g. Perl has the famous "apocalypses" that describe each language feature in detail.
>>
>>
>> I was thinking more of Python's PEPs, but yeh, same deal.  I like PEPs in that anyone can submit one, get it assigned an offical number.   As opposed to Larry's stone tablets brought down from the mountain top for the rest of us to ooh and ahh over.  Python's way is a little more egalitarian.
>>
>> Semi-official Enhancement Proposals are also a good way to handle the many repeated requests for 'feature X'.   Just tell 'em -- great idea, now go write a DEP for it.  If they actually do it, then great.  Next time someone asks for the feature you just point 'em to the previous DEP and the BFD ruling on it.
>>
>> My impression is that PEPs get weighted somehow based on how fleshed-out they are.  For instance it seems a majority of PEPs (at least the ones I've read) come complete with prototype implementations.  So PEP is more than just a vague wish like "Please implement Java-style serialization".  It has to contain the nitty gritty details.
>>
>> --bb
> 
> The PEP numbers are also a convenient way to refer to certain issues or features: "Oh the 'with' statement? That's PEP 343." The PEP can also serve as a starting point for the official documentation of the feature.
> 
> Most importantly, they serve as a fixed locus of discussion, and can vastly improve the low signal-to-noise ratio that running around in circles on a mailing list or newsgroup can cause.
> 
> In other words:
> DEPs++

I, too, think that's a great idea.

Andrei

Derek Parnell wrote:
> Hmmm ... so the difference between 'const' and 'invariant' is that const is
> only effective for one level deep, but 'invariant' is effective for all
> levels, right?

Nonono. Const means: it might have originated as a variable, and somebody, somewhere, might hold a mutating reference to this and call it "my precious".

Invariant is: nobody ever holds a mutating reference to it.

Example:

void Fun(const char[] a, char[] b)
{
  char c = a[0];
  b[0] = 'z';
  assert(a[0] == c); // will fail
}

void main()
{
  char[] x = "Wyda".dup;
  Fun(x, x);
}

Notice how the data referenced to by a is changing from under a's feet. Now let's rewrite Fun:

void Gun(invariant char[] a, char[] b)
{
  char c = a[0];
  b[0] = 'z';
  assert(a[0] == c); // never, ever, ever, ever fails
}

void main()
{
  char[] x = "Wyda".dup;
  Fun(x, x); // can't compile!
      // mutating references can't be bound to invariant references
}


Andrei

On Mon, 19 Mar 2007 17:50:28 -0700, Andrei Alexandrescu (See Website For
Email) wrote:

> Derek Parnell wrote:
>> Hmmm ... so the difference between 'const' and 'invariant' is that const is only effective for one level deep, but 'invariant' is effective for all levels, right?
> 
> Nonono. Const means: it might have originated as a variable, and somebody, somewhere, might hold a mutating reference to this and call it "my precious".
> 
> Invariant is: nobody ever holds a mutating reference to it.

Yeah ... but that's not what I was talking about. But be that as it may, it seems you are saying that the *only* difference between 'const' and 'invariant' is that 'invariant' means that every write attempt (direct and indirect) is denied, but 'const' only prevents direct write attempts.

By 'direct' I mean explicitly using the symbol in the statement, and not another symbol that happens to contain the same address of the data.


But I was talking earlier about the situation where something contains reference types or structs, and what effect 'const' and 'invariant' have on the things indirectly accessible via member data.

 struct Foo
 {
    char[] s;
 }

 void Func(const Foo fc, invariant Foo fi)
 {
    fc.s = "abc"; // fails ???
    fc.s[0] = 'a'; // okay ???

    fi.s = "abc"; // fails ???
    fi.s[0] = 'a'; // fails ???
 }

 Foo a;
 Foo b;
 Func(a,b); // fails ???

 const Foo ac;
 const Foo bc;
 Func(ac,bc); // fails ???

 invariant Foo ai;
 invariant Foo bi;
 Func(ai,bi); // fails ???

 Func(ac,bi); // okay ???


-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Justice for David Hicks!"
20/03/2007 12:04:01 PM

Derek Parnell wrote:
> On Mon, 19 Mar 2007 17:50:28 -0700, Andrei Alexandrescu (See Website For
> Email) wrote:
> 
>> Derek Parnell wrote:
>>> Hmmm ... so the difference between 'const' and 'invariant' is that const is
>>> only effective for one level deep, but 'invariant' is effective for all
>>> levels, right?
>> Nonono. Const means: it might have originated as a variable, and somebody, somewhere, might hold a mutating reference to this and call it "my precious".
>>
>> Invariant is: nobody ever holds a mutating reference to it.
> 
> Yeah ... but that's not what I was talking about. But be that as it may, it
> seems you are saying that the *only* difference between 'const' and
> 'invariant' is that 'invariant' means that every write attempt (direct and
> indirect) is denied, but 'const' only prevents direct write attempts.
> 
> By 'direct' I mean explicitly using the symbol in the statement, and not
> another symbol that happens to contain the same address of the data.
> 
> 
> But I was talking earlier about the situation where something contains
> reference types or structs, and what effect 'const' and 'invariant' have on
> the things indirectly accessible via member data.
> 
>  struct Foo
>  {
>     char[] s;
>  }
> 
>  void Func(const Foo fc, invariant Foo fi)
>  {
>     fc.s = "abc"; // fails ???

Works. Understand that fc is the private copy owned by Func.

>     fc.s[0] = 'a'; // okay ???

Fails. fc would change bits not belonging to itself, and "const" makes those bits unchangeable.

>     fi.s = "abc"; // fails ???

Works. Modifies Func's own private copy of fi.

>     fi.s[0] = 'a'; // fails ???

Fails.

>  }
> 
>  Foo a;
>  Foo b;
>  Func(a,b); // fails ???

Fails. Can't bind modifiable b to an invariant Foo.

>  const Foo ac;
>  const Foo bc;
>  Func(ac,bc); // fails ???

Fails. Can't bind const b (which conservatively may be aliased to a modifiable view) to an invariant Foo.

>  invariant Foo ai;
>  invariant Foo bi;
>  Func(ai,bi); // fails ???

Works. No problem to bind ai to a const Foo.

>  Func(ac,bi); // okay ???

Perfect.


Andrei

Walter Bright escribió:
> Andrei Alexandrescu (See Website For Email) wrote:
>> kris wrote:
>>> So, "invariant" is already a keyword ... what about that?
>> I completely missed that one. I think it's a good idea to look into it as a candidate for a qualifier name. Thank you.
> 
> I agree. I think:
> 
> final
> const
> invariant
> 
> for the three cases looks pretty good.

I haven't finished reading everything on this thread yet, so I don't know if this has been asked. Also, I'm failing to fully understand this, but I still have this question: how would the following be understood by the compiler?

class A
{
	invariant
	{
		int a = 4;
		float b = -10;
	}
}

What is that? A class invariant or two invariant class members? (If the latter even makes sense at all...)

-- 
Carlos Santander Bernal

I think I've got this.  Is the following right?

*final variables can be assigned only once in their lifetime, but are otherwise normal variables (i.e. they *do* have a memory address), except where constant folded by the compiler (this should be completely transparent to the programmer).

*const and invariant only make sense when applied to references, otherwise they are ignored or (preferably) errors.

*const references cannot be used to mutate the data they reference, but the data they reference may be mutated directly or through other non-const, non-invariant references.

*invariant references only reference data that will never be mutated (e.g. final variables).

*Both const and invariant references can be reassigned unless they are also final references.

*Conversely, when final is applied to the declaration of a reference, only that reference is protected from mutation, not the data it references, unless that reference is also const or invariant.

*Neither const nor invariant references can be cast to non-const or non-invariant references directly.

*The address of any variable or the value of any reference (of compatible type) can be assigned to const references.

*Only the addresses of final variables (I'm sure of this), the direct result of "new" statements (not sure on this one), and the values of other invariant references may be assigned to invariant references.

*const and invariant are viral (i.e. all references in the data referenced by a const or invariant reference is also const or invariant, respectively, when access through the const or invariant reference).

*final, when applied directly to POD structures (structs and array references), means no members of that structure can be modified after assignment, but is non-viral to references within the POD structure. (Not sure on this.)

*Function declarations cannot overloaded based on const or invariant. (Pretty sure on this.)
---------------------
Here are a whole bunch of contrived examples of most of these rules:

final int w = 5; 	//legal
final int x = 55;	//ditto
int y = 6; 		//legal
int z;			//ditto
const int i;		//either (preferably) illegal or ignored
invariant int i;	//ditto

x = 6; 			//illegal
y = 7; 			//legal

int* mut_pointer;
mut_pointer = &x; 	//illegal: this would allow x to be mutated
mut_pointer = &y; 	//obviously legal
*mut_pointer = 10;	//obviously legal

const int* const_pointer;
const_pointer = &x;		//legal
const_pointer = &y; 		//Also legal: const_pointer is mutable (non-final)
z = *const_pointer;		//Legal: you can read through const references
*const_pointer = 10;		//illegal: you can't mutate through const references

invariant int* inv_pointer = &x; 	//Legal: x is final and never changes
inv_pointer = &y 			//Illegal: y is non-final may change
inv_pointer = &w;			//Legal: inv_pointer is mutable (non-final)
y = *inv_pointer;			//Legal: data can be read through inv_pointer
*inv_pointer = y;			//Illegal: you can't mutate through invariant references

final int* final_pointer = &y;		//obviously legal
z = *final_pointer;			//obviously legal
*final_pointer = 2; 			//legal: y can be changed through final_pointer
final_pointer = &z; 			//illegal: final_pointer already assigned

final int* final_pointer2 = &x;		//illegal: final_pointer2 allows mutation

final const int* final_const_pointer = &x; //Legal: const pointer does not allow mutation
final_const_pointer = &w;		//Illegal: final_const_pointer is final and cannot be mutated
y = *final_const_pointer;		//Obviously legal
*final_const_pointer = y;		//Obviously illegal

final const int* final_const_pointer2 = &y; //Legal: neither y nor const care about underlying mutability

final invariant int* final_inv_pointer = &x; //Legal
final invariant int* final_inv_pointer2 = &y; //Illegal: y is mutable

struct S
{
	int a;
	char[] b;
	static S* opCall()
	{
		S* ret = new S;
		s.b = "test";
		return ret;
	}
}

final S final_s;	//final_s exists on the stack or in the global scope

final_s.a = 6;		//Illegal: s is final
final_s.b = "hello"; 	//ditto

final S final_s2 = *S();	//Legal
final_s2.b[2] = 'd';		//Legal: final is non-viral

final S* s_pointer = new S;	//Legal, *(s_pointer) exists on the heap
s_pointer.a = 6;		//Legal, *(s_pointer) can be mutated through s_pointer
s_pionter.b = "hello"; 		//ditto
s_pointer = new S;		//Illegal: s_pointer is already assigned

final S* s_pointer2 = S();	//Obviously legal

const S* const_s_pointer = new S;	//Legal
y = const_s_pointer.a;			//Legal
const_s_pointer.a = 5;			//Illegal
const_s_pointer = S();			//Legal: const_s_pionter can be reassigned

invariant S* inv_s_pointer = new S;	//Legal
y = inv_s_pionter.a;			//Legal
inv_s_pointer.a = 5;			//Illegal
inv_s_pointer = S();			//Not sure, but probably illegal: S* points to mutable data,
					//even though there are no other references to it.  How do
					//invariant pointers work with the static opCall() workaround for
					//missing struct constructors? Does NRVO deal with this?

class C
{
	int a;
	final S* s_in_c;
	this() { s_in_c = new S; } 	//Is it legal to assign final values in constructors?
					//What about in a static opCall() for structs?
	void increment_a() { this.a++; }
}

final C final_c = new C();
final_c = new C();		//Illegal: final_c already assigned.
y = final_c.a;			//Legal
final_c.a = x;			//Also legal
final_c.increment_a();		//Ditto

const C const_c = new C();
const_c = new C();		//Legal: reference const_c can be reassigned
const_c.a = y;			//Illegal: cannot mutate through const reference
const_c.increment_a();		//Legal??? Is the object's "this" reference non-const?

invariant C inv_c = new C();
inv_c = new C();		//legal: reference inv_c can be reassigned
ivn_c.a = y;			//Illegal: cannot mutate through invariant reference
inv_c.increment_a();		//Illegal???  The object shouldn't be able to be modified at all, but
				//how could the compiler catch this?

Tyler Knott wrote:
> I think I've got this.  Is the following right?
> 
> *final variables can be assigned only once in their lifetime, but are otherwise normal variables (i.e. they *do* have a memory address), except where constant folded by the compiler (this should be completely transparent to the programmer).

Correct.

> *const and invariant only make sense when applied to references, otherwise they are ignored or (preferably) errors.

Correct.

> *const references cannot be used to mutate the data they reference, but the data they reference may be mutated directly or through other non-const, non-invariant references.

Correct.

> *invariant references only reference data that will never be mutated (e.g. final variables).

Ambiguously expressed. But I think I see what you mean, e.g. you can take the address of a final int and bind it to an invariant int*, like so:

final int x = 42;
immutable int* p = &x; // correct

> *Both const and invariant references can be reassigned unless they are also final references.

Correct.

> *Conversely, when final is applied to the declaration of a reference, only that reference is protected from mutation, not the data it references, unless that reference is also const or invariant.

Correct.

> *Neither const nor invariant references can be cast to non-const or non-invariant references directly.

Correct.

> *The address of any variable or the value of any reference (of compatible type) can be assigned to const references.

Correct. (Just like in C++.)

> *Only the addresses of final variables (I'm sure of this), the direct result of "new" statements (not sure on this one), and the values of other invariant references may be assigned to invariant references.

Yes, yes (and more), and yes.

New statements can be solved easily by just requiring people to qualify the type created:

invariant int* p = new invariant int(42);

User-defined types will have a chance to define constructors for invariant objects. Those constructors will have the chance of modifying the object being constructed, but won't be able to alias 'this' and member addresses to modifiable pointers.

The deeper problem is with functions that return pointers that could be assigned to either mutable or invariant symbols. For example, array.dup. Take a look:

int[] array = [1, 2, 3];
invariant int[] brray = array.dup; // should work
invariant int[] crray = array.dup; // should work, too!

I am working on a solution for this issue. The problem is rather general. Consider a function char[] readln() that reads a line from the standard input, creating a fresh string each call. That string should be assigned to char[] or invariant char[].

> *const and invariant are viral (i.e. all references in the data referenced by a const or invariant reference is also const or invariant, respectively, when access through the const or invariant reference).

"Transitive." Yes.

> *final, when applied directly to POD structures (structs and array references), means no members of that structure can be modified after assignment, but is non-viral to references within the POD structure. (Not sure on this.)

Correct. "final x" freezes all of x's bits.

> *Function declarations cannot overloaded based on const or invariant. (Pretty sure on this.)

Walter is mildly unconvinced about such overloads' utility, I think they should be allowed without being required.

I'll snip your examples, which are correct and also raise a couple of interesting questions that are being worked on.


Andrei

On Mon, 19 Mar 2007 22:34:13 -0700, Andrei Alexandrescu (See Website For
Email) wrote:

> Tyler Knott wrote:
>> *const and invariant only make sense when applied to references, otherwise they are ignored or (preferably) errors.
> 
> Correct.

Can 'const' and 'invariant' apply to structs, especially ones that contain references.

  struct Foo
  {
      char[] s;
      char c;
  }

  const Foo f;  // Is this a mistake then?
  f.s[0] = 'a'; // okay???
  f.s = "xyz";  // okay???
  f.s = "def".dup; // okay ???
  f.s.length = 1;  // okay?
  f.s ~= 'a'; // okay??
  f.c = 'a';   // okay???

  const Foo somefunc();
  f = somefunc; // okay??

  Foo someotherfunc();
  f = someotherfunc; // okay??

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Justice for David Hicks!"
20/03/2007 4:45:27 PM

Derek Parnell wrote:
> On Mon, 19 Mar 2007 22:34:13 -0700, Andrei Alexandrescu (See Website For
> Email) wrote:
> 
>> Tyler Knott wrote:
>>> *const and invariant only make sense when applied to references, otherwise they are ignored or (preferably) errors.
>> Correct.
> 
> Can 'const' and 'invariant' apply to structs, especially ones that contain
> references.
> 
>   struct Foo
>   {
>       char[] s;
>       char c;
>   }
> 
>   const Foo f;  // Is this a mistake then?

That code is fine. I should have said "const and invariant make sense only when applied to types that are, or contain, references". Sorry.

>   f.s[0] = 'a'; // okay???

No.

>   f.s = "xyz";  // okay???

Yes.

>   f.s = "def".dup; // okay ???

Yes.

>   f.s.length = 1;  // okay?

Yes.

>   f.s ~= 'a'; // okay??

Yes.

>   f.c = 'a';   // okay???

Yes.

>   const Foo somefunc();
>   f = somefunc; // okay??

Yes.

>   Foo someotherfunc();
>   f = someotherfunc; // okay??

Yes. Non-const transforms into const no problem.


Andrei

Andrei Alexandrescu (See Website For Email) wrote:
> Don Clugston wrote:
>> Assuming that 'invariant' = really constant, 'const' = C++ pseudo-const...
> 
> Yah.
> 
>> It's better than super const. However:
>> (1) I would want to go through all my existing D code and change 100% of my usages of 'const' to 'invariant'.
> 
> Great to hear that!

Sorry, I didn't explain myself very well. I meant that this proposal breaks 100% of all existing D uses of const.

 >> (3) I concede the problem of association with 'readonly' and ROM. But
>> the association between 'const' and 'constant' is pretty strong.
> 
> I think there's universal agreement that you can't express tersely the two notions "const-as-in-C++" and "const-as-in-we-really-mean-it". Probably it's best to just go with two terms and move on.

I completely agree with this. The gripe I have is that 'const' is a completely inappropriate term for 'const-as-in-C++'. It's a bizarre misnomer to use 'const' to mean "don't touch". The only benefit (albeit a considerable one) of this nomenclature is compatibility with C++ nomenclature, especially the term "const correctness".
D currently has a 'const' which actually means "constant", it seems dreadful to exchange that for a clearly inferior "don't touch" meaning.
I'm concerned that you may have spent so much time in C++ that you've become accustomed to "const" == "don't touch"; all of my previous posts have been arguing that this is a grossly unnatural association.

It is possible that the C++ misnomer is so entrenched that D needs to perpetuate it, even at the expense of breaking all existing D code that uses 'const'.

But then this really surprises me:

> final int a = 2;
> 
> void f(final int b)
> {
> }
> The choice of "final" in the second case prevents f from changing its argument,
> and it's the free will choice of f's author. The "final" does not
> influence f's signature or how other people use it.
> It's just constraining f's implementation.

because it seems that that we still have 'const' surprises for C++ refugees.