On 11/10/13 01:43, Daniel Davidson wrote:
> That is probably a reasonable interpretation... but I think it will only get you
> pain. The fact is, regardless of your interpretation of "const" arguments - the
> general guideline is "prefer const because immutables and mutables can be passed
> in".

Which makes much sense for code that is intended to be generically useful.

> Think of it this way: you are writing a very generic function that requires
> inputs that you do not mutate. If you choose immutable you are making your life
> easier, I guess, since you know data is not being changed on you. But you are
> also preventing your function from being called by any arguments that are
> already mutable (assuming the type has mutable aliasing) because crossing the
> mutable to immutable divide is not part of the language. So, the only way then
> for mutable types to make it into your function is to copy them and this may or
> may not even be possible. And if you truly know no code is mutating your data it
> is a tougher sell.

If you are insisting that your function receive immutable inputs, you are in turn constraining the user of that code.  If you really want your function to be "generic", isn't the appropriate thing to allow the user the choice of whether they want to pass it immutable data, or to take the risk of passing it mutable data?

Surely there's a tradeoff -- you may well feel the resulting safety is worth it, and there may be other benefits, such as optimizations, that you can derive from having only immutable data.

But for the standard library of a multiparadigm language, it's not appropriate to force users to use immutable data.

> On the other hand if you go with const parameters, your data could in fact
> change on you. Unfortunately, it can cause terrible angst when the only options
> make you wish for other options.

Well, that depends.  If you go with const parameters but you pass it immutable data, you know it won't change on you.  You, the user, are in control.

> Have you actually written nested D data types with mutable aliasing (not just
> slices but assoc arrays) and used immutable in the signatures? I have tried and
> not succeeded.

Yes, I tried it, and ran into this problem.  But generally these days I use "in" to mark arguments that are not intended to be modified.  I personally prefer that -- it has a semantic meaning, and how it translates into code is for the compiler.

> I guess this just highlights the need for guidelines.

Guidelines are always nice. :-)

On Wednesday, 2 October 2013 at 13:09:34 UTC, Daniel Davidson wrote:
> I'm reviewing Ali's insightful presentation from 2013 DConf. I wonder has he or anyone else followed up on the concepts or formalized some guidelines that could achieve consensus. I definitely agree it would be helpful to have a 50 Ways To Improve Your D. The first thing I'd like to see is a set of guidelines on mutability along the lines he discussed in his talk. But as it stands, I don't know if there was any finalization/consensus. For instance, from the first two guidelines at the end (after several iterative reconstructions of those guidelines):
>
> 1. If a variable is never mutated, make it const, not immutable.
> 2. Make the parameter reference to immutable if that is how you will use it anyway. It is fine to ask a favor from the caller.
> ...
>
> If you follow (1) exclusively, why the need for immutable in the language at all?
>
> Maybe it is a philosophical question, but where does immutability really come from? Is it an aspect of some piece of data or is it a promise that function will not change it? Or is it a requirement by a function that data passed not be changed by anyone else?
>
> The two keywords cover all in some sense.
>
> I found the end of the video amusing, when one gentleman looking at a rather sophisticated "canonical" struct with three overloads for 'this(...)' and two overloads for opAssign, asked if all those methods were required. I think there was back and forth and head-scratching. Another asked if the language designers were happy with the resultant complexity. Naturally the answer was yes - it is a good mix. If that is the case I wonder if the reason is they don't write software in D like Ali was looking to develop. I imagine standard library code is much more functional than OO by its very nature. My money says you will not find a struct S constructed like Ali's in the wild - it is just too much boilerplate. But surely they have their own guidelines/approaches.
>
> By posting this I am not looking for a single answer to the simple question above as it is just one of many questions in the set of "how do you choose among the options in general". If you have such guidelines - please post them.
>
> Thanks
> Dan

After trying for several days to use immutable with types containing some mutable aliasing I have come to the conclusion that maybe rule number one should be:

If you have a type that has now or may ever have in the future any mutable aliasing (e.g. inclusion of T[] or T1[T1] where Ts are mutable) do not ever use the immutable keyword in any context as things just break down.

If you have had more success with a immutable with types containing mutable aliasing and can share your success story that would be great.

As D is growing are there any out there yet offering support services?

Thanks,
Dan

On 10/16/2013 10:23 AM, Daniel Davidson wrote:

> On Wednesday, 2 October 2013 at 13:09:34 UTC, Daniel Davidson wrote:

>> guidelines):
>>
>> 1. If a variable is never mutated, make it const, not immutable.
>> 2. Make the parameter reference to immutable if that is how you will
>> use it anyway. It is fine to ask a favor from the caller.

> After trying for several days to use immutable with types containing
> some mutable aliasing I have come to the conclusion that maybe rule
> number one should be:
>
> If you have a type that has now or may ever have in the future any
> mutable aliasing (e.g. inclusion of T[] or T1[T1] where Ts are mutable)
> do not ever use the immutable keyword in any context as things just
> break down.

I think this topic should be carried to the main newsgroup already. I am convinced that this is a language issue. :-/

> Thanks,
> Dan

Ali

On Wed, Oct 16, 2013 at 07:23:24PM +0200, Daniel Davidson wrote:
> On Wednesday, 2 October 2013 at 13:09:34 UTC, Daniel Davidson wrote:
[...]
> >Maybe it is a philosophical question, but where does immutability really come from? Is it an aspect of some piece of data or is it a promise that function will not change it? Or is it a requirement by a function that data passed not be changed by anyone else?

I think it helps to realize that D's const system is different from C++'s.

Immutable means the data will never change, ever. It means you can put that data in read-only memory, maybe burned into a ROM chip or something like that.

Const means *you* can't change the data, but somebody else may be able to.

Therefore:


[...]
> After trying for several days to use immutable with types containing some mutable aliasing I have come to the conclusion that maybe rule number one should be:

If you have mutable aliasing, that means the data cannot be immutable. Use const.


> If you have a type that has now or may ever have in the future any mutable aliasing (e.g. inclusion of T[] or T1[T1] where Ts are mutable) do not ever use the immutable keyword in any context as things just break down.

Yes, because immutable means nothing, no one, can change the data after it's constructed, ever. If you want mutable aliasing, what you want is const, not immutable.


> If you have had more success with a immutable with types containing mutable aliasing and can share your success story that would be great.
[...]

Maybe it's helpful to understand how D's const system works. The following diagram may help (please excuse the ASCII graphics):

	       const
	      /     \
	mutable     immutable

What this means is that const subsumes mutable and immutable. A mutable type can be implicitly converted to a const type (the receiver of the const can't modify the data, which is fine since the code holding the mutable reference can still mutate it), and so can immutable (immutable cannot be modified, ever, and const doesn't let you modify it either, so it's OK to make a const reference to immutable data). However, you cannot implicitly convert between mutable and immutable, unless you're copying the data by value.

So if you have immutable data and want to make changes, you have to first make a copy of the data, then mutate it at will.

What's the use of immutable, you ask? Immutable makes hard guarantees about the non-changeability of some piece of data. This makes it useful for implementing strings -- in fact, the 'string' type in D is just an alias for immutable(char)[]. You can take substrings (slices) of any given string freely, and be assured that your copy of the (sub)string will never unexpectedly change its value from somewhere else in the code. This saves the need for a lot of copying, which can be costly.

One interesting subtlety here is that 'string' is immutable(char)[], but not immutable(char[]). The latter would mean that the string itself can never be changed -- you couldn't assign to it, you couldn't append to it, etc., which would make strings a lot less useful than they are. But by making strings a *mutable* array of *immutable* chars, you allow the string to be appended to, substring'd, etc., all while guaranteeing that the underlying bytes themselves will never change. So you can have the best of both worlds: you can append to strings, take substrings, assign strings to each other, etc., yet at the same time be assured that the list of intermediate substrings you stored somewhere during the process will continue to retain the values you assigned to them, because the underlying bytes they point to are immutable, and therefore guaranteed never to change.


T

-- 
Just because you survived after you did it, doesn't mean it wasn't stupid!

On Wednesday, 16 October 2013 at 17:55:14 UTC, H. S. Teoh wrote:
> On Wed, Oct 16, 2013 at 07:23:24PM +0200, Daniel Davidson wrote:
>> On Wednesday, 2 October 2013 at 13:09:34 UTC, Daniel Davidson wrote:
> [...]
>> >Maybe it is a philosophical question, but where does immutability
>> >really come from? Is it an aspect of some piece of data or is it a
>> >promise that function will not change it? Or is it a requirement
>> >by a function that data passed not be changed by anyone else?
>
> I think it helps to realize that D's const system is different from
> C++'s.
>
> Immutable means the data will never change, ever. It means you can put
> that data in read-only memory, maybe burned into a ROM chip or something
> like that.
>
> Const means *you* can't change the data, but somebody else may be able
> to.
>
> Therefore:
>
>
> [...]
>> After trying for several days to use immutable with types containing
>> some mutable aliasing I have come to the conclusion that maybe rule
>> number one should be:
>
> If you have mutable aliasing, that means the data cannot be immutable.
> Use const.
>
>
>> If you have a type that has now or may ever have in the future any
>> mutable aliasing (e.g. inclusion of T[] or T1[T1] where Ts are
>> mutable) do not ever use the immutable keyword in any context as
>> things just break down.
>
> Yes, because immutable means nothing, no one, can change the data after
> it's constructed, ever. If you want mutable aliasing, what you want is
> const, not immutable.
>

I think the term "mutable aliasing" and "what you want" don't work together. "mutable aliasing" is not about context of usage or what you want, it is a compile time attribute of a struct. I want to use associative arrays in composition because I think they are the way I view and use my data. `struct T { string[string] i; }` has mutable aliasing, like it or not. So, it is not so much that I want mutable aliasing - in fact I fear it. But once you use AAs it is a fact of life.

>
>> If you have had more success with a immutable with types containing
>> mutable aliasing and can share your success story that would be
>> great.
> [...]
>
> Maybe it's helpful to understand how D's const system works. The
> following diagram may help (please excuse the ASCII graphics):
>
> 	       const
> 	      /     \
> 	mutable     immutable
>
> What this means is that const subsumes mutable and immutable. A mutable
> type can be implicitly converted to a const type (the receiver of the
> const can't modify the data, which is fine since the code holding the
> mutable reference can still mutate it), and so can immutable (immutable
> cannot be modified, ever, and const doesn't let you modify it either, so
> it's OK to make a const reference to immutable data). However, you
> cannot implicitly convert between mutable and immutable, unless you're
> copying the data by value.
>

yes - it requires transitive deep copy.

> So if you have immutable data and want to make changes, you have to
> first make a copy of the data, then mutate it at will.
>
> What's the use of immutable, you ask? Immutable makes hard guarantees
> about the non-changeability of some piece of data. This makes it useful
> for implementing strings -- in fact, the 'string' type in D is just an
> alias for immutable(char)[]. You can take substrings (slices) of any
> given string freely, and be assured that your copy of the (sub)string
> will never unexpectedly change its value from somewhere else in the
> code. This saves the need for a lot of copying, which can be costly.
>

I agree that string behaves as you say. I don't quite agree that immutable alone is the reason it does. I think it is an byproduct of the way immutable(T)[] is implemented. It is an implementation detail and relying on that could lead to bad deduction. We covered that here in this thread:
http://forum.dlang.org/post/jfjudswamyxlttgsdwva@forum.dlang.org


> One interesting subtlety here is that 'string' is immutable(char)[], but
> not immutable(char[]). The latter would mean that the string itself can
> never be changed -- you couldn't assign to it, you couldn't append to
> it, etc., which would make strings a lot less useful than they are. But
> by making strings a *mutable* array of *immutable* chars, you allow the
> string to be appended to, substring'd, etc., all while guaranteeing that
> the underlying bytes themselves will never change. So you can have the
> best of both worlds: you can append to strings, take substrings, assign
> strings to each other, etc., yet at the same time be assured that the
> list of intermediate substrings you stored somewhere during the process
> will continue to retain the values you assigned to them, because the
> underlying bytes they point to are immutable, and therefore guaranteed
> never to change.
>
>
> T

Agreed with the description of the behavior. But disagree on why. It works that way because T[] is modeled as contiguous memory and the api associated with slice of type immutable(T)[] means there is no unsafe sharing. So, `struct T { string[string] i; }` and `struct T { immutable(S)[string] }` do not have the same properties because they have a different layout model.

Is the suggestion here: use immutable in any composition context where you have slices and you want to not "worry about mutable aliasing". So, do like string does: any case where you have T[] as a member, prefer immutable(T)[] since then you don't have to worry about sharing. Well, not sure that works in general because the T in string is char and has no mutable aliasing itself. Suppose T itself has mutable aliasing, then what? Or, suppose it is a struct with no mutable aliasing *now*. Who's to say it won't change.

So, where does all this leave us w.r.t. a good set of guidelines?

On Wednesday, 16 October 2013 at 17:55:14 UTC, H. S. Teoh wrote:
> Maybe it's helpful to understand how D's const system works. The
> following diagram may help (please excuse the ASCII graphics):
>
> 	       const
> 	      /     \
> 	mutable     immutable

I think people in this thread know how const works, but some think it is broken. Scenario is this:

Library code:

struct Foo { int x; }

User code:

Foo f;
immutable f2 = f;

This works, even though the library writer might not have anticipated that someone makes Foo immutable. However, now the library writer obliviously releases a new version of the library, which extends it like this:

struct Foo {
  int x;
  private int[] history;
}

Unfortunately, now the user code is broken due to the freshly introduced mutable aliasing. Personally, I think is fine. Upon compilation the user code gives a  error message and user developer can adapt to the code to the new library version. Some think the library writer should have a possibility to make this work.

On Wed, Oct 16, 2013 at 08:49:51PM +0200, Daniel Davidson wrote:
> On Wednesday, 16 October 2013 at 17:55:14 UTC, H. S. Teoh wrote:
> >On Wed, Oct 16, 2013 at 07:23:24PM +0200, Daniel Davidson wrote:
[...]
> >>If you have a type that has now or may ever have in the future any mutable aliasing (e.g. inclusion of T[] or T1[T1] where Ts are mutable) do not ever use the immutable keyword in any context as things just break down.
> >
> >Yes, because immutable means nothing, no one, can change the data after it's constructed, ever. If you want mutable aliasing, what you want is const, not immutable.
> >
> 
> I think the term "mutable aliasing" and "what you want" don't work together. "mutable aliasing" is not about context of usage or what you want, it is a compile time attribute of a struct. I want to use associative arrays in composition because I think they are the way I view and use my data. `struct T { string[string] i; }` has mutable aliasing, like it or not. So, it is not so much that I want mutable aliasing - in fact I fear it. But once you use AAs it is a fact of life.
[...]
> Is the suggestion here: use immutable in any composition context where you have slices and you want to not "worry about mutable aliasing". So, do like string does: any case where you have T[] as a member, prefer immutable(T)[] since then you don't have to worry about sharing. Well, not sure that works in general because the T in string is char and has no mutable aliasing itself. Suppose T itself has mutable aliasing, then what? Or, suppose it is a struct with no mutable aliasing *now*. Who's to say it won't change.
> 
> So, where does all this leave us w.r.t. a good set of guidelines?

Sorry, I kinda barged into this conversation, so I'm not sure what exactly you're trying to achieve here. What kind of composition contexts do you have in mind? Maybe you could help me understand what you're trying to do?

Keeping in mind that AA's, as they are currently implemented, leaves a lot of room for improvement, to say the least. So you might be running into some AA-related issues, not the type system proper.


T

-- 
Creativity is not an excuse for sloppiness.

On Wednesday, 16 October 2013 at 18:52:23 UTC, qznc wrote:
> On Wednesday, 16 October 2013 at 17:55:14 UTC, H. S. Teoh wrote:
>> Maybe it's helpful to understand how D's const system works. The
>> following diagram may help (please excuse the ASCII graphics):
>>
>> 	       const
>> 	      /     \
>> 	mutable     immutable
>
> I think people in this thread know how const works, but some think it is broken. Scenario is this:
>

Thanks.

> Library code:
>
> struct Foo { int x; }
>
> User code:
>
> Foo f;
> immutable f2 = f;
>
> This works, even though the library writer might not have anticipated that someone makes Foo immutable. However, now the library writer obliviously releases a new version of the library, which extends it like this:
>
> struct Foo {
>   int x;
>   private int[] history;
> }
>
> Unfortunately, now the user code is broken due to the freshly introduced mutable aliasing. Personally, I think is fine. Upon compilation the user code gives a  error message and user developer can adapt to the code to the new library version. Some think the library writer should have a possibility to make this work.

I don't understand how it could be fine. As code grows it would lead to people not adding useful members like history just because of the huge repercussions.

struct User {
   immutable(Foo) foos;
}

How can I as a user adapt to that change? Before the change assignment worked equally well among all of Mutable, Immutable, Const. After that change any `foos ~= createFoo(...)` would require change. And it is not clear what the change would be.

On Wednesday, 16 October 2013 at 19:06:06 UTC, Daniel Davidson wrote:
> I don't understand how it could be fine. As code grows it would lead to people not adding useful members like history just because of the huge repercussions.
>
> struct User {
>    immutable(Foo) foos;
> }
>
> How can I as a user adapt to that change? Before the change assignment worked equally well among all of Mutable, Immutable, Const. After that change any `foos ~= createFoo(...)` would require change. And it is not clear what the change would be.

I think any usage of immutable with types/entities not initially designed for immutability is an potential mistake and in that sense it is good that change has broken the user code. Same goes for operating on immutable entity in generic code as if it is a value type without actually checking it via introspection.

On Wed, Oct 16, 2013 at 08:52:22PM +0200, qznc wrote:
> On Wednesday, 16 October 2013 at 17:55:14 UTC, H. S. Teoh wrote:
> >Maybe it's helpful to understand how D's const system works. The following diagram may help (please excuse the ASCII graphics):
> >
> >	       const
> >	      /     \
> >	mutable     immutable
> 
> I think people in this thread know how const works, but some think it is broken. Scenario is this:
> 
> Library code:
> 
> struct Foo { int x; }
> 
> User code:
> 
> Foo f;
> immutable f2 = f;
> 
> This works, even though the library writer might not have anticipated that someone makes Foo immutable. However, now the library writer obliviously releases a new version of the library, which extends it like this:
> 
> struct Foo {
>   int x;
>   private int[] history;
> }
> 
> Unfortunately, now the user code is broken due to the freshly introduced mutable aliasing. Personally, I think is fine. Upon compilation the user code gives a  error message and user developer can adapt to the code to the new library version. Some think the library writer should have a possibility to make this work.

Well, I think here the type system is working as advertised. Since the original version of Foo has no mutable aliasing, implicit conversion to immutable is OK (you're making a new binary copy of the data). But in the second case, it will obviously violate immutability guarantees, because in D, immutable is transitive, so immutable(Foo) in the second case is equivalent to:

	struct Foo {
		int x;
		private immutable(int[]) history;
	}

You can't implicitly convert int[] to immutable(int[]) because whoever
holds the original int[] reference can use it to change the data, which
breaks the immutable guarantee of immutable(int[]).

The only way this could work is if you made a copy of the int[]. So the library writer would have to provide a method for creating an immutable copy of the struct (like an .idup method or something).

You may argue that this is bad because now user code is broken and you need to rewrite it to use .idup, but I'd argue that relying on implicit conversion to immutable already introduces a dependency on implementation details of Foo, which should be avoided in the first place if you want to have a clean encapsulation.

I mean, given a Foo type exported by a library, if that type is meant to be an opaque type, then my code should make no assumptions about whether it can implicitly convert to immutable. I should rather require the library writer to provide an .idup method for creating an immutable instance of the struct than to blindly write the code to implicitly convert to immutable (thus introducing a hidden reliance on how Foo is implemented) and then have it broken later when the library writer changes its implementation.

OTOH, if it's not an opaque type, then it's no surprise that changing its implementation should also require changing the code (that depends on its implementation details).

So I don't see anything wrong with this particular scenario. What other scenarios were being considered?


T

-- 
He who does not appreciate the beauty of language is not worthy to bemoan its flaws.