View mode: basic / threaded / horizontal-split · Log in · Help
May 29, 2008
Poll: a nonstate keyword
What if D2 were to introduce a keyword that allows you to decouple a class 
member from the class state for the purposes of const?  This keyword 
basically means that the member is stored with the class data but is 
unaffected by the constancy of an instance.  In other words, a non-state 
member variable is not cast to const when the class instance is cast to 
const.  This is an implementation of the 'is associated with' OO 
relationship.  This is similar to the 'mutable' keyword in C++, or the way 
mutexes work in D.

If this were to happen, would you consider this to be a positive, negative, 
or inconsequential change?

Would you stop using D because of it?

Would you switch to D2 because of it?

Would you consider this concept hard to explain or understand?
May 29, 2008
Re: Poll: a nonstate keyword
On 2008-05-29 18:41:21 +0200, "Steven Schveighoffer" 
<schveiguy@yahoo.com> said:

> What if D2 were to introduce a keyword that allows you to decouple a class
> member from the class state for the purposes of const?  This keyword
> basically means that the member is stored with the class data but is
> unaffected by the constancy of an instance.  In other words, a non-state
> member variable is not cast to const when the class instance is cast to
> const.  This is an implementation of the 'is associated with' OO
> relationship.  This is similar to the 'mutable' keyword in C++, or the way
> mutexes work in D.
> 
> If this were to happen, would you consider this to be a positive, negative,
> or inconsequential change?

I already expressed my view about it in
	http://www.wikiservice.at/d/wiki.cgi?TransitiveConst

I think that it is a good idea, but I think that its name should be 
much uglier unsafe_mutable, and should be seen by everybody in the 
community as a bad, and not to be used.

Basically it is an unsafe hole that allows to implement nice an useful 
stuff (suspended values(lazy eval & cache), dataflow variables, 
memoization, performace/statistical information), but can break 
invariant and introduce subtle bugs.

> Would you stop using D because of it?
no
> Would you switch to D2 because of it?
no
> Would you consider this concept hard to explain or understand?
no, but hard to use correctly if it really breaks the invariance, and 
not much useful if it does not break invariance.

Fawzi
May 30, 2008
Re: a nonstate keyword
"Steven Schveighoffer" <schveiguy@yahoo.com> wrote in message 
news:g1mmbh$ftq$1@digitalmars.com...
> What if D2 were to introduce a keyword that allows you to decouple a class 
> member from the class state for the purposes of const?  This keyword 
> basically means that the member is stored with the class data but is 
> unaffected by the constancy of an instance.  In other words, a non-state 
> member variable is not cast to const when the class instance is cast to 
> const.  This is an implementation of the 'is associated with' OO 
> relationship.  This is similar to the 'mutable' keyword in C++, or the way 
> mutexes work in D.
>
> If this were to happen, would you consider this to be a positive, 
> negative, or inconsequential change?

This is certainly a feature I've felt the need for in the past, and caching 
to me is the obvious use-case.

Having said that, I recall arguments to the effect that there are better 
ways to implement such things that don't require the use of such a keyword 
and prevent the need to "break" the transitive-const model.

Given this, my current stance is that it should be left out unless it can be 
demonstrated that there is a useful design that cannot be implemented 
without it, and I will be interested to see if I continue to feel the need 
for it when optimising code in future.
May 30, 2008
Re: Poll: a nonstate keyword
On 29/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
> but can break invariant

That part is not correct. A couple of months back, I submitted a paper
to Walter and the gang, based on these ideas, in which I showed that
the transitive closure of const/invariant remained fully intact under
this model. Andrei confirmed that I was correct, and subsequently
tried to help explain it to Walter.

> and introduce subtle bugs.

That part is probably true, but it depends on whether or not people
"get it", which I think is what Steven was asking. Consider the
following (legal) code

   struct A
   {
       static int n;
   }

   invariant A a;
   a.n = 42;

I don't think anyone would argue that "static breaks invariant", but
could you say that code like the above "introduces subtle bugs"?
Maybe. Maybe not. It depends on understanding.

"nonstate" has been known by other names in previous discussions,
including "unpaintable". (There was quite a long thread with that
one). The general feeling over at Phobos is that the number of
use-cases is small enough that a handful of library solutions could
eliminate the need for the keyword. The problem, as I see it, is that
those library solutions have been talked about, but not yet
implemented. Consequently, there is still a perceived need for
"nonstate" - a perceived need which would go away as soon as those
alternative solutions were made available.
May 30, 2008
Re: Poll: a nonstate keyword
On 2008-05-30 08:08:35 +0200, "Janice Caron" <caron800@googlemail.com> said:

> On 29/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
>> but can break invariant
> 
> That part is not correct. A couple of months back, I submitted a paper
> to Walter and the gang, based on these ideas, in which I showed that
> the transitive closure of const/invariant remained fully intact under
> this model. Andrei confirmed that I was correct, and subsequently
> tried to help explain it to Walter.

Indeed there are two ways to introduce mutable space.
The first one that gives access to the mutable space also to pure 
functions (but potentially breaks everything).
The catch is that by default updates to this mutable state might get 
lost due to compiler optimizations (as the compiler is free to float 
pure functions on them out of a loop, or to copy invariant objects), 
and this should be considered ok (which means that the result of the 
program cannot depend on this mutable state).
This can be used to implement efficiently memoization and lazy caching 
also for pure functions (which I want).

The other possibility (that Steven and you propose, if I understood 
correctly) is to disallow pure functions to access mutable state. This 
diminished the usefulness of pure functions or mutable state depending 
from how you look at the problem.
Indeed this solution does not break  invariance.

If implicit automatic copies (without calling a copy method) of object 
or part of it are allowed then the picture becomes more complex, but 
such a thing has a questionable value.

>> and introduce subtle bugs.
> 
> That part is probably true, but it depends on whether or not people
> "get it", which I think is what Steven was asking. Consider the
> following (legal) code
> 
>     struct A
>     {
>         static int n;
>     }
> 
>     invariant A a;
>     a.n = 42;
> 
> I don't think anyone would argue that "static breaks invariant", but
> could you say that code like the above "introduces subtle bugs"?
> Maybe. Maybe not. It depends on understanding.
> 
> "nonstate" has been known by other names in previous discussions,
> including "unpaintable". (There was quite a long thread with that
> one). The general feeling over at Phobos is that the number of
> use-cases is small enough that a handful of library solutions could
> eliminate the need for the keyword. The problem, as I see it, is that
> those library solutions have been talked about, but not yet
> implemented. Consequently, there is still a perceived need for
> "nonstate" - a perceived need which would go away as soon as those
> alternative solutions were made available.

I agree that a good library implementation is what one should strive to-
The question is how to implement them.
Without keyword one could achieve this using fully unsafe casts, I 
think, but is something like that allowed in a pure function?
If the answer is no, then to be able to implement them in plain D in a 
library the keyword would still be needed, or a way to declare an 
unsafe function pure (basically haskell unsafePerformIO).
May 31, 2008
Re: Poll: a nonstate keyword
On 30/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
>  The other possibility (that Steven and you propose, if I understood
> correctly) is to disallow pure functions to access mutable state.

As currently planned, inputs and outputs to pure functions must be
invariant (or implicitly castable to invariant, e.g. int). You can use
mutable data inside the function, but the lifetime of that data is the
scope of the function - it will not persist beyond it.

> Without keyword one could achieve [a good library implementation] using fully
> unsafe casts, I think, but is something like that allowed in a pure function?

I think not. Also, caching could be achieved by a library
implementation without any unsafe casts at all - but it still wouldn't
be allowed in a pure function because it modifies global state.

If you think about it logically, Steven's proposed keyword "nostate"
is a good way of looking at it. A "nostate" variable is not part of
the class's state. It is well known that pure functions cannot access
global state, but more generally, pure functions cannot access
anything which is not part of the state of its inputs. "nostate"
variables would be, by definition, not part of the objects' state, and
therefore inaccessible to pure functions, same as static member
variables.


> If the answer is no, then to be able to implement them in plain D in a library
> the keyword would still be needed, or a way to declare an unsafe function pure
> (basically haskell unsafePerformIO).

I don't think I've completely understood what you're trying to get at.
Saying "basically haskell unsafePerformIO" tells me nothing, because I
don't speak Haskell and have no idea what that means.

One of the advantages of pure functions, however, is that (like
inlining) memoization can be done /by the compiler/, so you don't have
to do it yourself. How good a job the compiler will do remains to be
seen. My guess would be, poor at first, getting increasingly better
over time.
May 31, 2008
Re: Poll: a nonstate keyword
On 2008-05-31 07:40:56 +0200, "Janice Caron" <caron800@googlemail.com> said:

> On 30/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
[...]
>> If the answer is no, then to be able to implement them in plain D in a library
>> the keyword would still be needed, or a way to declare an unsafe function pure
>> (basically haskell unsafePerformIO).
> 
> I don't think I've completely understood what you're trying to get at.
> Saying "basically haskell unsafePerformIO" tells me nothing, because I
> don't speak Haskell and have no idea what that means.

Sorry , basically I want a hack to convince the compiler that even if a 
function looks not pure it actually is.
Maybe a cast, like this (or with an uglier name)
pure int function(T) x=cast(pure int function(T))&y;
basically it says, to the compiler "yes I was careful and you can treat 
this function as pure, just do it, an if there are bug they are my 
fault because I am doing something potentially unsafe".

If there is something like this then no_state becomes useful, otherwise 
t is much less useful.

> One of the advantages of pure functions, however, is that (like
> inlining) memoization can be done /by the compiler/, so you don't have
> to do it yourself. How good a job the compiler will do remains to be
> seen. My guess would be, poor at first, getting increasingly better
> over time.

Sorry that is not good enough for me I want functions lifted out of 
inner loops, and a compiler that says to me either you use the slow 
pure function, and I will lift it out of the loop, or the fast one, but 
it will stay in the loop, is not good enough for me.
I want a hole to be able to trick the compiler into floating out the 
fast function.
In haskell this hole is called unsafePerformIO, and I think that it is 
a good name because when you use it it makes it very clear that you are 
on your own on slippery soil... :)

Fawzi
May 31, 2008
Re: Poll: a nonstate keyword
On 31/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
>  Sorry , basically I want a hack to convince the compiler that even if a
> function looks not pure it actually is.

The one example of this which I can recall, which has been
demonstrated in this forum, is calling a "helper function" from within
a pure function. Is that what you're talking about? If so, it's been
mentioned before. Here's an example:

   void helper(char[] s) // not pure
   {
       s[0] = 'J';
   }

   string foo(string s) pure
   {
       char[] tmp = s.dup;
       helper(tmp);
       return assumeUnique(tmp);
   }

Currently, it seems that the above might not be legal code, since pure
functions cannot call non-pure functions. However, what makes this
case (and others like it) special is that, although helper itself is
not pure, if it were inlined, it wouldn't stop foo from being pure.

This has been pointed out before, and I am /sure/ that Walter will
come up with a solution which makes it possible (though not
necessarily in the first attempt).

I have absolutely no idea, however, what this has to do with Steven's
question. Is it relevant, or are we heading off-topic?



>  Maybe a cast, like this (or with an uglier name)
>  pure int function(T) x=cast(pure int function(T))&y;

The way I read it, the following will work, as is already legal syntax

   auto x = cast(function int(T) pure)&y;


> > One of the advantages of pure functions, however, is that (like
> > inlining) memoization can be done /by the compiler/, so you don't have
> > to do it yourself. How good a job the compiler will do remains to be
> > seen. My guess would be, poor at first, getting increasingly better
> > over time.
>
>  Sorry that is not good enough for me I want functions lifted out of inner
> loops, and a compiler that says to me either you use the slow pure function,
> and I will lift it out of the loop, or the fast one, but it will stay in the
> loop, is not good enough for me.
>  I want a hole to be able to trick the compiler into floating out the fast
> function.

I still don't get it. Can you show me what you mean with an example?
May 31, 2008
Re: Poll: a nonstate keyword
On 2008-05-31 11:37:44 +0200, "Janice Caron" <caron800@googlemail.com> said:

> On 31/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
>> Sorry , basically I want a hack to convince the compiler that even if a
>> function looks not pure it actually is.
> 
> The one example of this which I can recall, which has been
> demonstrated in this forum, is calling a "helper function" from within
> a pure function. Is that what you're talking about? If so, it's been
> mentioned before. Here's an example:
> 
>     void helper(char[] s) // not pure
>     {
>         s[0] = 'J';
>     }
> 
>     string foo(string s) pure
>     {
>         char[] tmp = s.dup;
>         helper(tmp);
>         return assumeUnique(tmp);
>     }
> 
> Currently, it seems that the above might not be legal code, since pure
> functions cannot call non-pure functions. However, what makes this
> case (and others like it) special is that, although helper itself is
> not pure, if it were inlined, it wouldn't stop foo from being pure.
> 
> This has been pointed out before, and I am /sure/ that Walter will
> come up with a solution which makes it possible (though not
> necessarily in the first attempt).
> 
> I have absolutely no idea, however, what this has to do with Steven's
> question. Is it relevant, or are we heading off-topic?

but yes with this we are heading off-topic, what I meant is
1) I want a loophole to be able to access this no state mutable state 
also in pure functions (even if I know that in general this should not 
be allowed as it potentially breaks everything)

2) if there is a way to convince the compiler that a non pure function 
should be treated as a pure function (and this is what I was after 
before) then the "safe" no_state can be used to achieve 1).

If the following works:
>> Maybe a cast, like this (or with an uglier name)
>> pure int function(T) x=cast(pure int function(T))&y;
> 
> The way I read it, the following will work, as is already legal syntax
> 
>     auto x = cast(function int(T) pure)&y;

then basically we have 2)
int potentiallyUnsafe(T x){ ...}

pure int ITellYouItIsSafe(T x){
 return (cast(function int(T) pure)&potentiallyUnsafe)(x);
}

(I am surprised at how harmlessly it looks)
and adding no_state is equivalent with the more powerful (and 
dangerous) version that I want.

>>> One of the advantages of pure functions, however, is that (like
>>> inlining) memoization can be done /by the compiler/, so you don't have
>>> to do it yourself. How good a job the compiler will do remains to be
>>> seen. My guess would be, poor at first, getting increasingly better
>>> over time.
>> 
>> Sorry that is not good enough for me I want functions lifted out of inner
>> loops, and a compiler that says to me either you use the slow pure function,
>> and I will lift it out of the loop, or the fast one, but it will stay in the
>> loop, is not good enough for me.
>> I want a hole to be able to trick the compiler into floating out the fast
>> function.
> 
> I still don't get it. Can you show me what you mean with an example?

This is like a generalization of the array indexing example

starting with a code like this:
 void g(int i, const Pippo x, Pippo y){
	auto t=pureF(x);
	y.doSomething(t,i);
 }

then this loop
 for (int i=0;i<100;i++)
   g(i,x,y);
should be transformed as follow:
inline g:
 for (int i=0;i<100;i++){
	auto t=pureF(x);
	y.doSomething(t,i);
}

move pureF(x) outside the loop:
 auto t=pureF(x);
 for (int i=0;i<100;i++){
	y.doSomething(t,i);
}

This last operation is valid *only* if pureF is pure.
Now imagine that the previous fragment is called several times, and 
that memoization in pureF can improve much its performance.
Then I either have to choose to use memoization (but then the function 
is not pure anymore for D, and it stays in the loop) or avoid 
memoization, and the function is pure but then pay the penalty later 
when the previous code snipped is calculated several times.
I want it all, I want it to be possible to have memoization a purity 
related optimizations.

It seems that with a safe no_state and a cast this is possible, so yes 
I very much want no_state.

Fawzi
May 31, 2008
Re: Poll: a nonstate keyword
On 31/05/2008, Fawzi Mohamed <fmohamed@mac.com> wrote:
>  It seems that with a safe no_state and a cast this is possible, so yes I
> very much want no_state.

But it's likely also to be possible /without/ nostate.

As I noted in my first post on this thread, only a small handful of
use-cases for nostate (aka mutable, aka unpaintable, etc) have been
identified. One of these uses is caching (aka memoization). The
current plan is to make caching available as a library solution, and
so *therefore*, you will not need "nostate" in order to do this.

I have no idea whether or not the library solution will be considered
pure. I imagine that if it can be proven safe, then it will count as
pure. If not, well, there's always explicit casting.

So, given that, do you /still/ want nostate? Remember that (like
explicit casts and unions) it opens the door to /serious/ abuse of the
const system, just like in C++. Inexperienced programmers may well
think to themselves: "I can't get this program to compile, unless I
make member x mutable, so I'll do that". It seems to me that instead
of opening that door, a better solution is (1) identify all reasonable
use-cases (and we've only managed to come up with three or four so
far, so even if we've missed a few, it's clearly not a big number),
and then (2) provide alternative solutions for each of those
use-cases.

What do you think?
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home