December 22, 2009
Walter Bright wrote:
> Yigal Chripun wrote:
>> But that doesn't mean the idea itself isn't valid. Perhaps a different language with different goals in mind can provide a much simpler non convoluted implementation and semantics for the same idea?
>> You've shown in the past that you're willing to break backward compatibility in the name of progress and experiment with new ideas. You can make decisions that the C++ committee will never approve.
>>
>> Doesn't that mean that this is at least worth a shot?
> 
> I believe that D's template constraint feature fills the bill, it does everything Concepts purported to do, and more, in a simple and easily explained manner, except check the template body against the constraint.
> 
> The latter is, in my not-so-humble opinion, a desirable feature but its desirability is overwhelmed by the payment in complexity and constrictions on the Concepts necessary to make it work.

I seriously wonder why you're saying that, while at the same time clinging on overcomplicated failures such as const/immutable/pure or auto ref etc...
December 22, 2009
grauzone wrote:
> Walter Bright wrote:
>> I believe that D's template constraint feature fills the bill, it does everything Concepts purported to do, and more, in a simple and easily explained manner, except check the template body against the constraint.
>>
>> The latter is, in my not-so-humble opinion, a desirable feature but its desirability is overwhelmed by the payment in complexity and constrictions on the Concepts necessary to make it work.
> 
> I seriously wonder why you're saying that, while at the same time clinging on overcomplicated failures such as const/immutable/pure or auto ref etc...

Because there is a large payoff to immutability and purity, and they are far simpler than Concepts.

Consider that Concepts required more pages to specify than the entire template feature in C++.

I can expound on the huge advantages immutability and purity offer, if you want.
December 22, 2009
Walter Bright Wrote:

> Yigal Chripun wrote:
> > But that doesn't mean the idea itself isn't valid. Perhaps a different
> > language with different goals in mind can provide a much simpler non
> > convoluted implementation and semantics for the same idea?
> > You've shown in the past that you're willing to break backward
> > compatibility in the name of progress and experiment with new ideas. You
> > can make decisions that the C++ committee will never approve.
> > 
> > Doesn't that mean that this is at least worth a shot?
> 
> I believe that D's template constraint feature fills the bill, it does everything Concepts purported to do, and more, in a simple and easily explained manner, except check the template body against the constraint.
> 
> The latter is, in my not-so-humble opinion, a desirable feature but its desirability is overwhelmed by the payment in complexity and constrictions on the Concepts necessary to make it work.

could you please expand on what are the main issues with implementing that check? I also wonder what's the situation regarding this in other languages - C# has constraints IIRC - is the check performed there?

IMO cconstraints are neat but they aren't perfect. For one, they need to be repeated for each template, it would be very awesome if that could be moved to the parameter of the template so instead of:

template foo(T) if isRange!T ...
template bar(T) if isRange!T ...

you could write something like:
struct Range if ... {}
template foo(r : Range) ...
template bar(r : Range) ...

inside both templates the parameter r satisfies all the constraits of Range.

does that sound reasonable at all?
December 22, 2009
Walter Bright wrote:
> I can expound on the huge advantages immutability and purity offer, if you want.

Yes, I'd like to hear about this. You can leave away the things that are not going to be implemented (like memorization of pure return values), are only micro-optimizations (common sub-expression elimination with pure functions?), which don't work (immutable was used to make strings read-only, but you can stomp over immutable arrays), which were thought to be useful, but nothing has materialized yet (something like immutable was supposed to be the cure for multithreading)...

Over two years have passed since immutable/const was added to dmd, but I couldn't see any benefit yet. But lots of new compiler bugs. There's the danger that those feature are all nothing but hot air in real programming. I like D, and it sure would relief me to hear that this is not the case.
December 22, 2009
yigal chripun wrote:
> Walter Bright Wrote:
> 
>> Yigal Chripun wrote:
>>> But that doesn't mean the idea itself isn't valid. Perhaps a
>>> different language with different goals in mind can provide a
>>> much simpler non convoluted implementation and semantics for the
>>> same idea? You've shown in the past that you're willing to break
>>> backward compatibility in the name of progress and experiment
>>> with new ideas. You can make decisions that the C++ committee
>>> will never approve.
>>> 
>>> Doesn't that mean that this is at least worth a shot?
>> I believe that D's template constraint feature fills the bill, it
>> does everything Concepts purported to do, and more, in a simple and
>> easily explained manner, except check the template body against the
>> constraint.
>> 
>> The latter is, in my not-so-humble opinion, a desirable feature but
>> its desirability is overwhelmed by the payment in complexity and constrictions on the Concepts necessary to make it work.
> 
> could you please expand on what are the main issues with implementing
> that check?

Because the constraint can have any computation in it. So how does one verify that all computations in the template body are represented in the constraint in the same way?

> I also wonder what's the situation regarding this in
> other languages - C# has constraints IIRC - is the check performed
> there?

I don't know C# constraints.


> IMO cconstraints are neat but they aren't perfect. For one, they need
> to be repeated for each template, it would be very awesome if that
> could be moved to the parameter of the template so instead of:
> 
> template foo(T) if isRange!T ... template bar(T) if isRange!T ...
> 
> you could write something like: struct Range if ... {} template foo(r
> : Range) ... template bar(r : Range) ...
> 
> inside both templates the parameter r satisfies all the constraits of
> Range.
> 
> does that sound reasonable at all?

The template parameter list was already fairly complicated, I thought that adding in the constraints would make it impenetrable. Also, the parameter list involves the individual parameters, whereas the constraint involves any combination of them.

Adding in the constraints to the parameters also has unknown consequences for figuring out partial ordering. Partial ordering is based on abstract types, whereas constraints are based on actual types.
December 22, 2009
grauzone wrote:
> Walter Bright wrote:
>> I can expound on the huge advantages immutability and purity offer, if you want.
> 
> Yes, I'd like to hear about this. You can leave away the things that are not going to be implemented (like memorization of pure return values), are only micro-optimizations (common sub-expression elimination with pure functions?), which don't work (immutable was used to make strings read-only, but you can stomp over immutable arrays), which were thought to be useful, but nothing has materialized yet (something like immutable was supposed to be the cure for multithreading)...

The short answer is: the benefits of functional programming.

The longer answer:

1. optimizations - yes, the optimizer does take advantage of immutability and purity, and yes, those optimizations are minor. But they do exist and do work.

2. immutable reference types can be treated as if they were value types - this advantage is most obvious in strings. Without immutability (and this happens in C, C++, and D1) programmers tend to make their own private copies of things "just in case" someone else changes them.

3. (2) has a great advantage in doing message passing between threads. This model was popularized by Erlang, and is very successful. You can do message passing without immutable references, but you've got to hope and pray that your programming team didn't make any mistakes with it. With immutable references, you have a statically enforced guarantee. Value types (and immutable references) do not need synchronization.

4. immutability and purity enable user reasoning about a program. Otherwise, you have to rely on the (probably wrong) documentation about a function to see what its effects are, and if it has any side effects.

Yes, there was a recently discovered bug which enabled modifying an immutable array. This was a bug, and has been fixed. A bug does not mean the concept is broken.


> Over two years have passed since immutable/const was added to dmd, but I couldn't see any benefit yet. But lots of new compiler bugs. There's the danger that those feature are all nothing but hot air in real programming. I like D, and it sure would relief me to hear that this is not the case.

The message passing threading advantage awaits the construction of a message passing library. Sean Kelly is working on it.
December 22, 2009
Walter Bright wrote:
> Yigal Chripun wrote:
>> But that doesn't mean the idea itself isn't valid. Perhaps a different language with different goals in mind can provide a much simpler non convoluted implementation and semantics for the same idea?
>> You've shown in the past that you're willing to break backward compatibility in the name of progress and experiment with new ideas. You can make decisions that the C++ committee will never approve.
>>
>> Doesn't that mean that this is at least worth a shot?
> 
> I believe that D's template constraint feature fills the bill, it does everything Concepts purported to do, and more, in a simple and easily explained manner, except check the template body against the constraint.
> 
> The latter is, in my not-so-humble opinion, a desirable feature but its desirability is overwhelmed by the payment in complexity and constrictions on the Concepts necessary to make it work.

I think a consequence of that, is that facilities for compile-time testing become quite important, since we're relying on testing rather than compile-time checks to eliminate bugs.
So I'm delighted that the static assert backtrace patch has been implemented.
(One very useful feature would be code-coverage of template instantiations -- which lines of a template have actually been instantiated?)
December 22, 2009
On 22.12.2009 5:15, Walter Bright wrote:
> Yigal Chripun wrote:
>> But that doesn't mean the idea itself isn't valid. Perhaps a different
>> language with different goals in mind can provide a much simpler non
>> convoluted implementation and semantics for the same idea?
>> You've shown in the past that you're willing to break backward
>> compatibility in the name of progress and experiment with new ideas.
>> You can make decisions that the C++ committee will never approve.
>>
>> Doesn't that mean that this is at least worth a shot?
>
> I believe that D's template constraint feature fills the bill, it does
> everything Concepts purported to do, and more, in a simple and easily
> explained manner, except check the template body against the constraint.

...and template overloading based on concept refinement. D requires hacks to accomplish that:

template isFoo(T)
{
}

template isBar(T)
{
    enum isBar = ... && isFoo!T;
}

template Foo(T) if (isFoo!T && !isBar!T /+ hack +/) {}
template Foo(T) if (isBar!T) {}

In the run-time domain, it would be analogous to requiring a parameter to specify which interfaces the argument *should not* implement:

interface IFoo
{
}

interface IBar : IFoo
{
}

void foo (IBar a)
{
}

void foo ((IFoo && !IBar) a) // meh
{
}

Maybe it is not a significant shortcoming but it needs to be mentioned.

>
> The latter is, in my not-so-humble opinion, a desirable feature but its
> desirability is overwhelmed by the payment in complexity and
> constrictions on the Concepts necessary to make it work.
December 23, 2009
Walter Bright wrote:
> grauzone wrote:
>> Walter Bright wrote:
>>> I can expound on the huge advantages immutability and purity offer, if you want.
>>
>> Yes, I'd like to hear about this. You can leave away the things that are not going to be implemented (like memorization of pure return values), are only micro-optimizations (common sub-expression elimination with pure functions?), which don't work (immutable was used to make strings read-only, but you can stomp over immutable arrays), which were thought to be useful, but nothing has materialized yet (something like immutable was supposed to be the cure for multithreading)...
> 
> The short answer is: the benefits of functional programming.

I can see how D benefits from some functional language features (like those that increase expressiveness), but not the immutability thing. It's a good idea, but there's some tradeoff. And immutability can still be handled as a concept outside of the language type system.

> The longer answer:
> 
> 1. optimizations - yes, the optimizer does take advantage of immutability and purity, and yes, those optimizations are minor. But they do exist and do work.
> 
> 2. immutable reference types can be treated as if they were value types - this advantage is most obvious in strings. Without immutability (and this happens in C, C++, and D1) programmers tend to make their own private copies of things "just in case" someone else changes them.

With string, you used to follow the copy-and-write protocol. Now you're doing the same (you have to re-instantiate immutable data to change it), just that the compiler forces you. This can go good for reliability, but it also takes a lot of flexibility. Plus you have to deal with the complications of the type system now.

> 3. (2) has a great advantage in doing message passing between threads. This model was popularized by Erlang, and is very successful. You can do message passing without immutable references, but you've got to hope and pray that your programming team didn't make any mistakes with it. With immutable references, you have a statically enforced guarantee. Value types (and immutable references) do not need synchronization.

But you need to allocate this data from a shared garbage collection, which again slow down the whole thing. Is there really an advantage over copying?

For large portions of data you could (at least in theory) make it _actually_ read-only by using mprotect (make the memory pages read-only).

> 
> 4. immutability and purity enable user reasoning about a program. Otherwise, you have to rely on the (probably wrong) documentation about a function to see what its effects are, and if it has any side effects.

If the program logic gets more complicated because of the type system, this isn't going to help much. Now I see you applying language hacks like DIP2 to reduce the damage. Is there an end to it?

> Yes, there was a recently discovered bug which enabled modifying an immutable array. This was a bug, and has been fixed. A bug does not mean the concept is broken.

Sure, but the question is: will all those bugs ever to be fixed? There are old and central language core features which _still_ can trigger dmd bugs as of today (like forward references and circular module imports). I don't mean to be insolent, but I think the language is going to be too huge for one compiler writer. Yes, you're the one who knows best how far he can go, but I as a clueless user suffering from dmd bugs think the limit must already have been reached.

(btw. here's another one, but it's also going to be fixed soon: http://d.puremagic.com/issues/show_bug.cgi?id=2093)

Also, how much is this reliability worth if you can just cast away immutable? It's even exactly the same syntax you have to use for relatively harmless things, like casting a float to an integer.

> 
>> Over two years have passed since immutable/const was added to dmd, but I couldn't see any benefit yet. But lots of new compiler bugs. There's the danger that those feature are all nothing but hot air in real programming. I like D, and it sure would relief me to hear that this is not the case.
> 
> The message passing threading advantage awaits the construction of a message passing library. Sean Kelly is working on it.

That's nice, but it would be possible without immutable.
December 23, 2009
grauzone wrote:
> Walter Bright wrote:
>> grauzone wrote:
>>> Walter Bright wrote:
>>>> I can expound on the huge advantages immutability and purity offer, if you want.
>>>
>>> Yes, I'd like to hear about this. You can leave away the things that are not going to be implemented (like memorization of pure return values), are only micro-optimizations (common sub-expression elimination with pure functions?), which don't work (immutable was used to make strings read-only, but you can stomp over immutable arrays), which were thought to be useful, but nothing has materialized yet (something like immutable was supposed to be the cure for multithreading)...
>>
>> The short answer is: the benefits of functional programming.
> 
> I can see how D benefits from some functional language features (like those that increase expressiveness), but not the immutability thing. It's a good idea, but there's some tradeoff. And immutability can still be handled as a concept outside of the language type system.

You can certainly do immutability as a convention, but I contend that is unreliable and does not scale. It's like saying you can write C code that doesn't have buffer overflows.

I have been using C since before it had function prototypes. It was just all kinds of win to add the prototypes, because then the manual checking got handed over to the compiler.


>> 2. immutable reference types can be treated as if they were value types - this advantage is most obvious in strings. Without immutability (and this happens in C, C++, and D1) programmers tend to make their own private copies of things "just in case" someone else changes them.
> 
> With string, you used to follow the copy-and-write protocol. Now you're doing the same (you have to re-instantiate immutable data to change it), just that the compiler forces you. This can go good for reliability, but it also takes a lot of flexibility. Plus you have to deal with the complications of the type system now.

My experience is that relying on convention to follow the protocol does not work. I think that the evidence in the field that it doesn't work is pretty compelling as well.


>> 3. (2) has a great advantage in doing message passing between threads. This model was popularized by Erlang, and is very successful. You can do message passing without immutable references, but you've got to hope and pray that your programming team didn't make any mistakes with it. With immutable references, you have a statically enforced guarantee. Value types (and immutable references) do not need synchronization.
> 
> But you need to allocate this data from a shared garbage collection, 

You do anyway.

> which again slow down the whole thing. Is there really an advantage over copying?

Copying will invoke the garbage collector. Since you argued that is slow, then avoiding the necessity of doing so will make it faster.


> For large portions of data you could (at least in theory) make it _actually_ read-only by using mprotect (make the memory pages read-only).

Compile time checking is better than runtime checking.


>> 4. immutability and purity enable user reasoning about a program. Otherwise, you have to rely on the (probably wrong) documentation about a function to see what its effects are, and if it has any side effects.
> 
> If the program logic gets more complicated because of the type system, this isn't going to help much. Now I see you applying language hacks like DIP2 to reduce the damage. Is there an end to it?

C function prototypes increased the complexity, but it was darn well worth it. You can either have more complexity in the language, or you can spend endless hours manually checking to see if convention was followed - and even then you can't be sure.


>> Yes, there was a recently discovered bug which enabled modifying an immutable array. This was a bug, and has been fixed. A bug does not mean the concept is broken.
> 
> Sure, but the question is: will all those bugs ever to be fixed?

Forgive me, but every month 20 to 40 bugs get fixed. You can see it in the change log. I don't understand these complaints.


> Also, how much is this reliability worth if you can just cast away immutable? It's even exactly the same syntax you have to use for relatively harmless things, like casting a float to an integer.

It's not allowed in @safe functions.

> That's nice, but it would be possible without immutable.

Again, relying on convention has shown, in practice, to NOT WORK when it comes to making reliable multithreaded programs.