Robert wrote:
> When reading stuff, like: "Yes this is bad, but people use it already, so we could only possible change this in D3 or something" and reading endless discussions about a new feature (e.g. ref semantics) of how it could break things and so on, I thought it might be a good idea to implement new features in an experimental version, which can then be thoroughly tested and only if nothing bad found they will be merged in the stable branch. People simply have to be aware that they should not rely on semantics implemented in experimental.
> 
> Discussions about new features before their are implemented would of course still be a very good idea, but it would reduce the pressure a bit, because you can simply try. This does not solve everything, because some issues will only pop up if used for a long time or only in real complicated production code, but I think it is better than the approach of having no way back?
> 
> I don't believe this idea is entirely new or maybe I am missing something. What do you think?

Sounds useful to me.
How would you implement it? I fear it complicates the compiler a lot
and hence introduces bugs.

Jens

On 10/24/2012 09:38 AM, Robert wrote:
> When reading stuff, like: "Yes this is bad, but people use it already,
> so we could only possible change this in D3 or something" and reading
> endless discussions about a new feature (e.g. ref semantics) of how it
> could break things and so on, I thought it might be a good idea to
> implement new features in an experimental version, which can then be
> thoroughly tested and only if nothing bad found they will be merged in
> the stable branch. People simply have to be aware that they should not
> rely on semantics implemented in experimental.

The problem AFAICS is not _new_ features, or even stability per se -- the problem is modifying the behaviour of existing features, even if the modification is an improvement.

If you modify an existing feature, then it means that you will break downstream users' code -- not because your modification is wrong per se, but because that downstream code assumes that this feature will behave a certain way.  The larger your downstream codebase, the more problematic that becomes, hence the reason why certain kinds of change have to be limited to "major version" updates.

Also, it's not just a problem of causing work for downstream developers, but also about the sense of security/stability people have with D, that they are building on solid ground and not quicksand.  If people see features changing too often, they are likely to be unwilling to adopt D as a development platform.

D already has a deprecation path for features that really need to be changed, and where necessary this _does_ happen, but it's clear from past list discussions that this comes at a cost to D's reputation as a ready-for-production-use development language.

An experimental branch is a nice idea for other purposes, but it doesn't solve the problem you've identified.

Simply don't release a new stable version with new features, semantics, but first release an experimental version. Encourage people to compile their code with it, try out the new features/semantics, ... but always let them keep in mind, that things can break with further improvements. The idea is that features in an experimental release can be changed at any time without having to worry about breaking users code, because they expect it. It seems to me that until now things are just discussed very thoroughly, implemented and released. If this is really the case, then I guess the only reason this works so surprisingly well is because the people (Andrei, Walter, ...) are amazingly smart and have learned from the mistakes of other languages. But introducing a (limited) way of learning from own mistakes without breaking things, might be useful especially for new concepts not found in other languages.

I don't see how this complicates the compiler?

People could choose between using the experimental version for the awesome feature xy and taking the risk of fixing code with incompatible changes or stick with the stable version.

Robert
> 
> Sounds useful to me.
> How would you implement it? I fear it complicates the compiler a lot
> and hence introduces bugs.
> 
> Jens

On 10/24/12 6:24 AM, Robert Klotzner wrote:
> People could choose between using the experimental version for the
> awesome feature xy and taking the risk of fixing code with incompatible
> changes or stick with the stable version.

That fosters balkanization.

At this point in D's evolution I very strongly think it's the time to put to good use its many features, instead of making it any easier to add more. Constraints are liberating. Innovation can be found in the minds of people, and the thought that there's always a new feature that can be added instead of an innovative solution within the existing language stifles creativity.


Andrei

> At this point in D's evolution I very strongly think it's the time to put to good use its many features, instead of making it any easier to add more. Constraints are liberating. Innovation can be found in the minds of people, and the thought that there's always a new feature that can be added instead of an innovative solution within the existing language stifles creativity.

I completely agree, D's feature set is already very good and well thought of. But having more thorough testing, might still be a good idea. In fact I am very happy with D's feature set, but as long as D is alive there will be improvements from time to time and having a way of testing them thoroughly, can't really harm the language. But I don't know. You and other people more involved in development are of course better judges of usefulness of this proposal. Basically I was just curious why it is not done this way.

Robert Klotzner wrote:
> Simply don't release a new stable version with new features, semantics, but first release an experimental version. Encourage people to compile their code with it, try out the new features/semantics, ... but always let them keep in mind, that things can break with further improvements. The idea is that features in an experimental release can be changed at any time without having to worry about breaking users code, because they expect it. It seems to me that until now things are just discussed very thoroughly, implemented and released. If this is really the case, then I guess the only reason this works so surprisingly well is because the people (Andrei, Walter, ...) are amazingly smart and have learned from the mistakes of other languages. But introducing a (limited) way of learning from own mistakes without breaking things, might be useful especially for new concepts not found in other languages.
> 
> I don't see how this complicates the compiler?
> 
> People could choose between using the experimental version for the awesome feature xy and taking the risk of fixing code with incompatible changes or stick with the stable version.

But this complicates the development of the compiler. There may be even
incompatible features. What do you do then?
Just because you release an experimental version of the compiler does
not ease integration of experimental features. Further at some point you
have to integrate a feature into the stable line.
I believe this is possible using a particular git work flow but I think
it's far from easy. Further there needs to be some API between parts of
the compiler. Maybe one can learn from the development of the Linux
kernel.
I'd like to have this but it's not as straightforward as it seems.

Jens

On Wednesday, 24 October 2012 at 11:01:13 UTC, Robert Klotzner wrote:
In fact I am very happy with D's feature set, but as long
> as D is
> alive there will be improvements from time to time and having a way of
> testing them thoroughly, can't really harm the language. But I don't
> know. You and other people more involved in development are of course
> better judges of usefulness of this proposal. Basically I was just
> curious why it is not done this way.

It was, it created the D1 and D2 split. So to do it again, even if we don't call it D3 and just say it is experimental, it is still the same split.

As for testing the changes going in thoroughly. All source is public and can be compiled and verified by anyone at any time for any project they have access to. There is a Beta released prior to the official release, it is a time for and one to speak up of regressions and other issues before the release is finalized. Extending this period isn't going to add much value to having strong verification.

D2 is at/near/moving toward being stable so this is a good system at this time. Not to say it can't be improved, but the improvement will really be viable when more people see the stability and begin participating and adding to the quantity of verification that regressions are not being introduced.

Andrei Alexandrescu:

> That fosters balkanization.
>
> At this point in D's evolution I very strongly think it's the time to put to good use its many features, instead of making it any easier to add more. Constraints are liberating. Innovation can be found in the minds of people, and the thought that there's always a new feature that can be added instead of an innovative solution within the existing language stifles creativity.

We should also think what's the purpose of D development, that is what does it mean for D to be "successful".

Its adoption patterns in more than ten years let me think that maybe D can't aim at becoming a very widely used language.

For a lot of time (15-20 years) Haskell has shown to seek for other meanings of the word "success". And indeed the GHC Haskell compiler contains many (I count 70) language extensions, some of them are major features:

http://www.haskell.org/ghc/docs/7.2.1/html/users_guide/flag-reference.html#id639065

There is plenty of creativity in inventing, improving, implementing and using such extensions too :-)

Bye,
bearophile

On 10/24/12 5:21 PM, bearophile wrote:
> Andrei Alexandrescu:
>
>> That fosters balkanization.
>>
>> At this point in D's evolution I very strongly think it's the time to
>> put to good use its many features, instead of making it any easier to
>> add more. Constraints are liberating. Innovation can be found in the
>> minds of people, and the thought that there's always a new feature
>> that can be added instead of an innovative solution within the
>> existing language stifles creativity.
>
> We should also think what's the purpose of D development, that is what
> does it mean for D to be "successful".
>
> Its adoption patterns in more than ten years let me think that maybe D
> can't aim at becoming a very widely used language.

This reasoning is flawed. We're talking about an evolving process, with a lot happening in the past couple of years.

Andrei

Andrei Alexandrescu:

> This reasoning is flawed. We're talking about an evolving process, with a lot happening in the past couple of years.

There is no reasoning flaw in looking at a bigger picture than just the last two years :-)

In the last years I've seen languages like Go, Scala, Clojure and even silly things like CoffeeScript pop out from almost nothing and become widely more used compared to D. Today D is not nearly as used as Scala.

So I can't tell the future, and maybe D2 will have a glorious future, but from the signs I can tell D2 is not having a fast spreading. It's a complex language, it requires a good amount of time to be learnt (D1 was an important stepping stone to learn it, because D1 was simpler) it has significant problems, new programmers are more trained in Java and JavaScript and they often don't like system languages (and many firms don't need system level programs).

Today a system language, even if widely successful, is going to be a niche, maybe 100 times less used compared to JavaScript-level languages.

So in my precedent post I have suggested to remove our blinders a moment and think about what does it means to be "successful". There is more than one meaning, as Algol and Haskell show very well :-)

Bye,
bearophile