Now that http://wiki.dlang.org/Release_Process is slowly adopted I want to rise this discussion again.

Two somewhat contradictory aims meet here, both frequently raised in IRC and newsgroup:

1) Breaking user code with release is incredibly painful and brings lot of dissatisfaction.
2) Language specification is not mature enough yet and it will need to be changed at some point unless we want to stay with same design issues forever (D3 is forever enough).

As far as I see it, both points have a quite wide support in the community.

It is important to note that when I speak of breaking code and language changes, I mean ALL changes. Currently weird hypocrisy has place when bug fixes that involve changing language semantics are not considered to be backwards-incompatible and this actually every release breaks code in practice. As dmd front-end IS the specification currently, there is no way for user to know if certain behavior is bug or feature thus any semantics change must be considered breaking change.

That is it, now we are in position when we both need to change stuff and can't. Every time this issue reappears, new hacks and workarounds are introduced without addressing it in general. I think this can be solved via some strict additions to the release process.

I'd gladly hear any proposals on topic, but here is mine:

Currently bug fixes go only to last major version branches. Considering how fast major version changes (and that we don't have minor versions in practice) that is hardly different from fixing only one branch. This is what makes backwards-compatibility problem that unpleasant - there are no versions of compiler user can stick to for a longer time period with hard 100% guarantees no semantics change will happen, receiving non-breaking fixes at the same time. There are no LTS releases. I think adding ones will allow to remove breaking change ban from master and allow continuous improvement of language design with no hard consequences for real-world users.

By LTS I mean certain announced version that gets back-ported all bug fixes that does not introduce any semantics change. To be usable in practice I thing those need to be released once a year with two LTS releases supported at time. Announcing, both via dlang.org and D.announce, is important here. That will require considerable efforts and may slow down development on master but at least it is step aside from current stalemate.

Opinions/proposals?

I don't know much about the development process, but as you
said some bugs may be features or vice versa. Sometimes real
bugs are fixed and peoples' code breaks. But keeping the bug
around isn't an option.
The next code breakage comes from making array slices
consistently rvalues (the slice structure itself, not the
data). It's not a new idea, like introducing immutable, just
wasn't correctly implemented from the start. I don't know if
this warrants a LTS release already. The problem will be
obvious and easy to fix by introducing a temp variable to pass
as lvalue into functions taking slices by ref. Or changing
those functions to auto-ref.

P.S. Short discussion in IRC have shown that intent of the post is not clear enough: I am not here to rant about stability or how bad breaking changes are. Only thing I do want is to define some means to differentiate needs of those who want changes to improve language and those that require absolute stability, so that then can happily co-exist.

On Friday, 8 March 2013 at 20:07:50 UTC, Dicebot wrote:
> Opinions/proposals?

I completely agree that the needs of users who want stability over everything are not being met. There's no way to choose to get just the updates that don't break code (such as non-breaking bug fixes), and I think you're right in that there should be a way to do that.

However, I just want to make clear that there's another (probably rather big) camp out there: the camp that thinks the current D platform needs a lot of improvement and are more than willing to accept breaking improvements.

I want more language features to be implemented and cleaned up. I want Phobos to be changed, cleaned up, trimmed as well as expanded. All this regardless of the cost of breaking code. I don't mind fixing broken code if the replacement code is simply better.

Thankfully, both the language and standard library are currently being improved in such ways, but not without a fair amount of inertia from parts of the community (particularly from Walter) who want stability at all costs.

For example, I don't think the current std.process is adequate at all and it would feel like a defeat if we have to name the new one std.process2 and have it live alongside the old trash (unless it's temporary). What do you think new users will think when they see or hear about the story around this? Many other standard modules are in the same position or are going to be in the same position at some point in the foreseeable future.

I just hope we can find a way to satisfy both camps, both ensuring the viability of the language at present by providing a stable interface, as well as securing the future of the language by iteratively improving upon it without compromising at every turn. Having LTS releases might be a good way to do that.

Marco Leise:

> I don't know much about the development process, but as you
> said some bugs may be features or vice versa. Sometimes real
> bugs are fixed and peoples' code breaks. But keeping the bug
> around isn't an option.

Fixing issues like this is right, despite they will cause some breakage in user code.

In DMD 2.063alpha this code gives a warning (http://d.puremagic.com/issues/show_bug.cgi?id=7444 ):

void main() {
    int[10] a;
    a = 1;
}


temp.d(3): Warning: explicit element-wise assignment (a)[] = 1 is better than a = 1


My suggestion for D users is to update their DMD compiler often, to avoid cumulating many breakages at the same time, that is quite bad. It's better to fix one thing at a time.

All bug fixes are potential code breakages, but not all breakages have the same magnitude an importance. My suggestion for D developers is to focus their efforts first on the D/DMD changes that will cause most breaking because the more time passes, the more D2 code is around, and the more breakage will happen.

This also means implementing things like in/scope sooner than later, that probably are sometimes used wrongly, and that will cause some breakage once implemented. Another similar case is "final switch" on ints that is currently broken (http://d.puremagic.com/issues/show_bug.cgi?id=5713) and likely misused in some D program. So once it's fixed it will break some user code.


Time ago I have suggested to statically disallow code like this (starting first with a deprecation, a warning, and then an error), that currently compiles without errors or warnings:


string[4] colors = ["red",
                    "blue"
                    "green",
                    "yellow"];
void main() {}


If similar code will be statically disallowed, it will break some user code (Walter agreed to forbid half of that combo-pitfall).


I have a bit of statistical proof that code like this is a common source of bugs, and I'd like this to be statically disallowed (http://d.puremagic.com/issues/show_bug.cgi?id=5409 ):

(!x & y)

But disallowing this will break some user code (not much, because it's likely a bug in the first place).

Bye,
bearophile

"bearophile" <bearophileHUGS@lycos.com> wrote in message news:hohjkspdzavtiqxqimms@forum.dlang.org...
>
> string[4] colors = ["red",
>                     "blue"
>                     "green",
>                     "yellow"];
> void main() {}
>

I hate this, a lot.

> I have a bit of statistical proof that code like this is a common source of bugs, and I'd like this to be statically disallowed (http://d.puremagic.com/issues/show_bug.cgi?id=5409 ):
>
> (!x & y)
>

This would be a good dmd patch for you to tackle bearophile.

Should be as simple as adding a check to the semantic routine of AndExp, looking at what type of expression e1 is.

On Friday, 8 March 2013 at 21:07:23 UTC, Jakob Ovrum wrote:
> On Friday, 8 March 2013 at 20:07:50 UTC, Dicebot wrote:
>> Opinions/proposals?
>
> I completely agree that the needs of users who want stability over everything are not being met. There's no way to choose to get just the updates that don't break code (such as non-breaking bug fixes), and I think you're right in that there should be a way to do that.
>
> However, I just want to make clear that there's another (probably rather big) camp out there: the camp that thinks the current D platform needs a lot of improvement and are more than willing to accept breaking improvements.
>
> I want more language features to be implemented and cleaned up. I want Phobos to be changed, cleaned up, trimmed as well as expanded. All this regardless of the cost of breaking code. I don't mind fixing broken code if the replacement code is simply better.
>
> Thankfully, both the language and standard library are currently being improved in such ways, but not without a fair amount of inertia from parts of the community (particularly from Walter) who want stability at all costs.
>
> For example, I don't think the current std.process is adequate at all and it would feel like a defeat if we have to name the new one std.process2 and have it live alongside the old trash (unless it's temporary). What do you think new users will think when they see or hear about the story around this? Many other standard modules are in the same position or are going to be in the same position at some point in the foreseeable future.
>
> I just hope we can find a way to satisfy both camps, both ensuring the viability of the language at present by providing a stable interface, as well as securing the future of the language by iteratively improving upon it without compromising at every turn. Having LTS releases might be a good way to do that.

Personally I think we need to consider D3 as breaking. We can leave out any major changes till then or at least that would be nice. That way we can use D2 for LTS once we begin on D3.
How ever this will bring back e.g. the old python 2.x vs 3.x divide which would be a shame. But at the same time it'll mean we can do some big stuff that would just not be acceptable in breaking old projects.

On Friday, 8 March 2013 at 20:33:20 UTC, Marco Leise wrote:
> I don't know much about the development process, but as you
> said some bugs may be features or vice versa. Sometimes real
> bugs are fixed and peoples' code breaks. But keeping the bug
> around isn't an option.

The problem isn't really breaking the code. It is known to be necessary on some subjects. It is more about how it is done.

> The next code breakage comes from making array slices
> consistently rvalues (the slice structure itself, not the
> data). It's not a new idea, like introducing immutable, just
> wasn't correctly implemented from the start. I don't know if
> this warrants a LTS release already. The problem will be
> obvious and easy to fix by introducing a temp variable to pass
> as lvalue into functions taking slices by ref. Or changing
> those functions to auto-ref.

Very good example (many code of mine broke on that one).

Yes this is the way things should have been from day 1. And solving that is clearly the way to go.

However, this change is right now into master and will go out with the next version of D, breaking a truly large amount of code (most code that uses slice as range will break).

This isn't the type of change you can release without any damage mitigation plan.

On Saturday, March 09, 2013 04:38:08 Rikki Cattermole wrote:
> Personally I think we need to consider D3 as breaking. We can
> leave out any major changes till then or at least that would be
> nice. That way we can use D2 for LTS once we begin on D3.
> How ever this will bring back e.g. the old python 2.x vs 3.x
> divide which would be a shame. But at the same time it'll mean we
> can do some big stuff that would just not be acceptable in
> breaking old projects.

D3 will not be happening any time soon. D2 will have to have been stable and around for a while before we plan to even consider it.

- Jonathan M Davis

On Friday, 8 March 2013 at 20:07:50 UTC, Dicebot wrote:
> Now that http://wiki.dlang.org/Release_Process is slowly adopted I want to rise this discussion again.
>
> Two somewhat contradictory aims meet here, both frequently raised in IRC and newsgroup:
>
> 1) Breaking user code with release is incredibly painful and brings lot of dissatisfaction.
> 2) Language specification is not mature enough yet and it will need to be changed at some point unless we want to stay with same design issues forever (D3 is forever enough).

When a bug in the language design or compiler/libraries is fixed; the changes needed to upgrade aren't nearly as draining because it is the way forward. It is where the language should be going. But hitting a regression, waiting for the next release in hopes that a new regression won't crop up (I'm just assuming compiler is released with all known regressions completed) is detrimental to expectations. Updating code to match the "new" way just to have it reverted the next week (in git, thus no release); those things are what really kill the idea of stability.

So I would hope that a release is maintained until the next release. Regression and non-breaking bug fixes get applied to the release branch and are either released before the next release or at the same time. The point being that whatever issue was found was hopefully fixed and likely nothing new will be in the way to upgrade. A more elaborate system is fine, but I believe this truly the issues that need addressed first.

As for those who want no breaking changes, don't update your compiler or pay DigitalMars to maintain it. D will get to a point breaking changes aren't so frequent. But right now we have issue that must go.