On 7/15/2012 2:58 PM, Adam Wilson wrote:
> The idea that bugs and new features can and should be rolled into the same
> release runs counter to every accepted best practice in both FOSS and Commercial
> wisdom. The two have VERY different velocities, bugs can be fixed in days, but
> new features take much longer. Consider COFF support for example, Walter has
> been hammering away at it for weeks now, and he isn't even 50% done, how many
> bugs have been fixed and confirmed resolved, in the same timespan?

Weeks is an exaggeration. And still, there have been a steady accumulation of fixes.

> Also,
> consider that adding new features makes it significantly harder to track down
> regressions (is a real regression or did the new feature upset the code in an
> unexpected way) and the new features themselves create new bugs. If the branches
> are separate then it becomes trivial to determine if the new feature caused the
> bug, because it will show up in one and not the other.
>
> How DARE we DEMAND that our users wait 4 MONTHS for regression fixes because we
> are afraid of a split or a little extra work? How many users could we lose if we
> significantly slowed down the release cycle (and therefore the bugfix cycle)
> such that people are waiting many months for their fixes? The language would be
> perceived as dead/dying and that would be just as bad as the D1/D2 split. If you
> allow your past experiences to paralyze you into inaction, you will bring about
> the very problem you seek to avoid.

Sigh. Half say we release too often, the other half not often enough.

On Sun, 15 Jul 2012 16:26:50 -0700, Walter Bright <newshound2@digitalmars.com> wrote:

> On 7/15/2012 2:58 PM, Adam Wilson wrote:
>> The idea that bugs and new features can and should be rolled into the same
>> release runs counter to every accepted best practice in both FOSS and Commercial
>> wisdom. The two have VERY different velocities, bugs can be fixed in days, but
>> new features take much longer. Consider COFF support for example, Walter has
>> been hammering away at it for weeks now, and he isn't even 50% done, how many
>> bugs have been fixed and confirmed resolved, in the same timespan?
>
> Weeks is an exaggeration. And still, there have been a steady accumulation of fixes.
>
>> Also,
>> consider that adding new features makes it significantly harder to track down
>> regressions (is a real regression or did the new feature upset the code in an
>> unexpected way) and the new features themselves create new bugs. If the branches
>> are separate then it becomes trivial to determine if the new feature caused the
>> bug, because it will show up in one and not the other.
>>
>> How DARE we DEMAND that our users wait 4 MONTHS for regression fixes because we
>> are afraid of a split or a little extra work? How many users could we lose if we
>> significantly slowed down the release cycle (and therefore the bugfix cycle)
>> such that people are waiting many months for their fixes? The language would be
>> perceived as dead/dying and that would be just as bad as the D1/D2 split. If you
>> allow your past experiences to paralyze you into inaction, you will bring about
>> the very problem you seek to avoid.
>
> Sigh. Half say we release too often, the other half not often enough.
>

This would solve both complaints overnight. The half that say "not often enough" have critical bugfixes they are waiting on, the "too often" camp has new things that they want now (ex. COFF) and see all this bugfixing as getting in the way. I agree its a problem, but this makes both happy. The new feature camp can use dev and put up with the breakages (what we do now), and bugfix camp can get back to work.

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/

On Sunday, July 15, 2012 16:07:40 Walter Bright wrote:
> On 7/15/2012 3:43 PM, Jonathan M Davis wrote:
> > On Sunday, July 15, 2012 15:30:57 Walter Bright wrote:
> >> On 7/15/2012 3:27 PM, Jonathan M Davis wrote:
> >>> The problem is that we're still ironing out too much, and most of the breakage relates to bug fixes, not new features.
> >> 
> >> There's been a lot of non-bug-fixing breakage, for example, renaming library functions.
> > 
> > Yeah, but those are always done through a deprecation path, so there's no immediate breakage. And we've done most of that already, so that should be happening less and less.
> 
> It needs to stop completely.

Most of the renaming of functions which has gone on has been because Phobos has been inconsistent with its naming, which makes it harder to use and learn. As that's sorted out (as has mostly been done), those changes will stop. But do you honestly expect that everything in the standard library is going to be frozen at some point? Is that what you're suggesting?

If we figure out that function X really should be replaced, we should be able to replace it. If we come up with a way better design for a module, we should be able to replace it. That may mean leaving the old version around for a long period of time, but we shouldn't be stuck for bad design decisions permanently. That's actually one area where having major and minor releases (in addition to D1 vs D2 vs D3 etc) can really help, because then you restrict the larger changes to the major releases. Major libraries do this all the time (e.g. Qt).

We definitely want to be more stable than we have been, and we want to reach the point where there are no longer any minor changes for naming and whatnot (and we're getting there), but if you want to freeze the API on Phobos permanently, making it so that we can only ever have additive changes, then there are going to be a number of people who are going to be very unhappy. In the long run, breaking changes should be better managed (e.g. restricted to only certain releases) and much rarer, but they still need to be able to happen.

- Jonathan M Davis

Walter Bright Wrote:

> On 7/15/2012 4:08 PM, Patrick Stewart wrote:
> > Second biggest flaw with D development is premature optimization opsession by large number of devs. "We haven't make it work quite yet as specs define, but lets us optimize it, so it can work incorrectly even faster!"
> 
> All versions pass the D test suite 100%. Any regressions that appeared were not in the test suite, but do get added as they are fixed.
> 
> 

100% passage of test suites do not make me more happy even a bit. Which is my business, of course.

What would make me happy is feature list on page describing milestones. And each version branched and kept stable, with bugfixes and *no* new features introduced.

Real question is where is D going? When will be enough with cramming new features down its throat each time a semi-good idea pops up? When will D leave beta and reach v1 ? Because D2 is still in beta as far as I see it.

On Sunday, July 15, 2012 16:26:50 Walter Bright wrote:
> Sigh. Half say we release too often, the other half not often enough.

Which is actually one argument for going to a model where you have frequent minor releases which only contain bug fixes and less frequent major releases with the larger changes. You can never make everyone happy, but by doing so, you get the bug fixes faster for the folks complaining about the lack of frequent releases, and you get increased stability as far as the new stuff goes, because it doesn't come with every release.

I'm only against the proposed versioning scheme because I think that we need to stabilize things better (e.g. actually have all of the features that TDPL lists fully implemented) before we move to it. But I fully support moving to this sort of scheme in the long run. It manages change much better, and I think that many, many existing projects have shown that it promotes stable code bases while still allowing for them to evolve as necessary.

- Jonathan M Davis

bearophile Wrote:

> Patrick Stewart:
> 
> > Both languages have programming bugs and bad decisions. Python fix them without disrupting schedule and usability. D says "suck it up for next X years while we fix it" or "You have some obscure 4 year old version without that bug".
> 
> Python C interpreter is also far simpler than designing a new system language, and writing its compiler and standard library.

Simpler - true. But having more complex problem to solve just makes more important to have *better* organization. And D community approach is of weekend developers that discovered VCS & test suite, but know nothing about branching and project planning.

> And behind Python there are far more developers and users.
> 

Completely not relevant. Number of developers have nothing to do with project organization. There is a lot of software there equal or more complex that are product of just a few or even single programmer.

On Sun, 15 Jul 2012 16:06:58 -0700, Walter Bright <newshound2@digitalmars.com> wrote:

> On 7/15/2012 3:52 PM, Adam Wilson wrote:
>> So the problem is semantics then? Because I dredge up another word to describe
>> what we are asking for if that's all it takes. But I don't think that anyone
>> else is going to read "stable" as "unchanging". Software is by definition
>> changing, or it's dead. It appears to my parsing of your sentence that you are
>> asserting that stable == static. By that definition of stable, Windows ME is
>> "stable" and ... ehrm, not a soul in the tech world would agree with that
>> summation of WinME.
>>
>> As I said earlier, no one else in FOSS or Commercial equates stable with "has no
>> bugs", it means no new features and no regressions. Not a single solitary person
>> I've talked too expects their software to be bug free.
>>
>> THIS is what we mean when we say "stable":
>> http://www.modernperlbooks.com/mt/2009/06/what-does-stable-mean.html
>> It's also how pretty much everyone else will read "stable".
>
> D does have a test suite, and it is a (almost always achieved) goal to keep it always passing, even on the dev branch. In fact, most of my work is running the test suite and making sure each change doesn't regress. (Regressions that do slip through were not in the test suite.)
>
> Frankly, I don't know how to do what you're asking for. D users, every single day, clamor for:
>
> 1. more bug fixes

Branch A. Rebase into Branch B as needed.

> 2. more new features

Branch B.

> 3. why aren't deprecated features removed more quickly?

Branch A, marked for deprecation.
Branch B, actually removed. Becomes active when merged into Branch A.
(Assuming Branch B is merged roughly every other month as per current processes.)

> 4. why don't we add this breaking feature?

Add it. Branch B.

> 5. why did you add that breaking feature which broke my code?

Why are you using Branch B you knucklehead?

> Often, these are the same people! Sometimes, even in the same post!

This concept is precisely designed to significantly mitigate all five problems.

Not everyone will test against Repo B, but this allows you to put the responsibility for not testing against it on them. They know how it works here, it's not your problem if you broke something that they had the chance to test for but didn't.

> And, to reiterate, we did release D1. Since its release, it has only received bug fixes. No breaking changes, no regressions. This, inevitably, has made many D1 users unhappy - they wanted new features folded in.
>
> So that was not satisfactory, either.
>
> Yes, I do feel a bit put upon by this, as I see no way to satisfy all these mutually contradictory requests.

I do apologize for that. It is not my intention to cause undue stress. I am pushing the for the change because I think it will mitigate much of your current stress in dealing with us. And I do recognize that we users can be pretty demanding as I sit on the other side of this equation at work. But because I sit on the other side, I get frustrated when I see developers actively resisting the proven concepts that will drastically reduce the very problem they are complaining about.

I should note that we use this exact model for every project we have where I work and that it is been highly successful at keeping those five points of tension moderated. And our users can actually get work done without waiting for weeks and months because thing X is just plain broken, which in turn makes us look good. (Improving Loyalty)

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/

On 16/07/2012 01:06, Walter Bright wrote:
> Frankly, I don't know how to do what you're asking for. D users, every
> single day, clamor for:
>
> 1. more bug fixes
> 2. more new features
> 3. why aren't deprecated features removed more quickly?
> 4. why don't we add this breaking feature?
> 5. why did you add that breaking feature which broke my code?
>
> Often, these are the same people! Sometimes, even in the same post!
>

I understand that this may seems messed up. It isn't as much as it seems. It simply show the need for a more elaborate versionning and releasing system for D.

Theses needs do all exists. But not all at the same time or all in the same situation.

When dealing with code, as in all engineering fields, you always make tradeoffs. And changing things in a codebase have a cost. Presumably, every feature which are included into D has also a benefit.

The larger your codebase, the more interesting it is to slow down inclusion of new feature in your work. The smaller, the quicker you want to do it because the cost of doing so isn't the same, and you can benefit from the news feature at a very little cost.

Different situations, different needs. The same person can have both need at the same time, because it have experienced both situations with different codebases.

> And, to reiterate, we did release D1. Since its release, it has only
> received bug fixes. No breaking changes, no regressions. This,
> inevitably, has made many D1 users unhappy - they wanted new features
> folded in.
>
> So that was not satisfactory, either.
>
> Yes, I do feel a bit put upon by this, as I see no way to satisfy all
> these mutually contradictory requests.

As of D1, the problem is different.

I'll use again the example of PHP, because it has proven to manage the issue quite well, and because I discussed that quite a lot with Ramsus recently, so I know the topic quite well.

PHP release PHP 5.2 . Then it released PHP6 . PHP introduced breaking changes, just like D2 does. We can compare D1 as PHP5.2 and D2 as PHP6 .

It happened that some new feature of PHP6 weren't breaking features (GC, closures, namespaces) and some other were (introducing unicode into source code).

And here is what is done then, and here what we should learn from PHP. After PHP6, PHP relased PHP5.3 . PHP5.3 was basically PHP5.2 with all new feature of PHP6, except the one that was breaking.

PHP5.2 continued to live for very conservative users, 5.3 for user that want to use new features, and 6 for users that feel like beta testers.

(note 6 was then canceled, but for reasons completely unrelated to what we are talking here. I may talk about that, but this is really off topic here, so let's not epilogue on that).

On 16/07/2012 01:07, Walter Bright wrote:
> On 7/15/2012 3:43 PM, Jonathan M Davis wrote:
>> On Sunday, July 15, 2012 15:30:57 Walter Bright wrote:
>>> On 7/15/2012 3:27 PM, Jonathan M Davis wrote:
>>>> The problem is that we're still ironing out too much, and most of the
>>>> breakage relates to bug fixes, not new features.
>>>
>>> There's been a lot of non-bug-fixing breakage, for example, renaming
>>> library
>>> functions.
>>
>> Yeah, but those are always done through a deprecation path, so there's no
>> immediate breakage. And we've done most of that already, so that
>> should be
>> happening less and less.
>
> It needs to stop completely.
>

No. It hasn't been made for no reasons.

But yes, some code is broken in the process. This is exactly why we need a more sophisticated versionning process (note the recurring pattern in my posts :D ).

The fact that some people have legacy code shouldn't stop D progress. But with the current system, D must either break code or make no progress.

On Sunday, July 15, 2012 19:43:44 Patrick Stewart wrote:
> Completely not relevant. Number of developers have nothing to do with project organization. There is a lot of software there equal or more complex that are product of just a few or even single programmer.

Actually, it's _very_ relevant. If release model A takes up more time and resources than release model B, then release model A will slow down development. For projects with more developers, it may be possible to mitigate those costs such that the benefits outweigh them, but for projects with fewer developers, a slower release model may be completely unacceptable. It all depends on what the various pros and cons are and how they will affect the project.

It may very well be that the proposed release model is well worth going to, but it could also be that its benefits aren't worth its costs, and the number of developers involved has a definite effect on that calculation.

- Jonathan M Davis