View mode: basic / threaded / horizontal-split · Log in · Help
January 10, 2013
Re: Transitioning to the new Release Process
On Thursday, 10 January 2013 at 15:51:13 UTC, Andrei Alexandrescu 
wrote:
> My understanding was that staging is worked on only during the 
> (short) time span from initiating a new release and finalizing 
> that release.
>
> Andrei

I know, that is my general understanding of how staging would be 
used. This is a release branch in most described models.

However I think that extending that to a longer period isn't bad. 
But it does mean maintaining 3 branches (released, staging, dev). 
If we only want the usual short prep period, staging should die 
and we just branch into a Version branch (release branch) and tag 
when it is released. This would not need a transition period.
January 10, 2013
Re: Transitioning to the new Release Process
On Thursday, 10 January 2013 at 15:51:13 UTC, Andrei Alexandrescu 
wrote:
>
> My understanding was that staging is worked on only during the 
> (short) time span from initiating a new release and finalizing 
> that release.
>
> Andrei

There are significant side effects of that approach:

1) There's no beta made available for release. While you can try 
really hard to claim that there is a beta, in practical terms 
there isn't one, because very few people will even know it is 
there, and even less will have the ability to test it. Even if 
someone managed to try it out, not enough time is available to do 
anything significant with it and report back the results for 
refinement.

2) You'll never gain a high degree of stability between releases. 
The reason why is because the poorly tested development copy, is 
the released version, and following point 1) above, it will never 
be sufficiently tested and refined prior to release.

The results we got with the 2.061 release is what the process 
will give you, which is an unstable compiler that no one will 
want to use in a production environment unless they don't mind 
getting fired, in other words what was released is in fact a beta 
release of the unstable development branch.

If our intention is to produce a stable release that is 
maintained, then the process needs to be designed in a way that 
produces one.

The only way stability can happen, is if all development stops on 
a release branch for a sufficient period of time, ie., it enters 
into a maintenance mode where only critical bug fixes are 
introduced.

We don't want to stop the clock however, there's always newer 
exciting things to be done, so the development must proceed, and 
new major releases must follow. The tricky part is to do it in a 
non-disruptive way, that smooths out the instability between 
releases, that's the purpose the beta branch servers, a middle 
ground between stability and instability, but it's far more than 
that, it can also provide the development team with crucial feed 
back from real-world users.

We have to realize that the release process is not intended for 
developers, you have to consider what the users of the compiler 
want to see.

1) Users want to have access to a well maintained release of the 
compiler, where incremental upgrades contain only bug fixes up 
until the next major release, which will contain new features and 
other major improvements.

2) Some users want to have access to a reasonably stable beta 
release. The only reason why a user will want to try a beta 
version, is when the stable released version does not yet have 
something that they want to try out that is available in the 
beta, such as a new feature like UDA's. The beta should be in 
packaged form and available on the download page because you have 
to make it very easy for people to try it.

How we achieve the results may not matter so much, but from what 
I can see, to achieve point 1), the release must come from the 
beta, and there must be a release branch that can receive bug 
fixes. To achieve point 2) which is providing a reasonably stable 
beta for the users to try out, there must be a beta branch. The 
beta branch receives only new features and bug fixes that have 
been fully agreed to and well thought out.

Meanwhile all of the half baked new concepts are contained in the 
development branch (we've pegged it into master), and all the 
experimental stuff resides in the various scattered feature 
branches that only developers and high adventures are willing to 
try out.

--rt
January 11, 2013
Re: Transitioning to the new Release Process
Short comment about cherry pick - it is only bad in sense that it
creates separate untrackable commit with same content, which may
easily result in merging issues. If there is only one-way
direction for commit transitioning ( stuff goes from master to
staging and LTS and never goes back ) there is nothing inherently
bad about cherry pick.

On topic: well, it was somewhat discussed in the very first
thread. One pure technical issue is that github choses master as
default target for pull requests and people gonna do it wrong by
a simple mistake rather often. Much more important point is that
it complicates development process for newcomers without any
reasonable benefit. And if we still gonna push for success as an
open-source project, being easy understandable by an outsider is
as important as a clear release process.

Come on, I am reading all new posts in D and D.announce almost
every day (for 2+ years), check most of github pull requests for
druntime and phobos on weekly basis and still have a hard day
figuring out how some stuff is supposed to be done regarding
process and intentions. Imagine some random guy who just fixed
some stuff for his project and now he is full of altruistic
intentions to provide this fix/feature to an upstream. We have
current scenario A - fork repo on github, do stuff, push "pull
request" button. And now scenario B is proposed - go to wiki,
read release process guidelines, chose branch to fork from, do
not forget to chose the same branch when doing pull request
shortly after. Minor details regarding people comfort matter a
lot when they are supposed to work for free.

On Thursday, 10 January 2013 at 15:47:30 UTC, Andrei Alexandrescu
wrote:
> On 1/10/13 1:28 AM, mist wrote:
>> Sounds like different people have different understanding of 
>> staging
>> concept here and each one is doing minor tweaks to own 
>> direction. I ran
>> through updated wiki and some stuff like "pull requests 
>> against staging"
>> just scares me a lot.
>
> Why is that scary?
>
> Thanks,
>
> Andrei
January 11, 2013
Re: Transitioning to the new Release Process
My understanding is that your understanding is somewhat different 
from initial proposal but that is where discussion has flow to 
since then and that makes me sad :)

They very reason to have staging is to have better replacement to 
beta process which simply does not work good enough currently. Is 
good to have a single branch all the time, which some obscure 
project maintainer can check from time to time to make sure his 
stuff still compiles and fire regression bug reports if needed. 
He may even have add this test to own continuous integration 
suite (I'd do this if I had a serious D project) to be notified 
when stuff goes wrong without paying any constant attention.

Attention is a key here. How many D projects are out there? How 
many of their maintainers pay close attention to newsgroup and 
read beta mail list? Compare this to being able to check your 
stuff on very next release at any moment you have time and want 
to.

I stand at the point that for open source projects release and 
development processes should care most about end users and 
developers and least - about maintainers. Maintainers should have 
perfect git and process knowledge anyway, or at some scales thing 
are doomed to be crewed (2.061 anyone?).

Thus I vote for a persistent staging branch.

>
> My understanding was that staging is worked on only during the 
> (short) time span from initiating a new release and finalizing 
> that release.
>
> Andrei
January 12, 2013
Re: Transitioning to the new Release Process
On Friday, 11 January 2013 at 18:03:33 UTC, mist wrote:
> My understanding is that your understanding is somewhat 
> different from initial proposal but that is where discussion 
> has flow to since then and that makes me sad :)
>
> They very reason to have staging is to have better replacement 
> to beta process which simply does not work good enough 
> currently. Is good to have a single branch all the time, which 
> some obscure project maintainer can check from time to time to 
> make sure his stuff still compiles and fire regression bug 
> reports if needed. He may even have add this test to own 
> continuous integration suite (I'd do this if I had a serious D 
> project) to be notified when stuff goes wrong without paying 
> any constant attention.
>
> Attention is a key here. How many D projects are out there? How 
> many of their maintainers pay close attention to newsgroup and 
> read beta mail list? Compare this to being able to check your 
> stuff on very next release at any moment you have time and want 
> to.
>
> I stand at the point that for open source projects release and 
> development processes should care most about end users and 
> developers and least - about maintainers. Maintainers should 
> have perfect git and process knowledge anyway, or at some 
> scales thing are doomed to be crewed (2.061 anyone?).
>
> Thus I vote for a persistent staging branch.
>
>>
>> My understanding was that staging is worked on only during the 
>> (short) time span from initiating a new release and finalizing 
>> that release.
>>
>> Andrei

I don't understand the problem you seem to see in the process.

I've been away from the PC so couldn't respond earlier but I did 
manage to read most of the wiki page and I think Johnathan did a 
very good job explaining the various concepts. The only thing 
missing IMO is CI:
It's already agreed that master is the development channel in the 
proposed process. The next logical step would be to generate 
nightly builds off of that branch. The staging branch is the 
"beta" channel - and also makes sense to have a periodical beta 
build. This depends on the release time span but something like 
fortnight or monthly beta builds makes sense to me.
the additional manual builds described on the wiki on staging or 
even before an official release on the version branch will be 
something like RCs.

Regarding pull requests targeting master, the current model *is* 
geared around that. Most contributions _should_ go to master (aka 
devel) and go through the full process. pull-requests for staging 
are meant for fixing regressions and critical bugs only, where 
there is _higher urgency_ for the fix that justifies the 
short-cut. Regular "bug fixes" should simply go through the 
regular process and will be released in the _next_ release.
January 12, 2013
Re: Transitioning to the new Release Process
Am Thu, 10 Jan 2013 20:36:40 +0100
schrieb "Jesse Phillips" <Jessekphillips+D@gmail.com>:

>[...]
> 
> However I think that extending that to a longer period isn't bad. 
> But it does mean maintaining 3 branches (released, staging, dev). 
> If we only want the usual short prep period, staging should die 
> and we just branch into a Version branch (release branch) and tag 
> when it is released. This would not need a transition period.

Staging has also 2 other small benefits: It is a central 'beta
branch', you can check out 'staging' any time and be sure this is going
to be the next release. You'll always have all the features that are in
the next release and the only updates till the next release will be bug
fixes. This is an interesting point imho.

And then there is the benefit that you'll have a central target for bug
fix pull requests:
Without staging, bug fixes usually go into master. But when a release
branch is created, bug fixes suddenly have to go into that release
branch. Then after the release, bug fixes have to go to master again
and only regression fixes can go into the release branch.

This is kinda annoying as it does not only mean the target branch for
pull requests on github has to be changed a lot, you might also have to
rebase the bugfix on the other branch.

With staging you always target staging for bug fixes. They can be
merged at any point in time.
January 12, 2013
Re: Transitioning to the new Release Process
Am Sat, 12 Jan 2013 09:22:27 +0100
schrieb "foobar" <foo@bar.com>:

> [...]
> 
> Regarding pull requests targeting master, the current model *is* 
> geared around that. Most contributions _should_ go to master (aka 
> devel) and go through the full process. pull-requests for staging 
> are meant for fixing regressions and critical bugs only, where 
> there is _higher urgency_ for the fix that justifies the 
> short-cut. Regular "bug fixes" should simply go through the 
> regular process and will be released in the _next_ release.

I also think targeting staging for some pull requests is not that bad.
In the end it's not that bad if a pull request was accidentally merged
into master instead of staging. It just means that it'll take more time
for the fix to appear in a release (It'll be delayed by one release),
but it won't mess anything up.

Regarding where "most" requests go: This also depends on the project. I
guess for phobos most requests are enhancements/new features and would
go to master. For druntime it's mostly bugfixes, so probably more
requests target staging. And for the dmd almost everything is a bug fix
and could target staging.

The wiki currently says all bug fixes should go to staging. This is a
concession to D's rapid development. But again, it's not that important
where the pull requests go. We should try to make breaking changes in
master and have only "harmless" changes in staging, but the exact
decision is always up to the contributor & maintainers.

But those 'minor' tweaks such as defining what exactly is merged to
master and what to staging can always be made later on.
January 12, 2013
Re: Transitioning to the new Release Process
Am Fri, 11 Jan 2013 18:54:12 +0100
schrieb "mist" <none@none.none>:

> Short comment about cherry pick - it is only bad in sense that it
> creates separate untrackable commit with same content, which may
> easily result in merging issues. If there is only one-way
> direction for commit transitioning ( stuff goes from master to
> staging and LTS and never goes back ) there is nothing inherently
> bad about cherry pick.

Having multiple commit ids for the same commit is also annoying. For
example if someone bisected a regression and you want to revert a commit
you have to figure out the commit id of the cherry picked commits.

> 
> On topic: well, it was somewhat discussed in the very first
> thread. One pure technical issue is that github choses master as
> default target for pull requests and people gonna do it wrong by
> a simple mistake rather often.
It doesn't harm though if a requests goes to master instead of staging
- the only difference with the approach described on the wiki page is
that the bug fix will be delayed by one release if it's merge into
master instead of staging.

> Much more important point is that
> it complicates development process for newcomers without any
> reasonable benefit. And if we still gonna push for success as an
> open-source project, being easy understandable by an outsider is
> as important as a clear release process.
The rule is quite simple and easy to understand though:
regression fix -> version branch
bug fix -> staging
enhancement -> master

The benefits have already been stated in other posts.
> 
> Come on, I am reading all new posts in D and D.announce almost
> every day (for 2+ years), check most of github pull requests for
> druntime and phobos on weekly basis and still have a hard day
> figuring out how some stuff is supposed to be done regarding
> process and intentions. Imagine some random guy who just fixed
> some stuff for his project and now he is full of altruistic
> intentions to provide this fix/feature to an upstream. We have
> current scenario A - fork repo on github, do stuff, push "pull
> request" button. And now scenario B is proposed - go to wiki,
> read release process guidelines, chose branch to fork from, do
> not forget to chose the same branch when doing pull request
> shortly after. Minor details regarding people comfort matter a
> lot when they are supposed to work for free.

It's always possible for a maintainer to get the changes from the pull
request into their personal repo, rebase on the correct branch, reopen.
And it's not that bad if a request goes to master accidentally.

Regression fixes have to go to version branches anyway, you can't avoid
this without cherry-picking. But newbies don't do regression fixes
anyway.

Separating between staging and master then has the nice benefit of a
more stable "beta" branch and a fast moving development branch.
January 14, 2013
Re: Transitioning to the new Release Process
On Wednesday, 9 January 2013 at 17:12:15 UTC, Jesse Phillips 
wrote:
> Before starting this I did come across Walter's complaint of 
> issues, but feel this needs a new thread. But didn't read all 
> replies.
>
> The new process introduces a new branch called staging. This is 
> _not_ used as one would first think.
>
> staging is created/updated _after_ release. This means a delay 
> in changes in master (breaking changes/new additions) and 
> release (a good thing). It also means that we have to do some 
> awkward delaying to get some useful changes into staging so 
> that it is different from this last release.

Since we don't have much opposition here for the staging process 
I will make my suggestion for this transition.

Since std.uni is up for review, lets plan that upon inclusion we 
move master into staging. This will be the next release as 
defined in our process. When another group of changes is 
considered ready we can make the release and update staging.

We could choose other changes, but I think this will be a good 
arbitrary point to make the transition (and not end up with an 
"empty" release).
January 14, 2013
Re: Transitioning to the new Release Process
On Saturday, 12 January 2013 at 12:30:05 UTC, Johannes Pfau wrote:
> Am Fri, 11 Jan 2013 18:54:12 +0100
> schrieb "mist" <none@none.none>:
>
>> Short comment about cherry pick - it is only bad in sense that 
>> it
>> creates separate untrackable commit with same content, which 
>> may
>> easily result in merging issues. If there is only one-way
>> direction for commit transitioning ( stuff goes from master to
>> staging and LTS and never goes back ) there is nothing 
>> inherently
>> bad about cherry pick.
>
> Having multiple commit ids for the same commit is also 
> annoying. For
> example if someone bisected a regression and you want to revert 
> a commit
> you have to figure out the commit id of the cherry picked 
> commits.
>

AFAIK reverts are content-based, so knowing any one commit id is 
enough. It is annoying mostly when comparison happens between two 
branches with those commits.

>> Much more important point is that
>> it complicates development process for newcomers without any
>> reasonable benefit. And if we still gonna push for success as 
>> an
>> open-source project, being easy understandable by an outsider 
>> is
>> as important as a clear release process.
> The rule is quite simple and easy to understand though:
> regression fix -> version branch
> bug fix -> staging
> enhancement -> master

Ye, it is easy.. except you need to actually check out wiki page 
to be aware about it. At the very least we need relevant excerpts 
for outer devs separated to one small wiki page and set links to 
this page in github marked with big "NB:".

I don't have a reason to argue about the rest.
1 2 3
Top | Discussion index | About this forum | D home