View mode: basic / threaded / horizontal-split · Log in · Help
December 10, 2012
Next focus: PROCESS
Hello all,


Walter and I have had a long discussion following his trip to Australia. 
Following the current sprint for Win64 (which we all, I think, agree was 
long overdue and had to be done), the main area we need to work on (as 
I'm sure many would agree) is improving our process, in particular the 
approach to releasing the compiler and standard library.

Walter has delegated me to lead this effort, and agreed to obey with 
whatever process comes forward as long as it's reasonably clear and simple.

In turn, I'll be collecting thoughts and opinions in this thread, 
distilled from previous discussions. We should develop a few simple git 
scripts (such as git-start-new-feature or git-start-bugfix etc) 
supported by an equally simple piece of documentation describing how to 
start a new release, fix a bug, experiment with a new feature, and such.

(One piece that has been brought forward is 
http://nvie.com/posts/a-successful-git-branching-model/ - something to 
keep in mind.)

Please chime in with ideas and thoughts.


Thanks,

Andrei
December 11, 2012
Re: Next focus: PROCESS
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
> Hello all,
>
>
> Walter and I have had a long discussion following his trip to 
> Australia. Following the current sprint for Win64 (which we 
> all, I think, agree was long overdue and had to be done), the 
> main area we need to work on (as I'm sure many would agree) is 
> improving our process, in particular the approach to releasing 
> the compiler and standard library.
>
> Walter has delegated me to lead this effort, and agreed to obey 
> with whatever process comes forward as long as it's reasonably 
> clear and simple.
>
> In turn, I'll be collecting thoughts and opinions in this 
> thread, distilled from previous discussions. We should develop 
> a few simple git scripts (such as git-start-new-feature or 
> git-start-bugfix etc) supported by an equally simple piece of 
> documentation describing how to start a new release, fix a bug, 
> experiment with a new feature, and such.
>
> (One piece that has been brought forward is 
> http://nvie.com/posts/a-successful-git-branching-model/ - 
> something to keep in mind.)
>

As someone who uses the git workflow described in this link every 
single day let me say I'm a big fan.  It's so well documented 
(and widely used) that it should be pretty easy for everyone to 
adopt and it provides a lot of real benefits to a project.  From 
the look of it most contributors already use feature branches for 
their pull requests so this should be a pretty simple transition.

> Please chime in with ideas and thoughts.
>
>

I like the idea of having a release manager (that changes to 
whoever volunteers for each release). It'll take some of the 
release time pressure off of Walter (at least for the times he 
isn't the release manager) and raise D's bus factor (which is 
very close to being above 1 these days) as the release process is 
documented and shared among several individuals.  I imagine the 
role of release manager being fairly laid back though (it is a 
volunteer job still).  Their responsibilities would be rolling 
the betas and actual releases and just being the point-man who 
goes through and makes sure all the pull requests that are ready 
to merge are merged and last minute regressions are acknowledged.

BA
December 11, 2012
Re: Next focus: PROCESS
On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:
> Hello all,
> 
> Walter and I have had a long discussion following his trip to
> Australia. Following the current sprint for Win64 (which we all, I
> think, agree was long overdue and had to be done), the main area we
> need to work on (as I'm sure many would agree) is improving our
> process, in particular the approach to releasing the compiler and
> standard library.

*applause* It's about time we did this right.


> Walter has delegated me to lead this effort, and agreed to obey with
> whatever process comes forward as long as it's reasonably clear and
> simple.

Good to know.


> In turn, I'll be collecting thoughts and opinions in this thread,
> distilled from previous discussions. We should develop a few simple
> git scripts (such as git-start-new-feature or git-start-bugfix etc)
> supported by an equally simple piece of documentation describing how
> to start a new release, fix a bug, experiment with a new feature,
> and such.

Sounds OK to me, though it seems unnecessarily complicated. What's wrong
with just doing things like:

	# New feature
	git checkout develop	# latest dev branch
	git branch -b brand_new_feature
	# implement said feature
	git commit -a

	# Merge feature into dev branch
	git checkout develop
	git merge brand_new_feature

	# New release candidate
	git checkout develop
	git branch -b 2.062-rc1
	# prepare code for release
	git commit -a

	# Release
	git checkout master
	git merge 2.062-rc1

Seems to me these are all straightforward everyday git commands. The
whole selling point of git is that branching/merging is so easy that you
could do it multiple times a day within a single implementation effort.
For example, I frequently find myself doing things like this:

	# I want to implement feature X today
	git checkout master

	# Implementation attempt #1
	git checkout -b feature_X_1
	vim feature.d  # code code code

	# While coding feature.d, I got inspired by an idea of an
	# alternative implementation. So, put current code aside and
	# test idea on new branch.
	git commit -a
	git checkout -b feature_X_alt_impl
	vim feature.d  # implement alternative approach

	# Turns out, alternate approach doesn't work. Go back to
	# original approach. (But leave alt_impl in case I decide later
	# to go back to it.)
	git checkout feature_X_1
	vim feature.d  # more code

	# I hit a difficult spot in the code. Put it aside and work on
	# other stuff first.
	git commit -a
	git checkout master
	vim main.d  # do some routine maintenance, bugfixes, etc.
	git commit -a

	# Take afternoon nap. While resting I got another idea for
	# feature X.
	git checkout feature_X_1
	vim feature.d  # try out new idea

	# Turns out, it needs a big change. So don't commit to
	# feature_X_1, but use a separate branch.
	git checkout -b feature_X_new_idea
	vim feature.d  # more code
	git commit -a

	# I decide that feature_X_new_idea is the way to go. So fold
	# into feature X branch.
	git checkout -b feature_X_1
	git merge feature_X_new_idea

	# I don't need the new idea branch anymore now that it's been
	# adopted as the "official" way to implement X. Also, the
	# previous alternative approach was a dead-end, so junk that as
	# well.
	git branch -d feature_X_new_idea
	git branch -d feature_X_alt_impl

	# Continue working on X
	vim feature.d
	git commit -a

	# I decide that it's now stable enough to merge into the
	# mainline code.
	git checkout master
	git merge feature_X_1

This can happen several times a day when I'm doing a lot of coding.


> (One piece that has been brought forward is
> http://nvie.com/posts/a-successful-git-branching-model/ - something
> to keep in mind.)
[...]

Glanced at it briefly. I think the basic ideas behind it are sound.
Whether or not we should adopt all of the specifics is up for
discussion, but I think we should definitely incorporate the main ideas:
(1) have a release-ready branch, (2) a separate dev branch, and (3)
always use a separate branch for new features so that if things go wrong
or it takes too long to finish implementation, development work on other
stuff can continue without being held up by bugs in the new feature.

Furthermore, in a git environment it's perfectly normal to have multiple
"exploratory" or "sub-feature" branches within a single feature branch,
like I illustrated above in exploring how to implement feature X. These
are usually just local branches, but if the feature is complex enough,
it can warrant pushing to github so that other devs can also play around
with the alternative implementation approaches. Eventually the best
approach can be merged back to the feature branch and the others
discarded.

Personally, I find that this approach is very profitable, because you
don't have to worry that a particular breaking change may turn out to be
a dead-end; do the experiment in a sub-branch and throw it away if it
doesn't work. The cheapness of doing this allows one to explore
ambitious or risky approaches without affecting the stable "traditional"
implementation, and so opens up the possibility of discovering new and
novel ways of doing things that are usually left unexplored.


T

-- 
It said to install Windows 2000 or better, so I installed Linux instead.
December 11, 2012
Re: Next focus: PROCESS
On 11-12-2012 01:19, H. S. Teoh wrote:
> On Mon, Dec 10, 2012 at 06:41:25PM -0500, Andrei Alexandrescu wrote:
>> Hello all,
>>
>> Walter and I have had a long discussion following his trip to
>> Australia. Following the current sprint for Win64 (which we all, I
>> think, agree was long overdue and had to be done), the main area we
>> need to work on (as I'm sure many would agree) is improving our
>> process, in particular the approach to releasing the compiler and
>> standard library.
>
> *applause* It's about time we did this right.
>
>
>> Walter has delegated me to lead this effort, and agreed to obey with
>> whatever process comes forward as long as it's reasonably clear and
>> simple.
>
> Good to know.
>
>
>> In turn, I'll be collecting thoughts and opinions in this thread,
>> distilled from previous discussions. We should develop a few simple
>> git scripts (such as git-start-new-feature or git-start-bugfix etc)
>> supported by an equally simple piece of documentation describing how
>> to start a new release, fix a bug, experiment with a new feature,
>> and such.
>
> Sounds OK to me, though it seems unnecessarily complicated. What's wrong
> with just doing things like:
>
> 	# New feature
> 	git checkout develop	# latest dev branch
> 	git branch -b brand_new_feature
> 	# implement said feature
> 	git commit -a
>
> 	# Merge feature into dev branch
> 	git checkout develop
> 	git merge brand_new_feature
>
> 	# New release candidate
> 	git checkout develop
> 	git branch -b 2.062-rc1
> 	# prepare code for release
> 	git commit -a
>
> 	# Release
> 	git checkout master
> 	git merge 2.062-rc1
>

master must be the development branch. This is a pitfall the Rust guys 
walked into: People accidentally send their pull requests to the master 
branch instead of the develop branch which overall ends up being a mess 
when a reviewer doesn't notice the error before merging.

Otherwise sounds good.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org
December 11, 2012
Re: Next focus: PROCESS
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
> (One piece that has been brought forward is 
> http://nvie.com/posts/a-successful-git-branching-model/ - 
> something to keep in mind.)
>

In this workflow, only one version of the software is produced. 
But we have an extra need : the one to combine language evolution 
(that may break) and long term support. This require some 
adjustment.

Here is how I see things :

New feature are develloped in their own branches.
When a feature is ready, it is included in the dev branch. The 
feature must be rebased on the dev branch to be included.
From time to time, dev branch is forked. The forked branch only 
receive bug fixes. The second digit of the version is incremented 
(minor). I'll call it stabilization branch.
When the fork branch is stable enough, a tag is made and we have 
a realease. The revision number is incremented.
In the future, more bug fixes can be done in that branch and new 
tag created for new release.

No let's focus on bug fixes. As feature, they are made in their 
own branches. The branch will then be merged in any stabilization 
branch where it make sense and in dev.

master is the highest numbered stabilization branch. By doing so, 
we ensure that user that get sources have a preview of the next 
thing that will be shipped.

I would advice against maintaining a lot of stabilization 
branches as this is more work and too much of them don't bring 
any benefit. 2 to 3 of them is the right amount.

The proposal works well with github, which is a plus considering 
we are using it.
December 11, 2012
Re: Next focus: PROCESS
On 12/10/12 7:19 PM, H. S. Teoh wrote:
> Sounds OK to me, though it seems unnecessarily complicated.

FWIW there's also this:

http://drewfradette.ca/a-simpler-successful-git-branching-model/


Andrei
December 11, 2012
Re: Next focus: PROCESS
On 12/10/2012 06:41 PM, Andrei Alexandrescu wrote:
>
> In turn, I'll be collecting thoughts and opinions in this thread,
> distilled from previous discussions. We should develop a few simple git
> scripts (such as git-start-new-feature or git-start-bugfix etc)
> supported by an equally simple piece of documentation describing how to
> start a new release, fix a bug, experiment with a new feature, and such.

One thing that I don't think I've seen mentioned: The person who started 
that git-flow concept created a git plugin to do exactly this for that 
workflow: https://github.com/nvie/gitflow

-Matt
December 11, 2012
Re: Next focus: PROCESS
On Monday, 10 December 2012 at 23:41:25 UTC, Andrei Alexandrescu 
wrote:
>
> Please chime in with ideas and thoughts.
>
>
> Thanks,
>
> Andrei

Andrei, as you know, a lot has already been discussed here:
http://forum.dlang.org/thread/k99coi$1q5e$1@digitalmars.com

In summary, here's where I understood things were left at ...

Instead of all of our ideas getting mixed around in multiple 
threads and eventually vanishing into the black hole, we had 
decided that we needed a fixed location to discuss and work on 
the issue, and a place where documents that describe the process 
can be developed and stored.

The poster named "1100110" started a git repo here
https://github.com/D-Programming-Language-Stable/dmd/wiki/Specification

I'm not sure it should be called "D-Programming-Language-Stable", 
it should be named something like 
"D-Programming-Language-Dev-Process", and it should only be 
concerned with developing the new development and release process 
for D.

--rt
December 11, 2012
Re: Next focus: PROCESS
Am 11.12.2012 03:02, schrieb Andrei Alexandrescu:
> On 12/10/12 7:19 PM, H. S. Teoh wrote:
>> Sounds OK to me, though it seems unnecessarily complicated.
> 
> FWIW there's also this:
> 
> http://drewfradette.ca/a-simpler-successful-git-branching-model/
> 
> 
> Andrei

Just throwing this in: http://vibed.org/temp/branch-model-small.png

It avoids the pull request problem that was mentioned (pull target for a new feature may often
erroneously be master vs. develop) and treats the release branches as the stable branches from which
release tags are directly published. This seems pretty close to the actual mode of development and a
single master for releases, as in the "successful git branching" models, seems to be insufficient
anyway; continuous parallel releases of 1.x and 2.x (or 2.0.x and 2.1.x) seem not to really fit in
there.
December 11, 2012
Re: Next focus: PROCESS
On Tuesday, 11 December 2012 at 02:02:54 UTC, Andrei Alexandrescu
wrote:
> On 12/10/12 7:19 PM, H. S. Teoh wrote:
>> Sounds OK to me, though it seems unnecessarily complicated.
>
> FWIW there's also this:
>
> http://drewfradette.ca/a-simpler-successful-git-branching-model/
>
>
> Andrei

First of all - Yay!

There are still a few open questions that need to be decided
before a suitable process can be defined.

a. Should we have more than one stable release and provide
support branches?
That means we have version X which is latest stable with all the
latest (stable) features included and we have an older Y version
that only receives bug-fixes and is intended for support (think
of Ubuntu's LTE versions). If we have such additional versions,
how many do we have, for how long?
My opinion - (based on Walter's) you can always download a
previous version of the compiler. I'd say we should _at most_
support _one_ previous stable version with critical bug fixes
only. we could always decide to support ad-hoc additional
specific versions, for example before we introduce major paradigm
shifting changes and this also depends on available manpower.

B. should we have public pre-release versions?
I think we should release additional "releases" - call them beta,
pre-release, release candidates, whatever. These are for staging
changes, allowing to field test new features and language changes
before they are made final. Also allowing users to adjust their
code-bases.

Given the above, I think the "simpler flow" model above looks
very promising and we should adopt it with minor changes:
1. Both "staging" and "release" should be "public" - the website
lists both the stable versions and the staging versions.
2. the development branch should be the master branch for easier
integration with pull requests.

The staging releases (let's call them beta) will provide previews
of upcoming and experimental features to try out without the
promise of a stable API and would be released _monthly_.

The stable releases will be every 3-4 months and will guaranty a
stable API. A user could download DMD-201212 (i.e. this month)
and play around with UDA and of course report bugs, but knowing
that this feature could change syntax or even be completely
removed thus allowing the developers to test stuff out in the
open without committing to anything.
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home