View mode: basic / threaded / horizontal-split · Log in · Help
December 20, 2012
Re: Next focus: PROCESS
On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:

> master : used as a base for development. New feature are merged 
> here.
> staging : used to provide a view of what the next version will 
> look like. Regular snapshot of that branch are made so public 
> can use the last features.
> version : used to contain a version that will have a support 
> for an extended period of time.

I do not see how development in master is moved to staging based 
on this description. I'll try and be more specific.

I realize you don't like features, but we need to talk about 
Phobos additions, @property mechanics, and other highly 
disruptive bugs. I'll call it the major-change.

The major-change branch is generally developed to completion and 
may just be a pull requests from developer342's master.

From the sound of it this request is pulled into master. We 
continue to pull many of these changes in. How do we decide they 
should be placed into staging, when we pull them into master?. If 
we wait for some 'magic time' how do we pull it into staging, 
does that mean we now cherry pick commits of master?

Another issue is it sounds like master becomes a "phantom" 
branch. At no point in time would master resemble what is 
released. I see this as a problem because it is the branch people 
are developing off of, it means nothing to test in master as 
staging has the actual state that will be released.
December 20, 2012
Re: Next focus: PROCESS
On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips 
wrote:
> On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:
>
>> master : used as a base for development. New feature are 
>> merged here.
>> staging : used to provide a view of what the next version will 
>> look like. Regular snapshot of that branch are made so public 
>> can use the last features.
>> version : used to contain a version that will have a support 
>> for an extended period of time.
>
> I do not see how development in master is moved to staging 
> based on this description. I'll try and be more specific.
>
> I realize you don't like features, but we need to talk about 
> Phobos additions, @property mechanics, and other highly 
> disruptive bugs. I'll call it the major-change.
>
> The major-change branch is generally developed to completion 
> and may just be a pull requests from developer342's master.
>
> From the sound of it this request is pulled into master. We 
> continue to pull many of these changes in. How do we decide 
> they should be placed into staging, when we pull them into 
> master?. If we wait for some 'magic time' how do we pull it 
> into staging, does that mean we now cherry pick commits of 
> master?
>
> Another issue is it sounds like master becomes a "phantom" 
> branch. At no point in time would master resemble what is 
> released. I see this as a problem because it is the branch 
> people are developing off of, it means nothing to test in 
> master as staging has the actual state that will be released.

That is very true. With the given modification master and 
stagging become redundant with one another. Why not get rid of 
stagging ?
December 20, 2012
Re: Next focus: PROCESS
On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips 
wrote:
>
> From the sound of it this request is pulled into master. We 
> continue to pull many of these changes in. How do we decide 
> they should be placed into staging, when we pull them into 
> master?. If we wait for some 'magic time' how do we pull it 
> into staging, does that mean we now cherry pick commits of 
> master?
>
> Another issue is it sounds like master becomes a "phantom" 
> branch. At no point in time would master resemble what is 
> released. I see this as a problem because it is the branch 
> people are developing off of, it means nothing to test in 
> master as staging has the actual state that will be released.

Most of the D users are not D devs, and they don't want to be 
testing highly unstable features, however some of them will want 
to use reasonably stable new features, even though there may be 
some bugs and there may be some changes. The reason for using the 
testing branch at all, is that they need something in it that's 
not available in the stable branch, so they may take a smallish 
risk and try out the testing or staging branch (need to decide on 
the terms here).

I placed a note in the wiki about the issue of making sure there 
are incentives for people to do the things you wish them to do 
for each major step in the process. If there is no incentives for 
people to test what comes out of development, few people  will 
test it, but you need plenty of testers. The carrot is that a 
staging tester gets a snapshot of what is most current that is 
also reasonably stable, ie., there has to be enough confidence 
that the code is stable enough to use for real, else the risk 
will be too high to bother with it.

I know this because I'm one of those people. I pick and choose 
stable vs
"reasonably" stable based on my needs and the risk assessment. 
You need to be quick about supporting the testing branch and 
careful about not dumping crap into it, it's a delicate balance 
yes, but you *need* testers, and there's only one good way to get 
them, you have to dangle a carrot out and hope they take the 
bait, and then you have to be prepared to help them out when 
things go wrong.

I expect very few people will take a chance on the dev branch 
other than the devs themselves and a few of the experienced 
people who know their way around the compiler, that's just the 
way things will be because it'll be too risky for the regular 
folks to bother with.

The user count will always be lowest for the dev branch, more for 
staging but you want to encourage highest use possible which 
means quick support, and the highest use will be for stable, but 
because it's stable means it's not really being tested because it 
should be mostly bug free, i.e., it's being "used" as opposed to 
being "tested".

I don't think you can afford to drop the testing branch unless 
you think there's no need for very many Average Joe testers, but 
those are the guys who will uncover a ton of unexpected bugs 
missed by the dev testers.

--rt
December 20, 2012
Re: Next focus: PROCESS
On Thu, Dec 20, 2012 at 05:48:13AM +0100, deadalnix wrote:
> On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips wrote:
> >On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix wrote:
> >
> >>master : used as a base for development. New feature are merged
> >>here.
> >>staging : used to provide a view of what the next version will
> >>look like. Regular snapshot of that branch are made so public
> >>can use the last features.
> >>version : used to contain a version that will have a support for
> >>an extended period of time.
[...]
> >From the sound of it this request is pulled into master. We
> >continue to pull many of these changes in. How do we decide they
> >should be placed into staging, when we pull them into master?. If
> >we wait for some 'magic time' how do we pull it into staging, does
> >that mean we now cherry pick commits of master?

In my mind, after a release, the contents of staging are updated to be
exactly the same as master. This can be done either via a merge, or
simply deleting the current staging and making a new one by branching
from master.


> >Another issue is it sounds like master becomes a "phantom" branch.
> >At no point in time would master resemble what is released. I see
> >this as a problem because it is the branch people are developing off
> >of, it means nothing to test in master as staging has the actual
> >state that will be released.
> 
> That is very true. With the given modification master and stagging
> become redundant with one another. Why not get rid of stagging ?

No, staging does not receive new features except immediately after
release. But it will get bugfixes that do not introduce new things.

Example timeline:
- time=0: we just made a release, so staging := master.

- time=1: a crash bug is discovered in master. Walter checks in a fix.
 Fix is propagated to staging because it doesn't introduce new things,
 only fixes existing problems.

- time=2: Walter decides to merge UDA implementation into master (for
 example). Staging does NOT get UDA merged in, because UDA is new.

- time=3: more bugs are found in master. Fixes are checked in. Fixes get
 propagated to staging.

- time=4: a UDA bug is fixed. Fix is NOT propagated to staging, because
 staging doesn't have UDA (yet).

- time=5: we decide that it's time to release. So:
  - we tag a release on current staging
  - we merge master into staging, so now staging gets UDA

- time=6: Walter adds HalfFloat to master. HalfFloat is NOT propagated
 to staging, because it's new.

- time=7: users running staging find a nasty design flaw in UDA. Walter
 makes a (breaking) fix in master, and the fix is propagated to
 staging, because UDA is already in staging.

- time=8: Walter finds bugs in HalfFloat. Fixes are checked into master,
 but NOT staging.

- time=9: Walter adds more new features to master. These are NOT
 propagated to staging, because they're new.

- time=10: it is decided that UDA design is stable now, and we're ready
 for release. So:
  - we tag a new release on current staging
  - we merge master into staging, now staging gets HalfFloat.

- etc.

Hopefully this makes it clearer what we're trying to achieve.


T

-- 
Arise, you prisoners of Windows 
Arise, you slaves of Redmond, Wash, 
The day and hour soon are coming 
When all the IT folks say "Gosh!" 
It isn't from a clever lawsuit 
That Windowsland will finally fall, 
But thousands writing open source code 
Like mice who nibble through a wall.
-- The Linux-nationale by Greg Baker
December 20, 2012
Re: Next focus: PROCESS
On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:
>
> Personally, I think the whole pre-development stage needs to 
> get looks at -
> Specifically having some sort of at least high-level *binding* 
> planning - road map, mile stones, todo lists. This should give 
> more weight to DIPs.
> DIPs at the moment have no weight whatsoever. The attributes 
> showed all the major flows in the decision making process:
>
> 1. Feature idea comes up in discussion.
> 2. Feature was discussed heavily by the community reaching some 
> design consensus.
> 3. Plan is abandoned/forgotten due to Walter's objections - not 
> to the design but the idea itself.
> 4. Feature comes up again in discussion, thus returning to 
> point 1 above in infinite loop.
>

A big fix is in order there too, it's actually what should be 
fixed first but from what I've seen going on in here if we can 
get something even basic implemented for the development and 
releases that will be a *huge* step forward. Once we have some 
kind of formalized process in place and prove to everyone how 
much better things are because of it, it'll be a first by the 
looks of things, and from that experience the other big holes 
will become easier to pick out and deal with.

I think we have to keep it simple for now, focus on a dev, 
staging, release process, implement something, work out the bugs, 
get people used to having it, and prove the value, then we can 
get on with tackling the other major issues in a similar way.

There's lots of room for improvement ahead, but we need to make 
at least one significant step forward in a successful way before 
we can hope to move on to the next one.

--rt
December 20, 2012
Re: Next focus: PROCESS
On Thursday, 20 December 2012 at 05:32:30 UTC, H. S. Teoh wrote:
> On Thu, Dec 20, 2012 at 05:48:13AM +0100, deadalnix wrote:
>> On Thursday, 20 December 2012 at 04:11:00 UTC, Jesse Phillips
> In my mind, after a release, the contents of staging are 
> updated to be
> exactly the same as master. This can be done either via a 
> merge, or
> simply deleting the current staging and making a new one by 
> branching
> from master.

Yes, exactly!

>
> No, staging does not receive new features except immediately 
> after
> release. But it will get bugfixes that do not introduce new 
> things.
>
> Example timeline:
> - time=0: we just made a release, so staging := master.
>
> - time=1: a crash bug is discovered in master. Walter checks in 
> a fix.
>   Fix is propagated to staging because it doesn't introduce new 
> things,
>   only fixes existing problems.
>
> - time=2: Walter decides to merge UDA implementation into 
> master (for
>   example). Staging does NOT get UDA merged in, because UDA is 
> new.
>
> - time=3: more bugs are found in master. Fixes are checked in. 
> Fixes get
>   propagated to staging.
>
> - time=4: a UDA bug is fixed. Fix is NOT propagated to staging, 
> because
>   staging doesn't have UDA (yet).
>
> - time=5: we decide that it's time to release. So:
>    - we tag a release on current staging
>    - we merge master into staging, so now staging gets UDA
>
> - time=6: Walter adds HalfFloat to master. HalfFloat is NOT 
> propagated
>   to staging, because it's new.
>
> - time=7: users running staging find a nasty design flaw in 
> UDA. Walter
>   makes a (breaking) fix in master, and the fix is propagated to
>   staging, because UDA is already in staging.
>
> - time=8: Walter finds bugs in HalfFloat. Fixes are checked 
> into master,
>   but NOT staging.
>
> - time=9: Walter adds more new features to master. These are NOT
>   propagated to staging, because they're new.
>
> - time=10: it is decided that UDA design is stable now, and 
> we're ready
>   for release. So:
>    - we tag a new release on current staging
>    - we merge master into staging, now staging gets HalfFloat.
>
> - etc.
>
> Hopefully this makes it clearer what we're trying to achieve.

Yes, fully agreeable with me!

>
>
> T

Thanks for writing that out so well. Excellent!

--rt
December 20, 2012
Re: Next focus: PROCESS
On Thursday, 20 December 2012 at 05:33:27 UTC, Rob T wrote:
> On Wednesday, 19 December 2012 at 21:58:12 UTC, foobar wrote:
>>
>> Personally, I think the whole pre-development stage needs to 
>> get looks at -
>> Specifically having some sort of at least high-level *binding* 
>> planning - road map, mile stones, todo lists. This should give 
>> more weight to DIPs.
>> DIPs at the moment have no weight whatsoever. The attributes 
>> showed all the major flows in the decision making process:
>>
>> 1. Feature idea comes up in discussion.
>> 2. Feature was discussed heavily by the community reaching 
>> some design consensus.
>> 3. Plan is abandoned/forgotten due to Walter's objections - 
>> not to the design but the idea itself.
>> 4. Feature comes up again in discussion, thus returning to 
>> point 1 above in infinite loop.
>>
>
> A big fix is in order there too, it's actually what should be 
> fixed first but from what I've seen going on in here if we can 
> get something even basic implemented for the development and 
> releases that will be a *huge* step forward. Once we have some 
> kind of formalized process in place and prove to everyone how 
> much better things are because of it, it'll be a first by the 
> looks of things, and from that experience the other big holes 
> will become easier to pick out and deal with.
>
> I think we have to keep it simple for now, focus on a dev, 
> staging, release process, implement something, work out the 
> bugs, get people used to having it, and prove the value, then 
> we can get on with tackling the other major issues in a similar 
> way.
>
> There's lots of room for improvement ahead, but we need to make 
> at least one significant step forward in a successful way 
> before we can hope to move on to the next one.
>
> --rt

I see both as going hand-in-hand, otherwise we have chicken-egg 
problem.
We need a better process to allow more developers to contribute 
code more easily *and* we need better planning to provide 
incentive for new developer to contribute code.
Implementing only the the planning provides incentive but no 
means. Implementing only the process provides the means but not 
the incentive.

While improving git mastery levels is beneficial of itself, it 
brings no benefit to the _users_ if only Walter is still the main 
developer. We end up wasting Walter's time on adjusting to a new 
system for a hypothetical non existent system of many developers, 
when he already has a working system for the actual existing 
situation of being the sole main developer.

We need an *initial high-level* design for both parts. We can 
define and refine the details later and we probably should defer 
it to later and adjust it as we go along based on experience.
Arguing now on the exact time span of a release just wastes mind 
cycles and detracts from the important bits - reaching a 
consensus regarding the goals of the project and defining a 
general workflow that achieves those goals with minimal amount of 
overhead.
December 20, 2012
Re: Next focus: PROCESS
On Thursday, 20 December 2012 at 05:32:30 UTC, H. S. Teoh wrote:
>> >On Wednesday, 19 December 2012 at 23:05:59 UTC, deadalnix 
>> >wrote:
>> >
>> >>master : used as a base for development. New feature are 
>> >>merged
>> >>here.
>> >>staging : used to provide a view of what the next version 
>> >>will
>> >>look like. Regular snapshot of that branch are made so public
>> >>can use the last features.
>> >>version : used to contain a version that will have a support 
>> >>for
>> >>an extended period of time.
>
> In my mind, after a release, the contents of staging are 
> updated to be
> exactly the same as master. This can be done either via a 
> merge, or
> simply deleting the current staging and making a new one by 
> branching
> from master.

I can get behind this.

deadalnix, can we please go back to calling "version" as 
"release" that is what it is and what everyone else uses I don't 
see a need make a new name because you see "release" as something 
different.
December 20, 2012
Re: Next focus: PROCESS
On Thursday, 20 December 2012 at 08:27:22 UTC, foobar wrote:
> I see both as going hand-in-hand, otherwise we have chicken-egg 
> problem.
> We need a better process to allow more developers to contribute 
> code more easily *and* we need better planning to provide 
> incentive for new developer to contribute code.
> Implementing only the the planning provides incentive but no 
> means. Implementing only the process provides the means but not 
> the incentive.
>
> While improving git mastery levels is beneficial of itself, it 
> brings no benefit to the _users_ if only Walter is still the 
> main developer. We end up wasting Walter's time on adjusting to 
> a new system for a hypothetical non existent system of many 
> developers, when he already has a working system for the actual 
> existing situation of being the sole main developer.
>
> We need an *initial high-level* design for both parts. We can 
> define and refine the details later and we probably should 
> defer it to later and adjust it as we go along based on 
> experience.
> Arguing now on the exact time span of a release just wastes 
> mind cycles and detracts from the important bits - reaching a 
> consensus regarding the goals of the project and defining a 
> general workflow that achieves those goals with minimal amount 
> of overhead.

I think what was proposed - to fast track this one, and not over 
engineer - is a good idea. That way we can at least get something 
that's far better than before, and be able to more quickly move 
on to the next big problem area which I would agree is what you 
are mentioning. There's lack of planning and how decisions are 
made is very inefficient and arbitrary, and I would also say we 
have a big problem with the specification being poorly managed, 
eg, it's not even available for download, has no process for 
revisions and releases, nothing at all - it's worse than the 
existing development process which we all agree is in need of a 
big overhaul.

Please, let's all agree to deal with that monster next, but it's 
a big problem to solve, and if we try to bundle it in with this 
one, we may not get any accomplished at all. I'd prefer the 
divide and conquer approach.

--rt
December 20, 2012
Re: Next focus: PROCESS
On Wednesday, 19 December 2012 at 21:30:44 UTC, Andrei 
Alexandrescu wrote:
>
> I agree with one "stable" branch.
>
> Andrei

Just some food for thought.

In the section about the "Branching model", the wiki currently 
has a staging branch in addition to the master branch. From what 
I understand, the idea seems to be to vet a release on staging 
until it is considered production level and then marked as the 
release.

Another idea could be to keep the quality of the master branch at 
a high level so as to be able to branch into a release at any 
time, directly from master. Before feature branches are merged 
back into master, their quality is vetted so the quality of 
master is maintained.

This idea seems similar to what is used for the vibe.d project 
(http://vibed.org/temp/branch-model-small.png). My apologies if I 
misunderstood their process.

It looks like Xamarin has been using this process for a while and 
it seems to be working for them. 
http://tirania.org/blog/archive/2011/Oct-14.html

Joseph
12 13 14 15 16 17
Top | Discussion index | About this forum | D home