On Mon, 16 Jul 2012 09:41:30 -0700, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:

> On 7/16/12 3:51 AM, Adam Wilson wrote:
>> As a result of the D Versioning thread, we have decided to create a new
>> organization on Github called dlang-stable. This organization will be
>> responsible for maintaining stable releases of DMD, DRuntime, and Phobos.
> [snip]
>> Thank you for reading and I hope you enjoy D's new Stable releases!
>
> This is a wonderful initiative that I hope will catch on.
>
> Walter and I are fully behind this experiment. We will make/accept changes to the download pages to take into account this new branch.
>
> The main threat I see is bugfixes that depend on code previously introduced with now features. I'm not sure how frequent that situation is.
>
> Finally, one suggestion: I suggest you relentlessly work to automate the process to the point where you issue e.g. the command
>
> ./merge 534d44c979 70dcb958ea
>
> and the script merges in the two specified commits, builds everything, runs the test suite, runs the Phobos unittests, and if all worked, builds packages and uploads them making them available right away.

The only problem I can see with this is that the build would only for whichever OS the merger happens to be running on. A two step process is about the best we can do.

./merge does the following
git fetch upstream
git cherry-pick 534d44c979
git cherry-pick 70dcb958ea

If the cherry pick is successful, push to master, which will need to be done manually in case any merge conflicts arise.

./build does the following for all projects
git fetch origin
git merge origin/master
make clean -f <system>.mak
make -f <system>.mak
make unittests -f <system>.mak

As for actually putting together the script, my Bash-fu, is not as strong as my PowerShell-fu, but with some effort it could be achieved. Both are not terribly difficult (expect for the uploading part).

> That way the team members (busy professionals with many other responsibilities) know that once the collective decision to merge a commit has been taken, there's virtually no overhead to deploying the decision.
>
> I'm a huge fan of automation and I believe we've made progress introducing it in dmd's process, but not as much as we could and should.
>
>
> Good luck!
>
> Andrei
>


-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/

On Monday, 16 July 2012 at 19:35:47 UTC, Andrei Alexandrescu wrote:
> [snip]
>
> I agree this would be more direct. But I fail to see how Walter cherry-picking stuff is "basically no additional work", whereas Adam doing essentially the same is an unenviable "amount of labor".
>

He wouldn't be cherry-picking anything. All he'd have to do is a checkout (one command) before switching between working on features (and by features I mean breaking or major changes) or bugs. Adam, on the other hand, would be running git cherry-pick over and over dodging changes he determines are too risky to include in stable. It also seems like there is a lot more room for mistakes to be made doing it this way.

> Besides, if Walter is at the same time doing things and deciding their category may work worse than a small team making the assessment by consensus.
>

Perhaps. More eyes are better. It just seems like way more work than is necessary to accomplish the goal of isolating breaking changes so that stable releases can go out easier. The end result is all I'm interested in personally since I'm not the one doing the work here. I'm not too concerned with whether you use my idea or not. I just thought you guys might like to hear an alternative that is much easier for you while remaining easy for Walter.

Regards,
Brad Anderson

On 2012-07-16 21:37, Adam Wilson wrote:

> The only problem I can see with this is that the build would only for
> whichever OS the merger happens to be running on. A two step process is
> about the best we can do.
>
> ./merge does the following
> git fetch upstream
> git cherry-pick 534d44c979
> git cherry-pick 70dcb958ea
>
> If the cherry pick is successful, push to master, which will need to be
> done manually in case any merge conflicts arise.

How would this work? Isn't the whole point of cherry picking to be able to choose what to pick from a branch and not picking everything?

> ./build does the following for all projects
> git fetch origin
> git merge origin/master
> make clean -f <system>.mak
> make -f <system>.mak
> make unittests -f <system>.mak
>
> As for actually putting together the script, my Bash-fu, is not as
> strong as my PowerShell-fu, but with some effort it could be achieved.
> Both are not terribly difficult (expect for the uploading part).

You could use DVM with the "compile" command to build everything.

https://bitbucket.org/doob/dvm

-- 
/Jacob Carlborg

On Mon, 16 Jul 2012 23:17:38 -0700, Jacob Carlborg <doob@me.com> wrote:

> On 2012-07-16 21:37, Adam Wilson wrote:
>
>> The only problem I can see with this is that the build would only for
>> whichever OS the merger happens to be running on. A two step process is
>> about the best we can do.
>>
>> ./merge does the following
>> git fetch upstream
>> git cherry-pick 534d44c979
>> git cherry-pick 70dcb958ea
>>
>> If the cherry pick is successful, push to master, which will need to be
>> done manually in case any merge conflicts arise.
>
> How would this work? Isn't the whole point of cherry picking to be able to choose what to pick from a branch and not picking everything?

Each parameter represents a specific commit ID to pick.

>> ./build does the following for all projects
>> git fetch origin
>> git merge origin/master
>> make clean -f <system>.mak
>> make -f <system>.mak
>> make unittests -f <system>.mak
>>
>> As for actually putting together the script, my Bash-fu, is not as
>> strong as my PowerShell-fu, but with some effort it could be achieved.
>> Both are not terribly difficult (expect for the uploading part).
>
> You could use DVM with the "compile" command to build everything.
>
> https://bitbucket.org/doob/dvm

Hmm, this sounds like an interesting method, can it be targeted at any repo and if so how? And does it run unittests?

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/

On Mon, 16 Jul 2012 23:24:04 -0700
"Adam Wilson" <flyboynw@gmail.com> wrote:
> On Mon, 16 Jul 2012 23:17:38 -0700, Jacob Carlborg <doob@me.com> wrote:
> 
> > On 2012-07-16 21:37, Adam Wilson wrote:
> >
> >> ./build does the following for all projects
> >> git fetch origin
> >> git merge origin/master
> >> make clean -f <system>.mak
> >> make -f <system>.mak
> >> make unittests -f <system>.mak
> >>
> >> As for actually putting together the script, my Bash-fu, is not as strong as my PowerShell-fu, but with some effort it could be achieved. Both are not terribly difficult (expect for the uploading part).
> >
> > You could use DVM with the "compile" command to build everything.
> >
> > https://bitbucket.org/doob/dvm
> 
> Hmm, this sounds like an interesting method, can it be targeted at any repo and if so how? And does it run unittests?
> 

dvm compile <path>

Where <path> is a directory that contains git clones of dmd, phobos, druntime, and optionally DMD's tools repo. So whatever branch/version is checked out in that directory, that's what dvm will compile.

It's not very flexible though. For example, it doesn't support
passing arbitrary flags/params to make. RDMD is the only program from
the tools repo it tries to compile. IIRC, it always does a clean
build (or at least to the extent that the "clean" in DMD's
makefiles actually works in the first place - ie, not always perfectly).
It doesn't run unittests, IIRC. And I think there's a thing or two (I
forget what) that it does slightly different between windows and posix
(this was due to the targets and features available/unavailable in
the makefiles being completely fucking different between windows and
posix).

Also, if you're compiling Git sources (instead of recompiling a DMD release), then it doesn't attempt to recreate the DMD release directory structure, instead it just puts the bins in <path>/dmd/bin(32|64)/ (or something like that, don't recall exactly) and sets up sc.ini/dmd.conf to use the phobos in <path>/phobos, etc.

And then there's a little magic to be aware of: For some stuff (like windows libs) we don't have source on Github, so it automagically pulls those in from whatever it detects is the latest official release (downloading it if necessary).

(tl;dr): Main thing to remember is, as it's written right now, it was really intended more for people building DMD/Druntime/Phobos themself and then just using it. Not much thought went into using it to create packaged releases. So if happens to do what you need it to, then fantastic, but you should probably try it out first (with the --verbose flag) and pay close attention to what it's doing. Then see if that matches what you need. If so, great! If not, then file a ticket, and with luck maybe Jacob or I may be able to find some time to take care of it. But I've been busy though, so I can't promise anything :(

Or just deal with bash ;) Either way.

On 2012-07-17 08:24, Adam Wilson wrote:

> Each parameter represents a specific commit ID to pick.

So you would pass those to the script?

-- 
/Jacob Carlborg

On Tue, 17 Jul 2012 02:08:55 -0700, Jacob Carlborg <doob@me.com> wrote:

> On 2012-07-17 08:24, Adam Wilson wrote:
>
>> Each parameter represents a specific commit ID to pick.
>
> So you would pass those to the script?

Yup, I think that's probably the safest way to do it. Given the goals of the project, we need to be explicit about which commits we choose so forcing the maintainers to write down every commit is good practice.

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/

On 2012-07-17 19:04, Adam Wilson wrote:

> Yup, I think that's probably the safest way to do it. Given the goals of
> the project, we need to be explicit about which commits we choose so
> forcing the maintainers to write down every commit is good practice.

Exactly. I took that example a bit too literal, that's what confused me.

-- 
/Jacob Carlborg

On Monday, 16 July 2012 at 19:35:47 UTC, Andrei Alexandrescu wrote:

> I agree this would be more direct. But I fail to see how Walter cherry-picking stuff is "basically no additional work", whereas Adam doing essentially the same is an unenviable "amount of labor".

Well, if things go well, there is a system that could make things much simpler. But my thought would that Walter wouldn't need to touch the stable branch. The only benefit for the new org is a separate set of contributes and separate pull requests.

If this system takes off, while still having a cost to Walter, there would be a stable branch which get all the bug fixes and the new feature branch. When adding a bug fix just merge the stable branch into the feature branch. This won't always be a clean merge.

Good luck Adam.

Just curious, if Walter is too busy to make DMD more stable, then what does he spend most of his time on?  I thought we were done with adding new features to the language (at least for now)?

On Mon, Jul 16, 2012 at 1:29 PM, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> On Monday, July 16, 2012 13:18:55 Caligo wrote:
>> What?!
>>
>> All he has to do is create a branch and call it stable, or whatever, and use cherry-pick to move small stables patches from other branches to the stable branch. What's so difficult about that? I don't get it.
>
> It takes up his time for him to do it, and I don't think that there's any chance whatsoever of him giving someone else commit access to the main dmd repository to do this (certainly not at this point). He wasn't interested in changing the versioning or branching stuff at all. He agreed to this plan (which should be minimally disruptive to him) as an experiment. Depending on how it goes, we may change how we deal with the main repo, but for now, we're trying this. And since the likely alternative was that we would continue on exactly as we have been, this is presumably an improvement for the folks who wanted a branch with bug fixes with more frequent releases.
>
> - Jonathan M Davis