Sent this to dmd-internals, opening for a broader discussion:

Hello,


I wonder how we can define a few aliases of project-wide usefulness to git. For example, I tried today to get the latest and greatest phobos into my repo, and got a bunch of conflicts. I searched a while on the net to figure what the command for "just get the latest remote into my local copy", which is entirely possible but not obvious and not easy to find. I got it from here: http://goo.gl/pWvVB and the invocation is:

git pull --rebase -s recursive -X ours

which does work like a charm. So I wanted to define a macro called "pull-force" like this:

# Inside ~/.gitconfig
[alias]
    pull-force = pull --rebase -s recursive -X ours
    ...

This, too, works great.

What I'd want to do next is define the alias such that other participants to the dlang project may use it. With time we'd collect a good set of macros that help our process.

What is the best way to share a few git config items among all participants to dmd, druntime, phobos etc?


Thanks,

Andrei

On Sunday, January 06, 2013 14:50:49 Andrei Alexandrescu wrote:
> Sent this to dmd-internals, opening for a broader discussion:
> 
> Hello,
> 
> 
> I wonder how we can define a few aliases of project-wide usefulness to git. For example, I tried today to get the latest and greatest phobos into my repo, and got a bunch of conflicts. I searched a while on the net to figure what the command for "just get the latest remote into my local copy", which is entirely possible but not obvious and not easy to find. I got it from here: http://goo.gl/pWvVB and the invocation is:
> 
> git pull --rebase -s recursive -X ours
> 
> which does work like a charm. So I wanted to define a macro called "pull-force" like this:
> 
> # Inside ~/.gitconfig
> [alias]
>      pull-force = pull --rebase -s recursive -X ours
>      ...
> 
> This, too, works great.
> 
> What I'd want to do next is define the alias such that other participants to the dlang project may use it. With time we'd collect a good set of macros that help our process.
> 
> What is the best way to share a few git config items among all participants to dmd, druntime, phobos etc?

To repeat what I just said in dmd-internals:

Maybe some macros would be useful, but I'm kind of inclined to think that people should just learn git, especially since many people will need it beyond our projects and learning an alias that doesn't exist elsewhere will just hamstring you. I think that it would be a better idea to have a page in the wiki listing useful commands so that people can learn them properly rather than relying on an alias that isn't standard git and which won't exist on any other projects.

But with regards to this particular command, I'd argue that you're doing something wrong if you need to force a rebase with a pull. How on earth did you end up in a situation where you needed to force a rebase like this on master or any branch that's in the main repo? Changes should be being done on separate branches, not on master. So, master doesn't need to be rebased. And generally, branches are your local branches, managed by you and which generally are relatively short-lived, so you won't need to do what you're describing here for them. If you're making changes on other branches in the main repository, I'd expect them to be treated the same as master. You make changes on your own, separate branch, push that to your repo on github for a pull request and get that merged into the branch in the main repo. So, in no case do you end up with any branch on your box being out of sync with the main repo. The rebasing that should be needed is when you have to rebase one of your local branches with master, because other changes were merged into master via pull requests, and the changes in your branch haven't been merged in on github yet.

The only time that you'd need to make a change directly on master or any branch which is in the main repository is when you actually push upstream from your local box instead of going through github, which we're not generally supposed to do. I have never needed the command that you just listed.

Maybe, you did something normal that I can't think of, but it seems to me that the problem here is your workflow, not the need for a macro.

- Jonathan M Davis

On 2013-01-06 21:47, Jonathan M Davis wrote:


> But with regards to this particular command, I'd argue that you're doing
> something wrong if you need to force a rebase with a pull.

I agree.

-- 
/Jacob Carlborg

On Sunday, 6 January 2013 at 19:50:47 UTC, Andrei Alexandrescu wrote:
> Sent this to dmd-internals, opening for a broader discussion:
>
> Hello,
>
>
> I wonder how we can define a few aliases of project-wide usefulness to git. For example, I tried today to get the latest and greatest phobos into my repo, and got a bunch of conflicts. I searched a while on the net to figure what the command for "just get the latest remote into my local copy", which is entirely possible but not obvious and not easy to find. I got it from here: http://goo.gl/pWvVB and the invocation is:
>
> git pull --rebase -s recursive -X ours

Question: Do you know what exactly does this command do?

Git is a very powerful and flexible system. Unfortunately, it is also comparatively more complicated than its competitors, svn and hg. Incorrect usage, especially by project maintainers, can result in history rewrites, duplicate commits, and lost changes. This has already happened multiple times in the past.

Earlier I noticed some discussion regarding how Walter attempted to follow a guide on the wiki regarding the release process. The problem is that (from what I gathered) he attempted to simply run the commands listed there, without understanding each one's exact effect and purpose. An observation was raised that the guide was incomplete - however, I believe the iteration at that point was written as a guideline (a general outline of what needs to be done) as opposed to strict step-by-step instructions. When stumped with a merge, he elected to use the "git cherry-pick" command to copy the commits on top of his HEAD - however, this duplicates the commits' contents and results in duplicate commits in the repository.

I think you're trying to solve the wrong problem...

Back to your question. Let's look at the command you posted:

> git pull --rebase -s recursive -X ours

From my understanding, this this is what will happen:

1. Fetch whatever commit objects that are absent from your tree but are present in the remote branch, and copy them to your tree. So far so good.
2. Rewind your HEAD to the tip of the branch you just pulled.
3. Take each commit that are in your branch's history, but are not in the remote's history, and re-apply them on top of the new HEAD. This creates new commit objects, with new SHA-1 hashes. Anything that builds upon the old SHA-1 hashes (e.g. if you've already uploaded these commits anywhere) will break. (--rebase rewrites history!)
4. Whenever there is a merge conflict, it will THROW AWAY your change and automatically apply the other change.

Was this your intention?

Now, the question you should ask: WHY did you get a merge conflict?

Was it a case of you and someone else actually changing the same bit of code? Or was it another instance of git abuse and someone force-pushing after someone else already pulled the code?

If the merge conflict is genuine, and you don't want to deal with the merge, you should abort the merge, save your work as a new branch, and reset the current branch to the remote's:

$ git merge --abort
$ git branch branch-name-for-my-work
$ git reset --hard origin/master

(substitute "origin" for the remote name, and "master" for the remote branch name).

If you want to throw away any commits that are in your local branch but not in the remote branch, just run the third command.

-- Vladimir, and too much coffee

On 1/6/13 4:03 PM, Vladimir Panteleev wrote:
> On Sunday, 6 January 2013 at 19:50:47 UTC, Andrei Alexandrescu wrote:
>> git pull --rebase -s recursive -X ours
>
> Question: Do you know what exactly does this command do?

I do now after having read about it at the end of several google searches.

> Git is a very powerful and flexible system. Unfortunately, it is also
> comparatively more complicated than its competitors, svn and hg.
> Incorrect usage, especially by project maintainers, can result in
> history rewrites, duplicate commits, and lost changes. This has already
> happened multiple times in the past.

I agree it is important to be fluent with git. I'm an okay user but currently unfit for an admin. Probably Walter is at a similar stage. I'm doing my best to improve my knowledge in the limited time I have, and part of it is discussing git here. So I hope you'll understand why I find it contradictory to be reprimanded for doing so with the reproach that I should be doing exactly what I'm doing.

> Earlier I noticed some discussion regarding how Walter attempted to
> follow a guide on the wiki regarding the release process. The problem is
> that (from what I gathered) he attempted to simply run the commands
> listed there, without understanding each one's exact effect and purpose.
> An observation was raised that the guide was incomplete - however, I
> believe the iteration at that point was written as a guideline (a
> general outline of what needs to be done) as opposed to strict
> step-by-step instructions. When stumped with a merge, he elected to use
> the "git cherry-pick" command to copy the commits on top of his HEAD -
> however, this duplicates the commits' contents and results in duplicate
> commits in the repository.

I agree we need to develop a better understanding of git. At the same time it would be great to learn the meaning of commands from a guide that actually works, as opposed to a sketch that doesn't quite.

> I think you're trying to solve the wrong problem...
>
> Back to your question. Let's look at the command you posted:
>
>> git pull --rebase -s recursive -X ours
>
>  From my understanding, this this is what will happen:
>
> 1. Fetch whatever commit objects that are absent from your tree but are
> present in the remote branch, and copy them to your tree. So far so good.
> 2. Rewind your HEAD to the tip of the branch you just pulled.
> 3. Take each commit that are in your branch's history, but are not in
> the remote's history, and re-apply them on top of the new HEAD. This
> creates new commit objects, with new SHA-1 hashes. Anything that builds
> upon the old SHA-1 hashes (e.g. if you've already uploaded these commits
> anywhere) will break. (--rebase rewrites history!)
> 4. Whenever there is a merge conflict, it will THROW AWAY your change
> and automatically apply the other change.
>
> Was this your intention?

Yes, that was exactly my intention.

> Now, the question you should ask: WHY did you get a merge conflict?
>
> Was it a case of you and someone else actually changing the same bit of
> code? Or was it another instance of git abuse and someone force-pushing
> after someone else already pulled the code?

I have no idea. To the best of my understanding I had no meaningful work in master proper. The one conflict I looked at was in std/variant.d. It was a conflict of an older version of the thing with a newer one (the lines in conflict were at the recent apply functionality). I'd written no lines of code in the conflict. It's not impossible I might have checked out a pull request to look at it. Anyway, my purpose was: whatever crap is in my master right now, make it the same as the origin's master.

> If the merge conflict is genuine, and you don't want to deal with the
> merge, you should abort the merge, save your work as a new branch, and
> reset the current branch to the remote's:
>
> $ git merge --abort
> $ git branch branch-name-for-my-work
> $ git reset --hard origin/master
>
> (substitute "origin" for the remote name, and "master" for the remote
> branch name).

I get these tidbits of commands - git idioms - all the time, and I forget them because I don't use them frequently. I don't see why it's a crime to want to define some macros for such idioms instead of essentially putting that in a text file that I'd be perusing now and then.

> If you want to throw away any commits that are in your local branch but
> not in the remote branch, just run the third command.

That's pretty much what I wanted but was unable to find easily by searching the net.


Andrei

On Sunday, January 06, 2013 17:45:29 Andrei Alexandrescu wrote:
> I get these tidbits of commands - git idioms - all the time, and I
> forget them because I don't use them frequently. I don't see why it's a
> crime to want to define some macros for such idioms instead of
> essentially putting that in a text file that I'd be perusing now and then.

If you want to define macros for your personal use, then that's your prerogative, but I don't think that it's a good idea to encourage them by putting them in the repo's .gitconfig file. And in this particular case, git- reset is basic enough and so useful (and so frequently needed in my experience), that while I can understand someone not knowing about it as they're learning, it's the sort of thing that anyone using git on a regular basis should know by heart.

My guess would be that part of your problem is the fact that you're functioning primarily in the capacity of an admin and reviewer these days rather than writing code yourself (or at least, if you're writing much code for Phobos and it's related projects, you're not submitting it). And if you're not using git aside from dmd/druntime/Phobos/d-programming-language.org, then it's much easier to forget the commands.

- Jonathan M Davis

On 1/6/13 6:02 PM, Jonathan M Davis wrote:
> On Sunday, January 06, 2013 17:45:29 Andrei Alexandrescu wrote:
>> I get these tidbits of commands - git idioms - all the time, and I
>> forget them because I don't use them frequently. I don't see why it's a
>> crime to want to define some macros for such idioms instead of
>> essentially putting that in a text file that I'd be perusing now and then.
>
> If you want to define macros for your personal use, then that's your
> prerogative, but I don't think that it's a good idea to encourage them by
> putting them in the repo's .gitconfig file.

Why? I don't quite buy the argument. You're essentially asking for rote memorization. Also, process should only be helped by macros that enforce it.

Andrei

On Sunday, January 06, 2013 18:12:20 Andrei Alexandrescu wrote:
> On 1/6/13 6:02 PM, Jonathan M Davis wrote:
> > On Sunday, January 06, 2013 17:45:29 Andrei Alexandrescu wrote:
> >> I get these tidbits of commands - git idioms - all the time, and I forget them because I don't use them frequently. I don't see why it's a crime to want to define some macros for such idioms instead of essentially putting that in a text file that I'd be perusing now and then.
> > 
> > If you want to define macros for your personal use, then that's your prerogative, but I don't think that it's a good idea to encourage them by putting them in the repo's .gitconfig file.
> 
> Why? I don't quite buy the argument. You're essentially asking for rote memorization. Also, process should only be helped by macros that enforce it.

I'm arguing that people should actually learn how to use git properly rather than creating crutches for themselves. The more you try and do with git without learning it properly, the more problems you're likely to run into (e.g. by running commands that you don't entirely understand and ending up with nasty side effects). So, I think that it's just plain bad practice to create such macros. And since those macros won't exist with any other git repo unless you copy them over, getting used to them will just create more problems for anyone using them, because they'll learn the macro rather than the proper command and won't know what to do when they work with other repos.

So, if you want to create macros for yourself, because you want that, then that's up to you. But I'm against putting them in the repo's .gitconfig file, because I think that it's bad pratice and that it encourages bad practices.

- Jonathan M Davis

On Sunday, 6 January 2013 at 22:45:27 UTC, Andrei Alexandrescu wrote:
> I do now after having read about it at the end of several google searches.

I found the git documentation to be an excellent resource once you're familiar with the basic concept of how git works. For that, I found this quite insightful: http://eagain.net/articles/git-for-computer-scientists/

> I agree it is important to be fluent with git. I'm an okay user but currently unfit for an admin. Probably Walter is at a similar stage. I'm doing my best to improve my knowledge in the limited time I have, and part of it is discussing git here. So I hope you'll understand why I find it contradictory to be reprimanded for doing so with the reproach that I should be doing exactly what I'm doing.

Sorry, your post came across to me as you advocating that everyone used a git command without having understood its exact function.

> Yes, that was exactly my intention.

But, why? If both histories contained conflicting changes, Git will throw only some away. In some cases, this might result in a broken codebase (the result might even compile, but git might've thrown away some initialization code (only), and you may not find out until it's too late).

> I get these tidbits of commands - git idioms - all the time, and I forget them because I don't use them frequently. I don't see why it's a crime to want to define some macros for such idioms instead of essentially putting that in a text file that I'd be perusing now and then.

I just don't think it's that simple. The outcome of any command can be affected by a number of factors, such as:
- Do you have any unstaged changes? Do you want to keep or discard them?
- Do you have any staged but uncommitted changes? (ditto)
- Are there any untracked files that may conflict with the operation? (ditto)
- Are there any local-only commits? (ditto)
- Which branch are you on?
- Does the branch track a remote-tracking branch? Is it tracking the intended one?
- How is the behavior of the commands set up in the user's .gitconfig?

So, the question is not as much "How do I get X done?", as it is "How do I get to X from Y?"

For example, the commands I posted will only work correctly if the current checked-out branch (i.e., HEAD) is "master", otherwise it'll move some other branch to match the remote's master. A hard reset throws away local changes - someone running such a macro found in a repository's .gitconfig may result in accidental loss of work.

FWIW, I work with git daily, and haven't found the need to set up macros such as you describe. I use a "gg" alias to launch git gui for interactive staging and reviewing, and the "git pullrequest" script I posted here earlier.

> That's pretty much what I wanted but was unable to find easily by searching the net.

You could always ask on IRC ;)

On 1/6/13 3:41 PM, Jonathan M Davis wrote:
> I'm arguing that people should actually learn how to use git properly rather
> than creating crutches for themselves.

False choice.

> The more you try and do with git
> without learning it properly, the more problems you're likely to run into
> (e.g. by running commands that you don't entirely understand and ending up
> with nasty side effects). So, I think that it's just plain bad practice to
> create such macros.

I know how the shell works quite well but that doesn't stop me from writing scripts and aliases.

This boils down to advocating one needs to type by hand sequences of commands instead of defining higher-level scripts that have a cohesive meaning.


Andrei