Submodules and 'git pull --rebase'

We recently switched to git and are trying to use Submodules to include our Common libraries.

No matter what we do we can not get 'git pull -开发者_开发问答-rebase' to work in the Super or Submodule.

We get:

james:libraries james$ git pull --rebase
Cannot pull with rebase: You have unstaged changes.
Please commit or stash them.

This happens even if we do not have any local changes at all and have a clean directory. Any thoughts on what we might be doing wrong?

Thanks! James

We can not get 'git pull --rebase' to work in the Super or Submodule

To make sure to involve submodule, use Git 2.14 (Q3 2017, 6 years after the OP's question) or more with "git pull --rebase --recurse-submodules", which learned to rebase the branch in the submodules to an updated base.

See commit e8906a9 (27 Jun 2017), and commit a6d7eb2, commit 8c69832, commit 886dc15 (23 Jun 2017) by Stefan Beller (stefanbeller).
^{(Merged by Junio C Hamano -- gitster -- in commit c9c63ee, 13 Jul 2017)}

pull: optionally rebase submodules (remote submodule changes only)

^{Signed-off-by: Brandon Williams
Signed-off-by: Stefan Beller}

Teach pull to optionally update submodules when '--recurse-submodules' is provided.

This will teach pull to run 'submodule update --rebase' when the '--recurse-submodules' and '--rebase' flags are given under specific circumstances.

On a rebase workflow:

1. Both sides change the submodule

Let's assume the following history in a submodule:

H---I---J---K---L local branch
     \
      M---N---O---P remote branch

and the following in the superproject (recorded submodule in parens):

A(H)---B(I)---F(K)---G(L)  local branch
        \
         C(N)---D(N)---E(P) remote branch

In an ideal world this would rebase the submodule and rewrite the submodule pointers that the superproject points at such that the superproject looks like

A(H)---B(I)              F(K')---G(L')  rebased branch
        \               /
         C(N)---D(N)---E(P) remote branch

and the submodule as:

      J---K---L (old dangeling tip)
     /
H---I               J'---K'---L' rebased branch
     \             /
      M---N---O---P remote branch

And if a conflict arises in the submodule the superproject rebase would stop at that commit at which the submodule conflict occurs.

Currently a "pull --rebase" in the superproject produces a merge conflict as the submodule pointer changes are conflicting and cannot be resolved.

2. Local submodule changes only

Assuming histories as above, except that the remote branch would not contain submodule changes, then a result as

A(H)---B(I)              F(K)---G(L)  rebased branch
        \               /
         C(I)---D(I)---E(I) remote branch

is desire-able. This is what currently happens in rebase.

If the recursive flag is given, the ideal git would produce a superproject as:

A(H)---B(I)               F(K')---G(L')  rebased branch (incl. sub rebase!)
         \               /
          C(I)---D(I)---E(I) remote branch

and the submodule as:

      J---K---L (old dangeling tip)
     /
H---I               J'---K'---L' Locally rebased branch
     \             /
      M---N---O---P advaced branch

This patch doesn't address this issue, however a test is added that this fails up front.

3. Remote submodule changes only

Assuming histories as in (1) except that the local superproject branch would not have touched the submodule the rebase already works out in the superproject with no conflicts:

A(H)---B(I)              F(P)---G(P)  rebased branch (no sub changes)
        \               /
         C(N)---D(N)---E(P) remote branch

The recurse flag as presented in this patch would additionally update the submodule as:

H---I               J'---K'---L' rebased branch
     \             /
      M---N---O---P remote branch

As neither J, K, L nor J', K', L' are referred to from the superproject, no rewriting of the superproject commits is required.

Conclusion for 'pull --rebase --recursive'

If there are no local superproject changes it is sufficient to call "submodule update --rebase" as this produces the desired results.
In case of conflicts, the behavior is the same as in 'submodule update --recursive' which is assumed to be sane.

This patch implements (3) only.

On a merge workflow:

We'll start off with the same underlying DAG as in (1) in the rebase workflow.
So in an ideal world a 'pull --merge --recursive' would produce this:

H---I---J---K---L---X
     \             /
      M---N---O---P

with X as the new merge-commit in the submodule and the superproject as:

A(H)---B(I)---F(K)---G(L)---Y(X)
        \                  /
         C(N)---D(N)---E(P)

However modifying the submodules on the fly is not supported in git merge such that Y(X) is not easy to produce in a single patch. In fact git merge doesn't know about submodules at all.

However when at least one side does not contain commits touching the submodule at all, then we do not need to perform the merge for the submodule but a fast-forward can be done via checking out either L or P in the submodule.

This strategy is implemented in 68d03e4a6e ("Implement automatic fast-forward merge for submodules", 2010-07-07, Git v1.7.3-rc0 -- merge) already, so to align with the rebase behavior we need to also update the worktree of the submodule.

Before Git 2.27 (Q2 2020), "git pull --rebase" tried to run a rebase even after noticing that the pull results in a fast-forward and no rebase is needed nor sensible, for the past few years due to a mistake nobody noticed.

See commit fbae70d (27 Mar 2020) by Elijah Newren (newren).
^{(Merged by Junio C Hamano -- gitster -- in commit dfdce31, 22 Apr 2020)}

pull: avoid running both merge and rebase

^{Signed-off-by: Elijah Newren}

When opt_rebase is true, we still first check if we can fast-forward.
If the branch is fast-forwardable, then we can avoid the rebase and just use merge to do the fast-forward logic.

However, when commit a6d7eb2c7a ("pull: optionally rebase submodules (remote submodule changes only)", 2017-06-23, Git v2.14.0-rc0 -- merge) added the ability to rebase submodules, it accidentally caused us to run BOTH a merge and a rebase.

Add a flag to avoid doing both.

This was found when a user had both pull.rebase and rebase.autosquash set to true.

In such a case, the running of both merge and rebase would cause ORIG_HEAD to be updated twice (and match HEAD at the end instead of the commit before the rebase started), against expectation.

"git pull --rebase --recurse-submodules" checked for local changes in a wrong range and failed to run correctly when it should, which has been fixed with Git 2.30 (Q1 2020)

pull: check for local submodule modifications with the right range

Ever since 'git pull' learned '--recurse-submodules' in a6d7eb2 (pull: optionally rebase submodules (remote submodule changes only), 2017-06-23, Git v2.14.0-rc0), we check if there are local submodule modifications by checking the revision range 'curr_head --not rebase_fork_point'.

The goal of this check is to abort the pull if there are submodule modifications in the local commits being rebased, since this scenario is not supported.

However, the actual range of commits being rebased is not 'rebase_fork_point..curr_head', as the logic in 'get_rebase_newbase_and_upstream' reveals, it is 'upstream..curr_head'.

If the 'git merge-base --fork-point' invocation in 'get_rebase_fork_point' fails to find a fork point between the current branch and the remote-tracking branch we are pulling from, 'rebase_fork_point' is null and since 4d36f88 (submodule: do not pass null OID to setup_revisions, 2018-05-24, Git v2.18.0-rc1), 'submodule_touches_in_range' checks 'curr_head' and all its ancestors for submodule modifications.

Since it is highly likely that there are submodule modifications in this range (which is in effect the whole history of the current branch), this prevents 'git pull --rebase --recurse-submodules' from succeeding if no fork point exists between the current branch and the remote-tracking branch being pulled.
This can happen, for example, when the current branch was forked from a commit which was never recorded in the reflog of the remote-tracking branch we are pulling, as the last two paragraphs of the "Discussion on fork-point mode" section in git-merge-base explain.

Fix this bug by passing 'upstream' instead of 'rebase_fork_point' as the 'excl_oid' argument to 'submodule_touches_in_range'.

With Git 2.32 (Q2 2021), the sHA-256 transition has an influence on submodule, when using null oid:

See commit 3dd7146, commit b8505ec, commit 71b7672, commit 72871b1, commit dd15f4f, commit 1422844, commit 5a6dce7, commit 0e5e228, commit 5951bf4, commit ab795f0, commit c3b4e4e, commit 92e2cab, commit cf09832 (26 Apr 2021) by brian m. carlson (bk2204).
^{(Merged by Junio C Hamano -- gitster -- in commit aaa3c80, 10 May 2021)}

hash: provide per-algorithm null OIDs

^{Signed-off-by: brian m. carlson}

Up until recently, object IDs did not have an algorithm member, only a hash.
Consequently, it was possible to share one null (all-zeros) object ID among all hash algorithms.
Now that we're going to be handling objects from multiple hash algorithms, it's important to make sure that all object IDs have a correct algorithm field.

Introduce a per-algorithm null OID, and add it to struct hash_algo.
Introduce a wrapper function as well, and use it everywhere we used to use the null_oid constant.

After testing here, having changes in a submodule does not stop you from doing pull --rebase on the parent. So either you are doing pull --rebase inside the submodule and you have non stashed changes there, or perhaps you added the submodule with git submodule add url but you did not then commit the submodule addition, which would cause the parent from refusing to pull. Can you post the output of git status ?