How to split a git repository while preserving subdirectories?
What I want is similar to this question. However, I want the directory that is split into a separate repo to remain a subdirectory in that repo:
I have this:
foo/
.git/
bar/
baz/
qux/
And I want to split it into two开发者_如何学编程 completely independent repositories:
foo/
.git/
bar/
baz/
quux/
.git/
qux/ # Note: still a subdirectory
How to do this in git?
I could use the method from this answer if there is some way to move all the new repo's contents into a subdirectory, throughout history.
You could indeed use the subdirectory filter followed by an index filter to put the contents back into a subdirectory, but why bother, when you could just use the index filter by itself?
Here's an example from the man page:
git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD
This just removes one filename; what you want to do is remove everything but a given subdirectory. If you want to be cautious, you could explicitly list each path to remove, but if you want to just go all-in, you can just do something like this:
git filter-branch --index-filter 'git ls-tree -z --name-only --full-tree $GIT_COMMIT | grep -zv "^directory-to-keep$" | xargs -0 git rm --cached -r' -- --all
I expect there's probably a more elegant way; if anyone has something please suggest it!
A few notes on that command:
- filter-branch internally sets GIT_COMMIT to the current commit SHA1
- I wouldn't have expected
--full-tree
to be necessary, but apparently filter-branch runs the index-filter from the.git-rewrite/t
directory instead of the top level of the repo. - grep is probably overkill, but I don't think it's a speed issue.
--all
applies this to all refs; I figure you really do want that. (the--
separates it from the filter-branch options)-z
and-0
tell ls-tree, grep, and xargs to use NUL termination to handle spaces in filenames.
Edit, much later: Thomas helpfully suggested a way to remove the now-empty commits, but it's now out of date. Look at the edit history if you've got an old version of git, but with modern git, all you need to do is tack on this option:
--prune-empty
That'll remove all commits which are empty after the application of the index filter.
I wanted to do a similar thing, but since the list of files that i wanted to keep was pretty long, it didn't make sense to do this using countless greps. I wrote a script that reads the list of files from a file:
#!/bin/bash
# usage:
# git filter-branch --prune-empty --index-filter \
# 'this-script file-with-list-of-files-to-be-kept' -- --all
if [ -z $1 ]; then
echo "Too few arguments."
echo "Please specify an absolute path to the file"
echo "which contains the list of files that should"
echo "remain in the repository after filtering."
exit 1
fi
# save a list of files present in the commit
# which is currently being modified.
git ls-tree -r --name-only --full-tree $GIT_COMMIT > files.txt
# delete all files that shouldn't be removed
while read string; do
grep -v "$string" files.txt > files.txt.temp
mv -f files.txt.temp files.txt
done < $1
# remove unwanted files (i.e. everything that remained in the list).
# warning: 'git rm' will exit with non-zero status if it gets
# an invalid (non-existent) filename OR if it gets no arguments.
# If something exits with non-zero status, filter-branch will abort.
# That's why we have to check carefully what is passed to git rm.
if [ "$(cat files.txt)" != "" ]; then
cat files.txt | \
# enclose filenames in "" in case they contain spaces
sed -e 's/^/"/g' -e 's/$/"/g' | \
xargs git rm --cached --quiet
fi
Quite suprisingly, this turned out to be much more work than i initially expected, so i decided to post it here.
This is what I ended up doing to solve this issue when I had it myself:
git filter-branch --index-filter \
'git ls-tree --name-only --full-tree $GIT_COMMIT | \
grep -v "^directory-to-keep$" | \
sed -e "s/^/\"/g" -e "s/$/\"/g" | \
xargs git rm --cached -r -f --ignore-unmatch \
' \
--prune-empty -- --all
The solution is based on Jefromi’s answer and on Detach (move) subdirectory into separate Git repository plus many comments here on SO.
The reason why Jefromi’s solution did not work for me was, that I had files and folders in my repo whose names contained special characters (mostly spaces). Additionally git rm
complained about unmatched files (resolved with --ignore-unmatch
).
You can keep the filtering agnostic to the directory not being in the repo’s root or being moved around:
grep --invert-match "^.*directory-to-keep$"
And finally, you can use this to filter out a fixed subset of files or directories:
egrep --invert-match "^(.*file-or-directory-to-keep-1$|.*file-or-directory-to-keep-2$|…)"
To clean up afterwards you can use these commands:
$ git reset --hard
$ git show-ref refs/original/* --hash | xargs -n 1 git update-ref -d
$ git reflog expire --expire=now --all
$ git gc --aggressive --prune=now
Use git-filter-repo This is not part of git as of version 2.25. This requires Python3 (>=3.5) and git 2.22.0
mkdir new_repoA
mkdir new_repoB
git clone originalRepo newRepoA
git clone originalRepo newRepoB
pushd
cd new_repoA
git filter-repo --path foo/bar --path foo/baz
popd
cd new_repoB
git filter-repo --path foo/qux
For my repo that contained ~12000 commits git-filter-branch took more than 24 hours and git-filter-repo took less than a minute.
A cleaner method:
git filter-branch --index-filter '
git read-tree --empty
git reset $GIT_COMMIT path/to/dir
' \
-- --all -- path/to/dir
or to stick with just core commands, sub in git read-tree --prefix=path/to/dir/ $GIT_COMMIT:path/to/dir
for the reset.
Specifying path/to/dir
on the rev-list args does the pruning early, with a filter this cheap it doesn't matter much but it's good to avoid the wasted effort anyway.
If you wish split out just single directory as separate git repository
git-filter-branch has --subdirectory-filter
option and it is much simpler then previous mentioned solutions, just:
git filter-branch --subdirectory-filter foodir -- --all
Additionally it change path and place content of directory on top of new repo, not just filter and remove other content.
I used git-filter-repo
with filename-callback
.
stephen@B450-AORUS-M:~/source/linux$ git filter-repo --force --filename-callback '
if b"it87.c" in filename:
return filename
else:
# Keep the filename and do not rename it
return None
'
warning: Tag points to object of unexpected type tree, skipping.
warning: Tag points to object of unexpected type tree, skipping.
Parsed 935794 commitswarning: Omitting tag 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c,
since tags of trees (or tags of tags of trees, etc.) are not supported.
warning: Omitting tag 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c,
since tags of trees (or tags of tags of trees, etc.) are not supported.
Parsed 937142 commits
New history written in 177.03 seconds; now repacking/cleaning...
Repacking your repo and cleaning out old unneeded objects
HEAD is now at a57e6edb85a3 treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157
Enumerating objects: 20210, done.
Counting objects: 100% (20210/20210), done.
Delta compression using up to 12 threads
Compressing objects: 100% (17718/17718), done.
Writing objects: 100% (20210/20210), done.
Total 20210 (delta 1841), reused 20038 (delta 1669), pack-reused 0
Completely finished after 179.76 seconds.
It didn't remove empty merge commits, probably due to a bunch of tags that were associated with one side of the tree.
I tried using the top most voted answer, and it didn't seem to remove anything, and took a long time.
Rewrite 3e80e1395bd4f410b79dc0f17113f5b6b409c7d8 (329/937142) (8 seconds passed, remaining 22779 predicted)
22779 seconds = 6.3275 hours
精彩评论