Extracting git history for part of a git repo to create a git submodule -- cherry-pick?

A similar question, How to cherry-pick multiple commits, assumes that the commits are consecutive.

I have a module that I downloaded as a tarball. I've included it in my main project's git repo and made changes to it. These changes are interspersed with other commits for the main project.

I've realized the error of my ways and want to replace the module with a git submodule. I could just add the current state, but I want to keep my git history.

How can I move each individual commit into my new repo?

I think I can do this with cherry-pick, but it's time-consuming. Here's what I have:

Setup my new repo with the main project as a remote (so I can cherry pick):

cd ~/snippets
git remote add main ~/.vim/
git fetch main

I can see the relevant commits in the main project with git log ~/.vim/snippets

I can make a script to cherry pick with

cd ~/.vim/snippets
git log --oneline --reverse --format="format:git cherry-pick %h #%s" .

But if I run the script, there are merge conflicts. After I resolve a merge and commit, I need to remove the successful bits from my cherry pick script and run it again. I'd like to do this automatically, like git rebase.

You may find git filter-branch to be helpful -- if you use --tree-filter to move the files around and --commit-filter to remove commits that have no effect on your branch, then you should be able to end up with a branch containing just the changes from your sub-project.

If you don't have time for that now, you could use my other answer as an interim measure :).

If you just want the history, but don't care so much if there's extra data there, you could just create a couple of branches, one for your main code and one for the snippets. Move the snippets up to the top level in their branch, removing the rest of your files. You should find that rename-tracking allows you to view the history of the files, although you'll obviously have extra files and commits in your history. You can then use this branch (possibly cloned to a separate repository) as your sub-module.

Then, in the future, when you have more time, you can use my other answer to clean up your history :).

It sounds like git-subtree might do that. Here's a relevant blog post from the developer.

This question shows a much easier way:

git filter-branch --subdirectory-filter repo/subdirectory -- --all

From git help filter-branch:

--subdirectory-filter <directory>

Only look at the history which touches the given subdirectory. The result will contain that directory (and only that) as its project root. Implies the section called “Remap to ancestor”.