Pruning old backups in several steps

I am looking for a way开发者_StackOverflow to thin out old backups. The backups are run on a daily basis, and I want to increase the interval as the backups become older.

After a couple of days I'd like to remove the daily backups, leaving only the "Sunday" backup. After a couple of weeks, only the first backup of a month that is available should be removed.

Since I am dealing with historic backups, I cannot just change the naming scheme.

I tried to use 'find' for it, but couldn't find the right options.

Anyone got anything that might help?

I know it is historical data, but you might prefer coming up with a naming scheme to assist this problem. It might be far easier to tackle this problem in two passes: first, renaming the directories based on the date, then selecting the directories to keep in the future.

You could make a quick approximation, if all the directory dates in ls -l output look good enough:

ls -l | awk '{print "mv " $8 " "  $6;}' > /tmp/runme

Look at /tmp/runme, and if it looks good, you can run it with sh /tmp/runme. You might wish to prune the entries or something like that, up to you.

If all the backups are stored in directories named, e.g:

2011-01-01/
2011-01-02/
2011-01-03/
...
2011-02-01/
2011-02-02/
...
2011-03-07/

then your problem would be reduced to computing the names to keep and delete. This problem is much easier to solve than searching through all your files and trying to select which ones to keep and delete based on when they were made. (See date "+%Y-%m-%d" output for a quick way to generate this sort of name.)

Once they are named conveniently, you can keep the first backup of every month with a script like this:

for y in `seq 2008 2010`
    do for m in `seq -w 1 12`
            do for d in `seq -w 2 31`
                    do echo "rm $y-$m-$d"
            done
    done
done

Save its output, inspect it :) and then run the output, similar to the rename script.

Once you've got the past backups under control, then you can generate the 2010 from date --date="Last Year" "+%Y", and other improvements so it handles "one a week" for the current month and maintains itself forever going forward.

I've developed a solution for my similar needs on top of @ajreal's starting point. My backups are named like "backup-2015-06-01T01:00:01" (using date "+%Y-%m-%dT%H:%M:%S").

Two simple steps: touch the files to keep using a shell glob pattern for first-of-each-month, and use find and xargs to delete anything more than 30 days old.

cd $BACKUPS_DIR

# touch backups from the first of each month
touch *-01T*

# delete backups more than 30 days old
echo "Deleting these backups:"
find -maxdepth 1 -mtime +30
find -maxdepth 1 -mtime +30 -print0 | xargs -0 rm -r

yup, for example

find -type f -mtime 30

details - http://www.gnu.org/software/findutils/manual/html_mono/find.html#Age-Ranges