How to make daily backups with rsync and cronjobs

It’s always a good idea to make backups of important data (documents, images, music, etc.), e.g. the worst case for students is to lose data of written reports or theses.
I wanted to make backups of my documents and source codes every day. The backup should be stored on a usb flash drive. But I also wanted to keep deleted files for a couple of days, in case of accidentally deleted files. There are many ways to complete this task, but I decided to use rsync and cronjobs for this purpose. Both tools make the backup process very flexible. It’s very easy to manage the time of the backup process with cronjobs and rsync makes it easy to change the destination (e.g. another folder, device or remote host).
My idea was to add a cronjob to start a bash script which uses rsync to copy the directories and files to the flash drive. But the device names of flash drives changes, if you plug them in a different order. To avoid this problem, I added a udev rule to create a symbolic link every time the flash drive is plugged.

Add a udev rule to create a symlink

The first thing was to figure out the device name of the flash drive. So I plugged it and took a look at the partitions:

$ cat /proc/partitions

These two lines were the most interesting of the output:

major minor  #blocks  name

   8        0    1007616 sda
   8        1    1007584 sda1
The name of my 1 GB usb thumb drive is /dev/sda1. But I needed more information (vendor id, product id and serial) of the flash drive to add the udev rule. I used
$ lsusb -v

to get detailed information of all usb devices which are plugged. Here is an excerpt of the output with all the information I needed:

Bus 001 Device 005: ID 03ce:2d4f Kingston Technology Company Inc.
Device Descriptor:
  idVendor           0x03ce Kingston Technology Company Inc.
  idProduct          0x2d4f DataTraveler
  iSerial                 3 917C140607D8

The udev rules are located in /etc/udev/rules.d and have a "special" file name. The prefix is a number followed by a name and the suffix is .rules.
All udev rules are processed in lexical order, i.e. you can set the order of the udev rules with the prefix of the rule. I created a file named 80-backup.rules in /etc/udev/rules.d.
The content of udev rule files are key, value pairs and for this purpose the most important keys are KERNEL and ATTR.
The KERNEL value is matched against the kernel device name and ATTR values are matched against the attributes of the device (I don't want to go into detail). I used a wild card "?" to avoid the changing device names of the flash drive. This is the content of my created udev rule:

KERNEL=="sd?1", ATTRS{idVendor}=="03ce", ATTRS{idProduct}=="2d4f", \
ATTRS{serial}=="917C140607D8", SYMLINK+="usbbackup"

The SYMLINK value usbbackup is the name of the symbolic link, which is created in /dev every time the device is plugged, i.e. the symbolic link points automatically to the correct device. You have to reload the udev rules or restart the udev daemon to "activate" the udev rule. In my case (Slackware Linux) I used:

$ /etc/rc.d/rc.udev restart

Now the udev rule is activated and will create the symbolic link every time the device is plugged.


On Arch Linux you don't have to reload the udev daemon, it automatically detects changes. Just plug in the usb thumb drive again and the symlink should be available.

The backup script

The following script mirrors the given directories in BACKUP_FOLDERS to the flash drive, i.e. if you delete a file in the source directory, it'll also be deleted on the flash drive and it would not be possible to recover these files.
But rsync has an option --backup-dir to move all deleted files within the source directory into another directory. This is a nice option, because the mirrored folder on the flash drive will be up to date and all deleted files will be stored in another directory.
I used the variable DELFILES to define the folder of all deleted files.
I also wanted to know whether the backup process was successful or an error occurred. The script appends each output to the LOGFILE which is surrounded by a start and stop backup message.


DEST_DRIVE="/dev/usbbackup" # Backup will be stored on this device 
DEST_MOUNT="/mnt/flashdrive"    # Mount point of flash drive
BACKUP_FOLDERS=("/path/to/dir1" "/path/to/dir2")    # Backup folders
LOGFILE="/var/log/mybackup.log" # Logfile
DELFILES="$DEST_MOUNT/delted_files" # Path to folder for deleted files
THRESHOLD=10    # keep deleted files for 10 days

# write first parameter $1 to logfile
function write_to_log 
  # get current date and time
  bkp_date=$(date +%Y-%m-%d@%H:%M:%S)
  echo -e "$bkp_date : $1" >> $LOGFILE

# Append log entry
write_to_log "====== Starting Backup ======"

# test whether symbolic link to flash drive exists
if [ ! -h $DEST_DRIVE ]; then
  write_to_log "[X] Error: Backup drive not available"
  write_to_log "====== Backup finished ======\n\n"
  exit 1

# check whether mount point is in use
if [ ! -z "$(mount -l | grep ${DEST_MOUNT})" ]; then 
  write_to_log "[X] Error: Mount point already in use"
  write_to_log "====== Backup finished ======\n\n"
  exit 1

# Mount flash drive
if [ $? -ne 0 ]; then
  write_to_log "[X] Error: Could not mount flash drive"
  write_to_log "====== Backup finished ======\n\n"
  exit 1

# delete all files from $DELFILES older than $THRESHOLD days
if [ -d "${DELFILES}" ]; then
  find $DELFILES -type f -mtime $THRESHOLD -delete >> $LOGFILE

# copy files (create backup)
for f in ${BACKUP_FOLDERS[@]}; do
  # check if given file is directory and readable
  if [ -d $f ] && [ -r $d ]; then
    rsync -avbuz --delete --backup-dir=$DELFILES $f $DEST_MOUNT >> $LOGFILE

# flush file system buffer

# unmount flash drive
umount $DEST_MOUNT
if [ $? -ne 0 ]; then
  write_to_log "[X] Error: Could not unmount flash drive"

write_to_log "====== Backup finished ======\n\n"

I used rsync in archive mode (-a) which is equal to the options -rlptgoD. This is an excerpt of the man page of all used options:

Option Explanation
-a archive mode, equal to -rlptgoD
-r recursive
-l copy symlinks as symlinks
-p preserve permissions
-t preserve modification times
-g preserve group
-o preserve owner (super-user only)
-D preserve device files (super-user only) and special files
-v increase verbosity
-b make backups
-u skip files that are newer on the receiver
-z compress file data during the transfer
--backup-dir=DIR make backups into hierarchy based in DIR
--delete delete extraneous files from dest dirs

The find command searches for files (-type f) within the directory $DELFILES which are older than $THRESHOLD (-mtime $THRESHOLD) and deletes them (-delete)

Setup cron job

The last step is to add a cronjob to run the script at a given time. The cron daemon provides four directories cron.hourly, cron.daily,cron.weekly and cron.monthly to run scripts/programs. I copied the script to the directory cron.daily to run the script daily.
Make sure that the script is executable ("chmod u+x") and the flash device is plugged. You can look up the execution time of daily cronjobs at the crontab:

$ crontab -l

Here is an excerpt of my output:

# Run hourly cron jobs at 47 minutes after the hour:
47 * * * * /usr/bin/run-parts /etc/cron.hourly 1> /dev/null
# Run daily cron jobs at 4:40 every day:
40 4 * * * /usr/bin/run-parts /etc/cron.daily 1> /dev/null
# Run weekly cron jobs at 4:30 on the first day of the week:
30 4 * * 0 /usr/bin/run-parts /etc/cron.weekly 1> /dev/null
# Run monthly cron jobs at 4:20 on the first day of the month:
20 4 1 * * /usr/bin/run-parts /etc/cron.monthly 1> /dev/null

In my case the backup script will be executed at 04:40 AM. If you want to change the time, you can use:

# crontab -e

That's it.

Last update: 14.06.2015

Posted on May 28, 2013, in Bash-Scripts, Command-Line, Security and tagged , , , , , , , , , . Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: