Alternative cron job backups in Dreamhost

Dock Street Media

I wrote about Dreamhost Cron Job backups using shell in a previous post.  If you ran into an error because of the size of the site GZIP archives, then the alternative system below may be better for you.  It may also work better for you if you do not want to send your site files in an email attachment, which is less secure than secure FTP.

In a couple of the sites that I backed up, I received the error message:

postdrop: warning: uid=123456: File too large
sendmail: fatal: username(123456): message file too big
Error sending message, child exited 75 (Deferred.).
Could not send the message.

This is because the SENDMAIL application cannot send attachments larger than 1MB.  This is not a problem for smaller sites that do not change very often, but becomes a problem for large sites that contain a lot of content in the database and accrue a lot of uploaded files.

As an alternative, you can use the script below to automatically backup (1) all of the directories on your account except those you designate to skip, and (2) all of the databases on a single hostname that use the same database username and password.

The downside of this system is that you must then manually log in to your FTP account and transfer the backups to your computer to satisfy the "two locations" backup rule.

Step 1: Designate the directories to skip

Navigate to your root directory using FileZilla or your favorite FTP client.  You should see a directory of your sites including some system directories, such as Maildir, backups, logs, sent, svn, etc.

root
-- Maildir
-- backups
-- examplesite.com
-- logs
-- secondexamplesite.com
-- sent
-- svn
-- thirdexamplesite.com

In each of the system directories (e.g. Maildir, backups, logs, sent, svn), create an empty file called DONT_BACKUP.  Do not add an extension to it. Do not add this file to the site directories you want to backup (e.g. examplesite.com, secondexamplesite.com, thirdexamplesite.com), but you can add it to a site directory that you do not want to backup.

Step 2: Create the automatic directory backup script

Special thanks for this automatic code backup script goes to Edward Webb at http://edwardawebb.com/linux/backup-subdirectories-bash-array-loop. I simply modified it to my own tastes.

Open Notepad++ or your favorite text editor, create a file called auto_directory_backup.sh and insert the following code:

#!/bin/bash
# config variables
HOME="/home/YOUR_USERNAME"
NOWDATE=$(date +"%y%m%d")
NOWDAY=$(date +"%d")
BACKUPDIR="backups"

# change directory to home (root)
cd $HOME

# backup directories that contain alpha-numeric characters, a space, a ".", or a "-"
for DIR in $(ls | grep ^[a-zA-Z0-9s.-]*$)
do
# skip directories containing the file "DONT_BACKUP"
if [ -f $DIR/DONT_BACKUP ]
then
printf "tSKIPPING $DIR as it contains ignore filen" | tee -a $LOGFILE
else
# check to see if target path exists
# if so, delete the old one and create a new one
# otherwise just create it
TARGETPATH=$HOME/$BACKUPDIR/$DIR/$NOWDAY
if [ -d $TARGETPATH ]
then
rm -r $TARGETPATH
mkdir -p $TARGETPATH
else
mkdir -p $TARGETPATH
fi

# save a GZIP of the directory
tar -zcf ${TARGETPATH}/${DIR}_$NOWDATE.tar.gz ./$DIR
printf "t$DIR backed up to $TARGETPATHn" | tee -a $LOGFILE
fi
done

Change YOUR_USERNAME to your username that matches your home directory in Dreamhost. This may be different on different web hosts. You must match the home directory to whatever the home directory is on your host provider.

Step 3: Create the MySQL database backup script

Using your text editor, create a new file called auto_database_backup.sh and add the following code:

#!/bin/bash

# specific config variables (EDIT THESE)
HOME="/home/YOUR_USERNAME"
DBHOST="YOUR_DB_HOST"
DBUSER="YOUR_DB_USERNAME"
DBPASS="YOUR_DB_PASSWORD"

# other config variables(DO NOT EDIT THESE)
NOWDATE=$(date +"%y%m%d")
NOWDAY=$(date +"%d")
BACKUPDIR="backups/mysql"
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"

# check to see if target path exists
# if so, delete the old one and create a new one
# otherwise just create it
TARGETPATH=$HOME/$BACKUPDIR/$NOWDAY
if [ -d $TARGETPATH ]
then
rm -r $TARGETPATH
mkdir -p $TARGETPATH
else
mkdir -p $TARGETPATH
fi

# automatically backup each database using the hostname
DBS="$($MYSQL -u $DBUSER -h $DBHOST -p$DBPASS -Bse 'show databases')"
for DBNAME in $DBS
do
# dump the data into a SQL file inside the target path
$MYSQLDUMP -u $DBUSER -h $DBHOST -p$DBPASS $DBNAME | gzip > $TARGETPATH/${DBNAME}_$NOWDATE.sql.gz

printf "t$DBNAME backed up to $TARGETPATHn" | tee -a $LOGFILE
done

Upload, set permissions, clean with dos2unix, and set up the Cron job Follow Steps 3-5 in Automatic site backups using cron jobs in Dreamhost to complete the automatic backup system.