Back Up for a Moment

James‘s team has a pretty complicated deployment process implemented as a series of bash scripts. The deployment is complicated, the scripts doing the deployment are complicated, and failures mid-deployment are common. That means they need to gracefully roll back, and they way they do that is by making backup copies of the modified files.

This is how they do that.

DATE=`date '+%Y%m%d'`
BACKUPDIR=`dirname ${DESTINATION}`/backup
if [ ! -d $BACKUPDIR ]
then
        echo "Creating backup directory ..."
        mkdir -p $BACKUPDIR
fi
FILENAME=`basename ${DESTINATION}`
BACKUPFILETYPE=${BACKUPDIR}/${FILENAME}.${DATE}
BACKUPFILE=${BACKUPFILETYPE}-1
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-2 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-3 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-4 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-5 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-6 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-7 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-8 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then BACKUPFILE=${BACKUPFILETYPE}-9 ; fi
if [ -f ${BACKUPFILE} ] || [ -f ${BACKUPFILE}.gz ] ; then
cat <<EOF
You have already had 9 rates releases in one day.

${BACKUPFILE} already exists, do it manually !!!
EOF
exit 2
fi

Look, I know that loops in bash can be annoying, but they’re not that annoying.

This code creates a backup directory (if it doesn’t already exist), and then creates a file name for the file we’re about to backup, in the form OriginalName.Ymd-n.gz. It tests to see if this file exists, and if it does, it increments n by one. It does this until either it finds a file name that doesn’t exist, or it hits 9, at which point it gives you a delightfully passive aggressive message:

You have already had 9 rates releases in one day.
${BACKUPFILE} already exists, do it manually !!!

Yeah, do it manually. Now, admittedly, I don’t think a lot of folks want to do more than 9 releases in a given day, but there’s no reason why they couldn’t just keep trying until they find a good filename. Or even better, require each release to have an identifier (like the commit or build number or whatever) and then use that for the filenames.

Of course, just fixing this copy doesn’t address the real WTF, because we laid out the real WTF in the first paragraph: deployment is a series of complicated bash scripts doing complicated steps that can fail all the time. I’ve worked in places like that, and it’s always a nightmare. There are better tools! Our very own Alex has his product, of course, but there are a million ways to get your builds repeatable and reliable that don’t involve BuildMaster but also don’t involve fragile scripts. Please, please use one of those.