One of the most basic uses of Ansible is the creation of configuration files, as we will see in this article, these are based on Python’s jinja2 templates , which will allow us great flexibility to adapt our files to our needs.
In my case, I am going to use Ansible to configure the backups of a physical server and the jails contained in it. As I don’t have an internal DNS server, the first thing will be to modify my hosts file to access them by name.
192.168.69.6 drwho
192.168.69.14 gdrive
192.168.69.12 infinity
192.168.69.5 potras
192.168.69.13 prometheus
192.168.69.15 synapse
192.168.69.10 rxwod
192.168.69.11 haproxy
192.168.69.2 mighty
Now we install Ansible:
We create the directory where we will have the global Ansible configuration:
chown -R root:kr0m /usr/local/etc/ansible/
chmod 775 /usr/local/etc/ansible/
Now we can perform the rest of the steps with our regular user.
We define the list of hosts indicating the minimum size of the backups on each of the servers and some variables of the tobackup_servers group where we define the backup retention policies:
[tobackup_servers]
mighty min_backup_size=21
drwho min_backup_size=249
potras min_backup_size=17873
infinity min_backup_size=2749625
haproxy min_backup_size=9
prometheus min_backup_size=17
rxwod min_backup_size=153657
[tobackup_servers:vars]
local_retention=30
digi_retention=10
gdrive_retention=30
We configure some parameters so that it does not complain about the
Python path
or the SSH key change.
[defaults]
interpreter_python=auto_silent
host_key_checking = False
Some useful Ansible commands are:
ansible all -m ping -> Ping a los hosts
ansible all -a "uptime" -> Comando en los hosts
ansible all -m setup -> Variables de cada host
All servers will start from a base script, where a specific part will be added according to their needs, for example, if it is a postgresql server, the database will be dumped, if it is a web server, the files from the documentroot will be copied, etc, etc. In addition, at the end of the script, an additional part will be added if it is the physical equipment since it has a special requirement.
#! /usr/local/bin/bash
# Script managed by Ansible
# pkg install ccrypt
function sendTelegramMessage {
message=${@:1}
curl -s -X POST https://api.telegram.org/botYYYYYYYYYYYY/sendMessage -d chat_id=XXXXXXXXXX -d text="$message"
}
# Max: 50MB
function sendTelegramFile {
path=${@:1}
curl -s -F "chat_id=XXXXXXXXXX" -F document=@"$path" https://api.telegram.org/botYYYYYYYYYYYY/sendDocument
}
DATE=$(date +%d_%m_%Y)
SHORT_HOSTNAME=$(hostname -s)
BACKUPSDIR="/storage/backups/$SHORT_HOSTNAME/BACKUP"
rm -rf $BACKUPSDIR 2>/dev/null
mkdir -p $BACKUPSDIR
echo "-- Copying backup files"
# -- Common data --
cp -rp /root/.scripts/ $BACKUPSDIR/scripts
crontab -l > $BACKUPSDIR/crontab
cp /etc/hosts $BACKUPSDIR/hosts
cp /etc/rc.conf $BACKUPSDIR/rc.conf
# -----------------
# -- Custom data --
{% include inventory_hostname + '_custom_data.j2' %}
# ------------------
# -- Kr0m data --
crontab -u kr0m -l > $BACKUPSDIR/kr0m_crontab
cp -r /home/kr0m/.scripts $BACKUPSDIR/kr0m_scripts
# -----------------
# -- Packaging backup --
DIRNAME=$(dirname $BACKUPSDIR)
cd $DIRNAME
FILENAME="$SHORT_HOSTNAME"_backup_"$DATE".tar.gz
echo "-- Compressing backup: $FILENAME"
tar czf $FILENAME BACKUP
chmod 600 $FILENAME
echo "-- Done"
# ------------------
# -- Encrypting backup --
echo "-- Encrypting backup"
ccrypt -f -e -k ~/.scripts/backup_passkey $FILENAME
CRYPTED_FILENAME=$FILENAME.cpt
# To decrypt: ccrypt -d FILENAME
echo "-- Done"
# ------------------
# -- Telegram Upload --
echo "-- Uploading backup to Telegram"
sendTelegramFile $CRYPTED_FILENAME
echo ""
echo "-- Done"
# ------------------
# -- DIGI Upload --
echo "-- Uploading backup to DIGI"
# DONT CREATE DIRECTORY BECAUSE ITS PARENT FSTAB ENTRY AND SHOULD EXIST
# IF DOESNT AND YOU CREATE IT, BACKUP DOESNT FAIL AND YOU GET A LOCAL COPY ONLY
#if [ ! -d /mnt/DIGI/Backups/$SHORT_HOSTNAME ]; then
# mkdir -p /mnt/DIGI/Backups/$SHORT_HOSTNAME
#fi
cp $CRYPTED_FILENAME /mnt/DIGI/Backups/$SHORT_HOSTNAME/
echo "-- Done"
# ------------------
# -- GDrive Upload --
echo "-- Uploading backup to GDrive"
# DONT CREATE DIRECTORY BECAUSE ITS PARENT FSTAB ENTRY AND SHOULD EXIST
# IF DOESNT AND YOU CREATE IT, BACKUP DOESNT FAIL AND YOU GET A LOCAL COPY ONLY
#if [ ! -d /mnt/gdrive/Backups/$SHORT_HOSTNAME ]; then
# mkdir -p /mnt/gdrive/Backups/$SHORT_HOSTNAME
#fi
cp $CRYPTED_FILENAME /mnt/gdrive/Backups/$SHORT_HOSTNAME/
echo "-- Done"
# ------------------
# -- Size check --
echo "-- Checking backup size"
sleep 10
BACKUPSIZE=$(du $CRYPTED_FILENAME|awk '{print$1}')
BACKUPSIZEHUMAN=$(du -h $CRYPTED_FILENAME|awk '{print$1}')
echo "BACKUPSIZE: $BACKUPSIZE"
echo "BACKUPSIZE HUMAN: $BACKUPSIZEHUMAN"
# 249 -> 249K
if [ $BACKUPSIZE -lt {{ min_backup_size }} ]; then
message="$SHORT_HOSTNAME: Abnormal backup size -> /storage/backups/$SHORT_HOSTNAME/$CRYPTED_FILENAME $BACKUPSIZEHUMAN"
sendTelegramMessage $message
fi
echo "-- Done"
# ------------------
# -- Clear temp files --
echo "-- Deleting temp backup files"
rm -rf BACKUP
echo "-- Done"
# ------------------
# -- Remove old local backups --
echo "-- LOCAL_CHECK: Checking if there are old backups to remove"
DAYS_OF_BACKUP='{{ hostvars[groups['tobackup_servers'][0]].local_retention }}'
# Make cleaning of old backups:
TOTAL=$(ls -lt *_backup_*.tar.gz.cpt|wc -l|awk '{print$1}')
if [ $TOTAL -gt $DAYS_OF_BACKUP ]; then
let TO_DELETE=$TOTAL-$DAYS_OF_BACKUP
for BACKUP in $(ls -lt *_backup_*.tar.gz.cpt|awk '{print$9}'|tail -n $TO_DELETE); do
echo "-- Deleting backup: $BACKUP"
rm $BACKUP
done
fi
echo "-- Done"
# ------------------
# -- Remove old DIGI backups --
echo "-- DIGI_CHECK: Checking if there are old backups to remove"
DAYS_OF_BACKUP='{{ hostvars[groups['tobackup_servers'][0]].digi_retention }}'
# Make cleaning of old backups:
TOTAL=$(ls -lt /mnt/DIGI/Backups/$SHORT_HOSTNAME/*_backup_*.tar.gz.cpt|wc -l|awk '{print$1}')
if [ $TOTAL -gt $DAYS_OF_BACKUP ]; then
let TO_DELETE=$TOTAL-$DAYS_OF_BACKUP
for BACKUP in $(ls -lt /mnt/DIGI/Backups/$SHORT_HOSTNAME/*_backup_*.tar.gz.cpt|awk '{print$9}'|tail -n $TO_DELETE); do
echo "-- Deleting backup: $BACKUP"
rm $BACKUP
done
fi
echo "-- Done"
# ------------------
# -- Remove old GDrive backups --
echo "-- GDrive_CHECK: Checking if there are old backups to remove"
DAYS_OF_BACKUP='{{ hostvars[groups['tobackup_servers'][0]].gdrive_retention }}'
# Make cleaning of old backups:
TOTAL=$(ls -lt /mnt/gdrive/Backups/$SHORT_HOSTNAME/*_backup_*.tar.gz.cpt|wc -l|awk '{print$1}')
if [ $TOTAL -gt $DAYS_OF_BACKUP ]; then
let TO_DELETE=$TOTAL-$DAYS_OF_BACKUP
for BACKUP in $(ls -lt /mnt/gdrive/Backups/$SHORT_HOSTNAME/*_backup_*.tar.gz.cpt|awk '{print$9}'|tail -n $TO_DELETE); do
echo "-- Deleting backup: $BACKUP"
rm $BACKUP
done
fi
echo "-- Done"
# ------------------
{% if inventory_hostname == "mighty" %}
{% include 'mighty_additional_backup.j2' %}
{% endif %}
For each server, we will generate a custom_data file:
Mighty is a bit special, in addition to the previous custom_data, it also needs an additional_backup:
Now that we have the templates, we proceed with the playbook:
---
- name: Ansible backup script
hosts: tobackup_servers
remote_user: kr0m
tasks:
- name: Create backup script
template:
src: "~/ansible/templates/backup/backup.j2"
dest: "/root/.scripts/backup.sh"
owner: "root"
group: "wheel"
mode: 0700
become: true
become_method: su
vars:
ansible_su_pass: "ZZZZZZZZZZ"
I consulted the name of the ansible_su_pass parameter using the following command:
- become_pass
Password to pass to su
[Default: (null)]
set_via:
env:
- name: ANSIBLE_BECOME_PASS
- name: ANSIBLE_SU_PASS
ini:
- key: password
section: su_become_plugin
vars:
- name: ansible_become_password
- name: ansible_become_pass
- name: ansible_su_pass
Finally, we run the playbook and if everything went well, the backup script will be deployed on all servers: