Reworked and improved borg role
This commit is contained in:
parent
2451b5ced3
commit
5d9a225014
11 changed files with 123 additions and 57 deletions
22
README
22
README
|
|
@ -1,22 +0,0 @@
|
|||
There are several variables you can define to configure a machines response to the borg role :
|
||||
- borg_server: a string that contains a borg servers hostname
|
||||
- borg_jobs: a list of dict, one item per job with the following keys:
|
||||
- name: the name of the borg job
|
||||
- path: an optional path containing the files to backup
|
||||
- command_to_pipe: an optional command to pipe the backup data from
|
||||
- pre_command: an optional command to run before a job
|
||||
- post_command: an optional command to run after a job
|
||||
- exclude: an optional list of paths containing locations to exclude
|
||||
|
||||
To be valid, a borg job entry needs to have exactly one of the path or command_to_pipe keys.
|
||||
|
||||
Here are some job examples :
|
||||
- { name: etc, path: "/etc", exclude: [ "/etc/firmware" ] }
|
||||
- { name: mysqldump, command_to_pipe: "/usr/bin/mysqldump -h {{ mysql_server }} -u{{ ansible_hostname }} -p{{ ansible_local.mysql_client.password }} --single-transaction --add-drop-database -B {{ ansible_hostname }}" }
|
||||
- { name: gitea, path: "/tmp/gitea.zip", pre_command: "echo '/usr/local/sbin/gitea -C /etc/gitea -c /etc/gitea/app.ini dump -f /tmp/gitea.zip' | su -l _gitea", post_command: "rm -f /tmp/gitea.zip" }
|
||||
|
||||
There is an action plugin that parses the borg_server entries from all host_vars and set the borg.is_server fact to True for any machine specified as a backup target
|
||||
|
||||
Usefull commands:
|
||||
=================
|
||||
ansible all -i hosts -m shell -a "/usr/local/bin/adyxax_backup.sh"
|
||||
85
README.md
Normal file
85
README.md
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
This is the ansible role I use to orchestrate the backups on my personal infrastructure with [borg](https://borgbackup.readthedocs.io/en/stable/).
|
||||
|
||||
## Introduction
|
||||
|
||||
I wanted a role that you can use to easily manage my backups. A mandatory feature for me was the ability to configure a client in only one place without having to configure a server : the server configuration will be derived from the clients that need to use it as a backup target.
|
||||
|
||||
This way configuring backups for a host named `yen.adyxax.org` is as simple as having the following `host_vars` :
|
||||
```
|
||||
julien@yen:~/git/adyxax/ansible$ cat host_vars/yen.adyxax.org
|
||||
---
|
||||
borg_server: cobsd-jde.nexen.net
|
||||
borg_jobs:
|
||||
- { name: etc, path: "/etc", exclude: [ "/etc/firmware" ] }
|
||||
- { name: gitea, path: "/tmp/gitea.zip", pre_command: "echo '/usr/local/sbin/gitea -C /etc/gitea -c /etc/gitea/app.ini dump -f /tmp/gitea.zip' | su -l _gitea", post_command: "
|
||||
rm -f /tmp/gitea.zip" }
|
||||
- { name: nethack, path: "/opt/nethack" }
|
||||
- { name: var_imap, path: "/var/imap" }
|
||||
- { name: var_spool_imap, path: "/var/spool/imap" }
|
||||
...
|
||||
```
|
||||
|
||||
Which can be used in a simple playbook like :
|
||||
```
|
||||
julien@yen:~/git/adyxax/ansible$ cat setup.yml
|
||||
---
|
||||
- name: Gather facts
|
||||
hosts: all
|
||||
tags: always
|
||||
tasks:
|
||||
- name: Gather facts
|
||||
setup:
|
||||
|
||||
- name: Enforce every configurations
|
||||
hosts: all
|
||||
roles:
|
||||
- { role: borg, tags: [ 'borg' ] }
|
||||
...
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
First of all you only need to configure hosts that are backup clients. There are several `host_vars` you can define to this effect :
|
||||
- `borg_server`: a string that contains a borg servers hostname
|
||||
- `borg_jobs`: a list of dict, one item per job with the following keys:
|
||||
- `name`: the name of the borg job, an alphanumeric string.
|
||||
- `path`: an optional path containing the files to backup
|
||||
- `command_to_pipe`: an optional command to pipe the backup data from
|
||||
- `pre_command`: an optional command to run before a job
|
||||
- `post_command`: an optional command to run after a job
|
||||
- `exclude`: an optional list of paths containing locations to exclude
|
||||
- `borg_prune_arguments`: a string passed to the `borg prune` command, defaults to `'--keep-within 30d'` for a 30 days backups retention
|
||||
|
||||
To be valid a borg job entry needs to have a name and exactly one of `path` or `command_to_pipe` key.
|
||||
|
||||
## Job examples
|
||||
|
||||
Here are some job examples :
|
||||
- `{ name: etc, path: "/etc", exclude: [ "/etc/firmware" ] }`
|
||||
- `{ name: mysqldump, command_to_pipe: "/usr/bin/mysqldump -h {{ mysql_server }} -u{{ ansible_hostname }} -p{{ ansible_local.mysql_client.password }} --single-transaction --add-drop-database -B {{ ansible_hostname }}" }`
|
||||
- `{ name: gitea, path: "/tmp/gitea.zip", pre_command: "echo '/usr/local/sbin/gitea -C /etc/gitea -c /etc/gitea/app.ini dump -f /tmp/gitea.zip' | su -l _gitea", post_command: "rm -f /tmp/gitea.zip" }`
|
||||
|
||||
## What the role does
|
||||
|
||||
### On the servers
|
||||
|
||||
On servers, the role creates a `borg` user with `/srv/borg` as a home directory where backups will be stored. For each client, a line in the borg user's `authorized_keys` file is generated to enforce and limit access to only one clients' repository.
|
||||
|
||||
### On the clients
|
||||
|
||||
On clients, the role creates a borg ssh key for the root user and generates a backup script in `/usr/local/bin/adyxax_backup.sh`. The role also adds a cron job that will run the backup script each nigh. Lastly it makes sure the client's borg repository is properly initialised on the server.
|
||||
|
||||
### Action plugin
|
||||
|
||||
There is an action plugin that parses the borg_server entries from all host_vars and set a borg fact for both machines to be backed up and for machines that are specified as backup targets (so that they do not require any manual configuration or variables). This action plugin also enforces the job rules to make sure those are valid and without ambiguities.
|
||||
|
||||
### Ansible fact
|
||||
|
||||
There is a fact script deployed on each server. It is used to retrieve the ssh public key of clients or the repository status of servers and used in tasks.
|
||||
|
||||
## Usefull command:
|
||||
|
||||
Manually schedule a backup run :
|
||||
```
|
||||
ansible all -i hosts -m shell -a "/usr/local/bin/adyxax_backup.sh"
|
||||
```
|
||||
|
|
@ -9,17 +9,46 @@ class ActionModule(ActionBase):
|
|||
result['changed'] = False
|
||||
result['failed'] = False
|
||||
|
||||
is_server = False
|
||||
error_msgs = []
|
||||
|
||||
### OS support #######################################################
|
||||
os_package_names = {
|
||||
'Alpine': 'borgbackup',
|
||||
'Debian': 'borgbackup',
|
||||
'Gentoo': 'app-backup/borgbackup',
|
||||
'OpenBSD': 'borgbackup',
|
||||
'RedHat': 'borgbackup',
|
||||
}
|
||||
if task_vars['ansible_os_family'] not in os_package_names:
|
||||
error_msgs.append(f"borg role does not support {task_vars['ansible_os_family']} os family clients yet")
|
||||
|
||||
### Borg server variables ############################################
|
||||
server = {
|
||||
'clients': [], # a list of hostnames
|
||||
}
|
||||
for hostname, hostvars in task_vars['hostvars'].items() :
|
||||
if 'borg_server' in hostvars.keys() and hostvars['borg_server'] == task_vars['ansible_host']:
|
||||
is_server = True
|
||||
server['clients'].append(hostname)
|
||||
|
||||
### Borg client variables ############################################
|
||||
client = {
|
||||
'server': '', # a server hostname
|
||||
}
|
||||
if 'borg_server' in task_vars:
|
||||
client['server'] = task_vars['borg_server']
|
||||
|
||||
### Results compilation ##############################################
|
||||
if error_msgs != []:
|
||||
result['msg'] = ' ; '.join(error_msgs)
|
||||
result['failed'] = True
|
||||
return result
|
||||
|
||||
result['ansible_facts'] = {
|
||||
'borg': {
|
||||
'is_server': is_server,
|
||||
'client': client,
|
||||
'package_name': os_package_names[task_vars['ansible_os_family']],
|
||||
'server': server,
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
|
||||
|
|
|
|||
|
|
@ -1,15 +1,7 @@
|
|||
---
|
||||
- name: set distro-specific server variables
|
||||
include_vars: '{{ ansible_os_family }}.yml'
|
||||
|
||||
- name: Check if borg is supported on distro
|
||||
fail:
|
||||
msg: "borg tasks are not supported on this operating system yet."
|
||||
when: borg_packages is not defined
|
||||
|
||||
- name: Ensure borg is installed
|
||||
package:
|
||||
name: "{{ borg_packages }}"
|
||||
name: "{{ borg.package_name }}"
|
||||
|
||||
- name: Push borg gathering fact on client
|
||||
copy:
|
||||
|
|
|
|||
|
|
@ -2,10 +2,10 @@
|
|||
- action: borg_init
|
||||
|
||||
- import_tasks: common.yml
|
||||
when: borg.is_server or borg_server is defined
|
||||
when: borg.server.clients != [] or borg_server is defined
|
||||
|
||||
- import_tasks: server.yml
|
||||
when: borg.is_server
|
||||
when: borg.server.clients != []
|
||||
|
||||
- import_tasks: client.yml
|
||||
when: borg_server is defined
|
||||
|
|
|
|||
|
|
@ -4,6 +4,8 @@
|
|||
# ~~~~ ~~~~ #
|
||||
###############################################################################
|
||||
|
||||
set -eu
|
||||
|
||||
export HOME=/root
|
||||
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
||||
export BORG_RSH="ssh -i /root/.ssh/borg"
|
||||
|
|
|
|||
|
|
@ -1,4 +0,0 @@
|
|||
---
|
||||
borg_packages:
|
||||
- borgbackup
|
||||
...
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
---
|
||||
borg_packages:
|
||||
- borgbackup
|
||||
...
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
---
|
||||
borg_packages:
|
||||
- app-backup/borgbackup
|
||||
...
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
---
|
||||
borg_packages:
|
||||
- borgbackup
|
||||
...
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
---
|
||||
borg_packages:
|
||||
- borgbackup
|
||||
...
|
||||
Reference in a new issue