From 34fbde1f81bc52b9b16b35056686399775b608c7 Mon Sep 17 00:00:00 2001 From: Julien Dessaux Date: Wed, 23 Apr 2025 09:36:29 +0200 Subject: [PATCH] doc(tfstated): import documentation file from the forgejo wiki --- doc/Administrator-Guide.md | 14 ++++++ doc/Backup.md | 39 +++++++++++++++ doc/Contributor-Guide.md | 33 +++++++++++++ doc/Example-Backup-With-Borg.md | 86 +++++++++++++++++++++++++++++++++ doc/Home.md | 7 +++ doc/Release-Process.md | 29 +++++++++++ doc/What-is-TfStated.md | 16 ++++++ 7 files changed, 224 insertions(+) create mode 100644 doc/Administrator-Guide.md create mode 100644 doc/Backup.md create mode 100644 doc/Contributor-Guide.md create mode 100644 doc/Example-Backup-With-Borg.md create mode 100644 doc/Home.md create mode 100644 doc/Release-Process.md create mode 100644 doc/What-is-TfStated.md diff --git a/doc/Administrator-Guide.md b/doc/Administrator-Guide.md new file mode 100644 index 0000000..e9e7060 --- /dev/null +++ b/doc/Administrator-Guide.md @@ -0,0 +1,14 @@ +# Administrator Guide + +These documents are targeted to people who run TfStated on their infrastructure. +A reasonable level of proficiency in using one's operating system of choice is +expected. Networking and Database administration basics are strongly +recommended. + +- [[Release Process]] +- [[Installation]] +- [[Configuration]] +- [[Backup]] +- [[Upgrading]] +- [[Troubleshooting]] +- [[Reverse Proxy]] diff --git a/doc/Backup.md b/doc/Backup.md new file mode 100644 index 0000000..82932b7 --- /dev/null +++ b/doc/Backup.md @@ -0,0 +1,39 @@ +# Backup + +All TfStated data is stored in its SQLite database. Backing up this database can +be achieved through different means. + +## VACUUM INTO + +SQLite has a `VACUUM` command that is used to optimize the structure of the +database. It also has the side effect of shrinking its size by reclaiming disk +space that is no longer used by table data. + +`VACUUM INTO` is a variant that runs this process without interfering with other +writes and also outputs a perfect working copy of your database at the point in +time you run this command. Everything is kept as is, including the `PRAGMA` and +indexes. You can then backup this output file. Example: + +``` sql +umask 077 +echo -n "VACUUM INTO '/tmp/tfstated.db';" | sqlite3 /var/lib/tfstated/tfstated.db +``` + +This is the recommended way to backup your TfStated database as you can pair it +up with the backup software of your choice (for example +[borg](./Example-Backup-With-Borg)). + +Another option is to output this backup file +directly to a remote storage mounted on your server. + +## Litestream + +[Litestream](https://litestream.io/) is a background service that continuously +monitors and replicates an SQLite database to an S3 compatible object storage. + +Litestream is a good backup solution that has the advantage of providing point +in time recovery. The downsides are that it is another service to manage and +that it requires a write lock on the database when its checkpointing occurs. + +Since Litestream is more complex to operate, the TfStated project recommends to +pair this solution with a `VACUUM INTO` style backup. diff --git a/doc/Contributor-Guide.md b/doc/Contributor-Guide.md new file mode 100644 index 0000000..8df7f1d --- /dev/null +++ b/doc/Contributor-Guide.md @@ -0,0 +1,33 @@ +# Contributor Guide + +## TfStated's code is open source, not open contribution + +[Similar to SQLite](https://www.sqlite.org/copyright.html), TfStated's code is +open source but not open contribution for multiple reasons: + +- It avoids potential intellectual property and licensing issues. +- It removes the burden of reviewing patches and maintaining the resulting code. +- It helps keep the software focused on a clear vision. + +While this might be disappointing to you, this choice helps me continue to build +and maintain TfStated. + +## Bug reports + +I am thankful for any bug report. Feel free to open issues or comment on +existing ones and include as much useful information as possible. Please try to +search through open issues first to avoid duplicates! + +I cannot guarantee that I will fix every bug. + +## Ideas and feature suggestions + +Ideas about existing features and suggestions for new ones are welcome, either +on by opening issues on this forge or by +[email](mailto:julien.dessaux@adyxax.org). I am also reachable on [mastodon +@adyxax@adyxax.org](https://fedi.adyxax.org/@adyxax). If you want us to have +some privacy, [here is my public gpg +key](https://www.adyxax.org/static/F92E51B86E07177E.pgp). + +You can also [hire me](mailto:julien.dessaux@adyxax.org) for support or to +develop specific features. diff --git a/doc/Example-Backup-With-Borg.md b/doc/Example-Backup-With-Borg.md new file mode 100644 index 0000000..2ab6bf7 --- /dev/null +++ b/doc/Example-Backup-With-Borg.md @@ -0,0 +1,86 @@ +# Example Backup with Borg + +Here is a complete example of how to backup a `/var/lib/tfstated/tfstated.db` +SQLite database file using [borg](https://www.borgbackup.org/) on a Debian 12 +bookworm server using a bash script, a systemd service and a systemd timer. + +### Script + +The `/etc/borg/tfstated.sh` script should belong to `root:root` with 0500 +permissions (`r-x------`): + +``` shell +#!/usr/bin/env bash +set -euo pipefail + +archiveSuffix=".failed" + +# Run borg init if the repo doesn't exist yet +if ! borg list > /dev/null; then + borg init --encryption none +fi + +archiveName="tfstated-sqlite3-$(date +%Y-%m-%dT%H:%M:%S)" +rm -f /tmp/tfstated.db; umask 077; printf '%s' "VACUUM INTO '/tmp/tfstated.db'" \ + | sqlite3 /srv/tfstated/sqlite.db +borg create \ + --compression auto,zstd \ + "::${archiveName}${archiveSuffix}" \ + /tmp/tfstated.db +rm -f /tmp/tfstated.db +borg rename "::${archiveName}${archiveSuffix}" "${archiveName}" +borg prune \ + --keep-daily=14 --keep-monthly=3 --keep-weekly=4 \ + --glob-archives '*-tfstated-sqlite3-*' + +borg compact +``` + +Please change the destination hostname and retention options to your liking. You +can also encrypt your borg backups for additional security, but remember that +your OpenTofu/terraform states are already encrypted at rest in the SQLite +database. + +### Systemd service +The `/etc/systemd/system/borg-job-tfstated.service` systemd service file should +belong to `root:root` with 0444 permissions (`r--r--r--`): + +``` ini +[Unit] +Description=BorgBackup job tfstated + +[Service] +Environment="BORG_REPO=ssh://borg@myth.adyxax.org/srv/borg/tfstated" +Environment="BORG_RSH=ssh -i /etc/borg/tfstated.key -o StrictHostKeyChecking=accept-new" +CPUSchedulingPolicy=idle +ExecStart=/etc/borg/tfstated.sh +Group=root +IOSchedulingClass=idle +PrivateTmp=true +ProtectSystem=strict +ReadWritePaths=/root/.cache/borg +ReadWritePaths=/root/.config/borg +User=root +``` + +This service file uses environment variables to pass information about the +`BORG_REPO` and the `BORG_RSH` command to use. Change them to your liking. + +### Systemd timer + +The `/etc/systemd/system/borg-job-tfstated.timer` systemd timer file should +belong to `root:root` with 0444 permissions (`r--r--r--`): + +``` ini +[Unit] +Description=BorgBackup job tfstated timer + +[Timer] +FixedRandomDelay=true +OnCalendar=daily +Persistent=true +RandomizedDelaySec=3600 + +[Install] +WantedBy=timers.target +``` diff --git a/doc/Home.md b/doc/Home.md new file mode 100644 index 0000000..e75d610 --- /dev/null +++ b/doc/Home.md @@ -0,0 +1,7 @@ +# TfStated documentation + +- [[What is TfStated]] +- [[Quickstart]] +- [[Administrator Guide]] +- [[User Guide]] +- [[Contributor Guide]] diff --git a/doc/Release-Process.md b/doc/Release-Process.md new file mode 100644 index 0000000..41ba47a --- /dev/null +++ b/doc/Release-Process.md @@ -0,0 +1,29 @@ +# Release process + +TfStated follows [semver](https://semver.org/) versioning. + +## Patch releases + +Patch releases are published frequently and provide fixes for bugs and security +vulnerabilities in TfStated or its dependencies. Only the latest published +feature released should be expected to be maintained. + +Fixes for older feature releases might happen if someone [hires +me](mailto:julien.dessaux@adyxax.org) to maintain them. + +## Feature releases + +Feature releases are less frequent and do not follow a strict schedule. +Administrators interested in new features should follow [the project's +milestones](https://git.adyxax.org/adyxax/tfstated/milestones) to get an idea of +when the next feature release might come. Everyone is welcome to voice their +interest in features by commenting on their respective issues. + +Administrators should [backup](./Backup) and thoroughly read the [release +notes](https://git.adyxax.org/adyxax/tfstated/releases) for each version before +[upgrading](./Upgrading). Only upgrades from one feature release to the next in +one of the deployment options listed in this guide will be tested. + +Skipping feature releases might or might not work: use your best judgement based +on the contents of the release notes. Another option is to [hire +me](mailto:julien.dessaux@adyxax.org) to help perform your upgrades. diff --git a/doc/What-is-TfStated.md b/doc/What-is-TfStated.md new file mode 100644 index 0000000..1c125f2 --- /dev/null +++ b/doc/What-is-TfStated.md @@ -0,0 +1,16 @@ +# What is TfStated + +TfStated is an [OpenTofu/Terraform states HTTP +backend](https://developer.hashicorp.com/terraform/language/backend/http). + +OpenTofu/Terraform state files are extremely sensitive: they describe the +complete state of one's infrastructure including secrets and other sensitive +data. TfStated's primary mission is to safely and securely manage these state +files. This entails: + +- Storing the state files fully encrypted. +- Controlling whose people or what systems have access to which state files. +- Keeping an audit log of state files access and modifications. +- Storing state files version history. +- Allowing to rollback to previous state files versions. +- Manage state file locks.