Skip to content

Backups & Recovery

Comprehensive backup and disaster recovery procedures for collabrains.eu.

Backup Strategy

Automated Daily Backups

Schedule: 2 AM UTC daily

Script: /usr/local/bin/backup-collabrains.sh

Contents: - PostgreSQL dumps (all databases) - Docker volumes (compressed tar.gz) - Service configurations

Location: /backups/YYYY-MM-DD/

Retention: Last 30 days (auto-cleanup)

Size: ~9MB per day average

Backup Contents

Each daily backup directory contains:

/backups/2026-05-06/
├── authentik-postgres.sql       # Authentik database
├── grist-postgres.sql           # Grist spreadsheets
├── immich-postgres.sql          # Photo library
├── n8n-postgres.sql             # Workflows
├── grafana-postgres.sql         # Monitoring
├── *_postgres.sql               # Other services
├── volumes/
│   ├── immich-data.tar.gz       # Photo volumes
│   ├── n8n-data.tar.gz          # n8n config
│   └── *.tar.gz                 # All service volumes
└── coolify-services.tar.gz      # Service configurations

Manual Backup

Run Backup Now

/usr/local/bin/backup-collabrains.sh

# Verify backup
ls -lh /backups/$(date +%Y-%m-%d)/
du -sh /backups/$(date +%Y-%m-%d)

Backup Specific Service

PostgreSQL database only:

# Example: Backup Immich
SERVICE_ID=IMMICH_ID
docker exec postgres-$SERVICE_ID pg_dump -U immich immich > immich-backup-$(date +%Y%m%d-%H%M%S).sql

# Verify
file immich-backup-*.sql
ls -lh immich-backup-*.sql

Volume data only:

# Example: Backup Immich media
docker run --rm -v VOLUME_NAME:/data -v $(pwd):/backup \
  alpine tar czf /backup/volume-backup.tar.gz -C /data .

# Verify
tar tzf volume-backup.tar.gz | head -20

Database Recovery

Restore PostgreSQL Database

From Backup File

# 1. List available backups
ls -lh /backups/*/
ls -lh /backups/2026-05-06/*postgres.sql

# 2. Identify the service
SERVICE_ID=IMMICH_ID          # Replace with actual service ID
BACKUP_DATE="2026-05-06"
BACKUP_FILE="/backups/$BACKUP_DATE/immich-postgres.sql"

# 3. Restore the database
docker exec -i postgres-$SERVICE_ID psql -U immich immich < $BACKUP_FILE

# 4. Verify restore
docker exec -it postgres-$SERVICE_ID psql -U immich -d immich
# Inside psql:
SELECT COUNT(*) FROM assets;  # For Immich, count photos
\q

Full Database Recovery (if DB container corrupted)

# 1. Verify backup file is readable
file /backups/2026-05-06/immich-postgres.sql

# 2. Restore to new database
docker exec -it postgres-SERVICE_ID psql -U immich
CREATE DATABASE immich_restore;
\q

# 3. Restore into new database
docker exec -i postgres-SERVICE_ID psql -U immich immich_restore < /backups/2026-05-06/immich-postgres.sql

# 4. Verify data
docker exec postgres-SERVICE_ID psql -U immich immich_restore -c "SELECT COUNT(*) FROM assets;"

# 5. Swap databases
docker exec -it postgres-SERVICE_ID psql -U immich
DROP DATABASE immich;
ALTER DATABASE immich_restore RENAME TO immich;
\q

# 6. Restart service
docker restart SERVICE_CONTAINER

Volume Recovery

Restore Docker Volume

From Backup Tar

# 1. Find the backup
ls -lh /backups/2026-05-06/*.tar.gz

# 2. Identify the volume
VOLUME_NAME="woq978nbzog6dmddhrmeujvk_paperless-media"
BACKUP_FILE="/backups/2026-05-06/paperless-media.tar.gz"

# 3. Remove old data (optional, if corrupted)
docker run --rm -v $VOLUME_NAME:/data alpine rm -rf /data/*

# 4. Restore volume
docker run --rm -v $VOLUME_NAME:/data -v /backups/2026-05-06:/backup \
  alpine tar xzf /backup/paperless-media.tar.gz -C /data

# 5. Verify
docker run --rm -v $VOLUME_NAME:/data alpine ls -la /data | head -20

# 6. Restart service
docker restart SERVICE_CONTAINER

List Volume Contents Before Restore

# See what's in the backup before restoring
tar tzf /backups/2026-05-06/immich-data.tar.gz | head -50

Full Service Recovery

Recover Service with Database + Volumes

Example: Recovering Immich after data corruption

# 1. Stop the service
cd /data/coolify/services/IMMICH_ID
docker compose down

# 2. Restore database
docker exec -i postgres-IMMICH_ID psql -U immich immich < /backups/2026-05-06/immich-postgres.sql

# 3. Restore volumes
BACKUP_DIR="/backups/2026-05-06"
docker run --rm -v immich-data:/data -v $BACKUP_DIR:/backup \
  alpine tar xzf /backup/immich-data.tar.gz -C /data

# 4. Restart service
docker compose up -d

# 5. Verify
docker logs immich-server --tail 20
docker ps | grep immich

Backup Management

List All Backups

ls -lh /backups/

# View detailed backup contents
for dir in /backups/*/; do
  date=$(basename $dir)
  size=$(du -sh $dir | cut -f1)
  files=$(ls -1 $dir | wc -l)
  echo "$date: $size ($files files)"
done

Delete Old Backups

# Delete backups older than 30 days (auto-cleanup runs nightly)
find /backups -maxdepth 1 -mtime +30 -type d -exec rm -rf {} \;

# Verify removal
ls -lh /backups/ | tail -10

Compress Backup for Transfer

# Create portable backup (useful for external storage)
tar czf collabrains-backup-$(date +%Y%m%d).tar.gz /backups/$(date +%Y-%m-%d)

# Transfer to external storage
scp collabrains-backup-20260506.tar.gz user@backup-server:/mnt/backups/

Disaster Recovery Scenarios

Scenario 1: Single Service Corrupted

Example: Paperless database corrupted

# 1. Check backup exists
ls -lh /backups/2026-05-06/paperless-postgres.sql

# 2. Restore database only
BACKUP_FILE="/backups/2026-05-06/paperless-postgres.sql"
docker exec -i postgres-woq978nbzog6dmddhrmeujvk psql -U paperless paperless < $BACKUP_FILE

# 3. Restart
docker restart paperless-woq978nbzog6dmddhrmeujvk

# 4. Verify
docker logs paperless-woq978nbzog6dmddhrmeujvk --tail 20

Scenario 2: Complete Data Loss

If all data lost, restore from dated backup

# 1. Find most recent backup
ls -lh /backups/ | sort -k6,7 | tail -1

# 2. Restore all databases
BACKUP_DIR="/backups/2026-05-05"
for db_file in $BACKUP_DIR/*-postgres.sql; do
  db_name=$(basename $db_file | sed 's/-postgres.sql//')
  echo "Restoring $db_name..."
  docker exec -i postgres-SERVICE_ID psql -U $db_name -d $db_name < $db_file
done

# 3. Restore all volumes
docker run --rm -v VOLUME_NAME:/data -v $BACKUP_DIR:/backup \
  alpine tar xzf /backup/volume.tar.gz -C /data

# 4. Restart all services
docker compose -f /data/coolify/services/*/docker-compose.yml up -d

Scenario 3: Server Hardware Failure

If entire server lost, use Hetzner backup or external backup

# On new server:
# 1. Set up Docker and Coolify
# 2. Transfer backup files
scp -r /backup-server/collabrains-backup-20260506.tar.gz root@new-server:/tmp/

# 3. Extract
tar xzf /tmp/collabrains-backup-20260506.tar.gz -C /backups/

# 4. Restore services (see above recovery procedures)

Testing Backups

Verify Backup Integrity

# Check SQL files are not corrupted
for sql_file in /backups/2026-05-06/*.sql; do
  head -1 $sql_file | grep -q "PostgreSQL" && echo "✓ $sql_file" || echo "✗ $sql_file CORRUPTED"
done

# Test tar.gz files
for tar_file in /backups/2026-05-06/*.tar.gz; do
  tar tzf $tar_file >/dev/null 2>&1 && echo "✓ $tar_file" || echo "✗ $tar_file CORRUPTED"
done

Monthly Restore Test

First Monday of each month:

# Test restore to temporary location
BACKUP_DIR="/backups/2026-05-06"
TEMP_DB="test_restore_$(date +%s)"

# Create temporary test database
docker exec -it postgres-IMMICH_ID psql -U immich
CREATE DATABASE $TEMP_DB;
\q

# Restore into temp database
docker exec -i postgres-IMMICH_ID psql -U immich $TEMP_DB < $BACKUP_DIR/immich-postgres.sql

# Verify data
docker exec postgres-IMMICH_ID psql -U immich $TEMP_DB -c "SELECT COUNT(*) FROM assets;"

# Clean up
docker exec -it postgres-IMMICH_ID psql -U immich
DROP DATABASE $TEMP_DB;
\q

Backup Retention Policy

  • Daily backups: 30 days (auto-cleanup)
  • Monthly archives: 12 months (external storage)
  • Critical backups: Retain indefinitely (off-site)

Create Monthly Archive

# First day of each month
if [[ $(date +%d) == "01" ]]; then
  BACKUP_MONTH=$(date -d "1 month ago" +%Y-%m)
  tar czf /archive/collabrains-$BACKUP_MONTH.tar.gz /backups/*
fi