chore(ops): script de backup quotidien workflows.db + audit
Some checks failed
security-audit / Bandit (scan statique) (push) Successful in 12s
security-audit / pip-audit (CVE dépendances) (push) Successful in 10s
security-audit / Scan secrets (grep) (push) Successful in 9s
tests / Lint (ruff + black) (push) Successful in 13s
tests / Tests unitaires (sans GPU) (push) Failing after 14s
tests / Tests sécurité (critique) (push) Has been skipped
Some checks failed
security-audit / Bandit (scan statique) (push) Successful in 12s
security-audit / pip-audit (CVE dépendances) (push) Successful in 10s
security-audit / Scan secrets (grep) (push) Successful in 9s
tests / Lint (ruff + black) (push) Successful in 13s
tests / Tests unitaires (sans GPU) (push) Failing after 14s
tests / Tests sécurité (critique) (push) Has been skipped
Critique avant POC Anouste — trou identifié par le challenge du 16 avril. Sans backup, une perte de workflows.db = perte directe du travail client (workflows, historique d'exécutions, ancres visuelles). Script scripts/backup_vwb_and_audit.sh : - Copie workflows.db via `sqlite3 .backup` (snapshot cohérent, même si le backend Flask tient la BDD ouverte) → ~/backups/vwb/ - Copie data/audit/*.jsonl → ~/backups/audit/audit_YYYY-MM-DD/ - Rétention automatique 30 jours (override via RETENTION_DAYS env) - Destination override : BACKUP_ROOT=/chemin env var - Log horodaté : ~/backups/backup.log Installation (non automatique — à la main, cf. consigne) : crontab -e 0 2 * * * /home/dom/ai/rpa_vision_v3/scripts/backup_vwb_and_audit.sh Procédure de restore documentée dans ~/backups/README.md (créé hors repo, volontairement). Testé : 458752 octets restaurés à partir de workflows.db actuel (3 workflows, 115 exécutions, 18 steps, intégrité OK). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
124
scripts/backup_vwb_and_audit.sh
Executable file
124
scripts/backup_vwb_and_audit.sh
Executable file
@@ -0,0 +1,124 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
################################################################################
|
||||||
|
# backup_vwb_and_audit.sh
|
||||||
|
#
|
||||||
|
# Backup quotidien critique avant POC Anouste.
|
||||||
|
# Cf. challenge du 16 avril 2026 : sans backup, perte de workflows.db =
|
||||||
|
# perte directe de travail client. Ce script doit tourner AVANT tout
|
||||||
|
# déploiement chez un client.
|
||||||
|
#
|
||||||
|
# Ce qu'il sauvegarde :
|
||||||
|
# - visual_workflow_builder/backend/instance/workflows.db
|
||||||
|
# → ~/backups/vwb/workflows_YYYY-MM-DD.db
|
||||||
|
# - data/audit/*.jsonl
|
||||||
|
# → ~/backups/audit/audit_YYYY-MM-DD/
|
||||||
|
#
|
||||||
|
# Rétention : 30 jours (suppression automatique des backups plus anciens).
|
||||||
|
# Log : ~/backups/backup.log (append, horodaté).
|
||||||
|
#
|
||||||
|
# Installation (non automatique — à faire à la main) :
|
||||||
|
# crontab -e
|
||||||
|
# 0 2 * * * /home/dom/ai/rpa_vision_v3/scripts/backup_vwb_and_audit.sh
|
||||||
|
# → s'exécute tous les jours à 2h du matin.
|
||||||
|
#
|
||||||
|
# Procédure de restore : voir ~/backups/README.md
|
||||||
|
#
|
||||||
|
# Auteur : Dom + Claude — 16 avril 2026
|
||||||
|
################################################################################
|
||||||
|
|
||||||
|
set -u # strict: variable non définie = erreur
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Chemins
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||||
|
|
||||||
|
SRC_WORKFLOWS_DB="$PROJECT_ROOT/visual_workflow_builder/backend/instance/workflows.db"
|
||||||
|
SRC_AUDIT_DIR="$PROJECT_ROOT/data/audit"
|
||||||
|
|
||||||
|
BACKUP_ROOT="${BACKUP_ROOT:-$HOME/backups}"
|
||||||
|
BACKUP_VWB_DIR="$BACKUP_ROOT/vwb"
|
||||||
|
BACKUP_AUDIT_DIR="$BACKUP_ROOT/audit"
|
||||||
|
BACKUP_LOG="$BACKUP_ROOT/backup.log"
|
||||||
|
|
||||||
|
RETENTION_DAYS="${RETENTION_DAYS:-30}"
|
||||||
|
DATE_TAG="$(date +%Y-%m-%d)"
|
||||||
|
NOW="$(date '+%Y-%m-%d %H:%M:%S')"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Helpers
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
log() {
|
||||||
|
local msg="$1"
|
||||||
|
echo "[$NOW] $msg" >> "$BACKUP_LOG"
|
||||||
|
echo "$msg"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# Préparation
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
mkdir -p "$BACKUP_VWB_DIR" "$BACKUP_AUDIT_DIR"
|
||||||
|
touch "$BACKUP_LOG"
|
||||||
|
|
||||||
|
log "=== Début backup VWB + audit ==="
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# 1. workflows.db
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
if [ -f "$SRC_WORKFLOWS_DB" ]; then
|
||||||
|
DEST_DB="$BACKUP_VWB_DIR/workflows_${DATE_TAG}.db"
|
||||||
|
# On utilise sqlite3 .backup si possible (safe, même si la DB est
|
||||||
|
# ouverte par le backend). Fallback : cp simple.
|
||||||
|
if command -v sqlite3 > /dev/null 2>&1; then
|
||||||
|
if sqlite3 "$SRC_WORKFLOWS_DB" ".backup '$DEST_DB'" 2>/dev/null; then
|
||||||
|
size=$(stat -c %s "$DEST_DB" 2>/dev/null || echo "?")
|
||||||
|
log " [OK] workflows.db → $DEST_DB (${size} octets) via sqlite3 .backup"
|
||||||
|
else
|
||||||
|
cp "$SRC_WORKFLOWS_DB" "$DEST_DB"
|
||||||
|
log " [OK fallback] workflows.db → $DEST_DB via cp"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
cp "$SRC_WORKFLOWS_DB" "$DEST_DB"
|
||||||
|
log " [OK] workflows.db → $DEST_DB via cp (sqlite3 absent)"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
log " [WARN] workflows.db introuvable : $SRC_WORKFLOWS_DB"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# 2. data/audit/*.jsonl
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
if [ -d "$SRC_AUDIT_DIR" ]; then
|
||||||
|
DEST_AUDIT="$BACKUP_AUDIT_DIR/audit_${DATE_TAG}"
|
||||||
|
mkdir -p "$DEST_AUDIT"
|
||||||
|
copied=0
|
||||||
|
# shellcheck disable=SC2045
|
||||||
|
for f in "$SRC_AUDIT_DIR"/*.jsonl; do
|
||||||
|
[ -f "$f" ] || continue
|
||||||
|
cp "$f" "$DEST_AUDIT/"
|
||||||
|
copied=$((copied + 1))
|
||||||
|
done
|
||||||
|
log " [OK] $copied fichiers audit → $DEST_AUDIT"
|
||||||
|
else
|
||||||
|
log " [WARN] dossier audit introuvable : $SRC_AUDIT_DIR"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# 3. Rétention : suppression des backups > RETENTION_DAYS jours
|
||||||
|
# ------------------------------------------------------------------------------
|
||||||
|
# On retire les fichiers .db du dossier vwb
|
||||||
|
if [ -d "$BACKUP_VWB_DIR" ]; then
|
||||||
|
deleted_db=$(find "$BACKUP_VWB_DIR" -maxdepth 1 -name "workflows_*.db" \
|
||||||
|
-type f -mtime +"$RETENTION_DAYS" -print -delete 2>/dev/null | wc -l)
|
||||||
|
[ "$deleted_db" -gt 0 ] && log " [CLEAN] $deleted_db backup(s) vwb > ${RETENTION_DAYS}j supprimé(s)"
|
||||||
|
fi
|
||||||
|
# On retire les répertoires audit daté
|
||||||
|
if [ -d "$BACKUP_AUDIT_DIR" ]; then
|
||||||
|
deleted_audit=$(find "$BACKUP_AUDIT_DIR" -maxdepth 1 -type d \
|
||||||
|
-name "audit_*" -mtime +"$RETENTION_DAYS" -print -exec rm -rf {} \; 2>/dev/null | wc -l)
|
||||||
|
[ "$deleted_audit" -gt 0 ] && log " [CLEAN] $deleted_audit backup(s) audit > ${RETENTION_DAYS}j supprimé(s)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
log "=== Fin backup ==="
|
||||||
|
exit 0
|
||||||
Reference in New Issue
Block a user