chore(ops): script de backup quotidien workflows.db + audit
Some checks failed
security-audit / Bandit (scan statique) (push) Successful in 12s
security-audit / pip-audit (CVE dépendances) (push) Successful in 10s
security-audit / Scan secrets (grep) (push) Successful in 9s
tests / Lint (ruff + black) (push) Successful in 13s
tests / Tests unitaires (sans GPU) (push) Failing after 14s
tests / Tests sécurité (critique) (push) Has been skipped
Some checks failed
security-audit / Bandit (scan statique) (push) Successful in 12s
security-audit / pip-audit (CVE dépendances) (push) Successful in 10s
security-audit / Scan secrets (grep) (push) Successful in 9s
tests / Lint (ruff + black) (push) Successful in 13s
tests / Tests unitaires (sans GPU) (push) Failing after 14s
tests / Tests sécurité (critique) (push) Has been skipped
Critique avant POC Anouste — trou identifié par le challenge du 16 avril. Sans backup, une perte de workflows.db = perte directe du travail client (workflows, historique d'exécutions, ancres visuelles). Script scripts/backup_vwb_and_audit.sh : - Copie workflows.db via `sqlite3 .backup` (snapshot cohérent, même si le backend Flask tient la BDD ouverte) → ~/backups/vwb/ - Copie data/audit/*.jsonl → ~/backups/audit/audit_YYYY-MM-DD/ - Rétention automatique 30 jours (override via RETENTION_DAYS env) - Destination override : BACKUP_ROOT=/chemin env var - Log horodaté : ~/backups/backup.log Installation (non automatique — à la main, cf. consigne) : crontab -e 0 2 * * * /home/dom/ai/rpa_vision_v3/scripts/backup_vwb_and_audit.sh Procédure de restore documentée dans ~/backups/README.md (créé hors repo, volontairement). Testé : 458752 octets restaurés à partir de workflows.db actuel (3 workflows, 115 exécutions, 18 steps, intégrité OK). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
124
scripts/backup_vwb_and_audit.sh
Executable file
124
scripts/backup_vwb_and_audit.sh
Executable file
@@ -0,0 +1,124 @@
|
||||
#!/bin/bash
|
||||
################################################################################
|
||||
# backup_vwb_and_audit.sh
|
||||
#
|
||||
# Backup quotidien critique avant POC Anouste.
|
||||
# Cf. challenge du 16 avril 2026 : sans backup, perte de workflows.db =
|
||||
# perte directe de travail client. Ce script doit tourner AVANT tout
|
||||
# déploiement chez un client.
|
||||
#
|
||||
# Ce qu'il sauvegarde :
|
||||
# - visual_workflow_builder/backend/instance/workflows.db
|
||||
# → ~/backups/vwb/workflows_YYYY-MM-DD.db
|
||||
# - data/audit/*.jsonl
|
||||
# → ~/backups/audit/audit_YYYY-MM-DD/
|
||||
#
|
||||
# Rétention : 30 jours (suppression automatique des backups plus anciens).
|
||||
# Log : ~/backups/backup.log (append, horodaté).
|
||||
#
|
||||
# Installation (non automatique — à faire à la main) :
|
||||
# crontab -e
|
||||
# 0 2 * * * /home/dom/ai/rpa_vision_v3/scripts/backup_vwb_and_audit.sh
|
||||
# → s'exécute tous les jours à 2h du matin.
|
||||
#
|
||||
# Procédure de restore : voir ~/backups/README.md
|
||||
#
|
||||
# Auteur : Dom + Claude — 16 avril 2026
|
||||
################################################################################
|
||||
|
||||
set -u # strict: variable non définie = erreur
|
||||
|
||||
# ------------------------------------------------------------------------------
|
||||
# Chemins
|
||||
# ------------------------------------------------------------------------------
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
SRC_WORKFLOWS_DB="$PROJECT_ROOT/visual_workflow_builder/backend/instance/workflows.db"
|
||||
SRC_AUDIT_DIR="$PROJECT_ROOT/data/audit"
|
||||
|
||||
BACKUP_ROOT="${BACKUP_ROOT:-$HOME/backups}"
|
||||
BACKUP_VWB_DIR="$BACKUP_ROOT/vwb"
|
||||
BACKUP_AUDIT_DIR="$BACKUP_ROOT/audit"
|
||||
BACKUP_LOG="$BACKUP_ROOT/backup.log"
|
||||
|
||||
RETENTION_DAYS="${RETENTION_DAYS:-30}"
|
||||
DATE_TAG="$(date +%Y-%m-%d)"
|
||||
NOW="$(date '+%Y-%m-%d %H:%M:%S')"
|
||||
|
||||
# ------------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ------------------------------------------------------------------------------
|
||||
log() {
|
||||
local msg="$1"
|
||||
echo "[$NOW] $msg" >> "$BACKUP_LOG"
|
||||
echo "$msg"
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------------------
|
||||
# Préparation
|
||||
# ------------------------------------------------------------------------------
|
||||
mkdir -p "$BACKUP_VWB_DIR" "$BACKUP_AUDIT_DIR"
|
||||
touch "$BACKUP_LOG"
|
||||
|
||||
log "=== Début backup VWB + audit ==="
|
||||
|
||||
# ------------------------------------------------------------------------------
|
||||
# 1. workflows.db
|
||||
# ------------------------------------------------------------------------------
|
||||
if [ -f "$SRC_WORKFLOWS_DB" ]; then
|
||||
DEST_DB="$BACKUP_VWB_DIR/workflows_${DATE_TAG}.db"
|
||||
# On utilise sqlite3 .backup si possible (safe, même si la DB est
|
||||
# ouverte par le backend). Fallback : cp simple.
|
||||
if command -v sqlite3 > /dev/null 2>&1; then
|
||||
if sqlite3 "$SRC_WORKFLOWS_DB" ".backup '$DEST_DB'" 2>/dev/null; then
|
||||
size=$(stat -c %s "$DEST_DB" 2>/dev/null || echo "?")
|
||||
log " [OK] workflows.db → $DEST_DB (${size} octets) via sqlite3 .backup"
|
||||
else
|
||||
cp "$SRC_WORKFLOWS_DB" "$DEST_DB"
|
||||
log " [OK fallback] workflows.db → $DEST_DB via cp"
|
||||
fi
|
||||
else
|
||||
cp "$SRC_WORKFLOWS_DB" "$DEST_DB"
|
||||
log " [OK] workflows.db → $DEST_DB via cp (sqlite3 absent)"
|
||||
fi
|
||||
else
|
||||
log " [WARN] workflows.db introuvable : $SRC_WORKFLOWS_DB"
|
||||
fi
|
||||
|
||||
# ------------------------------------------------------------------------------
|
||||
# 2. data/audit/*.jsonl
|
||||
# ------------------------------------------------------------------------------
|
||||
if [ -d "$SRC_AUDIT_DIR" ]; then
|
||||
DEST_AUDIT="$BACKUP_AUDIT_DIR/audit_${DATE_TAG}"
|
||||
mkdir -p "$DEST_AUDIT"
|
||||
copied=0
|
||||
# shellcheck disable=SC2045
|
||||
for f in "$SRC_AUDIT_DIR"/*.jsonl; do
|
||||
[ -f "$f" ] || continue
|
||||
cp "$f" "$DEST_AUDIT/"
|
||||
copied=$((copied + 1))
|
||||
done
|
||||
log " [OK] $copied fichiers audit → $DEST_AUDIT"
|
||||
else
|
||||
log " [WARN] dossier audit introuvable : $SRC_AUDIT_DIR"
|
||||
fi
|
||||
|
||||
# ------------------------------------------------------------------------------
|
||||
# 3. Rétention : suppression des backups > RETENTION_DAYS jours
|
||||
# ------------------------------------------------------------------------------
|
||||
# On retire les fichiers .db du dossier vwb
|
||||
if [ -d "$BACKUP_VWB_DIR" ]; then
|
||||
deleted_db=$(find "$BACKUP_VWB_DIR" -maxdepth 1 -name "workflows_*.db" \
|
||||
-type f -mtime +"$RETENTION_DAYS" -print -delete 2>/dev/null | wc -l)
|
||||
[ "$deleted_db" -gt 0 ] && log " [CLEAN] $deleted_db backup(s) vwb > ${RETENTION_DAYS}j supprimé(s)"
|
||||
fi
|
||||
# On retire les répertoires audit daté
|
||||
if [ -d "$BACKUP_AUDIT_DIR" ]; then
|
||||
deleted_audit=$(find "$BACKUP_AUDIT_DIR" -maxdepth 1 -type d \
|
||||
-name "audit_*" -mtime +"$RETENTION_DAYS" -print -exec rm -rf {} \; 2>/dev/null | wc -l)
|
||||
[ "$deleted_audit" -gt 0 ] && log " [CLEAN] $deleted_audit backup(s) audit > ${RETENTION_DAYS}j supprimé(s)"
|
||||
fi
|
||||
|
||||
log "=== Fin backup ==="
|
||||
exit 0
|
||||
Reference in New Issue
Block a user