feat(qw2): hook LoopDetector dans api_stream + extension replay_state
Some checks failed
tests / Lint (ruff + black) (push) Successful in 15s
tests / Tests unitaires (sans GPU) (push) Failing after 17s
tests / Tests sécurité (critique) (push) Has been skipped

replay_state enrichi de _screenshot_history (5 dernières images PIL) et
_action_history (5 dernières signatures action).

report_action_result :
- met à jour les deux anneaux après chaque action
- évalue le LoopDetector (singleton lazy avec _clip_embedder serveur)
- si detected → bascule paused_need_help avec pause_reason="loop_detected"
  et bus event lea:loop_detected (signal + evidence)

Tous les chemins d'erreur (embedder absent, OOM, exception) loggent et
laissent le replay continuer — aucun blocage par la couche détection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dom
2026-05-05 23:25:04 +02:00
parent fc01afa59c
commit ca0b436a61
3 changed files with 140 additions and 0 deletions

View File

@@ -3942,6 +3942,82 @@ async def report_action_result(report: ReplayResultReport):
f"— worker VLM autorisé à reprendre"
)
# ===================================================================
# QW2 — LoopDetector : alimentation des anneaux + évaluation
# ===================================================================
# On n'évalue que si le replay est encore "running" — inutile de
# pauser quelque chose de déjà completed/error/paused.
if replay_state["status"] == "running":
# Snapshot image (PIL) dans l'anneau
try:
from PIL import Image
ss_raw = screenshot_after or replay_state.get("last_screenshot")
img = None
if isinstance(ss_raw, str) and ss_raw:
if os.path.isfile(ss_raw):
img = Image.open(ss_raw).copy() # détache du file handle
else:
# Possible base64 — décoder
try:
import base64
import io as _io
img_bytes = base64.b64decode(ss_raw, validate=False)
img = Image.open(_io.BytesIO(img_bytes)).copy()
except Exception:
img = None
if img is not None:
replay_state.setdefault("_screenshot_history", []).append(img)
replay_state["_screenshot_history"] = replay_state["_screenshot_history"][-5:]
except Exception as e:
logger.debug("LoopDetector: snapshot historique échoué: %s", e)
# Snapshot signature de l'action courante
try:
_act_pos = report.actual_position or {}
action_sig = {
"type": (original_action or {}).get("type")
or replay_state.get("_last_action_type", ""),
"x_pct": _act_pos.get("x_pct") if isinstance(_act_pos, dict)
else (original_action or {}).get("x_pct"),
"y_pct": _act_pos.get("y_pct") if isinstance(_act_pos, dict)
else (original_action or {}).get("y_pct"),
}
replay_state.setdefault("_action_history", []).append(action_sig)
replay_state["_action_history"] = replay_state["_action_history"][-5:]
except Exception as e:
logger.debug("LoopDetector: snapshot action_sig échoué: %s", e)
# Évaluation (silencieux si rien)
try:
verdict = _get_loop_detector().evaluate(
replay_state,
screenshots=replay_state.get("_screenshot_history", []),
actions=replay_state.get("_action_history", []),
)
if verdict.detected:
replay_state["status"] = "paused_need_help"
replay_state["pause_reason"] = "loop_detected"
replay_state["pause_message"] = (
f"Léa semble bloquée — {verdict.signal} "
f"(détail: {verdict.evidence})"
)
logger.warning(
"LoopDetector: replay %s mis en pause — signal=%s evidence=%s",
replay_state["replay_id"], verdict.signal, verdict.evidence,
)
# Bus event d'observabilité (logger pattern QW1)
try:
logger.info(
"[BUS] lea:loop_detected replay=%s signal=%s evidence=%s",
replay_state["replay_id"],
verdict.signal,
verdict.evidence,
)
except Exception as _e_bus:
logger.debug("emit lea:loop_detected échec: %s", _e_bus)
except Exception as e:
logger.warning("LoopDetector: évaluation échouée (non bloquant): %s", e)
return {
"status": "recorded",
"action_id": action_id,

View File

@@ -1381,6 +1381,9 @@ def _create_replay_state(
# t2a_decision, etc.). Résolues via templating {{var}} ou {{var.field}}
# dans les paramètres des actions suivantes.
"variables": {},
# QW2 — Anneaux d'historique pour LoopDetector (5 derniers max)
"_screenshot_history": [], # images PIL des N derniers heartbeats (LoopDetector embed à chaque tick)
"_action_history": [], # N dernières actions exécutées (signature)
}