snapshot: WIP 5j replay reliability (B1 watchdog + dialog handlers + grounding drift)

Snapshot avant correction du blocage relance Léa (3 incidents 24h: SSH refusé,
polls morts ×2). Point de rollback stable.

Contenu:
- agent_v1/core/executor.py: 5 patchs dialog handling (saveas drift, close_tab
  hotkey fallback, confirm_save Unicode apostrophe, foreground dialog
  recontextualization, runtime_dialog in-loop) + helpers normalize_window_hint,
  requires_post_verify_window_transition
- agent_v1/core/grounding.py: garde drift template fix (fallback_x/y plumbed)
- server_v1/replay_watchdog.py (NEW): orphan watchdog B1, scan 10s timeout 30s
- server_v1/api_stream.py: dispatched_action plumbing, watchdog lifespan,
  metrics endpoint
- server_v1/replay_engine.py: _schedule_retry préserve original_action +
  dispatched_action
- stream_processor.py: gardes _infer_tab_switch_target (no false switch_tab
  on save_as dialog open) + _attach_expected_window_before
- tests/integration: test_replay_watchdog.py (8 cas), test_stream_processor.py
- tests/unit: test_executor_verify_window_guard.py (start_button, close_tab,
  runtime_dialog, post_verify, transition fallbacks)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dom
2026-05-24 16:48:37 +02:00
parent 5ea4960e65
commit 7df51d2c79
47 changed files with 9811 additions and 451 deletions

View File

@@ -158,14 +158,25 @@ class CaptureHandler(BaseHTTPRequestHandler):
"""Capture l'ecran principal et le renvoie en base64 JPEG."""
t0 = time.perf_counter()
try:
import mss
from PIL import Image
from ..vision.capturer import (
capture_foreground_window_image,
capture_screen_image,
)
with mss.mss() as sct:
monitor = sct.monitors[1] # ecran principal
raw = sct.grab(monitor)
img = Image.frombytes("RGB", raw.size, raw.bgra, "raw", "BGRX")
_monitor, img, meta = capture_screen_image()
if img is None:
img, win_meta = capture_foreground_window_image()
meta.update(win_meta)
if img is None:
elapsed_ms = (time.perf_counter() - t0) * 1000
logger.error("Erreur capture : aucun backend exploitable (%s)", meta)
self._send_json(503, {
"error": "capture_unavailable",
"source": meta.get("backend", "unknown"),
"capture_ms": round(elapsed_ms),
"diagnostics": meta,
})
return
# Floutage des données sensibles (conformité AI Act)
if BLUR_SENSITIVE:
@@ -180,15 +191,22 @@ class CaptureHandler(BaseHTTPRequestHandler):
img_b64 = base64.b64encode(buf.getvalue()).decode()
elapsed_ms = (time.perf_counter() - t0) * 1000
logger.info(f"Capture {img.width}x{img.height} en {elapsed_ms:.0f}ms")
logger.info(
"Capture %sx%s via %s en %.0fms",
img.width,
img.height,
meta.get("backend", "unknown"),
elapsed_ms,
)
self._send_json(200, {
"image": img_b64,
"width": img.width,
"height": img.height,
"format": "jpeg",
"source": "windows_live",
"source": meta.get("backend", "windows_live"),
"capture_ms": round(elapsed_ms),
"diagnostics": meta,
})
except Exception as e: