feat(qw1): enrichissement Agent V1 (monitor_index + monitors_geometry) + hook serveur
Some checks failed
tests / Lint (ruff + black) (push) Successful in 16s
tests / Tests unitaires (sans GPU) (push) Failing after 15s
tests / Tests sécurité (critique) (push) Has been skipped

Côté client Agent V1 :
- helpers _get_monitors_geometry() / _get_active_monitor_index() via screeninfo
  (fallback gracieux [] / None si screeninfo absent)
- _enrich_with_monitor_info() ajouté aux payloads dict de capture_dual,
  capture_active_window, et heartbeat_event poussé par main.py
- screeninfo>=0.8 ajouté aux requirements (source + deploy Windows)
- Deploy capturer.py reçoit l'enrichissement de manière additive (pas de
  copie verbatim qui aurait introduit BLUR_SENSITIVE absent côté deploy)

Côté serveur :
- import resolve_target_monitor depuis monitor_router (créé en QW1.1)
- /replay/next : enrichissement action.monitor_resolution avant envoi
  au client (idx, offset_x/y, w, h, source de la décision)
- live_session_manager.add_event : propagation monitor_index +
  monitors_geometry depuis window_capture ET depuis le payload event
  brut (cas heartbeat enrichi sans window/window_title)

Cascade de résolution (cf monitor_router.py) :
1. action.monitor_index (hérité de la session source)
2. session.last_focused_monitor (focus actif vu en dernier heartbeat)
3. composite_fallback (offset 0,0) — backward compat strict

Backward 100% : si geometry vide, fallback composite identique au
comportement actuel mss.monitors[0].

Tests : baseline 89/89 préservée, monitor_router 4/4 OK (total 93/93).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dom
2026-05-05 23:05:44 +02:00
parent fae95c5366
commit 2d71e2a249
7 changed files with 208 additions and 2 deletions

View File

@@ -8,12 +8,73 @@ import os
import time
import logging
import hashlib
from typing import Any, Dict, List, Optional
from PIL import Image, ImageFilter, ImageStat
import mss
from ..config import TARGETED_CROP_SIZE, SCREENSHOT_QUALITY
logger = logging.getLogger(__name__)
# QW1 — détection multi-écrans (fallback gracieux si screeninfo absent)
try:
from screeninfo import get_monitors as _screeninfo_get_monitors
_SCREENINFO_AVAILABLE = True
except ImportError:
_SCREENINFO_AVAILABLE = False
def _get_monitors_geometry() -> List[Dict[str, Any]]:
"""Retourne la liste des monitors physiques avec leurs offsets.
Returns:
List[dict] : [{idx, x, y, w, h, primary}, ...]. Vide si screeninfo
indisponible (le serveur tombera sur fallback composite).
"""
if not _SCREENINFO_AVAILABLE:
return []
try:
monitors = _screeninfo_get_monitors()
return [
{
"idx": i,
"x": int(m.x),
"y": int(m.y),
"w": int(m.width),
"h": int(m.height),
"primary": bool(getattr(m, "is_primary", False)),
}
for i, m in enumerate(monitors)
]
except Exception:
return []
def _get_active_monitor_index() -> Optional[int]:
"""Retourne l'index logique du monitor où se trouve le curseur (focus actif).
Returns:
int ou None si indéterminable.
"""
if not _SCREENINFO_AVAILABLE:
return None
try:
import pyautogui # import paresseux : évite la dépendance dure
cx, cy = pyautogui.position()
for i, m in enumerate(_screeninfo_get_monitors()):
if m.x <= cx < m.x + m.width and m.y <= cy < m.y + m.height:
return i
except Exception:
return None
return None
def _enrich_with_monitor_info(payload: dict) -> dict:
"""Ajoute monitor_index et monitors_geometry au payload (in-place + return)."""
if isinstance(payload, dict):
payload["monitor_index"] = _get_active_monitor_index()
payload["monitors_geometry"] = _get_monitors_geometry()
return payload
class VisionCapturer:
def __init__(self, session_dir: str):
self.session_dir = session_dir
@@ -72,7 +133,12 @@ class VisionCapturer:
# Mise à jour du hash pour le prochain heartbeat
self.last_img_hash = self._compute_quick_hash(img)
return {"full": full_path, "crop": crop_path}
result = {"full": full_path, "crop": crop_path}
# QW1 — enrichissement multi-écrans (additif, fallback gracieux)
_enrich_with_monitor_info(result)
return result
except Exception as e:
logger.error(f"Erreur Dual Capture: {e}")
return {}

View File

@@ -5,6 +5,7 @@ Pillow>=10.0.0 # Crops et processing image
requests>=2.31.0 # Streaming réseau
python-socketio[client]>=5.10,<6.0 # Bus feedback Léa 'lea:*' (compat Flask-SocketIO 5.3.x serveur)
psutil>=5.9.0 # Monitoring CPU/RAM
screeninfo>=0.8 # QW1 — détection des monitors physiques + offsets
pystray>=0.19.5 # Icône Tray UI
plyer>=2.1.0 # Notifications toast natives (remplace PyQt5)