Dom/rpa_vision_v3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: grounding uniquement dans les fenêtres, template pour la taskbar

Browse Source 
Les clics taskbar (sans window_capture.rect) ne passent plus par le
grounding VLM qui trouve "Rechercher" dans l'explorateur au lieu de
la taskbar. Le template matching du crop 80x80 est utilisé à la place.

Règle : fenêtre = grounding, taskbar = template matching.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dom
2026-04-05 21:19:36 +02:00
parent 8a1dfc6e8b
commit 8589e87a13
1 changed files with 4 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
agent_v0/server_v1/api_stream.py
View File

@@ -5230,8 +5230,10 @@ def _resolve_target_sync(
# ---------------------------------------------------------------
by_text_source = target_spec.get("by_text_source", "")
if by_text_strict and by_text_source in ("ocr", "vlm"):
# Texte visible (OCR ou lu par gemma4) → grounding VLM direct
has_window = bool(target_spec.get("window_capture", {}).get("rect"))
if by_text_strict and by_text_source in ("ocr", "vlm") and has_window:
# Texte visible DANS une fenêtre → grounding VLM sur fenêtre croppée
grounding_result = _resolve_by_grounding(
screenshot_path=screenshot_path,
target_spec=target_spec,