fix: replay visuel — fallback coordonnées bbox si template matching échoue
- Le proxy injecte x_pct/y_pct depuis le centre du bbox de l'ancre - Si le visual resolve timeout → clic aux coordonnées bbox (pas à 0,0) - Lookup replay_states par machine_id (premier replay fonctionne) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -921,10 +921,23 @@ def execute_windows():
|
||||
if vwb_type in _ANCHOR_CLICK_TYPES:
|
||||
anchor_id = action.get('anchor_id')
|
||||
if anchor_id:
|
||||
anchor_meta = _load_anchor_metadata(anchor_id)
|
||||
|
||||
# Calculer les coordonnées du centre du bbox (fallback si visual échoue)
|
||||
if anchor_meta:
|
||||
bbox = anchor_meta.get('bounding_box', {})
|
||||
orig = anchor_meta.get('original_size', {})
|
||||
orig_w = orig.get('width', 1920)
|
||||
orig_h = orig.get('height', 1080)
|
||||
if bbox.get('x') is not None and orig_w > 0 and orig_h > 0:
|
||||
cx = (bbox['x'] + bbox.get('width', 0) / 2) / orig_w
|
||||
cy = (bbox['y'] + bbox.get('height', 0) / 2) / orig_h
|
||||
action['x_pct'] = round(cx, 4)
|
||||
action['y_pct'] = round(cy, 4)
|
||||
|
||||
# Tenter aussi le visual_mode (template matching)
|
||||
anchor_b64 = _load_anchor_image_b64(anchor_id)
|
||||
if anchor_b64:
|
||||
# Charger les métadonnées pour le bounding_box de référence
|
||||
anchor_meta = _load_anchor_metadata(anchor_id)
|
||||
target_spec = {
|
||||
'anchor_image_base64': anchor_b64,
|
||||
'anchor_id': anchor_id,
|
||||
|
||||
Reference in New Issue
Block a user