fix(replay): bug TypeError log + flag pré-check OCR off par défaut (démo GHT)
Diagnostic post-bench E2E (rapport docs/E2E_TEST_RUN_2026-05-08.md) :
1. BUG SILENCIEUX MAJEUR (api_stream.py:4549) — quand le pré-check OCR
rejette, mon code de rejet hier soir met x_pct=None / y_pct=None.
Le log structuré faisait result.get('x_pct', 0):.4f → None:.4f →
TypeError → réponse "analysis_error" qui MASQUE le vrai motif
"rejected_text_mismatch". Conséquence : pendant toute la session
du 7 mai soir, les rejets pré-check ont été silencieusement
transformés en erreurs analyse → cascade locale Léa V1 → clic au pif.
Fix : `(result.get('x_pct') or 0):.4f` traite None | None | 0
uniformément.
2. FLAG ENV pré-check OFF par défaut — le pré-check
_validate_text_at_position introduit hier soir a 2 défauts
identifiés par le bench E2E sur 8 click_anchor :
* radius_px=200 trop petit pour les tabs à 2 tokens (Examens
cliniques, Synthèse Urgences) — OCR voit un crop tronqué
"Maquette POC ler en cours Codage Statistiques" qui n'inclut
pas "Examens" → fuzzy match 1/2 = 50% < seuil 0.60 → REJET.
À radius 300/400 le mot est inclus → match passe.
* min_token_ratio=0.60 trop strict pour cibles 2 tokens.
Solution démo : flag env RPA_ENABLE_TEXT_PRECHECK (défaut "false").
Le pré-check est désactivé par défaut → retour au comportement
stable d'avant-hier (hybrid_text_direct ≥ 0.80 utilisé direct,
exemption drift préservée). Code et fonction _validate_text_at_position
conservés en place pour reprise post-démo après calibrage radius
adaptatif (≈ 0.17 × min(screen_w, screen_h)) et token_ratio descendu
à 0.50.
Pour ré-activer en dev/test : `RPA_ENABLE_TEXT_PRECHECK=true`
dans .env.local ou env du service rpa-streaming.
Inclus aussi :
- docs/E2E_TEST_RUN_2026-05-08.md (rapport agent test E2E ~1700 mots)
- tests/e2e/urgence_aiva_demo_expected.yaml (tolérances re-écrites)
- tests/e2e/fixtures/urgence_aiva_demo/live/*.png (8 fixtures
recapturées headless 1920x1080 pour itérer demain)
- _ocr_inventory.json + _run_resolve_results.json (raw runs)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
152
tests/e2e/fixtures/urgence_aiva_demo/_ocr_inventory.json
Normal file
@@ -0,0 +1,152 @@
|
||||
{
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792436.png": {
|
||||
"found": [],
|
||||
"size": 415142,
|
||||
"ocr_dt": 5.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792383.png": {
|
||||
"found": [],
|
||||
"size": 412395,
|
||||
"ocr_dt": 5.2
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792331.png": {
|
||||
"found": [],
|
||||
"size": 407364,
|
||||
"ocr_dt": 5.2
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792278.png": {
|
||||
"found": [],
|
||||
"size": 409614,
|
||||
"ocr_dt": 5.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792225.png": {
|
||||
"found": [],
|
||||
"size": 410632,
|
||||
"ocr_dt": 5.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792172.png": {
|
||||
"found": [],
|
||||
"size": 601747,
|
||||
"ocr_dt": 6.1
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792119.png": {
|
||||
"found": [],
|
||||
"size": 524070,
|
||||
"ocr_dt": 5.6
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792066.png": {
|
||||
"found": [],
|
||||
"size": 495872,
|
||||
"ocr_dt": 5.2
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791846.png": {
|
||||
"found": [],
|
||||
"size": 349923,
|
||||
"ocr_dt": 4.7
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791581.png": {
|
||||
"found": [],
|
||||
"size": 351106,
|
||||
"ocr_dt": 5.0
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791381.png": {
|
||||
"found": [],
|
||||
"size": 469478,
|
||||
"ocr_dt": 5.7
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791283.png": {
|
||||
"found": [],
|
||||
"size": 419376,
|
||||
"ocr_dt": 5.9
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791053.png": {
|
||||
"found": [],
|
||||
"size": 451460,
|
||||
"ocr_dt": 7.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773790629.png": {
|
||||
"found": [],
|
||||
"size": 402427,
|
||||
"ocr_dt": 4.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773790480.png": {
|
||||
"found": [],
|
||||
"size": 403940,
|
||||
"ocr_dt": 5.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773789256.png": {
|
||||
"found": [],
|
||||
"size": 366536,
|
||||
"ocr_dt": 5.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773788105.png": {
|
||||
"found": [],
|
||||
"size": 414903,
|
||||
"ocr_dt": 5.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773787561.png": {
|
||||
"found": [],
|
||||
"size": 378032,
|
||||
"ocr_dt": 5.2
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773786527.png": {
|
||||
"found": [],
|
||||
"size": 1622254,
|
||||
"ocr_dt": 5.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773785450.png": {
|
||||
"found": [],
|
||||
"size": 353892,
|
||||
"ocr_dt": 5.2
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773785264.png": {
|
||||
"found": [],
|
||||
"size": 407159,
|
||||
"ocr_dt": 5.5
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773785115.png": {
|
||||
"found": [],
|
||||
"size": 375099,
|
||||
"ocr_dt": 5.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784779.png": {
|
||||
"found": [],
|
||||
"size": 1029130,
|
||||
"ocr_dt": 7.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784695.png": {
|
||||
"found": [],
|
||||
"size": 1729091,
|
||||
"ocr_dt": 5.5
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784592.png": {
|
||||
"found": [],
|
||||
"size": 357796,
|
||||
"ocr_dt": 4.7
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784539.png": {
|
||||
"found": [],
|
||||
"size": 420256,
|
||||
"ocr_dt": 4.4
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783685.png": {
|
||||
"found": [],
|
||||
"size": 558014,
|
||||
"ocr_dt": 6.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783627.png": {
|
||||
"found": [],
|
||||
"size": 582681,
|
||||
"ocr_dt": 5.2
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783543.png": {
|
||||
"found": [],
|
||||
"size": 1208817,
|
||||
"ocr_dt": 5.3
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783484.png": {
|
||||
"found": [],
|
||||
"size": 451052,
|
||||
"ocr_dt": 4.8
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0001_full.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | 883 | 0 | 0 | [ € | M | @ | *l * | * | Q | 2<1616 * *l | Claude (MCP) | 0 @ + | 0 X | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude"
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0002_full.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | {83 | Q | [ Q | 78777 | @ | X * * | Q | 2 0 0 * * * | Claude (MCP) | 6 @ + | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude | 88 | Al "
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0003_full.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites < Sortie de veille de l'accès vocal > ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Enregistrement automatique | 8 9 ~ @ | Document3 | Rechercher | DB | X | Claude (MCP) | 6 @ + | Fichier | Accueil | Insertion | Dessin | Conception | Mise en page | Référe"
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0004_full.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Enregistrement automatique | 8 2 ~ @ | Document3 | Rechercher | DB | X | Claude (MCP) | 6 @ + | Fichier | Accueil | Insertion | Dessin | Conception | Mise en page | Référe"
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0005_full.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Enregistrement automatique | 8 2 ~ @ | Document3 | Rechercher | DB | X | Claude (MCP) | 6 @ + | Fichier | Accueil | Insertion | Dessin | Conception | Mise en page | Référe"
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/heartbeat_1777966309.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | 883 | 0 | 0 | [ € | M | @ | *l * | * | Q | 2<1616 * *l | Claude (MCP) | 0 @ + | 0 X | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude"
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/heartbeat_1777966315.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | {83 | Q | [ Q | 78777 | @ | X * * | Q | 2 0 0 * * * | Claude (MCP) | 6 @ + | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude | 88 | Al "
|
||||
},
|
||||
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/heartbeat_1777966322.png": {
|
||||
"found": [],
|
||||
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Q | [ Q | 78777 | @ | X * * | Q | 2 0 0 * *l | Claude (MCP) | 6 @ + | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude | 88 | Al"
|
||||
}
|
||||
}
|
||||
168
tests/e2e/fixtures/urgence_aiva_demo/_run_resolve_results.json
Normal file
@@ -0,0 +1,168 @@
|
||||
[
|
||||
{
|
||||
"by_text": "25003284",
|
||||
"fixture": "live/landing.png",
|
||||
"result": {
|
||||
"resolved": true,
|
||||
"method": "hybrid_text_direct",
|
||||
"x_pct": 0.0302734375,
|
||||
"y_pct": 0.1987847222222222,
|
||||
"score": 1.0,
|
||||
"matched_text": "25003284",
|
||||
"_dt_ms": 1542.8798198699951,
|
||||
"_recorded": [
|
||||
0.4928,
|
||||
0.4512
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Examens cliniques",
|
||||
"fixture": "live/dossier_motif.png",
|
||||
"result": {
|
||||
"resolved": false,
|
||||
"method": "fallback",
|
||||
"reason": "analysis_error",
|
||||
"detail": "unsupported format string passed to NoneType.__format__",
|
||||
"x_pct": 0.498046875,
|
||||
"y_pct": 0.4928125,
|
||||
"_dt_ms": 1420.240879058838,
|
||||
"_recorded": [
|
||||
0.498,
|
||||
0.4928
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Imagerie",
|
||||
"fixture": "live/dossier_examens-cliniques.png",
|
||||
"result": {
|
||||
"resolved": true,
|
||||
"method": "hybrid_text_direct",
|
||||
"x_pct": 0.2255859375,
|
||||
"y_pct": 0.1267361111111111,
|
||||
"score": 0.8,
|
||||
"matched_text": "Motif d'admission Examens cliniques Imagerie Notes médicales Synthèse Urgences Codage >",
|
||||
"_dt_ms": 1372.1542358398438,
|
||||
"_recorded": [
|
||||
0.498,
|
||||
0.4928
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Notes médicales",
|
||||
"fixture": "live/dossier_imagerie.png",
|
||||
"result": {
|
||||
"resolved": true,
|
||||
"method": "hybrid_text_direct",
|
||||
"x_pct": 0.22265625,
|
||||
"y_pct": 0.12586805555555555,
|
||||
"score": 0.8,
|
||||
"matched_text": "Motif d'admission Examens cliniques Imagerie Notes médicales Synthèse Urgences Codage >>",
|
||||
"_dt_ms": 975.5856990814209,
|
||||
"_recorded": [
|
||||
0.202,
|
||||
0.28
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Synthèse Urgences",
|
||||
"fixture": "live/dossier_notes-medicales.png",
|
||||
"result": {
|
||||
"resolved": false,
|
||||
"method": "fallback",
|
||||
"reason": "analysis_error",
|
||||
"detail": "unsupported format string passed to NoneType.__format__",
|
||||
"x_pct": 0.2705078125,
|
||||
"y_pct": 0.279375,
|
||||
"_dt_ms": 1341.4692878723145,
|
||||
"_recorded": [
|
||||
0.2705,
|
||||
0.2794
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Codage",
|
||||
"fixture": "live/dossier_synthese-urgences.png",
|
||||
"result": {
|
||||
"resolved": true,
|
||||
"method": "hybrid_text_direct",
|
||||
"x_pct": 0.13916015625,
|
||||
"y_pct": 0.05381944444444445,
|
||||
"score": 0.8,
|
||||
"matched_text": "Patients Planning Dossier en cours Codage Statistiques",
|
||||
"_dt_ms": 1252.6636123657227,
|
||||
"_recorded": [
|
||||
0.3189,
|
||||
0.2281
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Coller ou saisir le dossier patient",
|
||||
"fixture": "live/dossier_codage.png",
|
||||
"result": {
|
||||
"resolved": false,
|
||||
"method": "strict_vlm_template_failed",
|
||||
"reason": "vlm_and_template_all_failed",
|
||||
"x_pct": 0.0748046875,
|
||||
"y_pct": 0.44125,
|
||||
"_dt_ms": 4233.16764831543,
|
||||
"_recorded": [
|
||||
0.0748,
|
||||
0.4412
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"by_text": "Justification de la décision",
|
||||
"fixture": "live/dossier_codage.png",
|
||||
"result": {
|
||||
"resolved": false,
|
||||
"method": "strict_vlm_template_failed",
|
||||
"reason": "vlm_and_template_all_failed",
|
||||
"x_pct": 0.6482421875,
|
||||
"y_pct": 0.6228125,
|
||||
"_dt_ms": 3586.3852500915527,
|
||||
"_recorded": [
|
||||
0.6482,
|
||||
0.6228
|
||||
],
|
||||
"_screen_size": [
|
||||
1920,
|
||||
1080
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
BIN
tests/e2e/fixtures/urgence_aiva_demo/live/dossier_codage.png
Normal file
|
After Width: | Height: | Size: 140 KiB |
|
After Width: | Height: | Size: 140 KiB |
BIN
tests/e2e/fixtures/urgence_aiva_demo/live/dossier_imagerie.png
Normal file
|
After Width: | Height: | Size: 66 KiB |
BIN
tests/e2e/fixtures/urgence_aiva_demo/live/dossier_motif.png
Normal file
|
After Width: | Height: | Size: 140 KiB |
|
After Width: | Height: | Size: 140 KiB |
|
After Width: | Height: | Size: 140 KiB |
BIN
tests/e2e/fixtures/urgence_aiva_demo/live/landing.png
Normal file
|
After Width: | Height: | Size: 149 KiB |