fix(replay): bug TypeError log + flag pré-check OCR off par défaut (démo GHT)
Some checks failed
tests / Lint (ruff + black) (push) Successful in 14s
tests / Tests unitaires (sans GPU) (push) Failing after 13s
tests / Tests sécurité (critique) (push) Has been skipped

Diagnostic post-bench E2E (rapport docs/E2E_TEST_RUN_2026-05-08.md) :

1. BUG SILENCIEUX MAJEUR (api_stream.py:4549) — quand le pré-check OCR
   rejette, mon code de rejet hier soir met x_pct=None / y_pct=None.
   Le log structuré faisait result.get('x_pct', 0):.4f → None:.4f →
   TypeError → réponse "analysis_error" qui MASQUE le vrai motif
   "rejected_text_mismatch". Conséquence : pendant toute la session
   du 7 mai soir, les rejets pré-check ont été silencieusement
   transformés en erreurs analyse → cascade locale Léa V1 → clic au pif.
   Fix : `(result.get('x_pct') or 0):.4f` traite None | None | 0
   uniformément.

2. FLAG ENV pré-check OFF par défaut — le pré-check
   _validate_text_at_position introduit hier soir a 2 défauts
   identifiés par le bench E2E sur 8 click_anchor :
   * radius_px=200 trop petit pour les tabs à 2 tokens (Examens
     cliniques, Synthèse Urgences) — OCR voit un crop tronqué
     "Maquette POC ler en cours Codage Statistiques" qui n'inclut
     pas "Examens" → fuzzy match 1/2 = 50% < seuil 0.60 → REJET.
     À radius 300/400 le mot est inclus → match passe.
   * min_token_ratio=0.60 trop strict pour cibles 2 tokens.

   Solution démo : flag env RPA_ENABLE_TEXT_PRECHECK (défaut "false").
   Le pré-check est désactivé par défaut → retour au comportement
   stable d'avant-hier (hybrid_text_direct ≥ 0.80 utilisé direct,
   exemption drift préservée). Code et fonction _validate_text_at_position
   conservés en place pour reprise post-démo après calibrage radius
   adaptatif (≈ 0.17 × min(screen_w, screen_h)) et token_ratio descendu
   à 0.50.

   Pour ré-activer en dev/test : `RPA_ENABLE_TEXT_PRECHECK=true`
   dans .env.local ou env du service rpa-streaming.

Inclus aussi :
- docs/E2E_TEST_RUN_2026-05-08.md (rapport agent test E2E ~1700 mots)
- tests/e2e/urgence_aiva_demo_expected.yaml (tolérances re-écrites)
- tests/e2e/fixtures/urgence_aiva_demo/live/*.png (8 fixtures
  recapturées headless 1920x1080 pour itérer demain)
- _ocr_inventory.json + _run_resolve_results.json (raw runs)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dom
2026-05-08 10:09:23 +02:00
parent f8dc3c3af4
commit 56e869c467
13 changed files with 970 additions and 82 deletions

View File

@@ -0,0 +1,152 @@
{
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792436.png": {
"found": [],
"size": 415142,
"ocr_dt": 5.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792383.png": {
"found": [],
"size": 412395,
"ocr_dt": 5.2
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792331.png": {
"found": [],
"size": 407364,
"ocr_dt": 5.2
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792278.png": {
"found": [],
"size": 409614,
"ocr_dt": 5.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792225.png": {
"found": [],
"size": 410632,
"ocr_dt": 5.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792172.png": {
"found": [],
"size": 601747,
"ocr_dt": 6.1
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792119.png": {
"found": [],
"size": 524070,
"ocr_dt": 5.6
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773792066.png": {
"found": [],
"size": 495872,
"ocr_dt": 5.2
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791846.png": {
"found": [],
"size": 349923,
"ocr_dt": 4.7
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791581.png": {
"found": [],
"size": 351106,
"ocr_dt": 5.0
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791381.png": {
"found": [],
"size": 469478,
"ocr_dt": 5.7
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791283.png": {
"found": [],
"size": 419376,
"ocr_dt": 5.9
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773791053.png": {
"found": [],
"size": 451460,
"ocr_dt": 7.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773790629.png": {
"found": [],
"size": 402427,
"ocr_dt": 4.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773790480.png": {
"found": [],
"size": 403940,
"ocr_dt": 5.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773789256.png": {
"found": [],
"size": 366536,
"ocr_dt": 5.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773788105.png": {
"found": [],
"size": 414903,
"ocr_dt": 5.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773787561.png": {
"found": [],
"size": 378032,
"ocr_dt": 5.2
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773786527.png": {
"found": [],
"size": 1622254,
"ocr_dt": 5.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773785450.png": {
"found": [],
"size": 353892,
"ocr_dt": 5.2
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773785264.png": {
"found": [],
"size": 407159,
"ocr_dt": 5.5
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773785115.png": {
"found": [],
"size": 375099,
"ocr_dt": 5.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784779.png": {
"found": [],
"size": 1029130,
"ocr_dt": 7.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784695.png": {
"found": [],
"size": 1729091,
"ocr_dt": 5.5
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784592.png": {
"found": [],
"size": 357796,
"ocr_dt": 4.7
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773784539.png": {
"found": [],
"size": 420256,
"ocr_dt": 4.4
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783685.png": {
"found": [],
"size": 558014,
"ocr_dt": 6.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783627.png": {
"found": [],
"size": 582681,
"ocr_dt": 5.2
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783543.png": {
"found": [],
"size": 1208817,
"ocr_dt": 5.3
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/bg_DESKTOP-58D5CAC_windows/shots/heartbeat_1773783484.png": {
"found": [],
"size": 451052,
"ocr_dt": 4.8
}
}

View File

@@ -0,0 +1,34 @@
{
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0001_full.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | 883 | 0 | 0 | [ € | M | @ | *l * | * | Q | 2<1616 * *l | Claude (MCP) | 0 @ + | 0 X | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude"
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0002_full.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | {83 | Q | [ Q | 78777 | @ | X * * | Q | 2 0 0 * * * | Claude (MCP) | 6 @ + | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude | 88 | Al "
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0003_full.png": {
"found": [],
"preview": "0 | Mode veille | Dites < Sortie de veille de l'accès vocal > ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Enregistrement automatique | 8 9 ~ @ | Document3 | Rechercher | DB | X | Claude (MCP) | 6 @ + | Fichier | Accueil | Insertion | Dessin | Conception | Mise en page | Référe"
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0004_full.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Enregistrement automatique | 8 2 ~ @ | Document3 | Rechercher | DB | X | Claude (MCP) | 6 @ + | Fichier | Accueil | Insertion | Dessin | Conception | Mise en page | Référe"
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/shot_0005_full.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Enregistrement automatique | 8 2 ~ @ | Document3 | Rechercher | DB | X | Claude (MCP) | 6 @ + | Fichier | Accueil | Insertion | Dessin | Conception | Mise en page | Référe"
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/heartbeat_1777966309.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | 883 | 0 | 0 | [ € | M | @ | *l * | * | Q | 2<1616 * *l | Claude (MCP) | 0 @ + | 0 X | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude"
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/heartbeat_1777966315.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez | le bouton micro pour activer l'accès vocal. | {83 | Q | [ Q | 78777 | @ | X * * | Q | 2 0 0 * * * | Claude (MCP) | 6 @ + | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude | 88 | Al "
},
"/home/dom/ai/rpa_vision_v3/data/training/live_sessions/DESKTOP-58D5CAC_windows/sess_20260505T093148_6bf7eb/shots/heartbeat_1777966322.png": {
"found": [],
"preview": "0 | Mode veille | Dites | Sortie de veille de l'accès vocal | ou appuyez sur le bouton micro pour activer l'accès vocal. | {83 | Q | [ Q | 78777 | @ | X * * | Q | 2 0 0 * *l | Claude (MCP) | 6 @ + | gitlabs laurinebazin design/?repo-search-query =1 | ABP | X | 8 | = | Claude (MCP) | Claude | 88 | Al"
}
}

View File

@@ -0,0 +1,168 @@
[
{
"by_text": "25003284",
"fixture": "live/landing.png",
"result": {
"resolved": true,
"method": "hybrid_text_direct",
"x_pct": 0.0302734375,
"y_pct": 0.1987847222222222,
"score": 1.0,
"matched_text": "25003284",
"_dt_ms": 1542.8798198699951,
"_recorded": [
0.4928,
0.4512
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Examens cliniques",
"fixture": "live/dossier_motif.png",
"result": {
"resolved": false,
"method": "fallback",
"reason": "analysis_error",
"detail": "unsupported format string passed to NoneType.__format__",
"x_pct": 0.498046875,
"y_pct": 0.4928125,
"_dt_ms": 1420.240879058838,
"_recorded": [
0.498,
0.4928
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Imagerie",
"fixture": "live/dossier_examens-cliniques.png",
"result": {
"resolved": true,
"method": "hybrid_text_direct",
"x_pct": 0.2255859375,
"y_pct": 0.1267361111111111,
"score": 0.8,
"matched_text": "Motif d'admission Examens cliniques Imagerie Notes médicales Synthèse Urgences Codage >",
"_dt_ms": 1372.1542358398438,
"_recorded": [
0.498,
0.4928
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Notes médicales",
"fixture": "live/dossier_imagerie.png",
"result": {
"resolved": true,
"method": "hybrid_text_direct",
"x_pct": 0.22265625,
"y_pct": 0.12586805555555555,
"score": 0.8,
"matched_text": "Motif d'admission Examens cliniques Imagerie Notes médicales Synthèse Urgences Codage >>",
"_dt_ms": 975.5856990814209,
"_recorded": [
0.202,
0.28
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Synthèse Urgences",
"fixture": "live/dossier_notes-medicales.png",
"result": {
"resolved": false,
"method": "fallback",
"reason": "analysis_error",
"detail": "unsupported format string passed to NoneType.__format__",
"x_pct": 0.2705078125,
"y_pct": 0.279375,
"_dt_ms": 1341.4692878723145,
"_recorded": [
0.2705,
0.2794
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Codage",
"fixture": "live/dossier_synthese-urgences.png",
"result": {
"resolved": true,
"method": "hybrid_text_direct",
"x_pct": 0.13916015625,
"y_pct": 0.05381944444444445,
"score": 0.8,
"matched_text": "Patients Planning Dossier en cours Codage Statistiques",
"_dt_ms": 1252.6636123657227,
"_recorded": [
0.3189,
0.2281
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Coller ou saisir le dossier patient",
"fixture": "live/dossier_codage.png",
"result": {
"resolved": false,
"method": "strict_vlm_template_failed",
"reason": "vlm_and_template_all_failed",
"x_pct": 0.0748046875,
"y_pct": 0.44125,
"_dt_ms": 4233.16764831543,
"_recorded": [
0.0748,
0.4412
],
"_screen_size": [
1920,
1080
]
}
},
{
"by_text": "Justification de la décision",
"fixture": "live/dossier_codage.png",
"result": {
"resolved": false,
"method": "strict_vlm_template_failed",
"reason": "vlm_and_template_all_failed",
"x_pct": 0.6482421875,
"y_pct": 0.6228125,
"_dt_ms": 3586.3852500915527,
"_recorded": [
0.6482,
0.6228
],
"_screen_size": [
1920,
1080
]
}
}
]

Binary file not shown.

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 149 KiB