Dom/rpa_vision_v3
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: replay E2E fonctionnel — 25/25 actions, 0 retries, SomEngine via serveur

Browse Source 
Validé sur PC Windows (DESKTOP-58D5CAC, 2560x1600) :
- 8 clics résolus visuellement (1 anchor_template, 1 som_text_match, 6 som_vlm)
- Score moyen 0.75, temps moyen 1.6s
- Texte tapé correctement (bonjour, test word, date, email)
- 0 retries, 2 actions non vérifiées (OK)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dom
2026-03-31 14:04:41 +02:00
parent 5e0b53cfd1
commit a7de6a488b
79542 changed files with 6091757 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
data/training/live_sessions/sess_20260314T161831_d1f86a/live_events.jsonl Normal file
View File

@@ -0,0 +1,9 @@
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501539.2090852, "event": {"type": "heartbeat", "image": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_heartbeat.png", "timestamp": 1773501511.9880414}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501542.2611043, "event": {"type": "heartbeat", "image": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_heartbeat.png", "timestamp": 1773501511.9880414}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501546.482679, "event": {"type": "heartbeat", "image": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_heartbeat.png", "timestamp": 1773501511.9880414}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501552.5135496, "event": {"type": "heartbeat", "image": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_heartbeat.png", "timestamp": 1773501511.9880414}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501581.9214787, "event": {"type": "window_focus_change", "from": null, "to": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "timestamp": 1773501511.8326368, "window": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "screenshot_context": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_focus_change.png"}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501584.941769, "event": {"type": "window_focus_change", "from": null, "to": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "timestamp": 1773501511.8326368, "window": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "screenshot_context": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_focus_change.png"}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501588.9629693, "event": {"type": "window_focus_change", "from": null, "to": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "timestamp": 1773501511.8326368, "window": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "screenshot_context": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501511_focus_change.png"}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501601.0951326, "event": {"type": "mouse_click", "button": "left", "pos": [852, 1558], "timestamp": 1773501514.298243, "window": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "screenshot_id": "shot_0001", "vision_info": {"full": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\shot_0001_full.png", "crop": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\shot_0001_crop.png"}}}
{"session_id": "sess_20260314T161831_d1f86a", "timestamp": 1773501608.199802, "event": {"type": "window_focus_change", "from": {"title": "RPA Vision V3 - Agent V1", "app_name": "WindowsTerminal.exe"}, "to": {"title": "Rechercher", "app_name": "SearchHost.exe"}, "timestamp": 1773501514.560857, "window": {"title": "Rechercher", "app_name": "SearchHost.exe"}, "screenshot_context": "C:\\rpa_vision\\agent_v1\\sessions\\sess_20260314T161831_d1f86a\\shots\\context_1773501514_focus_change.png"}}