rpa_vision_v3/docs/RAPPORT_CONFORMITE_AI_ACT.md

{"parentUuid":"27762a0f-3d42-438c-81c7-ff3201308904","isSidechain":true,"agentId":"a89e576514eb3a4ec","message":{"model":"claude-opus-4-6","id":"msg_014RyauXc2qaRQ25JJuargBZ","type":"message","role":"assistant","content":[{"type":"text","text":"I now have comprehensive information from both web research and codebase analysis. Here is the full compliance report.\n\n---\n\n# EU AI Act Compliance Assessment Report -- RPA Vision V3\n\nDate: 18 March 2026\nRegulation: Regulation (EU) 2024/1689 (EU Artificial Intelligence Act)\nSystem assessed: RPA Vision V3 -- Visual RPA with AI-based UI understanding\nAssessor: Claude Opus 4.6 (automated compliance research)\n\n---\n\n## 1. PRELIMINARY QUESTION: Does RPA Vision V3 Qualify as an "AI System"?\n\nAnswer: YES, unambiguously.\n\nArticle 3(1) defines an AI system as: "a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments."\n\nWhile traditional rule-based RPA falls outside this definition, RPA Vision V3 is fundamentally different. It uses:\n- CLIP (contrastive language-image pre-training) for semantic UI understanding and embedding\n- Qwen3-VL (vision-language model) for visual inference on screenshots\n- FAISS for similarity search and state matching\n- OWLv2 for object detection\n- Machine learning inference to match screen states, resolve targets semantically, and make decisions about workflow progression\n\nThe system infers how to interact with UIs from visual observations, operates with autonomy during replay mode, and exhibits adaptiveness through its learning pipeline. This clearly meets the AI system definition.\n\n---\n\n## 2. RISK CLASSIFICATION\n\n### 2.1 Primary Classification: HIGH-RISK (Annex III, Point 4(b))\n\nAnnex III, Point 4(b) classifies as high-risk: "AI intended to be used for making decisions on [...] task allocation and for monitoring and evaluating performance and behavior of persons in [work-related] relationships."\n\nRPA Vision V3 triggers this because:\n- It observes and records worker behavior (screenshots, mouse clicks, keyboard inputs) during the Shadow/learning phase\n- It captures behavioral patterns of how employees perform tasks\n- In autonomous mode, it takes over task execution, which constitutes AI-driven task allocation/execution\n- The system's learning pipeline creates a behavioral model of worker actions\n- It operates in the workplace (hospitals, clinics) targeting healthcare professionals and administrative staff\n\n### 2.2 Does the Article 6(3) Exception Apply?\n\nArticle 6(3) allows an Annex III system to be classified as non-high-risk if it does not pose a significant risk of harm. However, Article 6(3) last paragraph states: "an AI system referred to in Annex III shall always be considered to be high-risk where the AI system performs profiling of natural persons."\n\nRPA Vision V3 performs profiling under the GDPR definition (Article 4(4) GDPR, referenced by the AI Act): it processes personal data through automated means to analyze or predict aspects related to a person's behavior and work performance. The system captures exactly how a person works -- what they click, what they type, what screens they visit, and in what order.\n\nConclusion: The Article 6(3) exception does NOT apply. The system is definitively high-risk.\n\n### 2.3 Additional High-Risk Triggers\n\n| Annex III Category | Applicability | Rationale |\n|---|---|---|\n| 4(b) Employment/worker monitoring | YES | Core function: observe and learn from worker behavior |\n| 5(a) Essential public services (healthcare) | POSSIBLE | If the system is used to process/manage patient-related workflows, it touches healthcare service delivery |\n| 1 Biometrics | NO | No biometric identification/categorization |\n\n### 2.4 Prohibited Practices Check (Article 5)\n\n| Prohibition | Risk | Assessment |\n|---|---|---|\n| Art. 5(1)(f): Emotion recognition in workplace | LOW but must verify | The system captures screenshots and analyzes UI elements, not facial expressions or emotions. However, if screenshots contain webcam feeds or the VLM is ever prompted to infer user emotional state, this would be PROHIBITED. Currently not implemented, but no technical safeguard prevents it. |\n| Art. 5(1)(a): Subliminal manipulation | NO | System does not manipulate user behavior |\n| Art. 5(1)(c): Social scoring | NO | No scoring of individuals |\n| Art. 5(1)(d): Predictive policing | NO | Not applicable |\n\n---\n\n## 3. COMPLIANCE OBLIGATIONS FOR HIGH-RISK AI SYSTEMS\n\n### 3.1 Required by the AI Act (Provider Obligations -- Articles 8-25)\n\n| Requirement | Article | Current State | Gap |\n|---|---|---|---|\n| Risk Management System | Art. 9 | ABSENT | No formal risk management process documented. No continuous lifecycle risk assessment. |\n| Data Governance | Art. 10 | ABSENT | No data governance framework. Screenshots containing patient data are stored without classification. No bias detection. |\n| Technical Documentation | Art. 11 + Annex IV | ABSENT | No Annex IV-compliant technical documentation exists. The codebase has inline docstrings but no formal regulatory documentation. |\n| Record-keeping / Logging | Art. 12 | PARTIAL | Python logging module is used throughout, but there is no structured audit trail, no immutable logs, no mandatory 6-month retention policy for AI decisions. |\n| Transparency & User Information | Art. 13 | PARTIAL | The "Lea" chatbot says "Je suis Lea" but does not explicitly disclose it is an AI system. No documentation provided to deployers about capabilities, limitations, or risks. |\n| Human Oversight | Art. 14 | PARTIAL | The system has Shadow/Copilot/Autonomous modes suggesting graduated autonomy. However, no formal "stop button", no human-in-the-loop override mechanism is documented, no ability for a human to systematically review before actions are executed in autonomous mode. |\n| Accuracy, Robustness, Cybersecurity | Art. 15 | PARTIAL | Match confidence scores exist (match_confidence in StepResult), but no formal accuracy benchmarks, no adversarial testing, no cybersecurity assessment. Default security keys in config (dev_secret_key_not_for_production). CORS set to * by default. |\n| Quality Management System | Art. 17 | ABSENT | No QMS. No documented development procedures, no validation processes, no incident reporting framework. |\n| EU Declaration of Conformity | Art. 47 | ABSENT | No conformity declaration exists. |\n| CE Marking | Art. 48 | ABSENT | No CE marking process. |\n| Registration in EU Database | Art. 49 | ABSENT | Not registered. |\n| Post-market Monitoring | Art. 72 | ABSENT | No post-market monitoring plan. |\n\n### 3.2 Deployer Obligations (Article 26 -- for hospitals/clinics using the system)\n\n| Requirement | Status | Gap |\n|---|---|---|\n| Use according to instructions | NO instructions exist | No user manual or instructions for use compliant with Art. 13 |\n| Human oversight by competent persons | NOT FORMALIZED | No training program, no competency requirements defined |\n| Monitor operation | PARTIAL | Dashboard exists (port 5001) but lacks compliance-oriented monitoring |\n| Inform workers' representatives | NOT ADDRESSED | No guidance provided to deployers about worker notification obligations |\n| Inform affected workers | NOT ADDRESSED | Workers being observed are not systematically informed the AI is recording and learning from their actions |\n| DPIA (GDPR Art. 35) | NOT PERFORMED | Mandatory given the processing of health data and employee monitoring |\n| Keep logs for 6 months minimum | NOT GUARANTEED | SessionStorage has retention_days=1 by default -- this VIOLATES the minimum 6-month retention requirement |\n\n---\n\n## 4. GDPR INTERSECTION -- CRITICAL ISSUES\n\n### 4.1 Personal Data Processing\n\nRPA Vision V3 captures and processes:\n- Screenshots containing potentially: patient names, medical records, diagnoses, treatments, prescriptions, personal addresses, phone numbers, social security numbers, financial data\n- Keyboard inputs (keystrokes) which may include passwords, medical notes, personal communications\n- Mouse movements and click patterns which constitute behavioral data\n\n### 4.2 GDPR Classification\n\n| Data Category | GDPR Classification | Impact |\n|---|---|---|\n| Patient medical records in screenshots | Special category data (Art. 9 GDPR -- health data) | Requires explicit consent or specific legal basis. Additional safeguards mandatory. |\n| Employee behavior patterns | Personal data (Art. 4(1) GDPR) + Profiling (Art. 4(4) GDPR) | Subject to GDPR profiling rules. Right to object under Art. 21. |\n| Keyboard inputs | Personal data, potentially special category if medical notes | High sensitivity. May capture passwords. |\n\n### 4.3 Legal Basis Analysis\n\nThe system currently has no documented legal basis for processing any of this data. Potential bases:\n\n- Art. 6(1)(a) Consent: Would require explicit, informed, freely given consent from both employees being observed AND patients whose data appears in screenshots. In an employment context, consent is generally NOT considered freely given (power imbalance).\n- Art. 6(1)(f) Legitimate interest: Requires a documented Legitimate Interest Assessment (LIA). The employer's interest in automation must be balanced against employee privacy rights and patient data protection.\n- Art. 9(2) Health data: Requires one of the specific Art. 9(2) exceptions. Processing patient health data visible in screenshots for RPA training purposes does not easily fit any exception.\n\n### 4.4 Current Data Protection Measures -- INSUFFICIENT\n\n| Measure | Current State | Required |\n|---|---|---|\n| Anonymization of screenshots | Optional (anonymize=False by default in capture_dual()) -- only Gaussian blur on crop, not on full screenshot | Mandatory for screenshots containing health data. Should be default-on with opt-out requiring justification. |\n| Encryption at rest | Mentioned in code comments but not implemented | Screenshots containing patient data must be encrypted |\n| Data minimization | NOT IMPLEMENTED | System captures full screenshots. No mechanism to mask/redact sensitive areas before storage. |\n| Purpose limitation | NOT DOCUMENTED | Data captured for "learning" is stored without purpose limitation controls |\n| Storage limitation | retention_days=1 default, but AI Act requires 6-month log retention | Conflicting requirements need resolution through tiered retention |\n| Right to erasure | NOT IMPLEMENTED | No mechanism for a patient or employee to request deletion of their data from training sets, FAISS indexes, or learned workflows |\n\n---\n\n## 5. TRANSPARENCY OBLIGATIONS (Article 50)\n\n### 5.1 Chatbot "Lea" -- Article 50(1)\n\nArticle 50(1) requires: "Providers shall ensure that AI systems intended to interact directly with natural persons are designed and developed in such a way that the natural persons concerned are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use."\n\nCurrent state: "Lea" introduces herself as "Bonjour ! Je suis Lea. Je peux apprendre vos taches repetitives et les refaire a votre place." This is insufficient because:\n- It does not explicitly state she is an AI system / artificial intelligence\n- A user unfamiliar with the product might believe "Lea" is a human assistant\n- The disclosure must be unambiguous: "Je suis Lea, une assistante basee sur l'intelligence artificielle"\n\n### 5.2 Autonomous Execution Disclosure\n\nWhen the system operates autonomously (replaying workflows), users whose computer is being controlled should be clearly informed that an AI system is performing actions on their screen.\n\n---\n\n## 6. COMPLIANCE TIMELINE\n\n| Date | Obligation | Status |\n|---|---|---|\n| 2 Feb 2025 (PAST) | Prohibited AI practices in force | Must verify no emotion recognition in screenshots |\n| 2 Feb 2025 (PAST) | AI literacy obligations (Art. 4) | No AI literacy training program exists |\n| 2 Aug 2025 (PAST) | GPAI model obligations | Applies if Qwen3-VL or CLIP are considered GPAI -- primarily a concern for the model providers (Meta, OpenAI), not for deployers |\n| 2 Aug 2026 (5 months away) | High-risk AI system obligations FULLY APPLICABLE | CRITICAL DEADLINE: All Articles 8-15, 17, Annex IV documentation, risk management, conformity assessment, EU database registration |\n| 2 Aug 2026 | Transparency obligations (Art. 50) | Lea chatbot disclosure must be compliant |\n| 2 Aug 2027 | Final deadline for certain existing systems | Grace period for some legacy high-risk systems already on market |\n\n---\n\n## 7. COMPLIANCE GAPS SUMMARY (Prioritized)\n\n### CRITICAL (must fix before August 2026)\n\n1. No Risk Management System (Art. 9) -- Must establish a documented, continuous risk management process\n2. No Technical Documentation (Art. 11 + Annex IV) -- Must produce comprehensive documentation before market placement\n3. No DPIA (GDPR Art. 35) -- Mandatory for AI processing health data and monitoring employees\n4. No legal basis for data processing -- Must establish and document GDPR-compliant legal bases\n5. Screenshots with patient data are unprotected -- Default anonymization OFF, no encryption at rest, no data minimization\n6. Log retention of 1 day violates the 6-month minimum (Art. 12 + Art. 26(6))\n7. No Quality Management System (Art. 17)\n8. No EU Declaration of Conformity (Art. 47)\n9. No registration in EU database (Art. 49)\n\n### HIGH (should fix before August 2026)\n\n10. Worker notification gap -- No mechanism to inform employees they are being observed by an AI\n11. No human oversight mechanism for autonomous mode -- No documented kill switch / override / review-before-execute\n12. Lea chatbot does not disclose AI nature clearly (Art. 50)\n13. No post-market monitoring plan (Art. 72)\n14. No incident reporting framework\n15. Security hardening needed -- default dev keys, CORS *, no TLS enforcement\n\n### MEDIUM (recommended)\n\n16. No bias detection or fairness testing (Art. 10)\n17. No accuracy benchmarks published (Art. 15)\n18. No adversarial robustness testing (Art. 15)\n19. No formal instructions for deployers (Art. 13)\n20. No data subject rights implementation (GDPR Art. 15-22)\n\n---\n\n## 8. REQUIRED ACTIONS -- ROADMAP\n\n### Phase 1: Immediate (March-April 2026)\n\n1. Perform a Data Protection Impact Assessment (DPIA) -- This is overdue given the system processes health data. Engage a DPO.\n2. Document the legal basis for all personal data processing activities.\n3. Enable screenshot anonymization by default -- change anonymize=False to anonymize=True in capturer.py. Implement full-screenshot redaction (not just crop blur).\n4. Fix log retention -- increase minimum to 6 months for AI decision logs while implementing tiered retention (short for raw screenshots with patient data, long for AI audit logs).\n5. Add explicit AI disclosure to Lea: "Je suis Lea, une assistante propulsee par l'intelligence artificielle."\n6. Add worker notification feature -- Before recording begins, display clear notification that AI is observing.\n\n### Phase 2: Documentation (April-June 2026)\n\n7. Draft Annex IV Technical Documentation including: system description, intended purpose, design specifications, data requirements, risk management, testing results, performance metrics.\n8. Establish Risk Management System (Art. 9) -- continuous, documented, covering the full lifecycle.\n9. Create Quality Management System (Art. 17) -- development procedures, testing, validation, incident reporting.\n10. Write Instructions for Use for deployers (hospital IT departments).\n11. Create Human Oversight documentation -- define who oversees the system, what competencies are needed, how to intervene.\n\n### Phase 3: Conformity (June-August 2026)\n\n12. Conduct conformity assessment (self-assessment for Annex III systems under Art. 43).\n13. Draft EU Declaration of Conformity (Art. 47).\n14. Register in the EU database (Art. 49).\n15. Implement post-market monitoring plan (Art. 72).\n16. Conduct accuracy and robustness testing with documented benchmarks.\n17. Implement encryption at rest for all stored screenshots.\n18. Implement data subject rights (access, erasure, objection to profiling).\n\n---\n\n## 9. KEY RECOMMENDATIONS\n\n1. Engage a specialized AI Act compliance consultant or law firm -- The intersection of AI Act + GDPR + healthcare data + worker monitoring creates a complex regulatory landscape that requires legal expertise.\n\n2. Consider appointing a "responsible AI officer" -- Given the high-risk classification, having a designated person accountable for AI compliance is strongly recommended.\n\n3. Implement Privacy by Design -- The current architecture stores raw screenshots by default. Redesign to:\n - Detect and redact PII/PHI in screenshots before storage (using the VLM itself)\n - Store only anonymized versions for training\n - Keep identifiable data only for the minimum necessary period\n - Encrypt all data at rest\n\n4. Consider the SME simplified documentation -- Article 11 and Annex IV allow SMEs to use a simplified technical documentation form. The European Commission is expected to publish this template.\n\n5. Document the "no emotion recognition" guarantee -- Add a technical safeguard (code-level) that prevents the VLM from being prompted with emotion-detection queries on workplace screenshots. This eliminates any Art. 5(1)(f) risk.\n\n6. Separate AI logs from personal data logs -- Create a tiered retention system:\n - AI decision audit logs: 6+ months (Art. 12/26 compliance)\n - Raw screenshots with personal data: minimum necessary, encrypted, access-controlled\n - Anonymized training data: longer retention acceptable\n\n7. The "on-premise / no cloud" architecture is a positive -- Local processing via Ollama significantly reduces data transfer risks and simplifies GDPR compliance. Highlight this in documentation.\n\n---\n\n## 10. PENALTIES FOR NON-COMPLIANCE\n\n| Violation | Maximum Fine |\n|---|---|\n| Prohibited practices (Art. 5) | EUR 35 million or 7% of global turnover |\n| High-risk system obligations (Art. 8-15, 17) | EUR 15 million or 3% of global turnover |\n| Deployer obligations (Art. 26) | EUR 15 million or 3% of global turnover |\n| Transparency obligations (Art. 50) | EUR 15 million or 3% of global turnover |\n| Incorrect information to authorities | EUR 7.5 million or 1% of global turnover |\n| GDPR violations (separate) | EUR 20 million or 4% of global turnover |\n\n---\n\n## Sources\n\n- Article 3: Definitions | EU AI Act\n- Article 5: Prohibited AI Practices | EU AI Act\n- Article 6: Classification Rules for High-Risk AI Systems\n- Article 9: Risk Management System\n- Article 14: Human Oversight\n- Article 17: Quality Management System\n- Article 26: Obligations of Deployers of High-Risk AI Systems\n- Article 50: Transparency Obligations\n- Annex III: High-Risk AI Systems\n- Annex IV: Technical Documentation\n- Implementation Timeline | EU AI Act\n- High-Level Summary of the AI Act\n- EU AI Act Brief - Pt. 4, AI at Work (CDT)\n- Prohibition of AI Emotion Recognition in the Workplace\n- AI and Human Resources in the EU: 2026 Legal Overview\n- Navigating the EU AI Act for Healthcare (PMC)\n- GDPR and AI Act Interplay (IAPP)\n- EU AI Act 2026 Compliance Guide\n- High-Risk Escape Route via Article 6(3)\n- EU Commission Clarifies Definition of AI Systems\n- Regulation (EU) 2024/1689 Full Text (EUR-Lex)\n\n---\n\nKey project files examined during this assessment:\n- /home/dom/ai/rpa_vision_v3/core/config.py -- Central configuration (security defaults, no privacy controls)\n- /home/dom/ai/rpa_vision_v3/agent_v0/agent_v1/vision/capturer.py -- Screenshot capture (anonymize=False default)\n- /home/dom/ai/rpa_vision_v3/agent_v0/agent_v1/session/storage.py -- Session storage (retention_days=1, violates 6-month minimum)\n- /home/dom/ai/rpa_vision_v3/agent_v0/agent_v1/ui/chat_window.py -- Lea chatbot (insufficient AI disclosure)\n- /home/dom/ai/rpa_vision_v3/core/execution/workflow_runner.py -- Autonomous replay (no human oversight mechanism)\n- /home/dom/ai/rpa_vision_v3/core/extraction/extraction_engine.py -- Data extraction from screens (patient data risk)\n\nBottom line: RPA Vision V3 is classified as a high-risk AI system under the EU AI Act. The August 2, 2026 deadline for full compliance is 5 months away. The project currently has zero of the required formal compliance artifacts (risk management system, technical documentation, QMS, declaration of conformity, EU database registration). Significant engineering work is also needed for data protection (anonymization defaults, encryption, retention policies, data subject rights). Engaging legal/compliance expertise is strongly recommended."}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":1,"cache_creation_input_tokens":1117,"cache_read_input_tokens":50463,"output_tokens":6018,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":0,"ephemeral_5m_input_tokens":1117},"inference_geo":"","iterations":[],"speed":"standard"}},"requestId":"req_011CZAhNymwQrUGNrLd85zEj","type":"assistant","uuid":"782ef93a-c6d8-4ee0-9a66-937f38d0d196","timestamp":"2026-03-18T14:25:39.474Z","userType":"external","cwd":"/home/dom/ai/rpa_vision_v3","sessionId":"e3cde4b8-cef7-4e9d-8596-3d411082503d","version":"2.1.76","gitBranch":"main","slug":"whimsical-herding-kazoo"}