rpa_vision_v3/.kiro/specs/rpa-system-unification/design.md

# Design Document: RPA System Unification

## Overview

Cette conception unifie définitivement le système RPA Vision V3 en créant une architecture centralisée qui élimine les incohérences de configuration, les chemins de données dispersés, et les services mal synchronisés.

## Architecture

### Composants Principaux

```mermaid
graph TB
    CM[Configuration Manager] --> SR[Service Registry]
    CM --> DM[Data Manager]
    CM --> HM[Health Monitor]
    
    SR --> API[API Service]
    SR --> DASH[Dashboard Service]
    SR --> WORK[Worker Service]
    SR --> AGENT[Agent Service]
    
    DM --> SESS[Sessions Storage]
    DM --> WF[Workflows Storage]
    DM --> LOGS[Logs Storage]
    
    HM --> API
    HM --> DASH
    HM --> WORK
    HM --> AGENT
```

### Configuration Manager

Le Configuration Manager est le point central de toute configuration système :

```python
@dataclass
class SystemConfig:
    # Chemins unifiés
    base_path: Path
    data_path: Path
    logs_path: Path
    
    # Services
    api_port: int = 8000
    dashboard_port: int = 5001
    worker_threads: int = 4
    
    # Base de données
    sessions_path: Path
    workflows_path: Path
    embeddings_path: Path
    
    # Sécurité
    secret_key: str
    auth_enabled: bool = True
    
    # Monitoring
    health_check_interval: int = 30
    metrics_enabled: bool = True
```

### Service Registry

Le Service Registry maintient l'état de tous les services :

```python
class ServiceRegistry:
    def register_service(self, service_id: str, service_info: ServiceInfo)
    def unregister_service(self, service_id: str)
    def get_service_status(self, service_id: str) -> ServiceStatus
    def get_all_services(self) -> Dict[str, ServiceStatus]
    def wait_for_service(self, service_id: str, timeout: int = 30)
```

## Components and Interfaces

### 1. Configuration Manager

**Interface:**
```python
class ConfigurationManager:
    def load_config(self) -> SystemConfig
    def validate_config(self, config: SystemConfig) -> List[ValidationError]
    def apply_config(self, config: SystemConfig)
    def watch_config_changes(self, callback: Callable)
```

**Responsabilités:**
- Charger la configuration depuis les variables d'environnement et fichiers
- Valider la cohérence de la configuration
- Propager les changements aux services actifs
- Fournir des valeurs par défaut sensées

### 2. Data Manager

**Interface:**
```python
class DataManager:
    def initialize_directories(self)
    def get_sessions_path(self) -> Path
    def get_workflows_path(self) -> Path
    def migrate_data_if_needed(self)
    def validate_data_integrity(self) -> List[IntegrityError]
```

**Responsabilités:**
- Créer et maintenir la structure de répertoires
- Gérer les migrations de données
- Valider l'intégrité des données
- Fournir des chemins cohérents à tous les composants

### 3. Health Monitor

**Interface:**
```python
class HealthMonitor:
    def start_monitoring(self)
    def stop_monitoring(self)
    def check_service_health(self, service_id: str) -> HealthStatus
    def get_system_health(self) -> SystemHealthStatus
    def register_health_check(self, service_id: str, check_func: Callable)
```

**Responsabilités:**
- Surveiller la santé de tous les services
- Détecter les pannes et tenter la récupération
- Fournir des métriques de santé unifiées
- Alerter en cas de problèmes critiques

### 4. System Controller

**Interface:**
```python
class SystemController:
    def start_all_services(self)
    def stop_all_services(self)
    def restart_service(self, service_id: str)
    def get_system_status(self) -> SystemStatus
    def run_consistency_check(self) -> ConsistencyReport
```

**Responsabilités:**
- Coordonner le démarrage/arrêt des services
- Respecter les dépendances entre services
- Fournir des commandes de gestion système
- Exécuter les vérifications de cohérence

## Data Models

### Configuration Models

```python
@dataclass
class ServiceInfo:
    service_id: str
    service_type: str
    host: str
    port: int
    pid: Optional[int]
    started_at: datetime
    health_check_url: str

@dataclass
class ServiceStatus:
    service_id: str
    status: Literal["starting", "running", "stopping", "stopped", "error"]
    last_health_check: datetime
    error_message: Optional[str]
    uptime: timedelta

@dataclass
class SystemHealthStatus:
    overall_status: Literal["healthy", "degraded", "critical"]
    services: Dict[str, ServiceStatus]
    last_check: datetime
    issues: List[str]
```

### Migration Models

```python
@dataclass
class MigrationTask:
    migration_id: str
    description: str
    source_version: str
    target_version: str
    migration_func: Callable

@dataclass
class MigrationResult:
    migration_id: str
    success: bool
    duration: timedelta
    error_message: Optional[str]
    backup_path: Optional[Path]
```

## Correctness Properties

*A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*

### Property 1: Configuration Consistency
*For any* system startup, all components should use identical configuration values for shared settings
**Validates: Requirements 1.1, 1.2**

### Property 2: Data Path Unification
*For any* data access operation, all components should resolve to the same physical paths for the same logical data
**Validates: Requirements 2.1, 2.2, 2.3**

### Property 3: Service Registration Completeness
*For any* active service, it should be registered in the Service Registry with accurate status information
**Validates: Requirements 3.1, 3.2, 3.3**

### Property 4: Health Check Responsiveness
*For any* registered service, health checks should complete within the configured timeout period
**Validates: Requirements 4.1, 4.2**

### Property 5: Startup Order Correctness
*For any* system startup sequence, services should start in dependency order and all dependencies should be ready before dependent services start
**Validates: Requirements 5.1, 5.3**

### Property 6: Shutdown Graceful Completion
*For any* system shutdown sequence, all services should stop cleanly within the configured timeout
**Validates: Requirements 5.2, 5.4**

### Property 7: Error Correlation Consistency
*For any* error event, it should be logged with consistent format and correlation IDs across all components
**Validates: Requirements 6.1, 6.2, 6.3**

### Property 8: Data Consistency Validation
*For any* consistency check run, detected inconsistencies should be accurately reported with actionable repair suggestions
**Validates: Requirements 7.1, 7.2, 7.3**

### Property 9: Migration Safety
*For any* data migration, the original data should be backed up and the migration should be reversible
**Validates: Requirements 8.1, 8.3, 8.5**

### Property 10: Configuration Validation Completeness
*For any* invalid configuration, the system should detect and report all validation errors before attempting to start services
**Validates: Requirements 1.4, 1.5**

## Error Handling

### Configuration Errors
- Invalid paths → Create directories or fail with clear message
- Missing required values → Use defaults or fail with specific missing keys
- Type mismatches → Convert when safe or fail with type information

### Service Errors
- Service startup failure → Log error, mark as failed, don't start dependents
- Service health check failure → Attempt restart, escalate if repeated failures
- Service communication failure → Retry with backoff, mark as degraded

### Data Errors
- Missing directories → Create with correct permissions
- Corrupted data → Attempt repair, restore from backup if available
- Schema version mismatch → Run migrations automatically

### Migration Errors
- Migration failure → Restore from backup, log detailed error
- Backup failure → Abort migration, require manual intervention
- Validation failure → Report specific issues, provide repair options

## Testing Strategy

### Unit Tests
- Configuration loading and validation
- Service registry operations
- Health check logic
- Migration procedures
- Error handling paths

### Property-Based Tests
- Configuration consistency across components (Property 1)
- Data path resolution correctness (Property 2)
- Service lifecycle management (Properties 3, 5, 6)
- Error correlation and logging (Property 7)
- Migration safety and reversibility (Property 9)

### Integration Tests
- Full system startup/shutdown cycles
- Service failure and recovery scenarios
- Configuration change propagation
- Cross-component data access
- End-to-end health monitoring

### Performance Tests
- System startup time under various loads
- Health check response times
- Configuration change propagation speed
- Migration performance with large datasets

**Testing Configuration:**
- Minimum 100 iterations per property test
- Each property test tagged with: **Feature: rpa-system-unification, Property {number}: {property_text}**
- Integration tests run in isolated environments
- Performance tests with realistic data volumes