rpa_vision_v3/.kiro/specs/rpa-system-unification/design.md

Design Document: RPA System Unification

Overview

Cette conception unifie définitivement le système RPA Vision V3 en créant une architecture centralisée qui élimine les incohérences de configuration, les chemins de données dispersés, et les services mal synchronisés.

Architecture

Composants Principaux

graph TB
    CM[Configuration Manager] --> SR[Service Registry]
    CM --> DM[Data Manager]
    CM --> HM[Health Monitor]
    
    SR --> API[API Service]
    SR --> DASH[Dashboard Service]
    SR --> WORK[Worker Service]
    SR --> AGENT[Agent Service]
    
    DM --> SESS[Sessions Storage]
    DM --> WF[Workflows Storage]
    DM --> LOGS[Logs Storage]
    
    HM --> API
    HM --> DASH
    HM --> WORK
    HM --> AGENT

Configuration Manager

Le Configuration Manager est le point central de toute configuration système :

@dataclass
class SystemConfig:
    # Chemins unifiés
    base_path: Path
    data_path: Path
    logs_path: Path
    
    # Services
    api_port: int = 8000
    dashboard_port: int = 5001
    worker_threads: int = 4
    
    # Base de données
    sessions_path: Path
    workflows_path: Path
    embeddings_path: Path
    
    # Sécurité
    secret_key: str
    auth_enabled: bool = True
    
    # Monitoring
    health_check_interval: int = 30
    metrics_enabled: bool = True

Service Registry

Le Service Registry maintient l'état de tous les services :

class ServiceRegistry:
    def register_service(self, service_id: str, service_info: ServiceInfo)
    def unregister_service(self, service_id: str)
    def get_service_status(self, service_id: str) -> ServiceStatus
    def get_all_services(self) -> Dict[str, ServiceStatus]
    def wait_for_service(self, service_id: str, timeout: int = 30)

Components and Interfaces

1. Configuration Manager

Interface:

class ConfigurationManager:
    def load_config(self) -> SystemConfig
    def validate_config(self, config: SystemConfig) -> List[ValidationError]
    def apply_config(self, config: SystemConfig)
    def watch_config_changes(self, callback: Callable)

Responsabilités:

• Charger la configuration depuis les variables d'environnement et fichiers
• Valider la cohérence de la configuration
• Propager les changements aux services actifs
• Fournir des valeurs par défaut sensées

2. Data Manager

Interface:

class DataManager:
    def initialize_directories(self)
    def get_sessions_path(self) -> Path
    def get_workflows_path(self) -> Path
    def migrate_data_if_needed(self)
    def validate_data_integrity(self) -> List[IntegrityError]

Responsabilités:

• Créer et maintenir la structure de répertoires
• Gérer les migrations de données
• Valider l'intégrité des données
• Fournir des chemins cohérents à tous les composants

3. Health Monitor

Interface:

class HealthMonitor:
    def start_monitoring(self)
    def stop_monitoring(self)
    def check_service_health(self, service_id: str) -> HealthStatus
    def get_system_health(self) -> SystemHealthStatus
    def register_health_check(self, service_id: str, check_func: Callable)

Responsabilités:

• Surveiller la santé de tous les services
• Détecter les pannes et tenter la récupération
• Fournir des métriques de santé unifiées
• Alerter en cas de problèmes critiques

4. System Controller

Interface:

class SystemController:
    def start_all_services(self)
    def stop_all_services(self)
    def restart_service(self, service_id: str)
    def get_system_status(self) -> SystemStatus
    def run_consistency_check(self) -> ConsistencyReport

Responsabilités:

• Coordonner le démarrage/arrêt des services
• Respecter les dépendances entre services
• Fournir des commandes de gestion système
• Exécuter les vérifications de cohérence

Data Models

Configuration Models

@dataclass
class ServiceInfo:
    service_id: str
    service_type: str
    host: str
    port: int
    pid: Optional[int]
    started_at: datetime
    health_check_url: str

@dataclass
class ServiceStatus:
    service_id: str
    status: Literal["starting", "running", "stopping", "stopped", "error"]
    last_health_check: datetime
    error_message: Optional[str]
    uptime: timedelta

@dataclass
class SystemHealthStatus:
    overall_status: Literal["healthy", "degraded", "critical"]
    services: Dict[str, ServiceStatus]
    last_check: datetime
    issues: List[str]

Migration Models

@dataclass
class MigrationTask:
    migration_id: str
    description: str
    source_version: str
    target_version: str
    migration_func: Callable

@dataclass
class MigrationResult:
    migration_id: str
    success: bool
    duration: timedelta
    error_message: Optional[str]
    backup_path: Optional[Path]

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Configuration Consistency

For any system startup, all components should use identical configuration values for shared settings Validates: Requirements 1.1, 1.2

Property 2: Data Path Unification

For any data access operation, all components should resolve to the same physical paths for the same logical data Validates: Requirements 2.1, 2.2, 2.3

Property 3: Service Registration Completeness

For any active service, it should be registered in the Service Registry with accurate status information Validates: Requirements 3.1, 3.2, 3.3

Property 4: Health Check Responsiveness

For any registered service, health checks should complete within the configured timeout period Validates: Requirements 4.1, 4.2

Property 5: Startup Order Correctness

For any system startup sequence, services should start in dependency order and all dependencies should be ready before dependent services start Validates: Requirements 5.1, 5.3

Property 6: Shutdown Graceful Completion

For any system shutdown sequence, all services should stop cleanly within the configured timeout Validates: Requirements 5.2, 5.4

Property 7: Error Correlation Consistency

For any error event, it should be logged with consistent format and correlation IDs across all components Validates: Requirements 6.1, 6.2, 6.3

Property 8: Data Consistency Validation

For any consistency check run, detected inconsistencies should be accurately reported with actionable repair suggestions Validates: Requirements 7.1, 7.2, 7.3

Property 9: Migration Safety

For any data migration, the original data should be backed up and the migration should be reversible Validates: Requirements 8.1, 8.3, 8.5

Property 10: Configuration Validation Completeness

For any invalid configuration, the system should detect and report all validation errors before attempting to start services Validates: Requirements 1.4, 1.5

Error Handling

Configuration Errors

• Invalid paths → Create directories or fail with clear message
• Missing required values → Use defaults or fail with specific missing keys
• Type mismatches → Convert when safe or fail with type information

Service Errors

• Service startup failure → Log error, mark as failed, don't start dependents
• Service health check failure → Attempt restart, escalate if repeated failures
• Service communication failure → Retry with backoff, mark as degraded

Data Errors

• Missing directories → Create with correct permissions
• Corrupted data → Attempt repair, restore from backup if available
• Schema version mismatch → Run migrations automatically

Migration Errors

• Migration failure → Restore from backup, log detailed error
• Backup failure → Abort migration, require manual intervention
• Validation failure → Report specific issues, provide repair options

Testing Strategy

Unit Tests

• Configuration loading and validation
• Service registry operations
• Health check logic
• Migration procedures
• Error handling paths

Property-Based Tests

• Configuration consistency across components (Property 1)
• Data path resolution correctness (Property 2)
• Service lifecycle management (Properties 3, 5, 6)
• Error correlation and logging (Property 7)
• Migration safety and reversibility (Property 9)

Integration Tests

• Full system startup/shutdown cycles
• Service failure and recovery scenarios
• Configuration change propagation
• Cross-component data access
• End-to-end health monitoring

Performance Tests

• System startup time under various loads
• Health check response times
• Configuration change propagation speed
• Migration performance with large datasets

Testing Configuration:

• Minimum 100 iterations per property test
• Each property test tagged with: Feature: rpa-system-unification, Property {number}: {property_text}
• Integration tests run in isolated environments
• Performance tests with realistic data volumes