215 lines
6.8 KiB
Markdown
215 lines
6.8 KiB
Markdown
# Design Document - Workflow Detection Fix
|
|
|
|
## Overview
|
|
|
|
This design addresses the critical bugs preventing workflow detection in GeniusIA v2. The main issues are:
|
|
1. SessionManager and WorkflowDetector are not integrated into the Orchestrator
|
|
2. VisionAnalysis has an attribute name bug (`self.llm_manager` vs `self.llm`)
|
|
3. No workflow persistence mechanism exists
|
|
|
|
The solution integrates these components into the existing cognitive loop and fixes the bugs.
|
|
|
|
## Architecture
|
|
|
|
### Current Architecture (Broken)
|
|
```
|
|
Orchestrator
|
|
├─ EventCapture (captures actions)
|
|
├─ VisionAnalysis (BROKEN - wrong attribute)
|
|
├─ SuggestionManager (never receives workflows)
|
|
└─ LearningManager
|
|
```
|
|
|
|
### Fixed Architecture
|
|
```
|
|
Orchestrator
|
|
├─ EventCapture (captures actions)
|
|
├─ SessionManager (NEW - segments actions)
|
|
│ └─ WorkflowDetector (NEW - detects patterns)
|
|
├─ VisionAnalysis (FIXED - correct attribute)
|
|
├─ SuggestionManager (receives workflow notifications)
|
|
└─ LearningManager
|
|
```
|
|
|
|
## Components and Interfaces
|
|
|
|
### 1. Orchestrator Integration
|
|
|
|
**Changes Required:**
|
|
- Import SessionManager and WorkflowDetector
|
|
- Initialize both components in `__init__`
|
|
- Pass actions to SessionManager in the cognitive loop
|
|
- Connect WorkflowDetector callbacks to SuggestionManager
|
|
|
|
**New Attributes:**
|
|
```python
|
|
self.session_manager: SessionManager
|
|
self.workflow_detector: WorkflowDetector
|
|
```
|
|
|
|
**New Method:**
|
|
```python
|
|
def _on_workflow_detected(self, workflow: Dict[str, Any]) -> None:
|
|
"""Callback when a workflow is detected."""
|
|
# Notify suggestion manager
|
|
# Log workflow details
|
|
# Persist workflow
|
|
```
|
|
|
|
### 2. SessionManager Integration
|
|
|
|
**Initialization:**
|
|
```python
|
|
self.session_manager = SessionManager(
|
|
logger=logger,
|
|
config=config
|
|
)
|
|
```
|
|
|
|
**Usage in Cognitive Loop:**
|
|
```python
|
|
# After action is captured
|
|
session = self.session_manager.add_action(action_data)
|
|
if session and session.is_complete:
|
|
self.workflow_detector.analyze_session(session)
|
|
```
|
|
|
|
### 3. WorkflowDetector Integration
|
|
|
|
**Initialization:**
|
|
```python
|
|
self.workflow_detector = WorkflowDetector(
|
|
embeddings_manager=learning_manager.embeddings_manager,
|
|
logger=logger,
|
|
config=config
|
|
)
|
|
self.workflow_detector.on_workflow_detected = self._on_workflow_detected
|
|
```
|
|
|
|
**Workflow Persistence:**
|
|
- Save to `data/user_profiles/workflows/workflow_{id}.json`
|
|
- Load on startup
|
|
- Update confidence scores on execution
|
|
|
|
### 4. VisionAnalysis Bug Fix
|
|
|
|
**Current Bug:**
|
|
```python
|
|
if self.llm_manager: # WRONG - attribute is self.llm
|
|
response = self.llm_manager.generate_with_vision(...)
|
|
```
|
|
|
|
**Fixed:**
|
|
```python
|
|
if self.llm: # CORRECT
|
|
response = self.llm.generate_with_vision(...)
|
|
```
|
|
|
|
## Data Models
|
|
|
|
### Session Model (Existing)
|
|
```python
|
|
@dataclass
|
|
class Session:
|
|
session_id: str
|
|
start_time: datetime
|
|
end_time: Optional[datetime]
|
|
actions: List[Dict[str, Any]]
|
|
window: Optional[str]
|
|
```
|
|
|
|
### Workflow Model (New)
|
|
```python
|
|
@dataclass
|
|
class Workflow:
|
|
workflow_id: str
|
|
name: str
|
|
pattern: List[str] # Action signatures
|
|
confidence: float
|
|
repetitions: int
|
|
last_seen: datetime
|
|
window: str
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
### VisionAnalysis Errors
|
|
- Catch all exceptions in `analyze_action`
|
|
- Log errors with context
|
|
- Return partial signature on failure
|
|
- Continue processing other actions
|
|
|
|
### SessionManager Errors
|
|
- Handle invalid action data gracefully
|
|
- Log session creation failures
|
|
- Don't crash on timeout issues
|
|
|
|
### WorkflowDetector Errors
|
|
- Handle embedding comparison failures
|
|
- Log pattern matching errors
|
|
- Continue processing other sessions
|
|
|
|
## Correctness Properties
|
|
|
|
*A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
|
|
|
|
### Property 1: Action forwarding to SessionManager
|
|
*For any* valid user action processed by the Orchestrator, the action should be passed to the SessionManager for segmentation.
|
|
**Validates: Requirements 1.1**
|
|
|
|
### Property 2: Session grouping by time and context
|
|
*For any* sequence of actions with timestamps and window contexts, actions within the time window threshold and same window should be grouped into the same session.
|
|
**Validates: Requirements 1.2**
|
|
|
|
### Property 3: Completed sessions trigger workflow analysis
|
|
*For any* completed session, the WorkflowDetector should receive the session for pattern analysis.
|
|
**Validates: Requirements 1.3**
|
|
|
|
### Property 4: Workflow detection triggers notifications
|
|
*For any* detected workflow pattern, the SuggestionManager callback should be invoked with the workflow details.
|
|
**Validates: Requirements 1.4**
|
|
|
|
### Property 5: Workflow notifications generate suggestions
|
|
*For any* workflow notification received by SuggestionManager, at least one suggestion should be created for the user.
|
|
**Validates: Requirements 1.5**
|
|
|
|
### Property 6: VisionAnalysis error resilience
|
|
*For any* error encountered during VisionAnalysis, the system should log the error and continue processing without crashing.
|
|
**Validates: Requirements 2.2**
|
|
|
|
### Property 7: Action signature completeness
|
|
*For any* action analyzed by VisionAnalysis, the resulting signature should contain both visual embeddings and element descriptions (or defaults if LLM unavailable).
|
|
**Validates: Requirements 2.3**
|
|
|
|
### Property 8: Workflow persistence
|
|
*For any* detected workflow, a corresponding JSON file should be created in the user profile workflows directory with all workflow details.
|
|
**Validates: Requirements 3.1**
|
|
|
|
### Property 9: Confidence score updates
|
|
*For any* workflow that is executed successfully, its confidence score should increase.
|
|
**Validates: Requirements 3.3**
|
|
|
|
### Property 10: Session logging completeness
|
|
*For any* session created by SessionManager, a log entry should be created containing the session ID and action count.
|
|
**Validates: Requirements 4.1**
|
|
|
|
### Property 11: Workflow detection logging
|
|
*For any* workflow detected by WorkflowDetector, a log entry should be created containing the workflow pattern and confidence score.
|
|
**Validates: Requirements 4.2**
|
|
|
|
### Property 12: Error logging with context
|
|
*For any* workflow detection failure, a log entry should be created containing the error message and relevant context.
|
|
**Validates: Requirements 4.3**
|
|
|
|
## Testing Strategy
|
|
|
|
### Unit Tests
|
|
- Test SessionManager action grouping with specific examples
|
|
- Test WorkflowDetector pattern matching with known patterns
|
|
- Test VisionAnalysis attribute access (bug fix verification)
|
|
- Test Orchestrator integration points with mocks
|
|
|
|
### Property-Based Tests
|
|
We will use Hypothesis for property-based testing in Python. Each property above will be implemented as a property-based test that runs 100+ iterations with randomly generated inputs to verify the correctness properties hold across all valid inputs.
|
|
|