Dom/Geniusia_v2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Dom
2026-03-05 00:20:25 +01:00
commit dcd4de9945
1954 changed files with 669380 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
.kiro/specs/workflow-detection-fix/design.md Normal file
View File

@@ -0,0 +1,214 @@
# Design Document - Workflow Detection Fix
## Overview
This design addresses the critical bugs preventing workflow detection in GeniusIA v2. The main issues are:
1. SessionManager and WorkflowDetector are not integrated into the Orchestrator
2. VisionAnalysis has an attribute name bug (`self.llm_manager` vs `self.llm`)
3. No workflow persistence mechanism exists
The solution integrates these components into the existing cognitive loop and fixes the bugs.
## Architecture
### Current Architecture (Broken)
```
Orchestrator
├─ EventCapture (captures actions)
├─ VisionAnalysis (BROKEN - wrong attribute)
├─ SuggestionManager (never receives workflows)
└─ LearningManager
```
### Fixed Architecture
```
Orchestrator
├─ EventCapture (captures actions)
├─ SessionManager (NEW - segments actions)
│ └─ WorkflowDetector (NEW - detects patterns)
├─ VisionAnalysis (FIXED - correct attribute)
├─ SuggestionManager (receives workflow notifications)
└─ LearningManager
```
## Components and Interfaces
### 1. Orchestrator Integration
**Changes Required:**
- Import SessionManager and WorkflowDetector
- Initialize both components in `__init__`
- Pass actions to SessionManager in the cognitive loop
- Connect WorkflowDetector callbacks to SuggestionManager
**New Attributes:**
```python
self.session_manager: SessionManager
self.workflow_detector: WorkflowDetector
```
**New Method:**
```python
def _on_workflow_detected(self, workflow: Dict[str, Any]) -> None:
"""Callback when a workflow is detected."""
# Notify suggestion manager
# Log workflow details
# Persist workflow
```
### 2. SessionManager Integration
**Initialization:**
```python
self.session_manager = SessionManager(
logger=logger,
config=config
)
```
**Usage in Cognitive Loop:**
```python
# After action is captured
session = self.session_manager.add_action(action_data)
if session and session.is_complete:
self.workflow_detector.analyze_session(session)
```
### 3. WorkflowDetector Integration
**Initialization:**
```python
self.workflow_detector = WorkflowDetector(
embeddings_manager=learning_manager.embeddings_manager,
logger=logger,
config=config
)
self.workflow_detector.on_workflow_detected = self._on_workflow_detected
```
**Workflow Persistence:**
- Save to `data/user_profiles/workflows/workflow_{id}.json`
- Load on startup
- Update confidence scores on execution
### 4. VisionAnalysis Bug Fix
**Current Bug:**
```python
if self.llm_manager: # WRONG - attribute is self.llm
response = self.llm_manager.generate_with_vision(...)
```
**Fixed:**
```python
if self.llm: # CORRECT
response = self.llm.generate_with_vision(...)
```
## Data Models
### Session Model (Existing)
```python
@dataclass
class Session:
session_id: str
start_time: datetime
end_time: Optional[datetime]
actions: List[Dict[str, Any]]
window: Optional[str]
```
### Workflow Model (New)
```python
@dataclass
class Workflow:
workflow_id: str
name: str
pattern: List[str] # Action signatures
confidence: float
repetitions: int
last_seen: datetime
window: str
```
## Error Handling
### VisionAnalysis Errors
- Catch all exceptions in `analyze_action`
- Log errors with context
- Return partial signature on failure
- Continue processing other actions
### SessionManager Errors
- Handle invalid action data gracefully
- Log session creation failures
- Don't crash on timeout issues
### WorkflowDetector Errors
- Handle embedding comparison failures
- Log pattern matching errors
- Continue processing other sessions
## Correctness Properties
*A property is a characteristic or behavior that should hold true across all valid executions of a system-essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
### Property 1: Action forwarding to SessionManager
*For any* valid user action processed by the Orchestrator, the action should be passed to the SessionManager for segmentation.
**Validates: Requirements 1.1**
### Property 2: Session grouping by time and context
*For any* sequence of actions with timestamps and window contexts, actions within the time window threshold and same window should be grouped into the same session.
**Validates: Requirements 1.2**
### Property 3: Completed sessions trigger workflow analysis
*For any* completed session, the WorkflowDetector should receive the session for pattern analysis.
**Validates: Requirements 1.3**
### Property 4: Workflow detection triggers notifications
*For any* detected workflow pattern, the SuggestionManager callback should be invoked with the workflow details.
**Validates: Requirements 1.4**
### Property 5: Workflow notifications generate suggestions
*For any* workflow notification received by SuggestionManager, at least one suggestion should be created for the user.
**Validates: Requirements 1.5**
### Property 6: VisionAnalysis error resilience
*For any* error encountered during VisionAnalysis, the system should log the error and continue processing without crashing.
**Validates: Requirements 2.2**
### Property 7: Action signature completeness
*For any* action analyzed by VisionAnalysis, the resulting signature should contain both visual embeddings and element descriptions (or defaults if LLM unavailable).
**Validates: Requirements 2.3**
### Property 8: Workflow persistence
*For any* detected workflow, a corresponding JSON file should be created in the user profile workflows directory with all workflow details.
**Validates: Requirements 3.1**
### Property 9: Confidence score updates
*For any* workflow that is executed successfully, its confidence score should increase.
**Validates: Requirements 3.3**
### Property 10: Session logging completeness
*For any* session created by SessionManager, a log entry should be created containing the session ID and action count.
**Validates: Requirements 4.1**
### Property 11: Workflow detection logging
*For any* workflow detected by WorkflowDetector, a log entry should be created containing the workflow pattern and confidence score.
**Validates: Requirements 4.2**
### Property 12: Error logging with context
*For any* workflow detection failure, a log entry should be created containing the error message and relevant context.
**Validates: Requirements 4.3**
## Testing Strategy
### Unit Tests
- Test SessionManager action grouping with specific examples
- Test WorkflowDetector pattern matching with known patterns
- Test VisionAnalysis attribute access (bug fix verification)
- Test Orchestrator integration points with mocks
### Property-Based Tests
We will use Hypothesis for property-based testing in Python. Each property above will be implemented as a property-based test that runs 100+ iterations with randomly generated inputs to verify the correctness properties hold across all valid inputs.

60
.kiro/specs/workflow-detection-fix/requirements.md Normal file
View File

@@ -0,0 +1,60 @@
# Requirements Document - Workflow Detection Fix
## Introduction
This document specifies the requirements for fixing the workflow detection system in GeniusIA v2. The system currently fails to detect workflows because the SessionManager and WorkflowDetector components are not integrated into the main Orchestrator, and there is a critical bug in VisionAnalysis that prevents action analysis.
## Glossary
- **Orchestrator**: The main cognitive loop that coordinates all system components
- **SessionManager**: Component that segments user actions into logical sessions
- **WorkflowDetector**: Component that identifies repeated patterns in sessions
- **VisionAnalysis**: Component that analyzes visual regions around user actions
- **SuggestionManager**: Component that generates suggestions based on detected workflows
- **Action**: A user interaction event (click, keypress, etc.)
- **Session**: A group of related actions within a time window
- **Workflow**: A repeated pattern of actions that can be automated
## Requirements
### Requirement 1
**User Story:** As a user, I want the system to detect my repeated action sequences, so that it can learn my workflows and suggest automations.
#### Acceptance Criteria
1. WHEN the Orchestrator processes user actions THEN the system SHALL pass those actions to the SessionManager for segmentation
2. WHEN the SessionManager receives actions THEN the system SHALL group them into sessions based on time windows and window context
3. WHEN a session is completed THEN the system SHALL pass it to the WorkflowDetector for pattern analysis
4. WHEN the WorkflowDetector identifies a repeated pattern THEN the system SHALL notify the SuggestionManager
5. WHEN the SuggestionManager receives a workflow notification THEN the system SHALL generate appropriate suggestions for the user
### Requirement 2
**User Story:** As a developer, I want the VisionAnalysis component to work correctly, so that action signatures can be created and compared.
#### Acceptance Criteria
1. WHEN VisionAnalysis analyzes an action THEN the system SHALL use the correct attribute name for the LLM manager
2. WHEN VisionAnalysis encounters an error THEN the system SHALL log the error and continue processing
3. WHEN VisionAnalysis creates an action signature THEN the system SHALL include visual embeddings and element descriptions
### Requirement 3
**User Story:** As a user, I want the system to persist detected workflows, so that they are available across sessions.
#### Acceptance Criteria
1. WHEN a workflow is detected THEN the system SHALL save it to the user profile directory
2. WHEN the system starts THEN the system SHALL load previously detected workflows
3. WHEN a workflow is executed successfully THEN the system SHALL update its confidence score
### Requirement 4
**User Story:** As a developer, I want comprehensive logging of the workflow detection process, so that I can debug issues.
#### Acceptance Criteria
1. WHEN actions are segmented into sessions THEN the system SHALL log session creation with action count
2. WHEN workflows are detected THEN the system SHALL log workflow details including pattern and confidence
3. WHEN workflow detection fails THEN the system SHALL log the error with context

114
.kiro/specs/workflow-detection-fix/tasks.md Normal file
View File

@@ -0,0 +1,114 @@
# Implementation Plan - Workflow Detection Fix
- [x] 1. Fix VisionAnalysis attribute bug
- Fix the `self.llm_manager` to `self.llm` attribute references
- Add error handling around LLM calls
- Ensure action signatures include all required fields
- _Requirements: 2.1, 2.2, 2.3_
- [ ]* 1.1 Write unit test for VisionAnalysis attribute fix
- Verify no AttributeError is raised when LLM is provided
- Test with and without LLM manager
- _Requirements: 2.1_
- [ ]* 1.2 Write property test for VisionAnalysis error handling
- **Property 6: VisionAnalysis error resilience**
- **Validates: Requirements 2.2**
- [ ]* 1.3 Write property test for action signature completeness
- **Property 7: Action signature completeness**
- **Validates: Requirements 2.3**
- [x] 2. Integrate SessionManager into Orchestrator
- Import SessionManager in orchestrator.py
- Initialize SessionManager in Orchestrator.__init__
- Add action forwarding in the cognitive loop
- Connect session completion callback
- _Requirements: 1.1, 1.2_
- [ ]* 2.1 Write property test for action forwarding
- **Property 1: Action forwarding to SessionManager**
- **Validates: Requirements 1.1**
- [ ]* 2.2 Write property test for session grouping
- **Property 2: Session grouping by time and context**
- **Validates: Requirements 1.2**
- [x] 3. Integrate WorkflowDetector into Orchestrator
- Import WorkflowDetector in orchestrator.py
- Initialize WorkflowDetector in Orchestrator.__init__
- Connect SessionManager to WorkflowDetector
- Implement _on_workflow_detected callback
- _Requirements: 1.3, 1.4_
- [ ]* 3.1 Write property test for session forwarding to WorkflowDetector
- **Property 3: Completed sessions trigger workflow analysis**
- **Validates: Requirements 1.3**
- [ ]* 3.2 Write property test for workflow notifications
- **Property 4: Workflow detection triggers notifications**
- **Validates: Requirements 1.4**
- [x] 4. Connect WorkflowDetector to SuggestionManager
- Implement workflow notification in _on_workflow_detected
- Ensure SuggestionManager receives workflow data
- Add logging for workflow detection events
- _Requirements: 1.5, 4.2_
- [ ]* 4.1 Write property test for suggestion generation
- **Property 5: Workflow notifications generate suggestions**
- **Validates: Requirements 1.5**
- [ ]* 4.2 Write property test for workflow detection logging
- **Property 11: Workflow detection logging**
- **Validates: Requirements 4.2**
- [x] 5. Implement workflow persistence
- Create workflow save method in WorkflowDetector
- Save workflows to data/user_profiles/workflows/
- Implement workflow loading on startup
- Add confidence score update mechanism
- _Requirements: 3.1, 3.2, 3.3_
- [ ]* 5.1 Write property test for workflow persistence
- **Property 8: Workflow persistence**
- **Validates: Requirements 3.1**
- [ ]* 5.2 Write unit test for workflow loading
- Create test workflow files
- Verify they are loaded on startup
- _Requirements: 3.2_
- [ ]* 5.3 Write property test for confidence score updates
- **Property 9: Confidence score updates**
- **Validates: Requirements 3.3**
- [ ] 6. Add comprehensive logging
- Add session creation logging in SessionManager
- Add workflow detection logging in WorkflowDetector
- Add error logging with context
- _Requirements: 4.1, 4.3_
- [ ]* 6.1 Write property test for session logging
- **Property 10: Session logging completeness**
- **Validates: Requirements 4.1**
- [ ]* 6.2 Write property test for error logging
- **Property 12: Error logging with context**
- **Validates: Requirements 4.3**
- [x] 7. Checkpoint - Ensure all tests pass
- Ensure all tests pass, ask the user if questions arise.
- **Status**: ✅ Intégration complète effectuée
- EventCapture passe maintenant les actions au SessionManager
- SessionManager segmente les actions en sessions
- WorkflowDetector analyse les sessions et détecte les patterns
- SuggestionManager reçoit les notifications de workflows
- Orchestrator coordonne tous les composants
- [ ] 8. End-to-end integration test
- Create test script that simulates repeated actions
- Verify sessions are created
- Verify workflows are detected
- Verify suggestions are generated
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5_