v1.0 - Version stable: multi-PC, détection UI-DETR-1, 3 modes exécution
- Frontend v4 accessible sur réseau local (192.168.1.40) - Ports ouverts: 3002 (frontend), 5001 (backend), 5004 (dashboard) - Ollama GPU fonctionnel - Self-healing interactif - Dashboard confiance Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
149
examples/REAL_FUNCTIONALITY_TEST_IMPROVEMENTS.md
Normal file
149
examples/REAL_FUNCTIONALITY_TEST_IMPROVEMENTS.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# Real Functionality Test Improvements
|
||||
|
||||
## Analysis of test_faiss_reindex.py
|
||||
|
||||
### Key Issues with Original Tests
|
||||
|
||||
1. **Heavy Mock Usage**: Extensive use of `unittest.mock.Mock()` objects
|
||||
2. **Simulated Behavior**: Patching critical methods like `_train_ivf_index`
|
||||
3. **Fake Data**: Creating mock nodes instead of real `WorkflowNode` instances
|
||||
4. **No Integration**: Testing components in isolation
|
||||
|
||||
### Specific Improvements Made
|
||||
|
||||
#### 1. Replace Mocks with Real Model Instances
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
node = Mock()
|
||||
template = Mock()
|
||||
template.embedding_prototype = [0.1, 0.2, 0.3, 0.4]
|
||||
node.template = template
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
def _create_real_node_v1_format(self, embedding_list: list) -> WorkflowNode:
|
||||
embedding_proto = EmbeddingPrototype(
|
||||
provider="test_provider",
|
||||
vector_id="",
|
||||
min_cosine_similarity=0.8,
|
||||
sample_count=1
|
||||
)
|
||||
template = ScreenTemplate(
|
||||
window=WindowConstraint(),
|
||||
text=TextConstraint(),
|
||||
ui=UIConstraint(),
|
||||
embedding=embedding_proto
|
||||
)
|
||||
template.embedding_prototype = embedding_list
|
||||
return WorkflowNode(
|
||||
node_id="test_node_v1",
|
||||
name="Test Node V1",
|
||||
description="Test node with v1 format",
|
||||
template=template
|
||||
)
|
||||
```
|
||||
|
||||
#### 2. Test Real FAISS Operations
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
with patch.object(manager, '_train_ivf_index') as mock_train:
|
||||
count = manager.reindex(items, force_train_ivf=True)
|
||||
mock_train.assert_called_once()
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# Test actual IVF training with real data
|
||||
count = manager.reindex(items, force_train_ivf=True)
|
||||
assert manager.is_trained # Verify real training occurred
|
||||
assert manager.index.ntotal == 10
|
||||
# Test that search actually works after training
|
||||
results = manager.search_similar(query_vector, k=3)
|
||||
assert len(results) > 0
|
||||
assert results[0].similarity > 0.95
|
||||
```
|
||||
#### 3. Use Real File System Operations
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
# Mock file operations
|
||||
with tempfile.NamedTemporaryFile(suffix='.npy', delete=False) as tmp:
|
||||
test_vector = np.array([0.5, 0.6, 0.7, 0.8], dtype=np.float32)
|
||||
np.save(tmp.name, test_vector)
|
||||
tmp_path = tmp.name
|
||||
|
||||
# Mock node with embedding.vector_id
|
||||
node = Mock()
|
||||
template = Mock()
|
||||
embedding = Mock()
|
||||
embedding.vector_id = tmp_path
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# Real file operations with proper cleanup
|
||||
def setup_method(self):
|
||||
self.temp_dir = Path(tempfile.mkdtemp())
|
||||
|
||||
def teardown_method(self):
|
||||
if self.temp_dir.exists():
|
||||
shutil.rmtree(self.temp_dir)
|
||||
|
||||
# Real node creation with actual file
|
||||
test_vector = np.array([0.5, 0.6, 0.7, 0.8], dtype=np.float32)
|
||||
vector_file = self.temp_dir / "test_vector.npy"
|
||||
np.save(vector_file, test_vector)
|
||||
node = self._create_real_node_v2_format(str(vector_file))
|
||||
```
|
||||
|
||||
#### 4. Test Integration Between Components
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
# Mock faiss_manager.reindex
|
||||
with patch.object(pipeline.faiss_manager, 'reindex') as mock_reindex:
|
||||
mock_reindex.return_value = 2
|
||||
pipeline._index_workflow_embeddings(workflow)
|
||||
mock_reindex.assert_called_once()
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# Test real integration
|
||||
workflow = self._create_real_workflow_with_nodes()
|
||||
assert self.pipeline.faiss_manager.index.ntotal == 0
|
||||
|
||||
# Real indexing operation
|
||||
self.pipeline._index_workflow_embeddings(workflow)
|
||||
|
||||
# Verify real results
|
||||
assert self.pipeline.faiss_manager.index.ntotal == 2
|
||||
query_vector = np.array([0.1, 0.2, 0.3], dtype=np.float32)
|
||||
results = self.pipeline.faiss_manager.search_similar(query_vector, k=2)
|
||||
assert results[0].embedding_id == "node1"
|
||||
```
|
||||
|
||||
### Benefits of Real Functionality Tests
|
||||
|
||||
1. **Catches Real Bugs**: Tests actual behavior, not mocked behavior
|
||||
2. **Integration Testing**: Verifies components work together correctly
|
||||
3. **Performance Validation**: Tests with real data sizes and operations
|
||||
4. **Regression Prevention**: Changes to internal implementation don't break tests
|
||||
5. **Documentation Value**: Tests show how components actually work
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
- **Fast Execution**: Tests still run quickly (< 1s each)
|
||||
- **Isolated**: Each test uses temporary directories
|
||||
- **Cleanup**: Proper resource cleanup prevents test pollution
|
||||
- **Small Data**: Uses minimal data sizes for speed while testing real functionality
|
||||
|
||||
### Maintained Test Reliability
|
||||
|
||||
- **Deterministic**: Uses fixed random seeds where needed
|
||||
- **Independent**: Tests don't depend on each other
|
||||
- **Robust**: Handles edge cases gracefully
|
||||
- **Clear Assertions**: Specific, meaningful assertions about behavior
|
||||
Reference in New Issue
Block a user