v1.0 - Version stable: multi-PC, détection UI-DETR-1, 3 modes exécution
- Frontend v4 accessible sur réseau local (192.168.1.40) - Ports ouverts: 3002 (frontend), 5001 (backend), 5004 (dashboard) - Ollama GPU fonctionnel - Self-healing interactif - Dashboard confiance Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Dom
342
docs/guides/TRAINING_GUIDE.md
Normal file
342
docs/guides/TRAINING_GUIDE.md
Normal file
@@ -0,0 +1,342 @@
|
||||
# Guide d'Utilisation du Training System
|
||||
|
||||
## Vue d'Ensemble
|
||||
|
||||
Le Training System permet de collecter des données pendant l'utilisation réelle, puis d'entraîner un modèle personnalisé offline avant le déploiement en production.
|
||||
|
||||
## Workflow Complet
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 1: COLLECTE (1-2 semaines) │
|
||||
│ Mode: OBSERVATION │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 2: ENTRAÎNEMENT OFFLINE (1 jour) │
|
||||
│ Traitement des données collectées │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 3: VALIDATION (3-5 jours) │
|
||||
│ Mode: COACHING avec modèle entraîné │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 4: PRODUCTION (continu) │
|
||||
│ Mode: AUTO_CONFIRMED │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Collecte de Données
|
||||
|
||||
### Initialisation
|
||||
|
||||
```python
|
||||
from core.training.training_data_collector import TrainingDataCollector
|
||||
|
||||
# Créer le collecteur
|
||||
collector = TrainingDataCollector(output_dir="my_training_data")
|
||||
```
|
||||
|
||||
### Pendant l'Utilisation
|
||||
|
||||
```python
|
||||
# Démarrer une session
|
||||
collector.start_session(
|
||||
session_id="session_001",
|
||||
workflow_id="email_workflow" # Optionnel
|
||||
)
|
||||
|
||||
# Enregistrer les screenshots
|
||||
collector.record_screenshot("/path/to/screenshot.png")
|
||||
|
||||
# Enregistrer les actions
|
||||
collector.record_action({
|
||||
'type': 'click',
|
||||
'target': 'compose_button',
|
||||
'position': (100, 200)
|
||||
})
|
||||
|
||||
# Enregistrer les embeddings
|
||||
collector.record_embedding("/path/to/embedding.npy")
|
||||
|
||||
# Enregistrer les corrections utilisateur
|
||||
collector.record_correction({
|
||||
'type': 'wrong_target',
|
||||
'expected': 'send_button',
|
||||
'actual': 'cancel_button'
|
||||
})
|
||||
|
||||
# Terminer la session
|
||||
collector.end_session(
|
||||
success=True,
|
||||
metadata={'duration_ms': 1500}
|
||||
)
|
||||
```
|
||||
|
||||
### Export des Données
|
||||
|
||||
```python
|
||||
# Exporter le dataset complet
|
||||
training_set = collector.export_training_set("training_set.json")
|
||||
|
||||
print(f"Collecté: {training_set['metadata']['total_sessions']} sessions")
|
||||
print(f"Taux de succès: {training_set['metadata']['success_rate']:.2%}")
|
||||
```
|
||||
|
||||
**Objectif:** Collecter 50-100 sessions par workflow
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Entraînement Offline
|
||||
|
||||
### Configuration
|
||||
|
||||
```python
|
||||
from core.training.offline_trainer import OfflineTrainer, TrainingConfig
|
||||
|
||||
config = TrainingConfig(
|
||||
learning_rate=0.001,
|
||||
batch_size=32,
|
||||
num_epochs=10,
|
||||
validation_split=0.2,
|
||||
min_samples_per_workflow=5
|
||||
)
|
||||
|
||||
trainer = OfflineTrainer(config)
|
||||
```
|
||||
|
||||
### Entraînement Complet
|
||||
|
||||
```python
|
||||
# Charger les données
|
||||
training_data = trainer.load_training_data("training_set.json")
|
||||
|
||||
# Entraîner les prototypes
|
||||
prototypes = trainer.train_prototypes(training_data)
|
||||
print(f"Prototypes entraînés: {len(prototypes)} workflows")
|
||||
|
||||
# Entraîner les seuils optimaux
|
||||
thresholds = trainer.train_thresholds(training_data)
|
||||
print(f"Seuils optimisés: {len(thresholds)} workflows")
|
||||
|
||||
# Valider le modèle
|
||||
metrics = trainer.validate_model(training_data)
|
||||
print(f"Accuracy: {metrics['accuracy']:.2%}")
|
||||
|
||||
# Exporter le modèle
|
||||
model_path = trainer.export_trained_model("trained_model")
|
||||
print(f"Modèle exporté: {model_path}")
|
||||
```
|
||||
|
||||
### Pipeline Automatique
|
||||
|
||||
```python
|
||||
# Tout en une commande
|
||||
result = trainer.train_full_pipeline("training_set.json")
|
||||
|
||||
print(f"Workflows entraînés: {result.trained_workflows}")
|
||||
print(f"Accuracy: {result.validation_accuracy:.2%}")
|
||||
print(f"Temps: {result.training_time_seconds:.1f}s")
|
||||
print(f"Modèle: {result.model_path}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Validation
|
||||
|
||||
### Validation Basique
|
||||
|
||||
```python
|
||||
from core.training.model_validator import ModelValidator
|
||||
|
||||
validator = ModelValidator(
|
||||
min_accuracy=0.85,
|
||||
min_samples=20
|
||||
)
|
||||
|
||||
# Valider le modèle
|
||||
report = validator.validate_model(
|
||||
model_path="trained_model",
|
||||
test_data_path="test_set.json"
|
||||
)
|
||||
|
||||
print(f"Accuracy: {report.overall_accuracy:.2%}")
|
||||
print(f"Recommandation: {report.recommendation}")
|
||||
print(f"Issues: {len(report.issues)}")
|
||||
|
||||
# Sauvegarder le rapport
|
||||
validator.save_report(report, "validation_report.json")
|
||||
```
|
||||
|
||||
### Comparaison avec Baseline
|
||||
|
||||
```python
|
||||
# Comparer nouveau modèle vs baseline
|
||||
comparison = validator.compare_with_baseline(
|
||||
new_model_path="trained_model",
|
||||
baseline_model_path="baseline_model",
|
||||
test_data_path="test_set.json"
|
||||
)
|
||||
|
||||
print(f"Amélioration accuracy: {comparison['improvements']['accuracy_delta']:+.2%}")
|
||||
print(f"Recommandation: {comparison['recommendation']}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Déploiement en Production
|
||||
|
||||
### Charger le Modèle Entraîné
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
import json
|
||||
|
||||
# Charger les prototypes
|
||||
prototypes = np.load("trained_model/prototypes.npz")
|
||||
|
||||
# Charger les seuils
|
||||
with open("trained_model/thresholds.json", 'r') as f:
|
||||
thresholds = json.load(f)
|
||||
|
||||
# Utiliser dans NodeMatcher
|
||||
from core.graph.node_matcher import NodeMatcher
|
||||
|
||||
matcher = NodeMatcher(
|
||||
faiss_manager=faiss_mgr,
|
||||
prototypes=prototypes,
|
||||
thresholds=thresholds
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Métriques et Monitoring
|
||||
|
||||
### Statistiques de Collecte
|
||||
|
||||
```python
|
||||
stats = collector._calculate_statistics()
|
||||
|
||||
print(f"Sessions totales: {stats['total_sessions']}")
|
||||
print(f"Sessions réussies: {stats['successful_sessions']}")
|
||||
print(f"Actions moyennes/session: {stats['avg_actions_per_session']:.1f}")
|
||||
print(f"Taux de correction: {stats['correction_rate']:.2%}")
|
||||
```
|
||||
|
||||
### Métriques par Workflow
|
||||
|
||||
```python
|
||||
# Obtenir sessions d'un workflow spécifique
|
||||
email_sessions = collector.get_sessions_by_workflow("email_workflow")
|
||||
print(f"Sessions email: {len(email_sessions)}")
|
||||
|
||||
# Obtenir seulement les sessions réussies
|
||||
successful = collector.get_successful_sessions()
|
||||
print(f"Taux de succès: {len(successful)/len(collector.sessions):.2%}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bonnes Pratiques
|
||||
|
||||
### 1. Collecte de Données
|
||||
|
||||
- ✅ Collecter au moins 50 sessions par workflow
|
||||
- ✅ Varier les conditions (différents moments, états)
|
||||
- ✅ Enregistrer les corrections utilisateur
|
||||
- ✅ Marquer clairement succès/échec
|
||||
|
||||
### 2. Entraînement
|
||||
|
||||
- ✅ Utiliser validation split (20%)
|
||||
- ✅ Vérifier que min_samples_per_workflow est atteint
|
||||
- ✅ Surveiller les métriques de validation
|
||||
- ✅ Comparer avec baseline avant déploiement
|
||||
|
||||
### 3. Validation
|
||||
|
||||
- ✅ Tester sur données non vues
|
||||
- ✅ Vérifier accuracy > 85%
|
||||
- ✅ Analyser les workflows problématiques
|
||||
- ✅ Tester en mode COACHING avant AUTO
|
||||
|
||||
### 4. Production
|
||||
|
||||
- ✅ Déployer progressivement (COACHING → AUTO_CANDIDATE → AUTO_CONFIRMED)
|
||||
- ✅ Monitorer les performances
|
||||
- ✅ Collecter feedback continu
|
||||
- ✅ Ré-entraîner périodiquement
|
||||
|
||||
---
|
||||
|
||||
## Exemple Complet
|
||||
|
||||
```python
|
||||
# 1. COLLECTE (pendant 1-2 semaines)
|
||||
collector = TrainingDataCollector("production_data")
|
||||
|
||||
for session in user_sessions:
|
||||
collector.start_session(session.id, session.workflow_id)
|
||||
for action in session.actions:
|
||||
collector.record_action(action)
|
||||
collector.end_session(session.success)
|
||||
|
||||
collector.export_training_set("training_set.json")
|
||||
|
||||
# 2. ENTRAÎNEMENT (1 jour)
|
||||
trainer = OfflineTrainer()
|
||||
result = trainer.train_full_pipeline("training_set.json")
|
||||
|
||||
# 3. VALIDATION (avant déploiement)
|
||||
validator = ModelValidator()
|
||||
report = validator.validate_model(
|
||||
result.model_path,
|
||||
"test_set.json"
|
||||
)
|
||||
|
||||
if report.recommendation == "DEPLOY":
|
||||
print("✓ Modèle prêt pour production")
|
||||
else:
|
||||
print(f"⚠ {report.recommendation}: {report.issues}")
|
||||
|
||||
# 4. DÉPLOIEMENT
|
||||
# Charger et utiliser le modèle entraîné
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Accuracy Faible (<70%)
|
||||
|
||||
- Collecter plus de données
|
||||
- Vérifier la qualité des labels (succès/échec)
|
||||
- Augmenter min_samples_per_workflow
|
||||
- Vérifier que les workflows sont cohérents
|
||||
|
||||
### Overfitting
|
||||
|
||||
- Augmenter validation_split
|
||||
- Réduire num_epochs
|
||||
- Collecter données plus variées
|
||||
|
||||
### Workflows Non Détectés
|
||||
|
||||
- Vérifier min_samples_per_workflow
|
||||
- Collecter plus de sessions pour ce workflow
|
||||
- Vérifier que workflow_id est correct
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
Pour plus d'informations, voir:
|
||||
- `core/training/training_data_collector.py`
|
||||
- `core/training/offline_trainer.py`
|
||||
- `core/training/model_validator.py`
|
||||
- `examples/test_training_system.py`
|
||||
Reference in New Issue
Block a user