Initial commit
This commit is contained in:
Dom
338
EMBEDDING_SYSTEM_README.md
Normal file
338
EMBEDDING_SYSTEM_README.md
Normal file
@@ -0,0 +1,338 @@
|
||||
# Système d'Embeddings - Documentation Complète
|
||||
|
||||
## 📋 Vue d'Ensemble
|
||||
|
||||
Le système d'embeddings de GeniusIA v2 permet de:
|
||||
- Générer des représentations vectorielles d'images UI
|
||||
- Rechercher des workflows similaires via FAISS
|
||||
- S'adapter automatiquement aux workflows utilisateur via fine-tuning
|
||||
- Améliorer la précision au fil du temps
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# Installer les dépendances
|
||||
./installer_dependances_completes.sh
|
||||
|
||||
# Ou manuellement
|
||||
pip install torch torchvision open-clip-torch faiss-cpu
|
||||
```
|
||||
|
||||
### Utilisation Basique
|
||||
|
||||
```python
|
||||
from geniusia2.core.embedders import EmbeddingManager, FAISSIndex
|
||||
from PIL import Image
|
||||
|
||||
# 1. Initialiser
|
||||
manager = EmbeddingManager(model_name="clip")
|
||||
index = FAISSIndex(manager.get_dimension())
|
||||
|
||||
# 2. Générer un embedding
|
||||
image = Image.open("screenshot.png")
|
||||
embedding = manager.embed(image)
|
||||
|
||||
# 3. Indexer
|
||||
index.add(embedding.reshape(1, -1), [{"workflow_id": "submit_form"}])
|
||||
|
||||
# 4. Rechercher
|
||||
results = index.search(embedding, k=5)
|
||||
print(f"Meilleur match: {results[0]['metadata']}")
|
||||
```
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
### Guides
|
||||
|
||||
- **[Guide d'Intégration](EMBEDDING_SYSTEM_INTEGRATION_GUIDE.md)** - Comment intégrer dans l'Orchestrator
|
||||
- **[Optimisations](PERFORMANCE_OPTIMIZATIONS.md)** - Performance et optimisations
|
||||
- **[Benchmark Pix2Struct](PIX2STRUCT_BENCHMARK_RESULTS.md)** - Comparaison des modèles
|
||||
|
||||
### Specs
|
||||
|
||||
- **[Requirements](.kiro/specs/embedding-improvement/requirements.md)** - Exigences détaillées
|
||||
- **[Design](.kiro/specs/embedding-improvement/design.md)** - Architecture et design
|
||||
- **[Tasks](.kiro/specs/embedding-improvement/tasks.md)** - Plan d'implémentation
|
||||
|
||||
## 🧪 Tests
|
||||
|
||||
### Tests Unitaires
|
||||
|
||||
```bash
|
||||
# Test du système de base
|
||||
geniusia2/venv/bin/python test_embedding_system.py
|
||||
|
||||
# Test complet avec fine-tuning
|
||||
geniusia2/venv/bin/python test_complete_embedding_system.py
|
||||
|
||||
# Benchmark CLIP vs Pix2Struct
|
||||
geniusia2/venv/bin/python test_pix2struct_vs_clip.py
|
||||
```
|
||||
|
||||
### Outils de Debug
|
||||
|
||||
```bash
|
||||
# Debug un embedding
|
||||
python debug_embeddings.py embedding screenshot.png
|
||||
|
||||
# Debug une recherche FAISS
|
||||
python debug_embeddings.py search screenshot.png --index data/workflow_embeddings
|
||||
|
||||
# Debug le fine-tuning
|
||||
python debug_embeddings.py finetuning
|
||||
```
|
||||
|
||||
### Visualisation
|
||||
|
||||
```bash
|
||||
# Visualiser les embeddings en 2D (t-SNE)
|
||||
python visualize_embeddings.py --method tsne
|
||||
|
||||
# Ou avec UMAP (nécessite: pip install umap-learn)
|
||||
python visualize_embeddings.py --method umap
|
||||
```
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ EmbeddingManager │
|
||||
│ - Sélection de modèle (CLIP/Pix2Struct) │
|
||||
│ - Cache LRU (1000 entrées) │
|
||||
│ - Fallback automatique │
|
||||
└────────────────┬────────────────────────────────────────┘
|
||||
│
|
||||
┌────────┴────────┐
|
||||
▼ ▼
|
||||
┌──────────────┐ ┌──────────────────┐
|
||||
│ CLIPEmbedder │ │ LightweightFine │
|
||||
│ │ │ Tuner │
|
||||
│ - 512D │ │ - Auto-trigger │
|
||||
│ - 20ms/img │ │ - Non-blocking │
|
||||
│ - Fine-tune │ │ - Checkpoints │
|
||||
└──────┬───────┘ └──────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────┐
|
||||
│ FAISSIndex │
|
||||
│ - <10ms │
|
||||
│ - Persistence│
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
## 📊 Performance
|
||||
|
||||
| Opération | Temps | Notes |
|
||||
|-----------|-------|-------|
|
||||
| Embedding (CPU) | 20ms | Batch de 5 |
|
||||
| Embedding (GPU) | 5ms | 4x plus rapide |
|
||||
| Cache hit | <1ms | 20x plus rapide |
|
||||
| FAISS search (10k) | <10ms | k=5 |
|
||||
| Fine-tuning (6 ex) | 0.4s | Non-bloquant |
|
||||
|
||||
## 🎯 Modèles Disponibles
|
||||
|
||||
### CLIP ViT-B/32 (Recommandé) ✅
|
||||
|
||||
```python
|
||||
manager = EmbeddingManager(model_name="clip")
|
||||
```
|
||||
|
||||
**Avantages:**
|
||||
- ⚡ Rapide (20ms/image)
|
||||
- 🎯 Précis pour UI
|
||||
- 💾 Léger (2GB)
|
||||
- ✅ Fine-tuning supporté
|
||||
|
||||
**Inconvénients:**
|
||||
- Généraliste (pas spécialisé UI)
|
||||
|
||||
### Pix2Struct (Non Recommandé) ❌
|
||||
|
||||
```python
|
||||
manager = EmbeddingManager(model_name="pix2struct")
|
||||
```
|
||||
|
||||
**Avantages:**
|
||||
- Spécialisé pour documents/UI
|
||||
|
||||
**Inconvénients:**
|
||||
- 🐌 Très lent (2900ms/image, 146x plus lent)
|
||||
- 😕 Moins précis (9x moins de discrimination)
|
||||
- 💾 Lourd (4GB)
|
||||
|
||||
**Verdict:** Utiliser CLIP
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Recommandée (Production)
|
||||
|
||||
```python
|
||||
config = {
|
||||
"embedding": {
|
||||
"model": "clip",
|
||||
"cache_size": 1000,
|
||||
"device": "cpu", # ou "cuda" si GPU disponible
|
||||
"fallback_enabled": True
|
||||
},
|
||||
"fine_tuning": {
|
||||
"enabled": True,
|
||||
"trigger_threshold": 10,
|
||||
"max_examples": 1000
|
||||
},
|
||||
"faiss": {
|
||||
"index_path": "data/workflow_embeddings"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Haute Performance (GPU)
|
||||
|
||||
```python
|
||||
config = {
|
||||
"embedding": {
|
||||
"model": "clip",
|
||||
"cache_size": 5000, # Plus de cache
|
||||
"device": "cuda", # GPU
|
||||
"fallback_enabled": True
|
||||
},
|
||||
"fine_tuning": {
|
||||
"enabled": True,
|
||||
"trigger_threshold": 5, # Plus fréquent
|
||||
"max_examples": 2000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Problème: Dimension mismatch dans FAISS
|
||||
|
||||
```python
|
||||
# L'index se rebuild automatiquement
|
||||
if index.rebuild_if_needed(new_dimension):
|
||||
logger.warning("Index rebuilt")
|
||||
```
|
||||
|
||||
### Problème: Cache hit rate faible
|
||||
|
||||
```python
|
||||
stats = manager.get_stats()
|
||||
if stats['cache_hit_rate'] < 0.3:
|
||||
# Augmenter la taille du cache
|
||||
manager = EmbeddingManager(cache_size=5000)
|
||||
```
|
||||
|
||||
### Problème: Fine-tuning lent
|
||||
|
||||
```python
|
||||
# Vérifier qu'il tourne en thread séparé
|
||||
assert fine_tuner.training_thread.daemon == True
|
||||
|
||||
# Réduire le nombre d'exemples
|
||||
fine_tuner = LightweightFineTuner(trigger_threshold=5)
|
||||
```
|
||||
|
||||
### Problème: Mémoire insuffisante
|
||||
|
||||
```python
|
||||
# Réduire le cache
|
||||
manager = EmbeddingManager(cache_size=100)
|
||||
|
||||
# Ou utiliser GPU pour libérer RAM
|
||||
manager = EmbeddingManager(device="cuda")
|
||||
```
|
||||
|
||||
## 📈 Monitoring
|
||||
|
||||
### Statistiques du Cache
|
||||
|
||||
```python
|
||||
stats = manager.get_stats()
|
||||
print(f"Hit rate: {stats['cache_hit_rate']:.1%}")
|
||||
print(f"Size: {stats['cache_size']}/{stats['cache_capacity']}")
|
||||
```
|
||||
|
||||
### Statistiques du Fine-tuning
|
||||
|
||||
```python
|
||||
stats = fine_tuner.get_stats()
|
||||
print(f"Trainings: {stats['training_count']}")
|
||||
print(f"Examples: {stats['total_examples']}")
|
||||
|
||||
for metrics in stats['metrics_history']:
|
||||
print(f"Loss: {metrics['loss']:.4f}")
|
||||
```
|
||||
|
||||
### Statistiques FAISS
|
||||
|
||||
```python
|
||||
stats = index.get_stats()
|
||||
print(f"Embeddings: {stats['num_embeddings']}")
|
||||
print(f"Dimension: {stats['dimension']}")
|
||||
```
|
||||
|
||||
## 🔄 Migration depuis l'Ancien Système
|
||||
|
||||
### Avant
|
||||
|
||||
```python
|
||||
from .embeddings_manager import EmbeddingsManager
|
||||
|
||||
embeddings = EmbeddingsManager()
|
||||
embedding = embeddings.encode_image(numpy_bgr_image)
|
||||
```
|
||||
|
||||
### Après
|
||||
|
||||
```python
|
||||
from .embedders import EmbeddingManager
|
||||
from PIL import Image
|
||||
import cv2
|
||||
|
||||
manager = EmbeddingManager(model_name="clip")
|
||||
|
||||
# Convertir numpy BGR → PIL RGB
|
||||
image_rgb = cv2.cvtColor(numpy_bgr_image, cv2.COLOR_BGR2RGB)
|
||||
pil_image = Image.fromarray(image_rgb)
|
||||
|
||||
embedding = manager.embed(pil_image)
|
||||
```
|
||||
|
||||
## 📝 Changelog
|
||||
|
||||
### v2.0.0 (2024-11-19)
|
||||
|
||||
**Ajouté:**
|
||||
- ✅ Système d'embeddings modulaire (EmbedderBase)
|
||||
- ✅ CLIPEmbedder avec batch support
|
||||
- ✅ FAISSIndex corrigé (dimensions + persistence)
|
||||
- ✅ EmbeddingManager avec cache LRU
|
||||
- ✅ LightweightFineTuner (auto-trigger, non-bloquant)
|
||||
- ✅ Pix2StructEmbedder (optionnel)
|
||||
- ✅ Tests complets (100% passed)
|
||||
- ✅ Outils de debug et visualisation
|
||||
- ✅ Documentation complète
|
||||
|
||||
**Corrigé:**
|
||||
- ✅ Erreurs de dimension FAISS
|
||||
- ✅ Persistence FAISS
|
||||
- ✅ Performance (cache, batch)
|
||||
|
||||
**Déprécié:**
|
||||
- ⚠️ Ancien EmbeddingsManager (toujours supporté)
|
||||
|
||||
## 🤝 Support
|
||||
|
||||
Pour toute question:
|
||||
1. Consulter cette documentation
|
||||
2. Lire les guides dans le dossier racine
|
||||
3. Exécuter les tests de debug
|
||||
4. Vérifier les specs dans `.kiro/specs/embedding-improvement/`
|
||||
|
||||
## 📄 Licence
|
||||
|
||||
Propriétaire - GeniusIA v2
|
||||
|
||||
Reference in New Issue
Block a user