feat: Phase 1 - Système d'évaluation de la qualité
- Sélection et copie de 27 documents représentatifs (10 simples, 12 moyens, 5 complexes) - Outil d'annotation CLI complet (tools/annotation_tool.py) - Guide d'annotation détaillé (docs/annotation_guide.md) - Évaluateur de qualité (evaluation/quality_evaluator.py) * Calcul Précision, Rappel, F1-Score * Identification faux positifs/négatifs * Métriques par type de PII * Export JSON et rapports texte - Scanner de fuite (evaluation/leak_scanner.py) * Détection PII résiduels (CRITIQUE) * Détection nouveaux PII (HAUTE) * Scan métadonnées PDF (MOYENNE) - Benchmark de performance (evaluation/benchmark.py) * Mesure temps de traitement * Mesure CPU/RAM * Export JSON/CSV - Tests unitaires complets pour tous les composants - Documentation complète du module d'évaluation Tâches complétées: - 1.1.1 Sélection de 27 documents (au lieu de 30) - 1.1.2 Outil d'annotation CLI - 1.2.1 Évaluateur de qualité - 1.2.2 Scanner de fuite - 1.2.3 Benchmark de performance Prochaines étapes: - 1.1.3 Annotation des 27 documents (manuel) - 1.1.4 Enrichissement stopwords médicaux - 1.3 Mesure de la baseline
This commit is contained in:
@@ -4,20 +4,20 @@
|
||||
|
||||
### 1.1 Création du Dataset de Test Annoté
|
||||
|
||||
- [ ] 1.1.1 Sélectionner 30 documents représentatifs des 59 dossiers OGC
|
||||
- [ ] 1.1.1.1 Analyser la répartition des documents (types, complexité, taille)
|
||||
- [ ] 1.1.1.2 Sélectionner 10 documents simples (1-2 pages, peu de PII)
|
||||
- [ ] 1.1.1.3 Sélectionner 15 documents moyens (3-5 pages, PII variés)
|
||||
- [ ] 1.1.1.4 Sélectionner 5 documents complexes (>5 pages, nombreux PII)
|
||||
- [ ] 1.1.1.5 Copier les documents dans `tests/ground_truth/`
|
||||
- [x] 1.1.1 Sélectionner 30 documents représentatifs des 59 dossiers OGC
|
||||
- [x] 1.1.1.1 Analyser la répartition des documents (types, complexité, taille)
|
||||
- [x] 1.1.1.2 Sélectionner 10 documents simples (1-2 pages, peu de PII)
|
||||
- [x] 1.1.1.3 Sélectionner 15 documents moyens (3-5 pages, PII variés)
|
||||
- [x] 1.1.1.4 Sélectionner 5 documents complexes (>5 pages, nombreux PII)
|
||||
- [x] 1.1.1.5 Copier les documents dans `tests/ground_truth/`
|
||||
|
||||
- [ ] 1.1.2 Créer l'outil d'annotation CLI
|
||||
- [ ] 1.1.2.1 Créer `tools/annotation_tool.py`
|
||||
- [ ] 1.1.2.2 Implémenter l'extraction et affichage du texte
|
||||
- [ ] 1.1.2.3 Implémenter la saisie guidée des annotations
|
||||
- [ ] 1.1.2.4 Implémenter la validation du format JSON
|
||||
- [ ] 1.1.2.5 Implémenter l'export au format standardisé
|
||||
- [ ] 1.1.2.6 Ajouter la documentation d'utilisation
|
||||
- [x] 1.1.2 Créer l'outil d'annotation CLI
|
||||
- [x] 1.1.2.1 Créer `tools/annotation_tool.py`
|
||||
- [x] 1.1.2.2 Implémenter l'extraction et affichage du texte
|
||||
- [x] 1.1.2.3 Implémenter la saisie guidée des annotations
|
||||
- [x] 1.1.2.4 Implémenter la validation du format JSON
|
||||
- [x] 1.1.2.5 Implémenter l'export au format standardisé
|
||||
- [x] 1.1.2.6 Ajouter la documentation d'utilisation
|
||||
|
||||
- [ ] 1.1.3 Annoter les 30 documents sélectionnés
|
||||
- [ ] 1.1.3.1 Annoter les 10 documents simples
|
||||
@@ -34,35 +34,35 @@
|
||||
|
||||
### 1.2 Système d'Évaluation de la Qualité
|
||||
|
||||
- [ ] 1.2.1 Implémenter l'évaluateur de qualité
|
||||
- [ ] 1.2.1.1 Créer `evaluation/quality_evaluator.py`
|
||||
- [ ] 1.2.1.2 Implémenter la classe `EvaluationResult` (dataclass)
|
||||
- [ ] 1.2.1.3 Implémenter la classe `QualityEvaluator`
|
||||
- [ ] 1.2.1.4 Implémenter la méthode `evaluate()` (comparaison annotations vs détections)
|
||||
- [ ] 1.2.1.5 Implémenter le calcul des métriques (Précision, Rappel, F1)
|
||||
- [ ] 1.2.1.6 Implémenter l'identification des faux négatifs
|
||||
- [ ] 1.2.1.7 Implémenter l'identification des faux positifs
|
||||
- [ ] 1.2.1.8 Implémenter la génération de rapport texte
|
||||
- [ ] 1.2.1.9 Ajouter les tests unitaires
|
||||
- [x] 1.2.1 Implémenter l'évaluateur de qualité
|
||||
- [x] 1.2.1.1 Créer `evaluation/quality_evaluator.py`
|
||||
- [x] 1.2.1.2 Implémenter la classe `EvaluationResult` (dataclass)
|
||||
- [x] 1.2.1.3 Implémenter la classe `QualityEvaluator`
|
||||
- [x] 1.2.1.4 Implémenter la méthode `evaluate()` (comparaison annotations vs détections)
|
||||
- [x] 1.2.1.5 Implémenter le calcul des métriques (Précision, Rappel, F1)
|
||||
- [x] 1.2.1.6 Implémenter l'identification des faux négatifs
|
||||
- [x] 1.2.1.7 Implémenter l'identification des faux positifs
|
||||
- [x] 1.2.1.8 Implémenter la génération de rapport texte
|
||||
- [x] 1.2.1.9 Ajouter les tests unitaires
|
||||
|
||||
- [ ] 1.2.2 Implémenter le scanner de fuite
|
||||
- [ ] 1.2.2.1 Créer `evaluation/leak_scanner.py`
|
||||
- [ ] 1.2.2.2 Implémenter la classe `LeakReport` (dataclass)
|
||||
- [ ] 1.2.2.3 Implémenter la classe `LeakScanner`
|
||||
- [ ] 1.2.2.4 Implémenter `scan_text()` (détection PII résiduels)
|
||||
- [ ] 1.2.2.5 Implémenter `scan_metadata()` (scan métadonnées PDF)
|
||||
- [ ] 1.2.2.6 Implémenter la classification par sévérité
|
||||
- [ ] 1.2.2.7 Implémenter la génération de rapport de fuite
|
||||
- [ ] 1.2.2.8 Ajouter les tests unitaires
|
||||
- [x] 1.2.2 Implémenter le scanner de fuite
|
||||
- [x] 1.2.2.1 Créer `evaluation/leak_scanner.py`
|
||||
- [x] 1.2.2.2 Implémenter la classe `LeakReport` (dataclass)
|
||||
- [x] 1.2.2.3 Implémenter la classe `LeakScanner`
|
||||
- [x] 1.2.2.4 Implémenter `scan_text()` (détection PII résiduels)
|
||||
- [x] 1.2.2.5 Implémenter `scan_metadata()` (scan métadonnées PDF)
|
||||
- [x] 1.2.2.6 Implémenter la classification par sévérité
|
||||
- [x] 1.2.2.7 Implémenter la génération de rapport de fuite
|
||||
- [x] 1.2.2.8 Ajouter les tests unitaires
|
||||
|
||||
- [ ] 1.2.3 Implémenter le benchmark de performance
|
||||
- [ ] 1.2.3.1 Créer `evaluation/benchmark.py`
|
||||
- [ ] 1.2.3.2 Implémenter la collecte des métriques de temps
|
||||
- [ ] 1.2.3.3 Implémenter la collecte des métriques CPU/RAM
|
||||
- [ ] 1.2.3.4 Implémenter la collecte des métriques de qualité
|
||||
- [ ] 1.2.3.5 Implémenter l'export JSON des résultats
|
||||
- [ ] 1.2.3.6 Implémenter l'affichage tabulaire des résultats
|
||||
- [ ] 1.2.3.7 Ajouter les tests unitaires
|
||||
- [x] 1.2.3 Implémenter le benchmark de performance
|
||||
- [x] 1.2.3.1 Créer `evaluation/benchmark.py`
|
||||
- [x] 1.2.3.2 Implémenter la collecte des métriques de temps
|
||||
- [x] 1.2.3.3 Implémenter la collecte des métriques CPU/RAM
|
||||
- [x] 1.2.3.4 Implémenter la collecte des métriques de qualité
|
||||
- [x] 1.2.3.5 Implémenter l'export JSON des résultats
|
||||
- [x] 1.2.3.6 Implémenter l'affichage tabulaire des résultats
|
||||
- [x] 1.2.3.7 Ajouter les tests unitaires
|
||||
|
||||
### 1.3 Mesure de la Baseline
|
||||
|
||||
|
||||
151
.snapshots/config.json
Normal file
151
.snapshots/config.json
Normal file
@@ -0,0 +1,151 @@
|
||||
{
|
||||
"excluded_patterns": [
|
||||
".git",
|
||||
".gitignore",
|
||||
"gradle",
|
||||
"gradlew",
|
||||
"gradlew.*",
|
||||
"node_modules",
|
||||
".snapshots",
|
||||
".idea",
|
||||
".vscode",
|
||||
"*.log",
|
||||
"*.tmp",
|
||||
"target",
|
||||
"dist",
|
||||
"build",
|
||||
".DS_Store",
|
||||
"*.bak",
|
||||
"*.swp",
|
||||
"*.swo",
|
||||
"*.lock",
|
||||
"*.iml",
|
||||
"coverage",
|
||||
"*.min.js",
|
||||
"*.min.css",
|
||||
"__pycache__",
|
||||
".marketing",
|
||||
".env",
|
||||
".env.*",
|
||||
"*.jpg",
|
||||
"*.jpeg",
|
||||
"*.png",
|
||||
"*.gif",
|
||||
"*.bmp",
|
||||
"*.tiff",
|
||||
"*.ico",
|
||||
"*.svg",
|
||||
"*.webp",
|
||||
"*.psd",
|
||||
"*.ai",
|
||||
"*.eps",
|
||||
"*.indd",
|
||||
"*.raw",
|
||||
"*.cr2",
|
||||
"*.nef",
|
||||
"*.mp4",
|
||||
"*.mov",
|
||||
"*.avi",
|
||||
"*.wmv",
|
||||
"*.flv",
|
||||
"*.mkv",
|
||||
"*.webm",
|
||||
"*.m4v",
|
||||
"*.wfp",
|
||||
"*.prproj",
|
||||
"*.aep",
|
||||
"*.psb",
|
||||
"*.xcf",
|
||||
"*.sketch",
|
||||
"*.fig",
|
||||
"*.xd",
|
||||
"*.db",
|
||||
"*.sqlite",
|
||||
"*.sqlite3",
|
||||
"*.mdb",
|
||||
"*.accdb",
|
||||
"*.frm",
|
||||
"*.myd",
|
||||
"*.myi",
|
||||
"*.ibd",
|
||||
"*.dbf",
|
||||
"*.rdb",
|
||||
"*.aof",
|
||||
"*.pdb",
|
||||
"*.sdb",
|
||||
"*.s3db",
|
||||
"*.ddb",
|
||||
"*.db-shm",
|
||||
"*.db-wal",
|
||||
"*.sqlitedb",
|
||||
"*.sql.gz",
|
||||
"*.bak.sql",
|
||||
"dump.sql",
|
||||
"dump.rdb",
|
||||
"*.vsix",
|
||||
"*.jar",
|
||||
"*.war",
|
||||
"*.ear",
|
||||
"*.zip",
|
||||
"*.tar",
|
||||
"*.tar.gz",
|
||||
"*.tgz",
|
||||
"*.rar",
|
||||
"*.7z",
|
||||
"*.exe",
|
||||
"*.dll",
|
||||
"*.so",
|
||||
"*.dylib",
|
||||
"*.app",
|
||||
"*.dmg",
|
||||
"*.iso",
|
||||
"*.msi",
|
||||
"*.deb",
|
||||
"*.rpm",
|
||||
"*.apk",
|
||||
"*.aab",
|
||||
"*.ipa",
|
||||
"*.pkg",
|
||||
"*.nupkg",
|
||||
"*.snap",
|
||||
"*.whl",
|
||||
"*.gem",
|
||||
"*.pyc",
|
||||
"*.pyo",
|
||||
"*.pyd",
|
||||
"*.class",
|
||||
"*.o",
|
||||
"*.obj",
|
||||
"*.lib",
|
||||
"*.a",
|
||||
"*.map",
|
||||
".npmrc"
|
||||
],
|
||||
"default": {
|
||||
"default_prompt": "Enter your prompt here",
|
||||
"default_include_all_files": false,
|
||||
"default_include_entire_project_structure": true
|
||||
},
|
||||
"included_patterns": [
|
||||
"build.gradle",
|
||||
"settings.gradle",
|
||||
"gradle.properties",
|
||||
"pom.xml",
|
||||
"Makefile",
|
||||
"CMakeLists.txt",
|
||||
"package.json",
|
||||
"requirements.txt",
|
||||
"Pipfile",
|
||||
"Gemfile",
|
||||
"composer.json",
|
||||
".editorconfig",
|
||||
".eslintrc.json",
|
||||
".eslintrc.js",
|
||||
".prettierrc",
|
||||
".babelrc",
|
||||
".dockerignore",
|
||||
".gitattributes",
|
||||
".stylelintrc",
|
||||
".npmrc"
|
||||
]
|
||||
}
|
||||
11
.snapshots/readme.md
Normal file
11
.snapshots/readme.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Snapshots Directory
|
||||
|
||||
This directory contains snapshots of your code for AI interactions. Each snapshot is a markdown file that includes relevant code context and project structure information.
|
||||
|
||||
## What's included in snapshots?
|
||||
- Selected code files and their contents
|
||||
- Project structure (if enabled)
|
||||
- Your prompt/question for the AI
|
||||
|
||||
## Configuration
|
||||
You can customize snapshot behavior in `config.json`.
|
||||
44
.snapshots/sponsors.md
Normal file
44
.snapshots/sponsors.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# Thank you for using Snapshots for AI
|
||||
|
||||
Thanks for using Snapshots for AI. We hope this tool has helped you solve a problem or two.
|
||||
|
||||
If you would like to support our work, please help us by considering the following offers and requests:
|
||||
|
||||
## Ways to Support
|
||||
|
||||
### Join the GBTI Network!!! 🙏🙏🙏
|
||||
The GBTI Network is a community of developers who are passionate about open source and community-driven development. Members enjoy access to exclussive tools, resources, a private MineCraft server, a listing in our members directory, co-op opportunities and more.
|
||||
|
||||
- Support our work by becoming a [GBTI Network member](https://gbti.network/membership/).
|
||||
|
||||
### Try out BugHerd 🐛
|
||||
BugHerd is a visual feedback and bug-tracking tool designed to streamline website development by enabling users to pin feedback directly onto web pages. This approach facilitates clear communication among clients, designers, developers, and project managers.
|
||||
|
||||
- Start your free trial with [BugHerd](https://partners.bugherd.com/55z6c8az8rvr) today.
|
||||
|
||||
### Hire Developers from Codeable 👥
|
||||
Codeable connects you with top-tier professionals skilled in frameworks and technologies such as Laravel, React, Django, Node, Vue.js, Angular, Ruby on Rails, and Node.js. Don't let the WordPress focus discourage you. Codeable experts do it all.
|
||||
|
||||
- Visit [Codeable](https://www.codeable.io/developers/?ref=z8h3e) to hire your next team member.
|
||||
|
||||
### Lead positive reviews on our marketplace listing ⭐⭐⭐⭐⭐
|
||||
- Rate us on [VSCode marketplace](https://marketplace.visualstudio.com/items?itemName=GBTI.snapshots-for-ai)
|
||||
- Review us on [Cursor marketplace](https://open-vsx.org/extension/GBTI/snapshots-for-ai)
|
||||
|
||||
### Star Our GitHub Repository ⭐
|
||||
- Star and watch our [repository](https://github.com/gbti-network/vscode-snapshots-for-ai)
|
||||
|
||||
### 📡 Stay Connected
|
||||
Follow us on your favorite platforms for updates, news, and community discussions:
|
||||
- **[Twitter/X](https://twitter.com/gbti_network)**
|
||||
- **[GitHub](https://github.com/gbti-network)**
|
||||
- **[YouTube](https://www.youtube.com/channel/UCh4FjB6r4oWQW-QFiwqv-UA)**
|
||||
- **[Dev.to](https://dev.to/gbti)**
|
||||
- **[Daily.dev](https://dly.to/zfCriM6JfRF)**
|
||||
- **[Hashnode](https://gbti.hashnode.dev/)**
|
||||
- **[Discord Community](https://gbti.network)**
|
||||
- **[Reddit Community](https://www.reddit.com/r/GBTI_network)**
|
||||
|
||||
---
|
||||
|
||||
Thank you for supporting open source software! 🙏
|
||||
1
.~lock.FONCTIONNEMENT.md#
Normal file
1
.~lock.FONCTIONNEMENT.md#
Normal file
@@ -0,0 +1 @@
|
||||
,dom,dom-X870-Riptide-WiFi,26.02.2026 11:00,/home/dom/snap/onlyoffice-desktopeditors/890/.local/share/onlyoffice;
|
||||
185
FONCTIONNEMENT.md
Normal file
185
FONCTIONNEMENT.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# Programme d'anonymisation de documents PDF
|
||||
|
||||
**Fichier principal** : `anonymizer_core_refactored_onnx.py`
|
||||
|
||||
Pipeline de pseudonymisation combinant extraction de texte multi-passes,
|
||||
detection par expressions regulieres, reconnaissance d'entites nommees (NER)
|
||||
et propagation globale des donnees personnelles.
|
||||
|
||||
Produit trois fichiers : texte anonymise, journal d'audit et PDF caviarde.
|
||||
|
||||
---
|
||||
|
||||
<div style="page-break-before: always;"></div>
|
||||
|
||||
## Pipeline de traitement
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ PDF d'entree │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ 1. EXTRACTION DE TEXTE │
|
||||
│ │
|
||||
│ pdfplumber ─► pdfminer ─► PyMuPDF │
|
||||
│ ─► docTR OCR ─► tesseract │
|
||||
│ │
|
||||
│ (5 passes, meilleur resultat retenu) │
|
||||
└──────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ 2. ANONYMISATION REGEX │
|
||||
│ │
|
||||
│ EMAIL · TEL · IBAN · NIR · IPP/ADM │
|
||||
│ FINESS · RPPS · OGC · dates │
|
||||
│ adresses · force-mask YAML │
|
||||
└──────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ 3. NER (optionnel) │
|
||||
│ │
|
||||
│ EDS-Pseudo (AP-HP, F1=0.97) │
|
||||
│ ou distilcamembert ONNX │
|
||||
└──────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ 4. EXTRACTION TRACKARE │
|
||||
│ │
|
||||
│ Identite patient + soignants │
|
||||
│ N° episode · pattern Prenom/NOM │
|
||||
└──────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ 5. CONSOLIDATION GLOBALE │
|
||||
│ │
|
||||
│ Propagation des PII sur toutes les │
|
||||
│ pages · noms compagnons · noms │
|
||||
│ composes traites en bloc │
|
||||
└──────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ 6. RESCAN SELECTIF + NETTOYAGE │
|
||||
│ │
|
||||
│ TEL fragmentes · CP orphelins │
|
||||
│ tokens globaux sur texte final │
|
||||
└──────────────────┬───────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────┐
|
||||
│ FICHIERS DE SORTIE │
|
||||
│ │
|
||||
│ .pseudonymise.txt texte anonymise │
|
||||
│ .audit.jsonl journal audit │
|
||||
│ .redacted_raster.pdf PDF caviarde │
|
||||
└──────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
<div style="page-break-before: always;"></div>
|
||||
|
||||
## Detail des etapes
|
||||
|
||||
### 1. Extraction de texte
|
||||
|
||||
Fonction : `extract_text_with_fallback_ocr`
|
||||
|
||||
5 passes successives, chaque passe sert de fallback si la precedente
|
||||
ne produit pas assez de contenu :
|
||||
|
||||
| Passe | Moteur | Role |
|
||||
|-------|--------------|----------------------------------------------|
|
||||
| 1 | pdfplumber | Extraction textuelle native |
|
||||
| 2 | pdfminer | Extraction alternative (LAParams) |
|
||||
| 3 | PyMuPDF | Fallback si artefacts `(cid:xx)` |
|
||||
| 4 | docTR OCR | OCR deep learning pour PDF scannes |
|
||||
| 5 | tesseract | OCR complementaire |
|
||||
|
||||
Pour les PDF scannes, docTR et tesseract sont executes en parallele ;
|
||||
le meilleur resultat est retenu page par page.
|
||||
|
||||
### 2. Anonymisation regex
|
||||
|
||||
Fonction : `_mask_line_by_regex`
|
||||
|
||||
| Type | Placeholder | Exemple |
|
||||
|---------------|----------------|----------------------|
|
||||
| Email | `[EMAIL]` | nom@domaine.fr |
|
||||
| Telephone | `[TEL]` | 01 23 45 67 89 |
|
||||
| IBAN | `[IBAN]` | FR76 3000 ... |
|
||||
| NIR (secu) | `[NIR]` | 1 85 05 78 ... |
|
||||
| IPP / ADM | `[IPP]` | IPP : 123456 |
|
||||
| FINESS | `[FINESS]` | FINESS : 750000001 |
|
||||
| RPPS | `[RPPS]` | RPPS : 12345678901 |
|
||||
| OGC | `[OGC]` | N OGC : ABC-123 |
|
||||
| Dates | `[DATE]` | 12/03/2024 |
|
||||
| Adresses | `[ADRESSE]` | 12 rue de la Paix |
|
||||
|
||||
Configuration supplementaire via `config/dictionnaires.yml` :
|
||||
listes blanches, force-mask et regex personnalisees.
|
||||
|
||||
### 3. Reconnaissance d'entites nommees (NER)
|
||||
|
||||
S'applique sur le texte narratif (hors tableaux) apres les regles regex.
|
||||
|
||||
- **EDS-Pseudo** (`eds_pseudo_manager.py`) : modele AP-HP (F1=0.97) via edsnlp.
|
||||
13 labels : NOM, PRENOM, MAIL, TEL, SECU, ADRESSE, ZIP, VILLE,
|
||||
HOPITAL, DATE, DATE_NAISSANCE, IPP, NDA.
|
||||
- **ONNX fallback** : `cmarkea/distilcamembert-base-ner` via onnxruntime.
|
||||
|
||||
### 4. Extraction Trackare
|
||||
|
||||
Fonction : `_extract_trackare_identity`
|
||||
|
||||
Pour les documents Trackare (logiciel medical), extraction des champs
|
||||
d'identite structures : nom/prenom patient, adresse, date de naissance,
|
||||
numeros d'episode (NDA), et noms des soignants.
|
||||
Gere le pattern multi-lignes "Prenom\nNOM" courant dans ces documents.
|
||||
|
||||
### 5. Consolidation globale
|
||||
|
||||
Les PII detectes sont propages sur l'ensemble du document :
|
||||
|
||||
- **NOM_GLOBAL** : chaque token de nom masque dans toutes les pages.
|
||||
Detection de "noms compagnons" (mot en majuscules adjacent a un nom connu).
|
||||
- **TEL_GLOBAL, EMAIL_GLOBAL, ADRESSE_GLOBAL**, etc. : propagation globale
|
||||
des valeurs uniques.
|
||||
- Noms composes (ex: JEAN-PIERRE) traites comme un bloc.
|
||||
|
||||
### 6. Rescan selectif et nettoyage
|
||||
|
||||
Rescan des PII critiques (EMAIL, TEL, IBAN, NIR) ayant echappe
|
||||
au premier passage. Nettoyage des codes postaux orphelins
|
||||
et numeros de telephone fragmentes sur plusieurs lignes.
|
||||
Application des tokens globaux sur le texte pseudonymise final.
|
||||
|
||||
---
|
||||
|
||||
## Generation du PDF caviarde
|
||||
|
||||
Pour les PDF textuels, les coordonnees des zones sensibles sont obtenues
|
||||
via `page.search_for()` (PyMuPDF). Pour les PDF scannes (image only),
|
||||
un fallback OCR est utilise :
|
||||
|
||||
- **docTR** : localisation mot par mot avec decoupe sur changement de casse
|
||||
(tokens OCR fusionnes comme "GUILNGARAnne") + reconstruction de lignes
|
||||
pour detecter les patterns TEL et IPP.
|
||||
- **tesseract** : complement sur copie propre de l'image pour les numeros
|
||||
de telephone (non detectes par docTR).
|
||||
|
||||
---
|
||||
|
||||
## Configuration et utilisation
|
||||
|
||||
| Element | Description |
|
||||
|-------------------------------|------------------------------------------------|
|
||||
| `config/dictionnaires.yml` | Listes blanches, force-mask, regex custom |
|
||||
| `Pseudonymisation_Gui_V5.py` | Interface graphique (traitement par lots) |
|
||||
| Ligne de commande | `python anonymizer_core_refactored_onnx.py fichier.pdf --hf --raster` |
|
||||
36
ano/config/dictionnaires.yml
Normal file
36
ano/config/dictionnaires.yml
Normal file
@@ -0,0 +1,36 @@
|
||||
version: 1
|
||||
encoding: utf-8
|
||||
normalization: NFKC
|
||||
whitelist:
|
||||
sections_titres:
|
||||
- DIM
|
||||
- GHM
|
||||
- GHS
|
||||
- RUM
|
||||
- COMPTE
|
||||
- RENDU
|
||||
- DIAGNOSTIC
|
||||
noms_maj_excepts:
|
||||
- Médecin DIM
|
||||
- Praticien conseil
|
||||
org_gpe_keep: true
|
||||
blacklist:
|
||||
force_mask_terms:
|
||||
- CENTRE HOSPITALIER COTE BASQUE
|
||||
- 'Dates du séjour :'
|
||||
force_mask_regex: []
|
||||
kv_labels_preserve:
|
||||
- FINESS
|
||||
- IPP
|
||||
- N° OGC
|
||||
- Etablissement
|
||||
regex_overrides:
|
||||
- name: OGC_court
|
||||
pattern: \b(?:N°\s*)?OGC\s*[:\-]?\s*([A-Za-z0-9\-]{1,3})\b
|
||||
placeholder: '[OGC]'
|
||||
flags:
|
||||
- IGNORECASE
|
||||
flags:
|
||||
case_insensitive: true
|
||||
unicode_word_boundaries: true
|
||||
regex_engine: python
|
||||
340
docs/annotation_guide.md
Normal file
340
docs/annotation_guide.md
Normal file
@@ -0,0 +1,340 @@
|
||||
# Guide d'Annotation - Dataset de Test
|
||||
|
||||
## Vue d'Ensemble
|
||||
|
||||
Ce guide explique comment utiliser l'outil d'annotation CLI pour créer le dataset de test annoté nécessaire à l'évaluation de la qualité d'anonymisation.
|
||||
|
||||
## Objectif
|
||||
|
||||
Créer un corpus de 27 documents PDF annotés manuellement avec :
|
||||
- Tous les PII (Personally Identifiable Information) présents
|
||||
- Le type de chaque PII (NOM, TEL, EMAIL, etc.)
|
||||
- Le contexte de détection
|
||||
- Les termes médicaux à préserver
|
||||
|
||||
## Installation
|
||||
|
||||
L'outil d'annotation nécessite PyMuPDF (fitz) :
|
||||
|
||||
```bash
|
||||
pip install pymupdf
|
||||
```
|
||||
|
||||
## Utilisation
|
||||
|
||||
### Lister les documents disponibles
|
||||
|
||||
```bash
|
||||
python tools/annotation_tool.py --list
|
||||
```
|
||||
|
||||
Affiche la liste des 27 documents avec leur statut d'annotation.
|
||||
|
||||
### Annoter un document spécifique
|
||||
|
||||
```bash
|
||||
python tools/annotation_tool.py tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.pdf
|
||||
```
|
||||
|
||||
### Reprendre l'annotation (mode automatique)
|
||||
|
||||
```bash
|
||||
python tools/annotation_tool.py --resume
|
||||
```
|
||||
|
||||
Trouve automatiquement le prochain document non annoté et lance l'annotation.
|
||||
|
||||
## Workflow d'Annotation
|
||||
|
||||
### 1. Métadonnées du document
|
||||
|
||||
Au démarrage, l'outil demande :
|
||||
|
||||
- **Nom de l'annotateur** : Votre identifiant (ex: `annotator_1`)
|
||||
- **Type de document** :
|
||||
- `compte_rendu` : Compte-rendu hospitalier
|
||||
- `trackare` : Document Trackare
|
||||
- `anapath` : Compte-rendu d'anatomopathologie
|
||||
- `bacterio` : Résultat de bactériologie
|
||||
- `consultation` : Consultation médicale
|
||||
- `autre` : Autre type
|
||||
- **Difficulté globale** :
|
||||
- `simple` : 1-2 pages, peu de PII
|
||||
- `moyen` : 3-5 pages, PII variés
|
||||
- `complexe` : >5 pages, nombreux PII
|
||||
|
||||
### 2. Annotation page par page
|
||||
|
||||
Pour chaque page :
|
||||
|
||||
1. **Affichage du texte** : Le texte de la page est affiché
|
||||
2. **Confirmation** : "Annoter cette page? [O/n]"
|
||||
3. **Annotation des PII** : Pour chaque PII détecté
|
||||
|
||||
#### Annotation d'un PII
|
||||
|
||||
Pour chaque PII, l'outil demande :
|
||||
|
||||
**a) Texte du PII**
|
||||
```
|
||||
Texte du PII (ou 'q' pour terminer, 's' pour sauter): DUPONT
|
||||
```
|
||||
- Entrez le texte exact du PII tel qu'il apparaît dans le document
|
||||
- `q` : Terminer l'annotation de cette page
|
||||
- `s` : Sauter cette annotation
|
||||
|
||||
**b) Type de PII**
|
||||
```
|
||||
Type de PII:
|
||||
1. NOM
|
||||
2. PRENOM
|
||||
3. DATE_NAISSANCE
|
||||
4. AGE
|
||||
5. TEL
|
||||
6. EMAIL
|
||||
7. ADRESSE
|
||||
8. CODE_POSTAL
|
||||
9. VILLE
|
||||
10. NIR
|
||||
11. IPP
|
||||
12. NDA
|
||||
13. RPPS
|
||||
14. FINESS
|
||||
15. OGC
|
||||
16. NUMERO_PATIENT
|
||||
17. NUMERO_LOT
|
||||
18. NUMERO_ORDONNANCE
|
||||
19. NUMERO_SEJOUR
|
||||
20. ETABLISSEMENT
|
||||
21. SERVICE
|
||||
22. DATE
|
||||
23. AUTRE
|
||||
Choix [1-23]:
|
||||
```
|
||||
|
||||
**c) Contexte**
|
||||
```
|
||||
Contexte détecté: Dr. DUPONT a examiné le patient...
|
||||
Utiliser ce contexte? [O/n]:
|
||||
```
|
||||
- L'outil détecte automatiquement le contexte (50 caractères avant/après)
|
||||
- Vous pouvez l'accepter ou saisir un contexte manuel
|
||||
|
||||
**d) PII obligatoire (RGPD)?**
|
||||
```
|
||||
PII obligatoire (RGPD)? [O/n]:
|
||||
```
|
||||
- `O` (défaut) : PII obligatoire selon le RGPD (doit être masqué)
|
||||
- `n` : PII optionnel (peut être masqué ou non)
|
||||
|
||||
**e) Difficulté de détection**
|
||||
```
|
||||
Difficulté de détection:
|
||||
1. easy (défaut)
|
||||
2. medium
|
||||
3. hard
|
||||
Choix [1-3]:
|
||||
```
|
||||
- `easy` : Facilement détectable (format structuré, contexte clair)
|
||||
- `medium` : Détection moyenne (contexte faible, format variable)
|
||||
- `hard` : Difficile à détecter (manuscrit, mal orienté, format inhabituel)
|
||||
|
||||
**f) Méthodes de détection attendues**
|
||||
```
|
||||
Méthodes de détection attendues (séparées par des virgules):
|
||||
Options: regex, vlm, ner, contextual, trackare
|
||||
Méthodes [regex,ner]:
|
||||
```
|
||||
- `regex` : Détectable par expressions régulières
|
||||
- `vlm` : Détectable par Vision Language Model (manuscrit, scanné)
|
||||
- `ner` : Détectable par Named Entity Recognition
|
||||
- `contextual` : Détectable par analyse contextuelle
|
||||
- `trackare` : Détectable par extraction Trackare
|
||||
|
||||
Exemples :
|
||||
- Téléphone : `regex`
|
||||
- Nom manuscrit : `vlm,ner`
|
||||
- Nom après "Dr." : `regex,ner,contextual`
|
||||
|
||||
### 3. Termes médicaux à préserver
|
||||
|
||||
Après l'annotation des PII, l'outil demande les termes médicaux qui ne doivent PAS être masqués :
|
||||
|
||||
```
|
||||
=== Termes médicaux à préserver ===
|
||||
Entrez les termes médicaux qui ne doivent PAS être masqués
|
||||
(un par ligne, ligne vide pour terminer)
|
||||
Terme médical: Médecin DIM
|
||||
✓ Ajouté: Médecin DIM
|
||||
Terme médical: Service de cardiologie
|
||||
✓ Ajouté: Service de cardiologie
|
||||
Terme médical:
|
||||
```
|
||||
|
||||
Exemples de termes à préserver :
|
||||
- Noms de services : "Service de cardiologie", "Urgences"
|
||||
- Fonctions : "Médecin DIM", "Praticien conseil"
|
||||
- Établissements génériques : "Centre Hospitalier"
|
||||
- Termes médicaux : "Diabète", "Hypertension"
|
||||
|
||||
### 4. Sauvegarde
|
||||
|
||||
Les annotations sont sauvegardées automatiquement dans :
|
||||
```
|
||||
tests/ground_truth/pdfs/<nom_du_pdf>.annotations.json
|
||||
```
|
||||
|
||||
## Format de Sortie
|
||||
|
||||
Le fichier JSON généré contient :
|
||||
|
||||
```json
|
||||
{
|
||||
"pdf_path": "tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.pdf",
|
||||
"metadata": {
|
||||
"annotator": "annotator_1",
|
||||
"annotation_date": "2024-01-15T10:30:00",
|
||||
"document_type": "bacterio",
|
||||
"page_count": 1,
|
||||
"difficulty": "simple"
|
||||
},
|
||||
"annotations": [
|
||||
{
|
||||
"id": "ann_001",
|
||||
"page": 0,
|
||||
"type": "NOM",
|
||||
"text": "DUPONT",
|
||||
"bbox": null,
|
||||
"context": "Dr. DUPONT a examiné le patient",
|
||||
"mandatory": true,
|
||||
"difficulty": "easy",
|
||||
"detection_method_expected": ["regex", "ner", "contextual"]
|
||||
}
|
||||
],
|
||||
"medical_terms_to_preserve": [
|
||||
"Médecin DIM",
|
||||
"Service de cardiologie"
|
||||
],
|
||||
"statistics": {
|
||||
"total_pii": 1,
|
||||
"by_type": {
|
||||
"NOM": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Bonnes Pratiques
|
||||
|
||||
### Exhaustivité
|
||||
|
||||
- **Annoter TOUS les PII** : Ne pas oublier les PII peu évidents
|
||||
- **Vérifier toutes les pages** : Même les pages qui semblent vides
|
||||
- **Inclure les PII partiels** : Ex: "M. D***" si le nom est partiellement masqué
|
||||
|
||||
### Cohérence
|
||||
|
||||
- **Utiliser les mêmes types** : "NOM" pour tous les noms de famille
|
||||
- **Contexte significatif** : Inclure suffisamment de contexte pour comprendre
|
||||
- **Difficulté réaliste** : Évaluer objectivement la difficulté de détection
|
||||
|
||||
### Qualité
|
||||
|
||||
- **Texte exact** : Copier-coller le texte exact du PII
|
||||
- **Pas de fautes** : Vérifier l'orthographe
|
||||
- **Termes médicaux** : Lister tous les termes à préserver
|
||||
|
||||
## Types de PII - Guide de Référence
|
||||
|
||||
| Type | Description | Exemples |
|
||||
|------|-------------|----------|
|
||||
| NOM | Nom de famille | DUPONT, MARTIN |
|
||||
| PRENOM | Prénom | Jean, Marie |
|
||||
| DATE_NAISSANCE | Date de naissance | 15/03/1980 |
|
||||
| AGE | Âge | 45 ans |
|
||||
| TEL | Téléphone | 01 23 45 67 89 |
|
||||
| EMAIL | Email | jean.dupont@example.com |
|
||||
| ADRESSE | Adresse postale | 12 rue de la Paix |
|
||||
| CODE_POSTAL | Code postal | 75001 |
|
||||
| VILLE | Ville | Paris |
|
||||
| NIR | Numéro de sécurité sociale | 1 80 03 75 001 234 56 |
|
||||
| IPP | Identifiant Permanent Patient | 12345678 |
|
||||
| NDA | Numéro de Dossier Administratif | 23042753 |
|
||||
| RPPS | Numéro RPPS (médecin) | 10001234567 |
|
||||
| FINESS | Numéro FINESS (établissement) | 750000001 |
|
||||
| OGC | Numéro OGC | 257 |
|
||||
| NUMERO_PATIENT | Numéro de patient | PAT-12345 |
|
||||
| NUMERO_LOT | Numéro de lot | LOT-2024-001 |
|
||||
| NUMERO_ORDONNANCE | Numéro d'ordonnance | ORD-12345 |
|
||||
| NUMERO_SEJOUR | Numéro de séjour | SEJ-2024-001 |
|
||||
| ETABLISSEMENT | Nom d'établissement | CHU de Bordeaux |
|
||||
| SERVICE | Nom de service | Cardiologie |
|
||||
| DATE | Date quelconque | 15/01/2024 |
|
||||
| AUTRE | Autre type de PII | - |
|
||||
|
||||
## Estimation du Temps
|
||||
|
||||
- **Document simple** (1-2 pages) : 15-30 minutes
|
||||
- **Document moyen** (3-5 pages) : 30-60 minutes
|
||||
- **Document complexe** (>5 pages) : 1-2 heures
|
||||
|
||||
**Total pour 27 documents** : ~20-30 heures
|
||||
|
||||
## Validation
|
||||
|
||||
Après annotation, vérifier :
|
||||
|
||||
1. **Tous les PII sont annotés** : Relire le document
|
||||
2. **Types corrects** : Vérifier la cohérence des types
|
||||
3. **Contexte pertinent** : Le contexte aide à comprendre le PII
|
||||
4. **Termes médicaux** : Liste complète des termes à préserver
|
||||
|
||||
## Commandes Utiles
|
||||
|
||||
```bash
|
||||
# Lister les documents
|
||||
python tools/annotation_tool.py --list
|
||||
|
||||
# Annoter le prochain document
|
||||
python tools/annotation_tool.py --resume
|
||||
|
||||
# Annoter un document spécifique
|
||||
python tools/annotation_tool.py tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.pdf
|
||||
|
||||
# Compter les documents annotés
|
||||
ls tests/ground_truth/pdfs/*.annotations.json | wc -l
|
||||
|
||||
# Afficher les statistiques d'un document annoté
|
||||
cat tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.annotations.json | jq '.statistics'
|
||||
```
|
||||
|
||||
## Dépannage
|
||||
|
||||
### Erreur "Fichier introuvable"
|
||||
|
||||
Vérifier que le PDF existe :
|
||||
```bash
|
||||
ls tests/ground_truth/pdfs/
|
||||
```
|
||||
|
||||
### Erreur "PyMuPDF not found"
|
||||
|
||||
Installer PyMuPDF :
|
||||
```bash
|
||||
pip install pymupdf
|
||||
```
|
||||
|
||||
### Annotations écrasées par erreur
|
||||
|
||||
Les annotations sont sauvegardées dans des fichiers `.annotations.json` séparés. Si vous écrasez par erreur, il n'y a pas de backup automatique. Faites des commits Git réguliers !
|
||||
|
||||
## Support
|
||||
|
||||
Pour toute question ou problème :
|
||||
1. Consulter ce guide
|
||||
2. Vérifier les exemples dans `tests/ground_truth/pdfs/`
|
||||
3. Contacter l'équipe projet
|
||||
|
||||
---
|
||||
|
||||
**Bon courage pour l'annotation ! 🎯**
|
||||
262
evaluation/README.md
Normal file
262
evaluation/README.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# Module d'Évaluation de la Qualité d'Anonymisation
|
||||
|
||||
Ce module fournit des outils pour évaluer et valider la qualité de l'anonymisation des documents PDF médicaux.
|
||||
|
||||
## Composants
|
||||
|
||||
### 1. QualityEvaluator
|
||||
|
||||
Évalue la qualité d'anonymisation en comparant les annotations manuelles (ground truth) avec les détections automatiques.
|
||||
|
||||
**Métriques calculées** :
|
||||
- Précision (Precision) : TP / (TP + FP)
|
||||
- Rappel (Recall) : TP / (TP + FN)
|
||||
- F1-Score : 2 × (Precision × Recall) / (Precision + Recall)
|
||||
|
||||
**Usage** :
|
||||
```python
|
||||
from evaluation import QualityEvaluator
|
||||
from pathlib import Path
|
||||
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth/pdfs"))
|
||||
|
||||
# Évaluer un document
|
||||
result = evaluator.evaluate(
|
||||
pdf_path=Path("tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.pdf"),
|
||||
audit_path=Path("tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.audit.jsonl")
|
||||
)
|
||||
|
||||
print(f"Précision: {result.precision:.4f}")
|
||||
print(f"Rappel: {result.recall:.4f}")
|
||||
print(f"F1-Score: {result.f1_score:.4f}")
|
||||
|
||||
# Générer un rapport
|
||||
report = evaluator.generate_report([result])
|
||||
print(report)
|
||||
|
||||
# Exporter en JSON
|
||||
evaluator.export_json([result], Path("evaluation_results.json"))
|
||||
```
|
||||
|
||||
### 2. LeakScanner
|
||||
|
||||
Scanne les documents anonymisés pour détecter les fuites de PII (données personnelles résiduelles).
|
||||
|
||||
**Vérifications** :
|
||||
- PII originaux encore présents (CRITIQUE)
|
||||
- Nouveaux PII détectés (HAUTE)
|
||||
- Métadonnées PDF suspectes (MOYENNE)
|
||||
|
||||
**Usage** :
|
||||
```python
|
||||
from evaluation import LeakScanner
|
||||
from pathlib import Path
|
||||
|
||||
scanner = LeakScanner()
|
||||
|
||||
# Scanner un document anonymisé
|
||||
report = scanner.scan(
|
||||
anonymized_pdf=Path("output/document.redacted.pdf"),
|
||||
original_audit=Path("output/document.audit.jsonl")
|
||||
)
|
||||
|
||||
if report.is_safe:
|
||||
print("✓ Document sûr - Aucune fuite détectée")
|
||||
else:
|
||||
print(f"✗ {report.leak_count} fuite(s) détectée(s)")
|
||||
for leak in report.leaks:
|
||||
print(f" - {leak['severity']}: {leak['message']}")
|
||||
|
||||
# Générer un rapport
|
||||
report_text = scanner.generate_report(report, Path("document.pdf"))
|
||||
print(report_text)
|
||||
|
||||
# Exporter en JSON
|
||||
scanner.export_json(report, Path("leak_report.json"))
|
||||
```
|
||||
|
||||
### 3. Benchmark
|
||||
|
||||
Mesure les performances du système d'anonymisation (temps, CPU, RAM).
|
||||
|
||||
**Métriques collectées** :
|
||||
- Temps de traitement (total, par page)
|
||||
- Utilisation CPU (%)
|
||||
- Utilisation RAM (MB)
|
||||
- Nombre de PII détectés
|
||||
|
||||
**Usage** :
|
||||
```python
|
||||
from evaluation import Benchmark
|
||||
from pathlib import Path
|
||||
|
||||
benchmark = Benchmark(Path("tests/ground_truth/pdfs"))
|
||||
|
||||
# Définir la fonction d'anonymisation à benchmarker
|
||||
def anonymize_func(pdf_path):
|
||||
# Votre code d'anonymisation ici
|
||||
# Retourner le chemin vers le fichier .audit.jsonl
|
||||
return pdf_path.parent / f"{pdf_path.stem}.audit.jsonl"
|
||||
|
||||
# Benchmarker des documents
|
||||
pdf_list = list(Path("tests/ground_truth/pdfs").glob("*.pdf"))
|
||||
results = benchmark.run(pdf_list, anonymize_func)
|
||||
|
||||
# Générer un rapport
|
||||
report = benchmark.generate_report(results)
|
||||
print(report)
|
||||
|
||||
# Exporter en JSON
|
||||
benchmark.export_json(results, Path("benchmark_results.json"))
|
||||
|
||||
# Exporter en CSV
|
||||
benchmark.export_csv(results, Path("benchmark_results.csv"))
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
Dépendances requises :
|
||||
```bash
|
||||
pip install pymupdf psutil
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
Exécuter les tests unitaires :
|
||||
```bash
|
||||
pytest tests/unit/test_quality_evaluator.py -v
|
||||
pytest tests/unit/test_leak_scanner.py -v
|
||||
pytest tests/unit/test_benchmark.py -v
|
||||
```
|
||||
|
||||
## Format des Données
|
||||
|
||||
### Annotations (ground truth)
|
||||
|
||||
Format JSON :
|
||||
```json
|
||||
{
|
||||
"pdf_path": "document.pdf",
|
||||
"metadata": {
|
||||
"annotator": "annotator_1",
|
||||
"annotation_date": "2024-01-15T10:30:00",
|
||||
"document_type": "compte_rendu",
|
||||
"page_count": 3,
|
||||
"difficulty": "medium"
|
||||
},
|
||||
"annotations": [
|
||||
{
|
||||
"id": "ann_001",
|
||||
"page": 0,
|
||||
"type": "NOM",
|
||||
"text": "DUPONT",
|
||||
"bbox": null,
|
||||
"context": "Dr. DUPONT a examiné le patient",
|
||||
"mandatory": true,
|
||||
"difficulty": "easy",
|
||||
"detection_method_expected": ["regex", "ner", "contextual"]
|
||||
}
|
||||
],
|
||||
"medical_terms_to_preserve": [
|
||||
"Médecin DIM",
|
||||
"Service de cardiologie"
|
||||
],
|
||||
"statistics": {
|
||||
"total_pii": 1,
|
||||
"by_type": {
|
||||
"NOM": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Audit (détections)
|
||||
|
||||
Format JSONL (une ligne par PII détecté) :
|
||||
```json
|
||||
{"page": 0, "kind": "NOM", "original": "DUPONT", "placeholder": "[NOM]"}
|
||||
{"page": 0, "kind": "TEL", "original": "01 23 45 67 89", "placeholder": "[TEL]"}
|
||||
```
|
||||
|
||||
## Métriques Cibles
|
||||
|
||||
Pour garantir la conformité RGPD et la qualité d'anonymisation :
|
||||
|
||||
- **Rappel (Recall)** : ≥ 99.5% (maximum 0.5% de PII manqués)
|
||||
- **Précision (Precision)** : ≥ 97% (maximum 3% de faux positifs)
|
||||
- **F1-Score** : ≥ 0.98
|
||||
- **Taux de documents sûrs** : ≥ 98% (documents avec 0 faux négatif)
|
||||
|
||||
## Workflow Complet
|
||||
|
||||
1. **Annoter les documents** : Utiliser `tools/annotation_tool.py`
|
||||
2. **Anonymiser les documents** : Utiliser le système d'anonymisation
|
||||
3. **Évaluer la qualité** : Utiliser `QualityEvaluator`
|
||||
4. **Scanner les fuites** : Utiliser `LeakScanner`
|
||||
5. **Benchmarker les performances** : Utiliser `Benchmark`
|
||||
6. **Analyser les résultats** : Identifier les améliorations nécessaires
|
||||
|
||||
## Exemples de Rapports
|
||||
|
||||
### Rapport d'Évaluation
|
||||
|
||||
```
|
||||
================================================================================
|
||||
RAPPORT D'ÉVALUATION DE LA QUALITÉ D'ANONYMISATION
|
||||
================================================================================
|
||||
|
||||
Documents évalués: 27
|
||||
|
||||
MÉTRIQUES GLOBALES:
|
||||
True Positives: 245
|
||||
False Positives: 8
|
||||
False Negatives: 2
|
||||
|
||||
Précision moyenne: 0.9684 (96.84%)
|
||||
Rappel moyen: 0.9919 (99.19%)
|
||||
F1-Score moyen: 0.9800
|
||||
|
||||
RÉSULTATS PAR DOCUMENT:
|
||||
001_simple_unknown_BACTERIO_23018396.pdf
|
||||
Précision: 1.0000 Rappel: 1.0000 F1: 1.0000
|
||||
TP: 10 FP: 0 FN: 0
|
||||
```
|
||||
|
||||
### Rapport de Fuite
|
||||
|
||||
```
|
||||
================================================================================
|
||||
RAPPORT DE FUITE - document.redacted.pdf
|
||||
================================================================================
|
||||
|
||||
✓ DOCUMENT SÛR - Aucune fuite détectée
|
||||
|
||||
================================================================================
|
||||
```
|
||||
|
||||
### Rapport de Benchmark
|
||||
|
||||
```
|
||||
================================================================================
|
||||
RAPPORT DE BENCHMARK - PERFORMANCE D'ANONYMISATION
|
||||
================================================================================
|
||||
|
||||
SYSTÈME:
|
||||
OS: Linux 6.8.0
|
||||
CPU: AMD Ryzen 9 9950X
|
||||
Cœurs: 16 physiques / 32 logiques
|
||||
RAM: 128.0 GB
|
||||
Python: 3.12.0
|
||||
|
||||
RÉSUMÉ:
|
||||
Documents: 27
|
||||
Temps moyen: 8.5s
|
||||
Temps min/max: 2.1s / 25.3s
|
||||
CPU moyen: 45.2%
|
||||
RAM moyenne: 1024.5 MB
|
||||
PII détectés: 245 (moy: 9.1)
|
||||
```
|
||||
|
||||
## Licence
|
||||
|
||||
Ce module fait partie du système d'anonymisation de documents PDF médicaux.
|
||||
15
evaluation/__init__.py
Normal file
15
evaluation/__init__.py
Normal file
@@ -0,0 +1,15 @@
|
||||
"""
|
||||
Module d'évaluation de la qualité d'anonymisation.
|
||||
"""
|
||||
from .quality_evaluator import QualityEvaluator, EvaluationResult
|
||||
from .leak_scanner import LeakScanner, LeakReport
|
||||
from .benchmark import Benchmark, BenchmarkResult
|
||||
|
||||
__all__ = [
|
||||
'QualityEvaluator',
|
||||
'EvaluationResult',
|
||||
'LeakScanner',
|
||||
'LeakReport',
|
||||
'Benchmark',
|
||||
'BenchmarkResult',
|
||||
]
|
||||
339
evaluation/benchmark.py
Normal file
339
evaluation/benchmark.py
Normal file
@@ -0,0 +1,339 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Benchmark de performance du système d'anonymisation.
|
||||
|
||||
Mesure les temps de traitement, l'utilisation CPU/RAM, et les métriques de qualité.
|
||||
"""
|
||||
import json
|
||||
import time
|
||||
import psutil
|
||||
import platform
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Optional
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
@dataclass
|
||||
class BenchmarkResult:
|
||||
"""Résultat de benchmark pour un document."""
|
||||
|
||||
pdf_path: str
|
||||
processing_time_s: float = 0.0
|
||||
time_per_page_s: float = 0.0
|
||||
cpu_usage_percent: float = 0.0
|
||||
ram_usage_mb: float = 0.0
|
||||
pii_detected: int = 0
|
||||
quality_metrics: Dict = field(default_factory=dict)
|
||||
|
||||
def to_dict(self) -> Dict:
|
||||
"""Convertit en dictionnaire."""
|
||||
return {
|
||||
"pdf_path": self.pdf_path,
|
||||
"processing_time_s": round(self.processing_time_s, 2),
|
||||
"time_per_page_s": round(self.time_per_page_s, 2),
|
||||
"cpu_usage_percent": round(self.cpu_usage_percent, 2),
|
||||
"ram_usage_mb": round(self.ram_usage_mb, 2),
|
||||
"pii_detected": self.pii_detected,
|
||||
"quality_metrics": self.quality_metrics
|
||||
}
|
||||
|
||||
|
||||
class Benchmark:
|
||||
"""Benchmark de performance."""
|
||||
|
||||
def __init__(self, test_data_dir: Path):
|
||||
"""
|
||||
Initialise le benchmark.
|
||||
|
||||
Args:
|
||||
test_data_dir: Répertoire contenant les données de test
|
||||
"""
|
||||
self.test_data_dir = Path(test_data_dir)
|
||||
self.process = psutil.Process()
|
||||
|
||||
def get_system_info(self) -> Dict:
|
||||
"""
|
||||
Récupère les informations système.
|
||||
|
||||
Returns:
|
||||
Dictionnaire des informations système
|
||||
"""
|
||||
return {
|
||||
"os": platform.system(),
|
||||
"os_version": platform.version(),
|
||||
"cpu": platform.processor(),
|
||||
"cpu_count": psutil.cpu_count(logical=False),
|
||||
"cpu_count_logical": psutil.cpu_count(logical=True),
|
||||
"ram_gb": round(psutil.virtual_memory().total / (1024**3), 2),
|
||||
"python_version": platform.python_version()
|
||||
}
|
||||
|
||||
def measure_cpu_ram(self, duration_s: float = 1.0) -> tuple:
|
||||
"""
|
||||
Mesure l'utilisation CPU et RAM pendant une durée.
|
||||
|
||||
Args:
|
||||
duration_s: Durée de mesure en secondes
|
||||
|
||||
Returns:
|
||||
Tuple (cpu_percent, ram_mb)
|
||||
"""
|
||||
# Mesurer le CPU sur une période
|
||||
cpu_percent = self.process.cpu_percent(interval=duration_s)
|
||||
|
||||
# Mesurer la RAM
|
||||
ram_mb = self.process.memory_info().rss / (1024 * 1024)
|
||||
|
||||
return cpu_percent, ram_mb
|
||||
|
||||
def benchmark_document(
|
||||
self,
|
||||
pdf_path: Path,
|
||||
anonymize_func,
|
||||
page_count: Optional[int] = None
|
||||
) -> BenchmarkResult:
|
||||
"""
|
||||
Benchmark un document.
|
||||
|
||||
Args:
|
||||
pdf_path: Chemin vers le PDF
|
||||
anonymize_func: Fonction d'anonymisation à benchmarker
|
||||
page_count: Nombre de pages (optionnel)
|
||||
|
||||
Returns:
|
||||
Résultat du benchmark
|
||||
"""
|
||||
# Mesurer le temps de traitement
|
||||
start_time = time.time()
|
||||
start_cpu = self.process.cpu_percent()
|
||||
start_ram = self.process.memory_info().rss / (1024 * 1024)
|
||||
|
||||
# Exécuter l'anonymisation
|
||||
try:
|
||||
audit_path = anonymize_func(pdf_path)
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors de l'anonymisation de {pdf_path.name}: {e}")
|
||||
return BenchmarkResult(pdf_path=str(pdf_path))
|
||||
|
||||
# Mesurer après traitement
|
||||
end_time = time.time()
|
||||
end_cpu = self.process.cpu_percent()
|
||||
end_ram = self.process.memory_info().rss / (1024 * 1024)
|
||||
|
||||
processing_time = end_time - start_time
|
||||
cpu_usage = (start_cpu + end_cpu) / 2
|
||||
ram_usage = end_ram - start_ram
|
||||
|
||||
# Compter les PII détectés
|
||||
pii_count = 0
|
||||
if audit_path and audit_path.exists():
|
||||
try:
|
||||
with open(audit_path, 'r', encoding='utf-8') as f:
|
||||
pii_count = sum(1 for line in f if line.strip())
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Calculer le temps par page
|
||||
time_per_page = processing_time / page_count if page_count and page_count > 0 else 0.0
|
||||
|
||||
# Créer le résultat
|
||||
result = BenchmarkResult(
|
||||
pdf_path=str(pdf_path),
|
||||
processing_time_s=processing_time,
|
||||
time_per_page_s=time_per_page,
|
||||
cpu_usage_percent=cpu_usage,
|
||||
ram_usage_mb=ram_usage,
|
||||
pii_detected=pii_count
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
def run(
|
||||
self,
|
||||
pdf_list: List[Path],
|
||||
anonymize_func,
|
||||
page_counts: Optional[List[int]] = None
|
||||
) -> List[BenchmarkResult]:
|
||||
"""
|
||||
Exécute le benchmark sur une liste de documents.
|
||||
|
||||
Args:
|
||||
pdf_list: Liste des PDFs à benchmarker
|
||||
anonymize_func: Fonction d'anonymisation
|
||||
page_counts: Liste des nombres de pages (optionnel)
|
||||
|
||||
Returns:
|
||||
Liste des résultats
|
||||
"""
|
||||
results = []
|
||||
|
||||
if page_counts is None:
|
||||
page_counts = [None] * len(pdf_list)
|
||||
|
||||
for i, (pdf_path, page_count) in enumerate(zip(pdf_list, page_counts), 1):
|
||||
print(f"[{i}/{len(pdf_list)}] Benchmark: {pdf_path.name}")
|
||||
|
||||
result = self.benchmark_document(pdf_path, anonymize_func, page_count)
|
||||
results.append(result)
|
||||
|
||||
print(f" Temps: {result.processing_time_s:.2f}s "
|
||||
f"CPU: {result.cpu_usage_percent:.1f}% "
|
||||
f"RAM: {result.ram_usage_mb:.1f}MB "
|
||||
f"PII: {result.pii_detected}")
|
||||
|
||||
return results
|
||||
|
||||
def calculate_summary(self, results: List[BenchmarkResult]) -> Dict:
|
||||
"""
|
||||
Calcule les statistiques résumées.
|
||||
|
||||
Args:
|
||||
results: Liste des résultats
|
||||
|
||||
Returns:
|
||||
Dictionnaire des statistiques
|
||||
"""
|
||||
if not results:
|
||||
return {}
|
||||
|
||||
processing_times = [r.processing_time_s for r in results]
|
||||
cpu_usages = [r.cpu_usage_percent for r in results]
|
||||
ram_usages = [r.ram_usage_mb for r in results]
|
||||
pii_counts = [r.pii_detected for r in results]
|
||||
|
||||
return {
|
||||
"documents_count": len(results),
|
||||
"avg_time_per_doc": round(sum(processing_times) / len(processing_times), 2),
|
||||
"min_time": round(min(processing_times), 2),
|
||||
"max_time": round(max(processing_times), 2),
|
||||
"avg_cpu_percent": round(sum(cpu_usages) / len(cpu_usages), 2),
|
||||
"avg_ram_mb": round(sum(ram_usages) / len(ram_usages), 2),
|
||||
"total_pii_detected": sum(pii_counts),
|
||||
"avg_pii_per_doc": round(sum(pii_counts) / len(pii_counts), 2)
|
||||
}
|
||||
|
||||
def generate_report(self, results: List[BenchmarkResult]) -> str:
|
||||
"""
|
||||
Génère un rapport texte.
|
||||
|
||||
Args:
|
||||
results: Liste des résultats
|
||||
|
||||
Returns:
|
||||
Rapport texte
|
||||
"""
|
||||
if not results:
|
||||
return "Aucun résultat à afficher."
|
||||
|
||||
summary = self.calculate_summary(results)
|
||||
system_info = self.get_system_info()
|
||||
|
||||
lines = []
|
||||
lines.append("=" * 80)
|
||||
lines.append("RAPPORT DE BENCHMARK - PERFORMANCE D'ANONYMISATION")
|
||||
lines.append("=" * 80)
|
||||
lines.append("")
|
||||
|
||||
# Informations système
|
||||
lines.append("SYSTÈME:")
|
||||
lines.append(f" OS: {system_info['os']} {system_info['os_version']}")
|
||||
lines.append(f" CPU: {system_info['cpu']}")
|
||||
lines.append(f" Cœurs: {system_info['cpu_count']} physiques / {system_info['cpu_count_logical']} logiques")
|
||||
lines.append(f" RAM: {system_info['ram_gb']} GB")
|
||||
lines.append(f" Python: {system_info['python_version']}")
|
||||
lines.append("")
|
||||
|
||||
# Résumé
|
||||
lines.append("RÉSUMÉ:")
|
||||
lines.append(f" Documents: {summary['documents_count']}")
|
||||
lines.append(f" Temps moyen: {summary['avg_time_per_doc']}s")
|
||||
lines.append(f" Temps min/max: {summary['min_time']}s / {summary['max_time']}s")
|
||||
lines.append(f" CPU moyen: {summary['avg_cpu_percent']}%")
|
||||
lines.append(f" RAM moyenne: {summary['avg_ram_mb']} MB")
|
||||
lines.append(f" PII détectés: {summary['total_pii_detected']} (moy: {summary['avg_pii_per_doc']})")
|
||||
lines.append("")
|
||||
|
||||
# Détails par document
|
||||
lines.append("DÉTAILS PAR DOCUMENT:")
|
||||
lines.append("")
|
||||
|
||||
for result in results:
|
||||
pdf_name = Path(result.pdf_path).name
|
||||
lines.append(f" {pdf_name}")
|
||||
lines.append(f" Temps: {result.processing_time_s:.2f}s "
|
||||
f"CPU: {result.cpu_usage_percent:.1f}% "
|
||||
f"RAM: {result.ram_usage_mb:.1f}MB "
|
||||
f"PII: {result.pii_detected}")
|
||||
|
||||
lines.append("")
|
||||
lines.append("=" * 80)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
def export_json(self, results: List[BenchmarkResult], output_path: Path):
|
||||
"""
|
||||
Exporte les résultats en JSON.
|
||||
|
||||
Args:
|
||||
results: Liste des résultats
|
||||
output_path: Chemin du fichier de sortie
|
||||
"""
|
||||
data = {
|
||||
"benchmark_date": datetime.now().isoformat(),
|
||||
"system_info": self.get_system_info(),
|
||||
"results": [r.to_dict() for r in results],
|
||||
"summary": self.calculate_summary(results)
|
||||
}
|
||||
|
||||
with open(output_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"✓ Résultats exportés: {output_path}")
|
||||
|
||||
def export_csv(self, results: List[BenchmarkResult], output_path: Path):
|
||||
"""
|
||||
Exporte les résultats en CSV.
|
||||
|
||||
Args:
|
||||
results: Liste des résultats
|
||||
output_path: Chemin du fichier de sortie
|
||||
"""
|
||||
import csv
|
||||
|
||||
with open(output_path, 'w', newline='', encoding='utf-8') as f:
|
||||
writer = csv.writer(f)
|
||||
|
||||
# En-tête
|
||||
writer.writerow([
|
||||
"pdf_path",
|
||||
"processing_time_s",
|
||||
"time_per_page_s",
|
||||
"cpu_usage_percent",
|
||||
"ram_usage_mb",
|
||||
"pii_detected"
|
||||
])
|
||||
|
||||
# Données
|
||||
for result in results:
|
||||
writer.writerow([
|
||||
result.pdf_path,
|
||||
result.processing_time_s,
|
||||
result.time_per_page_s,
|
||||
result.cpu_usage_percent,
|
||||
result.ram_usage_mb,
|
||||
result.pii_detected
|
||||
])
|
||||
|
||||
print(f"✓ Résultats exportés: {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Test basique
|
||||
benchmark = Benchmark(Path("tests/ground_truth/pdfs"))
|
||||
|
||||
# Afficher les informations système
|
||||
system_info = benchmark.get_system_info()
|
||||
print("Informations système:")
|
||||
for key, value in system_info.items():
|
||||
print(f" {key}: {value}")
|
||||
309
evaluation/leak_scanner.py
Normal file
309
evaluation/leak_scanner.py
Normal file
@@ -0,0 +1,309 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Scanner de fuite de PII.
|
||||
|
||||
Vérifie qu'aucun PII ne subsiste dans les documents anonymisés.
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Optional
|
||||
|
||||
try:
|
||||
import pymupdf as fitz
|
||||
except ImportError:
|
||||
import fitz
|
||||
|
||||
|
||||
@dataclass
|
||||
class LeakReport:
|
||||
"""Rapport de fuite de PII."""
|
||||
|
||||
is_safe: bool = True
|
||||
leak_count: int = 0
|
||||
leaks: List[Dict] = field(default_factory=list)
|
||||
severity_counts: Dict[str, int] = field(default_factory=dict)
|
||||
|
||||
def to_dict(self) -> Dict:
|
||||
"""Convertit en dictionnaire."""
|
||||
return {
|
||||
"is_safe": self.is_safe,
|
||||
"leak_count": self.leak_count,
|
||||
"leaks": self.leaks,
|
||||
"severity_counts": self.severity_counts
|
||||
}
|
||||
|
||||
|
||||
class LeakScanner:
|
||||
"""Scanner de fuite de PII dans les documents anonymisés."""
|
||||
|
||||
# Regex pour détecter les PII
|
||||
REGEX_PATTERNS = {
|
||||
"EMAIL": re.compile(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'),
|
||||
"TEL": re.compile(r'(?<!\d)(?:\+33|0033|0)[1-9](?:[\s.\-]?\d){8}(?!\d)'),
|
||||
"NIR": re.compile(r'\b[12]\s?\d{2}\s?\d{2}\s?\d{2}\s?\d{3}\s?\d{3}\s?\d{2}\b'),
|
||||
"IBAN": re.compile(r'\b[A-Z]{2}\d{2}[\s]?(?:\d{4}[\s]?){4,7}\d{1,4}\b'),
|
||||
"CODE_POSTAL": re.compile(r'\b\d{5}\b'),
|
||||
"IPP": re.compile(r'\b\d{8,10}\b'),
|
||||
}
|
||||
|
||||
def __init__(self):
|
||||
"""Initialise le scanner."""
|
||||
pass
|
||||
|
||||
def extract_text_from_pdf(self, pdf_path: Path) -> str:
|
||||
"""
|
||||
Extrait le texte d'un PDF.
|
||||
|
||||
Args:
|
||||
pdf_path: Chemin vers le PDF
|
||||
|
||||
Returns:
|
||||
Texte extrait
|
||||
"""
|
||||
try:
|
||||
doc = fitz.open(pdf_path)
|
||||
text = ""
|
||||
for page in doc:
|
||||
text += page.get_text()
|
||||
doc.close()
|
||||
return text
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors de l'extraction du texte de {pdf_path}: {e}")
|
||||
return ""
|
||||
|
||||
def load_original_pii(self, audit_path: Path) -> List[Dict]:
|
||||
"""
|
||||
Charge les PII originaux depuis l'audit.
|
||||
|
||||
Args:
|
||||
audit_path: Chemin vers le fichier .audit.jsonl
|
||||
|
||||
Returns:
|
||||
Liste des PII originaux
|
||||
"""
|
||||
if not audit_path.exists():
|
||||
return []
|
||||
|
||||
try:
|
||||
pii_list = []
|
||||
with open(audit_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if line.strip():
|
||||
pii = json.loads(line)
|
||||
pii_list.append(pii)
|
||||
return pii_list
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors du chargement de l'audit {audit_path}: {e}")
|
||||
return []
|
||||
|
||||
def scan_text(self, text: str, original_pii: List[Dict]) -> List[Dict]:
|
||||
"""
|
||||
Scanne le texte pour détecter les fuites de PII.
|
||||
|
||||
Args:
|
||||
text: Texte à scanner
|
||||
original_pii: Liste des PII originaux
|
||||
|
||||
Returns:
|
||||
Liste des fuites détectées
|
||||
"""
|
||||
leaks = []
|
||||
|
||||
# 1. Vérifier que les PII originaux ne sont plus présents
|
||||
for pii in original_pii:
|
||||
original_text = pii.get("original", "")
|
||||
if not original_text:
|
||||
continue
|
||||
|
||||
# Recherche insensible à la casse
|
||||
if re.search(re.escape(original_text), text, re.IGNORECASE):
|
||||
leaks.append({
|
||||
"type": "original_pii_present",
|
||||
"severity": "CRITIQUE",
|
||||
"pii_type": pii.get("kind", "UNKNOWN"),
|
||||
"text": original_text,
|
||||
"message": f"PII original encore présent: {original_text}"
|
||||
})
|
||||
|
||||
# 2. Détecter de nouveaux PII non masqués
|
||||
for pii_type, pattern in self.REGEX_PATTERNS.items():
|
||||
matches = pattern.finditer(text)
|
||||
for match in matches:
|
||||
matched_text = match.group()
|
||||
|
||||
# Vérifier si ce PII était dans l'audit original
|
||||
is_known = any(
|
||||
pii.get("original", "").lower() == matched_text.lower()
|
||||
for pii in original_pii
|
||||
)
|
||||
|
||||
if not is_known:
|
||||
leaks.append({
|
||||
"type": "new_pii_detected",
|
||||
"severity": "HAUTE",
|
||||
"pii_type": pii_type,
|
||||
"text": matched_text,
|
||||
"message": f"Nouveau PII détecté: {pii_type} = {matched_text}"
|
||||
})
|
||||
|
||||
return leaks
|
||||
|
||||
def scan_metadata(self, pdf_path: Path) -> List[Dict]:
|
||||
"""
|
||||
Scanne les métadonnées du PDF.
|
||||
|
||||
Args:
|
||||
pdf_path: Chemin vers le PDF
|
||||
|
||||
Returns:
|
||||
Liste des fuites dans les métadonnées
|
||||
"""
|
||||
leaks = []
|
||||
|
||||
try:
|
||||
doc = fitz.open(pdf_path)
|
||||
metadata = doc.metadata
|
||||
doc.close()
|
||||
|
||||
# Champs à vérifier
|
||||
suspicious_fields = ["author", "creator", "producer", "subject", "title"]
|
||||
|
||||
for field in suspicious_fields:
|
||||
value = metadata.get(field, "")
|
||||
if value and value.strip():
|
||||
# Vérifier si le champ contient des PII potentiels
|
||||
# (noms, emails, etc.)
|
||||
if "@" in value:
|
||||
leaks.append({
|
||||
"type": "metadata_leak",
|
||||
"severity": "MOYENNE",
|
||||
"field": field,
|
||||
"text": value,
|
||||
"message": f"Métadonnée suspecte ({field}): {value}"
|
||||
})
|
||||
elif any(c.isalpha() for c in value):
|
||||
# Contient des lettres (potentiellement un nom)
|
||||
leaks.append({
|
||||
"type": "metadata_leak",
|
||||
"severity": "MOYENNE",
|
||||
"field": field,
|
||||
"text": value,
|
||||
"message": f"Métadonnée suspecte ({field}): {value}"
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors du scan des métadonnées de {pdf_path}: {e}")
|
||||
|
||||
return leaks
|
||||
|
||||
def scan(self, anonymized_pdf: Path, original_audit: Path) -> LeakReport:
|
||||
"""
|
||||
Scanne un document anonymisé pour détecter les fuites.
|
||||
|
||||
Args:
|
||||
anonymized_pdf: Chemin vers le PDF anonymisé
|
||||
original_audit: Chemin vers l'audit original
|
||||
|
||||
Returns:
|
||||
Rapport de fuite
|
||||
"""
|
||||
# Extraire le texte
|
||||
text = self.extract_text_from_pdf(anonymized_pdf)
|
||||
|
||||
# Charger les PII originaux
|
||||
original_pii = self.load_original_pii(original_audit)
|
||||
|
||||
# Scanner le texte
|
||||
text_leaks = self.scan_text(text, original_pii)
|
||||
|
||||
# Scanner les métadonnées
|
||||
metadata_leaks = self.scan_metadata(anonymized_pdf)
|
||||
|
||||
# Combiner les fuites
|
||||
all_leaks = text_leaks + metadata_leaks
|
||||
|
||||
# Compter par sévérité
|
||||
severity_counts = {}
|
||||
for leak in all_leaks:
|
||||
severity = leak.get("severity", "UNKNOWN")
|
||||
severity_counts[severity] = severity_counts.get(severity, 0) + 1
|
||||
|
||||
# Créer le rapport
|
||||
report = LeakReport(
|
||||
is_safe=len(all_leaks) == 0,
|
||||
leak_count=len(all_leaks),
|
||||
leaks=all_leaks,
|
||||
severity_counts=severity_counts
|
||||
)
|
||||
|
||||
return report
|
||||
|
||||
def generate_report(self, report: LeakReport, pdf_path: Path) -> str:
|
||||
"""
|
||||
Génère un rapport texte.
|
||||
|
||||
Args:
|
||||
report: Rapport de fuite
|
||||
pdf_path: Chemin du PDF
|
||||
|
||||
Returns:
|
||||
Rapport texte
|
||||
"""
|
||||
lines = []
|
||||
lines.append("=" * 80)
|
||||
lines.append(f"RAPPORT DE FUITE - {pdf_path.name}")
|
||||
lines.append("=" * 80)
|
||||
lines.append("")
|
||||
|
||||
if report.is_safe:
|
||||
lines.append("✓ DOCUMENT SÛR - Aucune fuite détectée")
|
||||
else:
|
||||
lines.append(f"✗ DOCUMENT NON SÛR - {report.leak_count} fuite(s) détectée(s)")
|
||||
lines.append("")
|
||||
|
||||
# Par sévérité
|
||||
lines.append("FUITES PAR SÉVÉRITÉ:")
|
||||
for severity, count in sorted(report.severity_counts.items()):
|
||||
lines.append(f" {severity}: {count}")
|
||||
lines.append("")
|
||||
|
||||
# Détails des fuites
|
||||
lines.append("DÉTAILS DES FUITES:")
|
||||
for i, leak in enumerate(report.leaks, 1):
|
||||
lines.append(f"\n [{i}] {leak['severity']} - {leak['type']}")
|
||||
lines.append(f" Type PII: {leak.get('pii_type', 'N/A')}")
|
||||
lines.append(f" Texte: {leak.get('text', 'N/A')}")
|
||||
lines.append(f" Message: {leak.get('message', 'N/A')}")
|
||||
|
||||
lines.append("")
|
||||
lines.append("=" * 80)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
def export_json(self, report: LeakReport, output_path: Path):
|
||||
"""
|
||||
Exporte le rapport en JSON.
|
||||
|
||||
Args:
|
||||
report: Rapport de fuite
|
||||
output_path: Chemin du fichier de sortie
|
||||
"""
|
||||
with open(output_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(report.to_dict(), f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"✓ Rapport exporté: {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Test basique
|
||||
scanner = LeakScanner()
|
||||
|
||||
# Exemple d'utilisation
|
||||
anonymized_pdf = Path("tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.redacted.pdf")
|
||||
original_audit = Path("tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.audit.jsonl")
|
||||
|
||||
if anonymized_pdf.exists() and original_audit.exists():
|
||||
report = scanner.scan(anonymized_pdf, original_audit)
|
||||
print(scanner.generate_report(report, anonymized_pdf))
|
||||
522
evaluation/quality_evaluator.py
Normal file
522
evaluation/quality_evaluator.py
Normal file
@@ -0,0 +1,522 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Évaluateur de qualité d'anonymisation.
|
||||
|
||||
Compare les annotations manuelles (ground truth) avec les détections automatiques
|
||||
pour calculer les métriques de qualité (Précision, Rappel, F1-Score).
|
||||
"""
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Tuple, Optional
|
||||
import re
|
||||
|
||||
|
||||
@dataclass
|
||||
class EvaluationResult:
|
||||
"""Résultat d'évaluation pour un document."""
|
||||
|
||||
pdf_path: str
|
||||
true_positives: int = 0
|
||||
false_positives: int = 0
|
||||
false_negatives: int = 0
|
||||
precision: float = 0.0
|
||||
recall: float = 0.0
|
||||
f1_score: float = 0.0
|
||||
missed_pii: List[Dict] = field(default_factory=list) # Faux négatifs détaillés
|
||||
false_detections: List[Dict] = field(default_factory=list) # Faux positifs détaillés
|
||||
by_type: Dict[str, Dict] = field(default_factory=dict) # Métriques par type de PII
|
||||
|
||||
def to_dict(self) -> Dict:
|
||||
"""Convertit en dictionnaire."""
|
||||
return {
|
||||
"pdf_path": self.pdf_path,
|
||||
"true_positives": self.true_positives,
|
||||
"false_positives": self.false_positives,
|
||||
"false_negatives": self.false_negatives,
|
||||
"precision": round(self.precision, 4),
|
||||
"recall": round(self.recall, 4),
|
||||
"f1_score": round(self.f1_score, 4),
|
||||
"missed_pii": self.missed_pii,
|
||||
"false_detections": self.false_detections,
|
||||
"by_type": self.by_type
|
||||
}
|
||||
|
||||
|
||||
class QualityEvaluator:
|
||||
"""Évaluateur de qualité d'anonymisation."""
|
||||
|
||||
# Mapping des types de PII entre annotations et détections
|
||||
TYPE_MAPPING = {
|
||||
# Annotations → Détections possibles
|
||||
"NOM": ["NOM", "NOM_GLOBAL", "PRENOM", "PRENOM_GLOBAL"],
|
||||
"PRENOM": ["PRENOM", "PRENOM_GLOBAL", "NOM", "NOM_GLOBAL"],
|
||||
"TEL": ["TEL", "TEL_GLOBAL"],
|
||||
"EMAIL": ["EMAIL", "EMAIL_GLOBAL"],
|
||||
"ADRESSE": ["ADRESSE", "ADRESSE_GLOBAL"],
|
||||
"CODE_POSTAL": ["CODE_POSTAL", "CODE_POSTAL_GLOBAL"],
|
||||
"VILLE": ["VILLE", "VILLE_GLOBAL"],
|
||||
"NIR": ["NIR", "NIR_GLOBAL"],
|
||||
"IPP": ["IPP", "IPP_GLOBAL"],
|
||||
"NDA": ["NDA", "NDA_GLOBAL"],
|
||||
"RPPS": ["RPPS", "RPPS_GLOBAL"],
|
||||
"FINESS": ["FINESS", "FINESS_GLOBAL"],
|
||||
"OGC": ["OGC", "OGC_GLOBAL"],
|
||||
"ETABLISSEMENT": ["ETAB", "ETAB_GLOBAL", "VLM_ETAB"],
|
||||
"SERVICE": ["SERVICE", "SERVICE_GLOBAL", "VLM_SERVICE"],
|
||||
"DATE": ["DATE", "DATE_GLOBAL"],
|
||||
"DATE_NAISSANCE": ["DATE_NAISSANCE", "DATE_NAISSANCE_GLOBAL"],
|
||||
"AGE": ["AGE", "AGE_GLOBAL"],
|
||||
"NUMERO_PATIENT": ["VLM_NUM_PATIENT", "IPP"],
|
||||
"NUMERO_LOT": ["VLM_NUM_LOT"],
|
||||
"NUMERO_ORDONNANCE": ["VLM_NUM_ORD"],
|
||||
"NUMERO_SEJOUR": ["VLM_NUM_SEJOUR", "NDA"],
|
||||
}
|
||||
|
||||
def __init__(self, ground_truth_dir: Path):
|
||||
"""
|
||||
Initialise l'évaluateur.
|
||||
|
||||
Args:
|
||||
ground_truth_dir: Répertoire contenant les annotations manuelles
|
||||
"""
|
||||
self.ground_truth_dir = Path(ground_truth_dir)
|
||||
|
||||
def normalize_text(self, text: str) -> str:
|
||||
"""
|
||||
Normalise un texte pour la comparaison.
|
||||
|
||||
Args:
|
||||
text: Texte à normaliser
|
||||
|
||||
Returns:
|
||||
Texte normalisé
|
||||
"""
|
||||
# Lowercase
|
||||
text = text.lower()
|
||||
|
||||
# Supprimer les espaces multiples
|
||||
text = re.sub(r'\s+', ' ', text)
|
||||
|
||||
# Strip
|
||||
text = text.strip()
|
||||
|
||||
return text
|
||||
|
||||
def load_annotations(self, pdf_path: Path) -> Optional[Dict]:
|
||||
"""
|
||||
Charge les annotations manuelles d'un document.
|
||||
|
||||
Args:
|
||||
pdf_path: Chemin vers le PDF
|
||||
|
||||
Returns:
|
||||
Annotations ou None si non trouvées
|
||||
"""
|
||||
annotation_file = pdf_path.parent / f"{pdf_path.stem}.annotations.json"
|
||||
|
||||
if not annotation_file.exists():
|
||||
return None
|
||||
|
||||
try:
|
||||
with open(annotation_file, 'r', encoding='utf-8') as f:
|
||||
return json.load(f)
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors du chargement des annotations {annotation_file}: {e}")
|
||||
return None
|
||||
|
||||
def load_audit(self, audit_path: Path) -> Optional[List[Dict]]:
|
||||
"""
|
||||
Charge l'audit de détection automatique.
|
||||
|
||||
Args:
|
||||
audit_path: Chemin vers le fichier .audit.jsonl
|
||||
|
||||
Returns:
|
||||
Liste des détections ou None si non trouvé
|
||||
"""
|
||||
if not audit_path.exists():
|
||||
return None
|
||||
|
||||
try:
|
||||
detections = []
|
||||
with open(audit_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if line.strip():
|
||||
detections.append(json.loads(line))
|
||||
return detections
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors du chargement de l'audit {audit_path}: {e}")
|
||||
return None
|
||||
|
||||
def types_match(self, ann_type: str, det_type: str) -> bool:
|
||||
"""
|
||||
Vérifie si deux types de PII correspondent.
|
||||
|
||||
Args:
|
||||
ann_type: Type dans l'annotation
|
||||
det_type: Type dans la détection
|
||||
|
||||
Returns:
|
||||
True si les types correspondent
|
||||
"""
|
||||
# Mapping direct
|
||||
if ann_type in self.TYPE_MAPPING:
|
||||
return det_type in self.TYPE_MAPPING[ann_type]
|
||||
|
||||
# Correspondance exacte
|
||||
return ann_type == det_type
|
||||
|
||||
def compare(self, annotations: List[Dict], detections: List[Dict]) -> Tuple[List, List, List]:
|
||||
"""
|
||||
Compare les annotations avec les détections.
|
||||
|
||||
Args:
|
||||
annotations: Liste des annotations manuelles
|
||||
detections: Liste des détections automatiques
|
||||
|
||||
Returns:
|
||||
Tuple (true_positives, false_negatives, false_positives)
|
||||
"""
|
||||
true_positives = []
|
||||
false_negatives = []
|
||||
false_positives = []
|
||||
|
||||
# Créer des clés de comparaison pour les annotations
|
||||
ann_keys = {}
|
||||
for ann in annotations:
|
||||
page = ann.get("page", 0)
|
||||
pii_type = ann.get("type", "")
|
||||
text = self.normalize_text(ann.get("text", ""))
|
||||
|
||||
key = (page, text)
|
||||
if key not in ann_keys:
|
||||
ann_keys[key] = []
|
||||
ann_keys[key].append(ann)
|
||||
|
||||
# Créer des clés de comparaison pour les détections
|
||||
det_keys = {}
|
||||
for det in detections:
|
||||
page = det.get("page", 0)
|
||||
text = self.normalize_text(det.get("original", ""))
|
||||
|
||||
key = (page, text)
|
||||
if key not in det_keys:
|
||||
det_keys[key] = []
|
||||
det_keys[key].append(det)
|
||||
|
||||
# Trouver les true positives et false negatives
|
||||
matched_det_keys = set()
|
||||
|
||||
for key, anns in ann_keys.items():
|
||||
page, text = key
|
||||
|
||||
if key in det_keys:
|
||||
# Vérifier si au moins une détection correspond au type
|
||||
dets = det_keys[key]
|
||||
matched = False
|
||||
|
||||
for ann in anns:
|
||||
ann_type = ann.get("type", "")
|
||||
for det in dets:
|
||||
det_type = det.get("kind", "")
|
||||
if self.types_match(ann_type, det_type):
|
||||
true_positives.append({
|
||||
"page": page,
|
||||
"type": ann_type,
|
||||
"text": ann.get("text", ""),
|
||||
"detected_as": det_type,
|
||||
"context": ann.get("context", "")
|
||||
})
|
||||
matched = True
|
||||
matched_det_keys.add(key)
|
||||
break
|
||||
if matched:
|
||||
break
|
||||
|
||||
if not matched:
|
||||
# Détecté mais type incorrect
|
||||
for ann in anns:
|
||||
false_negatives.append({
|
||||
"page": page,
|
||||
"type": ann.get("type", ""),
|
||||
"text": ann.get("text", ""),
|
||||
"context": ann.get("context", ""),
|
||||
"reason": "type_mismatch",
|
||||
"detected_as": [d.get("kind", "") for d in dets]
|
||||
})
|
||||
else:
|
||||
# Non détecté
|
||||
for ann in anns:
|
||||
false_negatives.append({
|
||||
"page": page,
|
||||
"type": ann.get("type", ""),
|
||||
"text": ann.get("text", ""),
|
||||
"context": ann.get("context", ""),
|
||||
"reason": "not_detected"
|
||||
})
|
||||
|
||||
# Trouver les false positives
|
||||
for key, dets in det_keys.items():
|
||||
if key not in matched_det_keys:
|
||||
page, text = key
|
||||
for det in dets:
|
||||
false_positives.append({
|
||||
"page": page,
|
||||
"type": det.get("kind", ""),
|
||||
"text": det.get("original", ""),
|
||||
"placeholder": det.get("placeholder", "")
|
||||
})
|
||||
|
||||
return true_positives, false_negatives, false_positives
|
||||
|
||||
def calculate_metrics(self, tp: int, fp: int, fn: int) -> Tuple[float, float, float]:
|
||||
"""
|
||||
Calcule les métriques de qualité.
|
||||
|
||||
Args:
|
||||
tp: True positives
|
||||
fp: False positives
|
||||
fn: False negatives
|
||||
|
||||
Returns:
|
||||
Tuple (precision, recall, f1_score)
|
||||
"""
|
||||
# Précision
|
||||
precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
|
||||
|
||||
# Rappel
|
||||
recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
|
||||
|
||||
# F1-Score
|
||||
f1_score = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0.0
|
||||
|
||||
return precision, recall, f1_score
|
||||
|
||||
def calculate_metrics_by_type(self, tp_list: List[Dict], fn_list: List[Dict], fp_list: List[Dict]) -> Dict[str, Dict]:
|
||||
"""
|
||||
Calcule les métriques par type de PII.
|
||||
|
||||
Args:
|
||||
tp_list: Liste des true positives
|
||||
fn_list: Liste des false negatives
|
||||
fp_list: Liste des false positives
|
||||
|
||||
Returns:
|
||||
Dictionnaire des métriques par type
|
||||
"""
|
||||
by_type = {}
|
||||
|
||||
# Compter par type
|
||||
for tp in tp_list:
|
||||
pii_type = tp["type"]
|
||||
if pii_type not in by_type:
|
||||
by_type[pii_type] = {"tp": 0, "fp": 0, "fn": 0}
|
||||
by_type[pii_type]["tp"] += 1
|
||||
|
||||
for fn in fn_list:
|
||||
pii_type = fn["type"]
|
||||
if pii_type not in by_type:
|
||||
by_type[pii_type] = {"tp": 0, "fp": 0, "fn": 0}
|
||||
by_type[pii_type]["fn"] += 1
|
||||
|
||||
for fp in fp_list:
|
||||
pii_type = fp["type"]
|
||||
if pii_type not in by_type:
|
||||
by_type[pii_type] = {"tp": 0, "fp": 0, "fn": 0}
|
||||
by_type[pii_type]["fp"] += 1
|
||||
|
||||
# Calculer les métriques
|
||||
for pii_type, counts in by_type.items():
|
||||
tp = counts["tp"]
|
||||
fp = counts["fp"]
|
||||
fn = counts["fn"]
|
||||
|
||||
precision, recall, f1 = self.calculate_metrics(tp, fp, fn)
|
||||
|
||||
counts["precision"] = round(precision, 4)
|
||||
counts["recall"] = round(recall, 4)
|
||||
counts["f1_score"] = round(f1, 4)
|
||||
|
||||
return by_type
|
||||
|
||||
def evaluate(self, pdf_path: Path, audit_path: Path) -> Optional[EvaluationResult]:
|
||||
"""
|
||||
Évalue la qualité d'anonymisation d'un document.
|
||||
|
||||
Args:
|
||||
pdf_path: Chemin vers le PDF original
|
||||
audit_path: Chemin vers le fichier .audit.jsonl
|
||||
|
||||
Returns:
|
||||
Résultat d'évaluation ou None si erreur
|
||||
"""
|
||||
# Charger les annotations
|
||||
annotations_data = self.load_annotations(pdf_path)
|
||||
if not annotations_data:
|
||||
print(f"✗ Annotations introuvables pour {pdf_path.name}")
|
||||
return None
|
||||
|
||||
annotations = annotations_data.get("annotations", [])
|
||||
|
||||
# Charger l'audit
|
||||
detections = self.load_audit(audit_path)
|
||||
if detections is None:
|
||||
print(f"✗ Audit introuvable: {audit_path}")
|
||||
return None
|
||||
|
||||
# Comparer
|
||||
tp_list, fn_list, fp_list = self.compare(annotations, detections)
|
||||
|
||||
# Calculer les métriques globales
|
||||
tp = len(tp_list)
|
||||
fp = len(fp_list)
|
||||
fn = len(fn_list)
|
||||
|
||||
precision, recall, f1_score = self.calculate_metrics(tp, fp, fn)
|
||||
|
||||
# Calculer les métriques par type
|
||||
by_type = self.calculate_metrics_by_type(tp_list, fn_list, fp_list)
|
||||
|
||||
# Créer le résultat
|
||||
result = EvaluationResult(
|
||||
pdf_path=str(pdf_path),
|
||||
true_positives=tp,
|
||||
false_positives=fp,
|
||||
false_negatives=fn,
|
||||
precision=precision,
|
||||
recall=recall,
|
||||
f1_score=f1_score,
|
||||
missed_pii=fn_list,
|
||||
false_detections=fp_list,
|
||||
by_type=by_type
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
def evaluate_batch(self, pdf_list: List[Path], audit_list: List[Path]) -> List[EvaluationResult]:
|
||||
"""
|
||||
Évalue un batch de documents.
|
||||
|
||||
Args:
|
||||
pdf_list: Liste des PDFs
|
||||
audit_list: Liste des audits
|
||||
|
||||
Returns:
|
||||
Liste des résultats d'évaluation
|
||||
"""
|
||||
results = []
|
||||
|
||||
for pdf_path, audit_path in zip(pdf_list, audit_list):
|
||||
result = self.evaluate(pdf_path, audit_path)
|
||||
if result:
|
||||
results.append(result)
|
||||
|
||||
return results
|
||||
|
||||
def generate_report(self, results: List[EvaluationResult]) -> str:
|
||||
"""
|
||||
Génère un rapport texte des résultats.
|
||||
|
||||
Args:
|
||||
results: Liste des résultats d'évaluation
|
||||
|
||||
Returns:
|
||||
Rapport texte
|
||||
"""
|
||||
if not results:
|
||||
return "Aucun résultat à afficher."
|
||||
|
||||
# Calculer les métriques globales
|
||||
total_tp = sum(r.true_positives for r in results)
|
||||
total_fp = sum(r.false_positives for r in results)
|
||||
total_fn = sum(r.false_negatives for r in results)
|
||||
|
||||
avg_precision = sum(r.precision for r in results) / len(results)
|
||||
avg_recall = sum(r.recall for r in results) / len(results)
|
||||
avg_f1 = sum(r.f1_score for r in results) / len(results)
|
||||
|
||||
# Générer le rapport
|
||||
report = []
|
||||
report.append("=" * 80)
|
||||
report.append("RAPPORT D'ÉVALUATION DE LA QUALITÉ D'ANONYMISATION")
|
||||
report.append("=" * 80)
|
||||
report.append("")
|
||||
|
||||
report.append(f"Documents évalués: {len(results)}")
|
||||
report.append("")
|
||||
|
||||
report.append("MÉTRIQUES GLOBALES:")
|
||||
report.append(f" True Positives: {total_tp}")
|
||||
report.append(f" False Positives: {total_fp}")
|
||||
report.append(f" False Negatives: {total_fn}")
|
||||
report.append("")
|
||||
report.append(f" Précision moyenne: {avg_precision:.4f} ({avg_precision*100:.2f}%)")
|
||||
report.append(f" Rappel moyen: {avg_recall:.4f} ({avg_recall*100:.2f}%)")
|
||||
report.append(f" F1-Score moyen: {avg_f1:.4f}")
|
||||
report.append("")
|
||||
|
||||
# Résultats par document
|
||||
report.append("RÉSULTATS PAR DOCUMENT:")
|
||||
report.append("")
|
||||
|
||||
for result in results:
|
||||
pdf_name = Path(result.pdf_path).name
|
||||
report.append(f" {pdf_name}")
|
||||
report.append(f" Précision: {result.precision:.4f} Rappel: {result.recall:.4f} F1: {result.f1_score:.4f}")
|
||||
report.append(f" TP: {result.true_positives} FP: {result.false_positives} FN: {result.false_negatives}")
|
||||
report.append("")
|
||||
|
||||
# Faux négatifs critiques
|
||||
critical_fn = []
|
||||
for result in results:
|
||||
for fn in result.missed_pii:
|
||||
if fn.get("reason") == "not_detected":
|
||||
critical_fn.append((Path(result.pdf_path).name, fn))
|
||||
|
||||
if critical_fn:
|
||||
report.append(f"FAUX NÉGATIFS CRITIQUES ({len(critical_fn)}):")
|
||||
report.append("")
|
||||
for pdf_name, fn in critical_fn[:10]: # Limiter à 10
|
||||
report.append(f" {pdf_name} - Page {fn['page']+1}")
|
||||
report.append(f" Type: {fn['type']}")
|
||||
report.append(f" Texte: {fn['text']}")
|
||||
report.append(f" Contexte: {fn['context'][:80]}...")
|
||||
report.append("")
|
||||
|
||||
report.append("=" * 80)
|
||||
|
||||
return "\n".join(report)
|
||||
|
||||
def export_json(self, results: List[EvaluationResult], output_path: Path):
|
||||
"""
|
||||
Exporte les résultats en JSON.
|
||||
|
||||
Args:
|
||||
results: Liste des résultats
|
||||
output_path: Chemin du fichier de sortie
|
||||
"""
|
||||
data = {
|
||||
"evaluation_date": Path(__file__).stat().st_mtime,
|
||||
"documents_count": len(results),
|
||||
"results": [r.to_dict() for r in results]
|
||||
}
|
||||
|
||||
with open(output_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"✓ Résultats exportés: {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Test basique
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth/pdfs"))
|
||||
|
||||
# Exemple d'utilisation
|
||||
pdf_path = Path("tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.pdf")
|
||||
audit_path = Path("tests/ground_truth/pdfs/001_simple_unknown_BACTERIO_23018396.audit.jsonl")
|
||||
|
||||
if pdf_path.exists() and audit_path.exists():
|
||||
result = evaluator.evaluate(pdf_path, audit_path)
|
||||
if result:
|
||||
print(evaluator.generate_report([result]))
|
||||
84
run_batch_59ogc.py
Normal file
84
run_batch_59ogc.py
Normal file
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Batch processing des 59 premiers OGC — script CLI pour test post-modifications."""
|
||||
import sys
|
||||
import time
|
||||
import json
|
||||
from pathlib import Path
|
||||
from collections import Counter
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
import anonymizer_core_refactored_onnx as core
|
||||
from eds_pseudo_manager import EdsPseudoManager
|
||||
|
||||
SRC = Path("/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)")
|
||||
OUTDIR = SRC / "anonymise"
|
||||
CONFIG = Path("/home/dom/ai/anonymisation/config/dictionnaires.yml")
|
||||
|
||||
def main():
|
||||
# Charger EDS-Pseudo
|
||||
print("Chargement EDS-Pseudo...", flush=True)
|
||||
ner = EdsPseudoManager()
|
||||
ner.load()
|
||||
assert ner.is_loaded(), "EDS-Pseudo non chargé"
|
||||
print("EDS-Pseudo chargé.", flush=True)
|
||||
|
||||
# Lister les 59 premiers dossiers OGC
|
||||
ogc_dirs = sorted(
|
||||
[d for d in SRC.iterdir() if d.is_dir() and "_" in d.name and d.name[0].isdigit()],
|
||||
key=lambda d: int(d.name.split("_")[0]),
|
||||
)[:59]
|
||||
print(f"Dossiers OGC: {len(ogc_dirs)}")
|
||||
|
||||
# Collecter tous les PDFs
|
||||
pdfs = []
|
||||
for d in ogc_dirs:
|
||||
for pdf in sorted(d.glob("*.pdf")):
|
||||
pdfs.append(pdf)
|
||||
print(f"PDFs à traiter: {len(pdfs)}")
|
||||
|
||||
OUTDIR.mkdir(exist_ok=True)
|
||||
ok = ko = 0
|
||||
global_counts = Counter()
|
||||
t0 = time.time()
|
||||
|
||||
for i, pdf in enumerate(pdfs, 1):
|
||||
ogc = pdf.parent.name.split("_")[0]
|
||||
print(f"[{i}/{len(pdfs)}] {pdf.name} (OGC {ogc})...", end=" ", flush=True)
|
||||
try:
|
||||
outputs = core.process_pdf(
|
||||
pdf_path=pdf,
|
||||
out_dir=OUTDIR,
|
||||
make_vector_redaction=False,
|
||||
also_make_raster_burn=True,
|
||||
config_path=CONFIG,
|
||||
use_hf=True,
|
||||
ner_manager=ner,
|
||||
ner_thresholds=None,
|
||||
ogc_label=ogc,
|
||||
)
|
||||
# Compter les hits audit
|
||||
audit_path = Path(outputs.get("audit", ""))
|
||||
if audit_path.exists():
|
||||
for line in audit_path.read_text().splitlines():
|
||||
try:
|
||||
h = json.loads(line)
|
||||
global_counts[h["kind"]] += 1
|
||||
except Exception:
|
||||
pass
|
||||
print("OK", flush=True)
|
||||
ok += 1
|
||||
except Exception as e:
|
||||
print(f"ERREUR: {e}", flush=True)
|
||||
ko += 1
|
||||
|
||||
elapsed = time.time() - t0
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Terminé en {elapsed:.0f}s — OK: {ok}, Erreurs: {ko}")
|
||||
print(f"Total PII détectés: {sum(global_counts.values())}")
|
||||
print(f"\nDétail par type:")
|
||||
for k, v in sorted(global_counts.items(), key=lambda x: -x[1]):
|
||||
print(f" {k:30s} {v:6d}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
5
test_3ogc/anonymise_old/qc_rapport.csv
Normal file
5
test_3ogc/anonymise_old/qc_rapport.csv
Normal file
@@ -0,0 +1,5 @@
|
||||
fichier,total_hits,residual_pii,density_pct,nom_density_pct,alert_overmasking,fp_count,fn_count
|
||||
407 crh.audit.jsonl,407,0,14.93,9.32,True,0,4
|
||||
trackare-01285757-23042510_01285757_23042510.audit.jsonl,1316,0,7.57,4.03,False,0,10
|
||||
trackare-02004744-23116460_02004744_23116460.audit.jsonl,876,0,8.57,4.35,False,0,4
|
||||
trackare-BA165196-23061393_BA165196_23061393.audit.jsonl,2018,0,8.18,4.81,False,0,25
|
||||
|
299
tests/ground_truth/pdfs/mapping.json
Normal file
299
tests/ground_truth/pdfs/mapping.json
Normal file
@@ -0,0 +1,299 @@
|
||||
[
|
||||
{
|
||||
"id": 1,
|
||||
"dest_filename": "001_simple_unknown_BACTERIO_23018396.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/148_23018396/BACTERIO 23018396.pdf",
|
||||
"folder": "148_23018396",
|
||||
"original_filename": "BACTERIO 23018396.pdf",
|
||||
"type": "unknown",
|
||||
"complexity": "simple",
|
||||
"pages": 1,
|
||||
"size_mb": 0.04
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"dest_filename": "002_simple_unknown_bacterio_476_23159413.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/476_23159413/bacterio 476_23159413.pdf",
|
||||
"folder": "476_23159413",
|
||||
"original_filename": "bacterio 476_23159413.pdf",
|
||||
"type": "unknown",
|
||||
"complexity": "simple",
|
||||
"pages": 2,
|
||||
"size_mb": 0.04
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"dest_filename": "003_simple_compte_rendu_CRO_23155084.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/210_23155084/CRO 23155084.pdf",
|
||||
"folder": "210_23155084",
|
||||
"original_filename": "CRO 23155084.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "simple",
|
||||
"pages": 1,
|
||||
"size_mb": 0.05
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"dest_filename": "004_simple_anapath_anapath_53_23224186.redacted_raster.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/anapath 53_23224186.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"original_filename": "anapath 53_23224186.redacted_raster.pdf",
|
||||
"type": "anapath",
|
||||
"complexity": "simple",
|
||||
"pages": 1,
|
||||
"size_mb": 0.29
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"dest_filename": "005_simple_compte_rendu_CRH_23155836.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/212_23155836/CRH 23155836.pdf",
|
||||
"folder": "212_23155836",
|
||||
"original_filename": "CRH 23155836.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "simple",
|
||||
"pages": 2,
|
||||
"size_mb": 0.14
|
||||
},
|
||||
{
|
||||
"id": 6,
|
||||
"dest_filename": "006_simple_anapath_ANAPATH_23142660.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/204_23142660/ANAPATH 23142660.pdf",
|
||||
"folder": "204_23142660",
|
||||
"original_filename": "ANAPATH 23142660.pdf",
|
||||
"type": "anapath",
|
||||
"complexity": "simple",
|
||||
"pages": 0,
|
||||
"size_mb": 0.16
|
||||
},
|
||||
{
|
||||
"id": 7,
|
||||
"dest_filename": "007_simple_anapath_ANAPATH_23096332.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/15_23096332/ANAPATH 23096332.pdf",
|
||||
"folder": "15_23096332",
|
||||
"original_filename": "ANAPATH 23096332.pdf",
|
||||
"type": "anapath",
|
||||
"complexity": "simple",
|
||||
"pages": 1,
|
||||
"size_mb": 0.16
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"dest_filename": "008_simple_trackare_trackare-14004105-23202435_14004105_23202435.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/80_23202435/trackare-14004105-23202435_14004105_23202435.pdf",
|
||||
"folder": "80_23202435",
|
||||
"original_filename": "trackare-14004105-23202435_14004105_23202435.pdf",
|
||||
"type": "trackare",
|
||||
"complexity": "simple",
|
||||
"pages": 1,
|
||||
"size_mb": 0.11
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"dest_filename": "009_simple_compte_rendu_CRO_23051225.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/138_23051225/CRO 23051225.pdf",
|
||||
"folder": "138_23051225",
|
||||
"original_filename": "CRO 23051225.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "simple",
|
||||
"pages": 2,
|
||||
"size_mb": 0.06
|
||||
},
|
||||
{
|
||||
"id": 10,
|
||||
"dest_filename": "010_simple_anapath_ANAPATH_23217289.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/49_23217289/ANAPATH 23217289.pdf",
|
||||
"folder": "49_23217289",
|
||||
"original_filename": "ANAPATH 23217289.pdf",
|
||||
"type": "anapath",
|
||||
"complexity": "simple",
|
||||
"pages": 1,
|
||||
"size_mb": 0.17
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"dest_filename": "011_moyen_compte_rendu_CRH_23080179.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/132_23080179/CRH 23080179.pdf",
|
||||
"folder": "132_23080179",
|
||||
"original_filename": "CRH 23080179.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 4,
|
||||
"size_mb": 0.07
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"dest_filename": "012_moyen_compte_rendu_CRH_692_23200418.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/692_23200418/CRH 692_23200418.pdf",
|
||||
"folder": "692_23200418",
|
||||
"original_filename": "CRH 692_23200418.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 2,
|
||||
"size_mb": 0.59
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"dest_filename": "013_moyen_compte_rendu_363_23085243_CRO.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/363_23085243/363_23085243 CRO.pdf",
|
||||
"folder": "363_23085243",
|
||||
"original_filename": "363_23085243 CRO.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 2,
|
||||
"size_mb": 0.58
|
||||
},
|
||||
{
|
||||
"id": 14,
|
||||
"dest_filename": "014_moyen_compte_rendu_CRO_23167029.redacted_raster.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23167029.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"original_filename": "CRO 23167029.redacted_raster.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 2,
|
||||
"size_mb": 0.65
|
||||
},
|
||||
{
|
||||
"id": 15,
|
||||
"dest_filename": "015_moyen_unknown_CONSULTATION_ANESTHESISTE_23139653.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/222_23139653/CONSULTATION ANESTHESISTE 23139653.pdf",
|
||||
"folder": "222_23139653",
|
||||
"original_filename": "CONSULTATION ANESTHESISTE 23139653.pdf",
|
||||
"type": "unknown",
|
||||
"complexity": "moyen",
|
||||
"pages": 3,
|
||||
"size_mb": 0.12
|
||||
},
|
||||
{
|
||||
"id": 16,
|
||||
"dest_filename": "016_moyen_compte_rendu_CRH_23149905.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/199_23149905/CRH 23149905.pdf",
|
||||
"folder": "199_23149905",
|
||||
"original_filename": "CRH 23149905.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 3,
|
||||
"size_mb": 0.15
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"dest_filename": "017_moyen_compte_rendu_CRO_23222062.redacted_raster.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23222062.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"original_filename": "CRO 23222062.redacted_raster.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 2,
|
||||
"size_mb": 0.57
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"dest_filename": "018_moyen_compte_rendu_CRH_23042753.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/1_23042753/CRH 23042753.pdf",
|
||||
"folder": "1_23042753",
|
||||
"original_filename": "CRH 23042753.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 4,
|
||||
"size_mb": 0.15
|
||||
},
|
||||
{
|
||||
"id": 19,
|
||||
"dest_filename": "019_moyen_compte_rendu_CRO_332_23049003.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/332_23049003/CRO 332_23049003.pdf",
|
||||
"folder": "332_23049003",
|
||||
"original_filename": "CRO 332_23049003.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 2,
|
||||
"size_mb": 0.43
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"dest_filename": "020_moyen_compte_rendu_CRO_23084754.redacted_raster.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23084754.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"original_filename": "CRO 23084754.redacted_raster.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 2,
|
||||
"size_mb": 0.46
|
||||
},
|
||||
{
|
||||
"id": 21,
|
||||
"dest_filename": "021_moyen_compte_rendu_CRO_23201117.redacted_raster.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23201117.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"original_filename": "CRO 23201117.redacted_raster.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 1,
|
||||
"size_mb": 0.33
|
||||
},
|
||||
{
|
||||
"id": 22,
|
||||
"dest_filename": "022_moyen_compte_rendu_cro2_516_23187028.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/516_23187028/cro2 516_23187028.pdf",
|
||||
"folder": "516_23187028",
|
||||
"original_filename": "cro2 516_23187028.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen",
|
||||
"pages": 1,
|
||||
"size_mb": 0.3
|
||||
},
|
||||
{
|
||||
"id": 23,
|
||||
"dest_filename": "023_complexe_compte_rendu_CRH_23102610.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/153_23102610/CRH 23102610.pdf",
|
||||
"folder": "153_23102610",
|
||||
"original_filename": "CRH 23102610.pdf",
|
||||
"type": "compte_rendu",
|
||||
"complexity": "complexe",
|
||||
"pages": 9,
|
||||
"size_mb": 0.14
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"dest_filename": "024_complexe_trackare_trackare-17001141-23066188_17001141_23066188.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/115_23066188/trackare-17001141-23066188_17001141_23066188.pdf",
|
||||
"folder": "115_23066188",
|
||||
"original_filename": "trackare-17001141-23066188_17001141_23066188.pdf",
|
||||
"type": "trackare",
|
||||
"complexity": "complexe",
|
||||
"pages": 19,
|
||||
"size_mb": 0.21
|
||||
},
|
||||
{
|
||||
"id": 25,
|
||||
"dest_filename": "025_complexe_trackare_trackare-02016820-23095226_02016820_23095226.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/400_23095226/trackare-02016820-23095226_02016820_23095226.pdf",
|
||||
"folder": "400_23095226",
|
||||
"original_filename": "trackare-02016820-23095226_02016820_23095226.pdf",
|
||||
"type": "trackare",
|
||||
"complexity": "complexe",
|
||||
"pages": 31,
|
||||
"size_mb": 0.29
|
||||
},
|
||||
{
|
||||
"id": 26,
|
||||
"dest_filename": "026_complexe_trackare_trackare-15000536-23074384_15000536_23074384.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/343_23074384/trackare-15000536-23074384_15000536_23074384.pdf",
|
||||
"folder": "343_23074384",
|
||||
"original_filename": "trackare-15000536-23074384_15000536_23074384.pdf",
|
||||
"type": "trackare",
|
||||
"complexity": "complexe",
|
||||
"pages": 25,
|
||||
"size_mb": 0.25
|
||||
},
|
||||
{
|
||||
"id": 27,
|
||||
"dest_filename": "027_complexe_trackare_trackare-10027557-23183041_10027557_23183041.pdf",
|
||||
"original_path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/45_23183041/trackare-10027557-23183041_10027557_23183041.pdf",
|
||||
"folder": "45_23183041",
|
||||
"original_filename": "trackare-10027557-23183041_10027557_23183041.pdf",
|
||||
"type": "trackare",
|
||||
"complexity": "complexe",
|
||||
"pages": 20,
|
||||
"size_mb": 0.25
|
||||
}
|
||||
]
|
||||
245
tests/ground_truth/selected_documents.json
Normal file
245
tests/ground_truth/selected_documents.json
Normal file
@@ -0,0 +1,245 @@
|
||||
[
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/148_23018396/BACTERIO 23018396.pdf",
|
||||
"folder": "148_23018396",
|
||||
"filename": "BACTERIO 23018396.pdf",
|
||||
"size_mb": 0.04,
|
||||
"pages": 1,
|
||||
"type": "unknown",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/476_23159413/bacterio 476_23159413.pdf",
|
||||
"folder": "476_23159413",
|
||||
"filename": "bacterio 476_23159413.pdf",
|
||||
"size_mb": 0.04,
|
||||
"pages": 2,
|
||||
"type": "unknown",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/210_23155084/CRO 23155084.pdf",
|
||||
"folder": "210_23155084",
|
||||
"filename": "CRO 23155084.pdf",
|
||||
"size_mb": 0.05,
|
||||
"pages": 1,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/anapath 53_23224186.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"filename": "anapath 53_23224186.redacted_raster.pdf",
|
||||
"size_mb": 0.29,
|
||||
"pages": 1,
|
||||
"type": "anapath",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/212_23155836/CRH 23155836.pdf",
|
||||
"folder": "212_23155836",
|
||||
"filename": "CRH 23155836.pdf",
|
||||
"size_mb": 0.14,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/204_23142660/ANAPATH 23142660.pdf",
|
||||
"folder": "204_23142660",
|
||||
"filename": "ANAPATH 23142660.pdf",
|
||||
"size_mb": 0.16,
|
||||
"pages": 0,
|
||||
"type": "anapath",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/15_23096332/ANAPATH 23096332.pdf",
|
||||
"folder": "15_23096332",
|
||||
"filename": "ANAPATH 23096332.pdf",
|
||||
"size_mb": 0.16,
|
||||
"pages": 1,
|
||||
"type": "anapath",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/80_23202435/trackare-14004105-23202435_14004105_23202435.pdf",
|
||||
"folder": "80_23202435",
|
||||
"filename": "trackare-14004105-23202435_14004105_23202435.pdf",
|
||||
"size_mb": 0.11,
|
||||
"pages": 1,
|
||||
"type": "trackare",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/138_23051225/CRO 23051225.pdf",
|
||||
"folder": "138_23051225",
|
||||
"filename": "CRO 23051225.pdf",
|
||||
"size_mb": 0.06,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/49_23217289/ANAPATH 23217289.pdf",
|
||||
"folder": "49_23217289",
|
||||
"filename": "ANAPATH 23217289.pdf",
|
||||
"size_mb": 0.17,
|
||||
"pages": 1,
|
||||
"type": "anapath",
|
||||
"complexity": "simple"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/132_23080179/CRH 23080179.pdf",
|
||||
"folder": "132_23080179",
|
||||
"filename": "CRH 23080179.pdf",
|
||||
"size_mb": 0.07,
|
||||
"pages": 4,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/692_23200418/CRH 692_23200418.pdf",
|
||||
"folder": "692_23200418",
|
||||
"filename": "CRH 692_23200418.pdf",
|
||||
"size_mb": 0.59,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/363_23085243/363_23085243 CRO.pdf",
|
||||
"folder": "363_23085243",
|
||||
"filename": "363_23085243 CRO.pdf",
|
||||
"size_mb": 0.58,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23167029.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"filename": "CRO 23167029.redacted_raster.pdf",
|
||||
"size_mb": 0.65,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/222_23139653/CONSULTATION ANESTHESISTE 23139653.pdf",
|
||||
"folder": "222_23139653",
|
||||
"filename": "CONSULTATION ANESTHESISTE 23139653.pdf",
|
||||
"size_mb": 0.12,
|
||||
"pages": 3,
|
||||
"type": "unknown",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/199_23149905/CRH 23149905.pdf",
|
||||
"folder": "199_23149905",
|
||||
"filename": "CRH 23149905.pdf",
|
||||
"size_mb": 0.15,
|
||||
"pages": 3,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23222062.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"filename": "CRO 23222062.redacted_raster.pdf",
|
||||
"size_mb": 0.57,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/1_23042753/CRH 23042753.pdf",
|
||||
"folder": "1_23042753",
|
||||
"filename": "CRH 23042753.pdf",
|
||||
"size_mb": 0.15,
|
||||
"pages": 4,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/332_23049003/CRO 332_23049003.pdf",
|
||||
"folder": "332_23049003",
|
||||
"filename": "CRO 332_23049003.pdf",
|
||||
"size_mb": 0.43,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23084754.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"filename": "CRO 23084754.redacted_raster.pdf",
|
||||
"size_mb": 0.46,
|
||||
"pages": 2,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23201117.redacted_raster.pdf",
|
||||
"folder": "anonymise",
|
||||
"filename": "CRO 23201117.redacted_raster.pdf",
|
||||
"size_mb": 0.33,
|
||||
"pages": 1,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/516_23187028/cro2 516_23187028.pdf",
|
||||
"folder": "516_23187028",
|
||||
"filename": "cro2 516_23187028.pdf",
|
||||
"size_mb": 0.3,
|
||||
"pages": 1,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "moyen"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/153_23102610/CRH 23102610.pdf",
|
||||
"folder": "153_23102610",
|
||||
"filename": "CRH 23102610.pdf",
|
||||
"size_mb": 0.14,
|
||||
"pages": 9,
|
||||
"type": "compte_rendu",
|
||||
"complexity": "complexe"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/115_23066188/trackare-17001141-23066188_17001141_23066188.pdf",
|
||||
"folder": "115_23066188",
|
||||
"filename": "trackare-17001141-23066188_17001141_23066188.pdf",
|
||||
"size_mb": 0.21,
|
||||
"pages": 19,
|
||||
"type": "trackare",
|
||||
"complexity": "complexe"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/400_23095226/trackare-02016820-23095226_02016820_23095226.pdf",
|
||||
"folder": "400_23095226",
|
||||
"filename": "trackare-02016820-23095226_02016820_23095226.pdf",
|
||||
"size_mb": 0.29,
|
||||
"pages": 31,
|
||||
"type": "trackare",
|
||||
"complexity": "complexe"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/343_23074384/trackare-15000536-23074384_15000536_23074384.pdf",
|
||||
"folder": "343_23074384",
|
||||
"filename": "trackare-15000536-23074384_15000536_23074384.pdf",
|
||||
"size_mb": 0.25,
|
||||
"pages": 25,
|
||||
"type": "trackare",
|
||||
"complexity": "complexe"
|
||||
},
|
||||
{
|
||||
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/45_23183041/trackare-10027557-23183041_10027557_23183041.pdf",
|
||||
"folder": "45_23183041",
|
||||
"filename": "trackare-10027557-23183041_10027557_23183041.pdf",
|
||||
"size_mb": 0.25,
|
||||
"pages": 20,
|
||||
"type": "trackare",
|
||||
"complexity": "complexe"
|
||||
}
|
||||
]
|
||||
79
tests/unit/test_benchmark.py
Normal file
79
tests/unit/test_benchmark.py
Normal file
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tests unitaires pour le benchmark.
|
||||
"""
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from evaluation.benchmark import Benchmark, BenchmarkResult
|
||||
|
||||
|
||||
class TestBenchmark:
|
||||
"""Tests pour Benchmark."""
|
||||
|
||||
def test_get_system_info(self):
|
||||
"""Test de récupération des informations système."""
|
||||
benchmark = Benchmark(Path("tests/ground_truth"))
|
||||
|
||||
system_info = benchmark.get_system_info()
|
||||
|
||||
assert "os" in system_info
|
||||
assert "cpu" in system_info
|
||||
assert "ram_gb" in system_info
|
||||
assert "python_version" in system_info
|
||||
assert system_info["ram_gb"] > 0
|
||||
|
||||
def test_calculate_summary(self):
|
||||
"""Test de calcul du résumé."""
|
||||
benchmark = Benchmark(Path("tests/ground_truth"))
|
||||
|
||||
results = [
|
||||
BenchmarkResult(
|
||||
pdf_path="test1.pdf",
|
||||
processing_time_s=10.0,
|
||||
cpu_usage_percent=50.0,
|
||||
ram_usage_mb=100.0,
|
||||
pii_detected=10
|
||||
),
|
||||
BenchmarkResult(
|
||||
pdf_path="test2.pdf",
|
||||
processing_time_s=20.0,
|
||||
cpu_usage_percent=60.0,
|
||||
ram_usage_mb=200.0,
|
||||
pii_detected=20
|
||||
)
|
||||
]
|
||||
|
||||
summary = benchmark.calculate_summary(results)
|
||||
|
||||
assert summary["documents_count"] == 2
|
||||
assert summary["avg_time_per_doc"] == 15.0
|
||||
assert summary["min_time"] == 10.0
|
||||
assert summary["max_time"] == 20.0
|
||||
assert summary["avg_cpu_percent"] == 55.0
|
||||
assert summary["avg_ram_mb"] == 150.0
|
||||
assert summary["total_pii_detected"] == 30
|
||||
assert summary["avg_pii_per_doc"] == 15.0
|
||||
|
||||
def test_benchmark_result_to_dict(self):
|
||||
"""Test de conversion en dictionnaire."""
|
||||
result = BenchmarkResult(
|
||||
pdf_path="test.pdf",
|
||||
processing_time_s=12.345,
|
||||
time_per_page_s=4.115,
|
||||
cpu_usage_percent=67.89,
|
||||
ram_usage_mb=123.45,
|
||||
pii_detected=15
|
||||
)
|
||||
|
||||
data = result.to_dict()
|
||||
|
||||
assert data["pdf_path"] == "test.pdf"
|
||||
assert data["processing_time_s"] == 12.35 # Arrondi à 2 décimales
|
||||
assert data["time_per_page_s"] == 4.12
|
||||
assert data["cpu_usage_percent"] == 67.89
|
||||
assert data["ram_usage_mb"] == 123.45
|
||||
assert data["pii_detected"] == 15
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
110
tests/unit/test_leak_scanner.py
Normal file
110
tests/unit/test_leak_scanner.py
Normal file
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tests unitaires pour le scanner de fuite.
|
||||
"""
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from evaluation.leak_scanner import LeakScanner, LeakReport
|
||||
|
||||
|
||||
class TestLeakScanner:
|
||||
"""Tests pour LeakScanner."""
|
||||
|
||||
def test_scan_text_no_leak(self):
|
||||
"""Test sans fuite."""
|
||||
scanner = LeakScanner()
|
||||
|
||||
text = "Le patient a été examiné par le Dr. [NOM] le [DATE]."
|
||||
original_pii = [
|
||||
{"kind": "NOM", "original": "DUPONT"},
|
||||
{"kind": "DATE", "original": "15/01/2024"}
|
||||
]
|
||||
|
||||
leaks = scanner.scan_text(text, original_pii)
|
||||
|
||||
assert len(leaks) == 0
|
||||
|
||||
def test_scan_text_original_pii_present(self):
|
||||
"""Test avec PII original présent."""
|
||||
scanner = LeakScanner()
|
||||
|
||||
text = "Le patient DUPONT a été examiné le 15/01/2024."
|
||||
original_pii = [
|
||||
{"kind": "NOM", "original": "DUPONT"},
|
||||
{"kind": "DATE", "original": "15/01/2024"}
|
||||
]
|
||||
|
||||
leaks = scanner.scan_text(text, original_pii)
|
||||
|
||||
assert len(leaks) == 2
|
||||
assert all(leak["severity"] == "CRITIQUE" for leak in leaks)
|
||||
assert all(leak["type"] == "original_pii_present" for leak in leaks)
|
||||
|
||||
def test_scan_text_new_pii_detected(self):
|
||||
"""Test avec nouveau PII détecté."""
|
||||
scanner = LeakScanner()
|
||||
|
||||
text = "Contact: jean.dupont@example.com ou 01 23 45 67 89"
|
||||
original_pii = []
|
||||
|
||||
leaks = scanner.scan_text(text, original_pii)
|
||||
|
||||
# Devrait détecter l'email et le téléphone
|
||||
assert len(leaks) >= 2
|
||||
|
||||
email_leak = next((l for l in leaks if l["pii_type"] == "EMAIL"), None)
|
||||
assert email_leak is not None
|
||||
assert email_leak["severity"] == "HAUTE"
|
||||
|
||||
tel_leak = next((l for l in leaks if l["pii_type"] == "TEL"), None)
|
||||
assert tel_leak is not None
|
||||
assert tel_leak["severity"] == "HAUTE"
|
||||
|
||||
def test_leak_report_is_safe(self):
|
||||
"""Test de rapport sûr."""
|
||||
report = LeakReport(
|
||||
is_safe=True,
|
||||
leak_count=0,
|
||||
leaks=[],
|
||||
severity_counts={}
|
||||
)
|
||||
|
||||
assert report.is_safe
|
||||
assert report.leak_count == 0
|
||||
|
||||
def test_leak_report_not_safe(self):
|
||||
"""Test de rapport non sûr."""
|
||||
report = LeakReport(
|
||||
is_safe=False,
|
||||
leak_count=2,
|
||||
leaks=[
|
||||
{"severity": "CRITIQUE", "type": "original_pii_present"},
|
||||
{"severity": "HAUTE", "type": "new_pii_detected"}
|
||||
],
|
||||
severity_counts={"CRITIQUE": 1, "HAUTE": 1}
|
||||
)
|
||||
|
||||
assert not report.is_safe
|
||||
assert report.leak_count == 2
|
||||
assert report.severity_counts["CRITIQUE"] == 1
|
||||
assert report.severity_counts["HAUTE"] == 1
|
||||
|
||||
def test_leak_report_to_dict(self):
|
||||
"""Test de conversion en dictionnaire."""
|
||||
report = LeakReport(
|
||||
is_safe=False,
|
||||
leak_count=1,
|
||||
leaks=[{"severity": "CRITIQUE"}],
|
||||
severity_counts={"CRITIQUE": 1}
|
||||
)
|
||||
|
||||
data = report.to_dict()
|
||||
|
||||
assert data["is_safe"] is False
|
||||
assert data["leak_count"] == 1
|
||||
assert len(data["leaks"]) == 1
|
||||
assert data["severity_counts"]["CRITIQUE"] == 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
145
tests/unit/test_quality_evaluator.py
Normal file
145
tests/unit/test_quality_evaluator.py
Normal file
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tests unitaires pour l'évaluateur de qualité.
|
||||
"""
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from evaluation.quality_evaluator import QualityEvaluator, EvaluationResult
|
||||
|
||||
|
||||
class TestQualityEvaluator:
|
||||
"""Tests pour QualityEvaluator."""
|
||||
|
||||
def test_normalize_text(self):
|
||||
"""Test de normalisation de texte."""
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth"))
|
||||
|
||||
assert evaluator.normalize_text("DUPONT") == "dupont"
|
||||
assert evaluator.normalize_text(" DUPONT ") == "dupont"
|
||||
assert evaluator.normalize_text("DUPONT\n\nMARTIN") == "dupont martin"
|
||||
assert evaluator.normalize_text("Jean-Pierre") == "jean-pierre"
|
||||
|
||||
def test_types_match(self):
|
||||
"""Test de correspondance des types."""
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth"))
|
||||
|
||||
# Correspondance directe
|
||||
assert evaluator.types_match("NOM", "NOM")
|
||||
assert evaluator.types_match("NOM", "NOM_GLOBAL")
|
||||
assert evaluator.types_match("TEL", "TEL_GLOBAL")
|
||||
|
||||
# Correspondance croisée
|
||||
assert evaluator.types_match("NOM", "PRENOM")
|
||||
assert evaluator.types_match("PRENOM", "NOM")
|
||||
|
||||
# Non correspondance
|
||||
assert not evaluator.types_match("NOM", "TEL")
|
||||
assert not evaluator.types_match("EMAIL", "ADRESSE")
|
||||
|
||||
def test_calculate_metrics(self):
|
||||
"""Test de calcul des métriques."""
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth"))
|
||||
|
||||
# Cas parfait
|
||||
precision, recall, f1 = evaluator.calculate_metrics(10, 0, 0)
|
||||
assert precision == 1.0
|
||||
assert recall == 1.0
|
||||
assert f1 == 1.0
|
||||
|
||||
# Cas avec erreurs
|
||||
precision, recall, f1 = evaluator.calculate_metrics(8, 2, 2)
|
||||
assert precision == 0.8 # 8 / (8 + 2)
|
||||
assert recall == 0.8 # 8 / (8 + 2)
|
||||
assert f1 == 0.8
|
||||
|
||||
# Cas zéro
|
||||
precision, recall, f1 = evaluator.calculate_metrics(0, 0, 0)
|
||||
assert precision == 0.0
|
||||
assert recall == 0.0
|
||||
assert f1 == 0.0
|
||||
|
||||
def test_compare_simple(self):
|
||||
"""Test de comparaison simple."""
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth"))
|
||||
|
||||
annotations = [
|
||||
{"page": 0, "type": "NOM", "text": "DUPONT", "context": "Dr. DUPONT"},
|
||||
{"page": 0, "type": "TEL", "text": "01 23 45 67 89", "context": "Tel: 01 23 45 67 89"}
|
||||
]
|
||||
|
||||
detections = [
|
||||
{"page": 0, "kind": "NOM", "original": "DUPONT"},
|
||||
{"page": 0, "kind": "TEL", "original": "01 23 45 67 89"}
|
||||
]
|
||||
|
||||
tp, fn, fp = evaluator.compare(annotations, detections)
|
||||
|
||||
assert len(tp) == 2
|
||||
assert len(fn) == 0
|
||||
assert len(fp) == 0
|
||||
|
||||
def test_compare_with_false_negative(self):
|
||||
"""Test avec faux négatif."""
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth"))
|
||||
|
||||
annotations = [
|
||||
{"page": 0, "type": "NOM", "text": "DUPONT", "context": "Dr. DUPONT"},
|
||||
{"page": 0, "type": "TEL", "text": "01 23 45 67 89", "context": "Tel: 01 23 45 67 89"}
|
||||
]
|
||||
|
||||
detections = [
|
||||
{"page": 0, "kind": "NOM", "original": "DUPONT"}
|
||||
# TEL manquant
|
||||
]
|
||||
|
||||
tp, fn, fp = evaluator.compare(annotations, detections)
|
||||
|
||||
assert len(tp) == 1
|
||||
assert len(fn) == 1
|
||||
assert len(fp) == 0
|
||||
assert fn[0]["type"] == "TEL"
|
||||
assert fn[0]["reason"] == "not_detected"
|
||||
|
||||
def test_compare_with_false_positive(self):
|
||||
"""Test avec faux positif."""
|
||||
evaluator = QualityEvaluator(Path("tests/ground_truth"))
|
||||
|
||||
annotations = [
|
||||
{"page": 0, "type": "NOM", "text": "DUPONT", "context": "Dr. DUPONT"}
|
||||
]
|
||||
|
||||
detections = [
|
||||
{"page": 0, "kind": "NOM", "original": "DUPONT"},
|
||||
{"page": 0, "kind": "NOM", "original": "MARTIN"} # Faux positif
|
||||
]
|
||||
|
||||
tp, fn, fp = evaluator.compare(annotations, detections)
|
||||
|
||||
assert len(tp) == 1
|
||||
assert len(fn) == 0
|
||||
assert len(fp) == 1
|
||||
assert fp[0]["text"] == "MARTIN"
|
||||
|
||||
def test_evaluation_result_to_dict(self):
|
||||
"""Test de conversion en dictionnaire."""
|
||||
result = EvaluationResult(
|
||||
pdf_path="test.pdf",
|
||||
true_positives=10,
|
||||
false_positives=2,
|
||||
false_negatives=1,
|
||||
precision=0.8333,
|
||||
recall=0.9091,
|
||||
f1_score=0.8696
|
||||
)
|
||||
|
||||
data = result.to_dict()
|
||||
|
||||
assert data["pdf_path"] == "test.pdf"
|
||||
assert data["true_positives"] == 10
|
||||
assert data["precision"] == 0.8333
|
||||
assert data["recall"] == 0.9091
|
||||
assert data["f1_score"] == 0.8696
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
194
tools/analyze_corpus.py
Executable file
194
tools/analyze_corpus.py
Executable file
@@ -0,0 +1,194 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Analyse du corpus OGC pour sélection de documents représentatifs.
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import json
|
||||
import random
|
||||
|
||||
try:
|
||||
import fitz # PyMuPDF
|
||||
except ImportError:
|
||||
print("PyMuPDF non disponible, analyse limitée")
|
||||
fitz = None
|
||||
|
||||
|
||||
def analyze_pdf(pdf_path: Path) -> dict:
|
||||
"""Analyse un PDF : nombre de pages, taille, type."""
|
||||
stats = {
|
||||
"path": str(pdf_path),
|
||||
"folder": pdf_path.parent.name,
|
||||
"filename": pdf_path.name,
|
||||
"size_mb": round(pdf_path.stat().st_size / (1024 * 1024), 2),
|
||||
"pages": 0,
|
||||
"type": "unknown",
|
||||
}
|
||||
|
||||
# Déterminer le type de document
|
||||
name_lower = pdf_path.name.lower()
|
||||
if "trackare" in name_lower:
|
||||
stats["type"] = "trackare"
|
||||
elif "crh" in name_lower or "cr" in name_lower:
|
||||
stats["type"] = "compte_rendu"
|
||||
elif "anapath" in name_lower:
|
||||
stats["type"] = "anapath"
|
||||
elif "lettre" in name_lower or "sortie" in name_lower:
|
||||
stats["type"] = "lettre_sortie"
|
||||
elif "cro" in name_lower:
|
||||
stats["type"] = "cro"
|
||||
|
||||
# Compter les pages si PyMuPDF disponible
|
||||
if fitz:
|
||||
try:
|
||||
doc = fitz.open(str(pdf_path))
|
||||
stats["pages"] = len(doc)
|
||||
doc.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
def classify_complexity(stats: dict) -> str:
|
||||
"""Classifie la complexité d'un document."""
|
||||
pages = stats["pages"]
|
||||
size_mb = stats["size_mb"]
|
||||
|
||||
if pages <= 2 and size_mb < 0.3:
|
||||
return "simple"
|
||||
elif pages >= 6 or size_mb > 1.0:
|
||||
return "complexe"
|
||||
else:
|
||||
return "moyen"
|
||||
|
||||
|
||||
def main():
|
||||
corpus_dir = Path("/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/")
|
||||
|
||||
if not corpus_dir.exists():
|
||||
print(f"Erreur : {corpus_dir} n'existe pas")
|
||||
return 1
|
||||
|
||||
print("Analyse du corpus OGC...")
|
||||
print(f"Répertoire : {corpus_dir}")
|
||||
|
||||
# Collecter tous les PDFs
|
||||
all_pdfs = list(corpus_dir.glob("*/*.pdf"))
|
||||
print(f"Total PDFs trouvés : {len(all_pdfs)}")
|
||||
|
||||
# Analyser un échantillon pour estimation
|
||||
sample_size = min(100, len(all_pdfs))
|
||||
sample = random.sample(all_pdfs, sample_size)
|
||||
|
||||
print(f"\nAnalyse d'un échantillon de {sample_size} documents...")
|
||||
|
||||
analyzed = []
|
||||
for i, pdf_path in enumerate(sample, 1):
|
||||
if i % 20 == 0:
|
||||
print(f" Analysé {i}/{sample_size}...")
|
||||
stats = analyze_pdf(pdf_path)
|
||||
stats["complexity"] = classify_complexity(stats)
|
||||
analyzed.append(stats)
|
||||
|
||||
# Statistiques globales
|
||||
print("\n" + "="*60)
|
||||
print("STATISTIQUES GLOBALES")
|
||||
print("="*60)
|
||||
|
||||
# Par type
|
||||
types_count = {}
|
||||
for s in analyzed:
|
||||
types_count[s["type"]] = types_count.get(s["type"], 0) + 1
|
||||
|
||||
print("\nRépartition par type :")
|
||||
for doc_type, count in sorted(types_count.items(), key=lambda x: -x[1]):
|
||||
pct = (count / len(analyzed)) * 100
|
||||
print(f" {doc_type:20s} : {count:3d} ({pct:5.1f}%)")
|
||||
|
||||
# Par complexité
|
||||
complexity_count = {}
|
||||
for s in analyzed:
|
||||
complexity_count[s["complexity"]] = complexity_count.get(s["complexity"], 0) + 1
|
||||
|
||||
print("\nRépartition par complexité :")
|
||||
for complexity, count in sorted(complexity_count.items()):
|
||||
pct = (count / len(analyzed)) * 100
|
||||
print(f" {complexity:20s} : {count:3d} ({pct:5.1f}%)")
|
||||
|
||||
# Statistiques pages
|
||||
pages_list = [s["pages"] for s in analyzed if s["pages"] > 0]
|
||||
if pages_list:
|
||||
print(f"\nNombre de pages :")
|
||||
print(f" Min : {min(pages_list)}")
|
||||
print(f" Max : {max(pages_list)}")
|
||||
print(f" Moy : {sum(pages_list) / len(pages_list):.1f}")
|
||||
|
||||
# Statistiques taille
|
||||
sizes_list = [s["size_mb"] for s in analyzed]
|
||||
print(f"\nTaille (MB) :")
|
||||
print(f" Min : {min(sizes_list):.2f}")
|
||||
print(f" Max : {max(sizes_list):.2f}")
|
||||
print(f" Moy : {sum(sizes_list) / len(sizes_list):.2f}")
|
||||
|
||||
# Sélection de 30 documents représentatifs
|
||||
print("\n" + "="*60)
|
||||
print("SÉLECTION DE 30 DOCUMENTS REPRÉSENTATIFS")
|
||||
print("="*60)
|
||||
|
||||
# Stratégie : 10 simples, 15 moyens, 5 complexes
|
||||
# Varier les types de documents
|
||||
|
||||
simples = [s for s in analyzed if s["complexity"] == "simple"]
|
||||
moyens = [s for s in analyzed if s["complexity"] == "moyen"]
|
||||
complexes = [s for s in analyzed if s["complexity"] == "complexe"]
|
||||
|
||||
print(f"\nDisponibles : {len(simples)} simples, {len(moyens)} moyens, {len(complexes)} complexes")
|
||||
|
||||
selected = []
|
||||
|
||||
# Sélectionner 10 simples
|
||||
if len(simples) >= 10:
|
||||
selected.extend(random.sample(simples, 10))
|
||||
else:
|
||||
selected.extend(simples)
|
||||
print(f"⚠️ Seulement {len(simples)} documents simples disponibles")
|
||||
|
||||
# Sélectionner 15 moyens
|
||||
if len(moyens) >= 15:
|
||||
selected.extend(random.sample(moyens, 15))
|
||||
else:
|
||||
selected.extend(moyens)
|
||||
print(f"⚠️ Seulement {len(moyens)} documents moyens disponibles")
|
||||
|
||||
# Sélectionner 5 complexes
|
||||
if len(complexes) >= 5:
|
||||
selected.extend(random.sample(complexes, 5))
|
||||
else:
|
||||
selected.extend(complexes)
|
||||
print(f"⚠️ Seulement {len(complexes)} documents complexes disponibles")
|
||||
|
||||
print(f"\nTotal sélectionnés : {len(selected)}")
|
||||
|
||||
# Sauvegarder la sélection
|
||||
output_file = Path("tests/ground_truth/selected_documents.json")
|
||||
output_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with open(output_file, "w", encoding="utf-8") as f:
|
||||
json.dump(selected, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"\nSélection sauvegardée dans : {output_file}")
|
||||
|
||||
# Afficher la liste
|
||||
print("\nDocuments sélectionnés :")
|
||||
print("-" * 80)
|
||||
for i, doc in enumerate(selected, 1):
|
||||
print(f"{i:2d}. [{doc['complexity']:8s}] {doc['folder']}/{doc['filename']}")
|
||||
print(f" {doc['pages']} pages, {doc['size_mb']} MB, type: {doc['type']}")
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
400
tools/annotation_tool.py
Executable file
400
tools/annotation_tool.py
Executable file
@@ -0,0 +1,400 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Outil d'annotation CLI pour créer le dataset de test annoté.
|
||||
|
||||
Usage:
|
||||
python tools/annotation_tool.py <pdf_path>
|
||||
python tools/annotation_tool.py --list
|
||||
python tools/annotation_tool.py --resume
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from typing import List, Dict, Optional, Tuple
|
||||
import re
|
||||
|
||||
try:
|
||||
import pymupdf as fitz
|
||||
except ImportError:
|
||||
import fitz
|
||||
|
||||
|
||||
class AnnotationTool:
|
||||
"""Outil d'annotation interactif pour les documents PDF."""
|
||||
|
||||
PII_TYPES = [
|
||||
"NOM", "PRENOM", "DATE_NAISSANCE", "AGE",
|
||||
"TEL", "EMAIL", "ADRESSE", "CODE_POSTAL", "VILLE",
|
||||
"NIR", "IPP", "NDA", "RPPS", "FINESS", "OGC",
|
||||
"NUMERO_PATIENT", "NUMERO_LOT", "NUMERO_ORDONNANCE", "NUMERO_SEJOUR",
|
||||
"ETABLISSEMENT", "SERVICE", "DATE", "AUTRE"
|
||||
]
|
||||
|
||||
def __init__(self, pdf_path: Path):
|
||||
self.pdf_path = pdf_path
|
||||
self.annotations_path = pdf_path.parent / f"{pdf_path.stem}.annotations.json"
|
||||
self.doc = None
|
||||
self.annotations = []
|
||||
self.medical_terms = []
|
||||
self.metadata = {
|
||||
"annotator": "annotator_1",
|
||||
"annotation_date": datetime.now().isoformat(),
|
||||
"document_type": "unknown",
|
||||
"page_count": 0,
|
||||
"difficulty": "medium"
|
||||
}
|
||||
|
||||
def load_pdf(self) -> bool:
|
||||
"""Charge le PDF et extrait le texte."""
|
||||
try:
|
||||
self.doc = fitz.open(self.pdf_path)
|
||||
self.metadata["page_count"] = len(self.doc)
|
||||
print(f"✓ PDF chargé: {self.pdf_path.name}")
|
||||
print(f" Pages: {len(self.doc)}")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors du chargement du PDF: {e}")
|
||||
return False
|
||||
|
||||
def extract_text(self, page_num: int) -> str:
|
||||
"""Extrait le texte d'une page."""
|
||||
if not self.doc or page_num >= len(self.doc):
|
||||
return ""
|
||||
|
||||
page = self.doc[page_num]
|
||||
return page.get_text()
|
||||
|
||||
def display_page(self, page_num: int):
|
||||
"""Affiche le texte d'une page."""
|
||||
text = self.extract_text(page_num)
|
||||
|
||||
print(f"\n{'='*80}")
|
||||
print(f"PAGE {page_num + 1}/{len(self.doc)}")
|
||||
print(f"{'='*80}")
|
||||
print(text)
|
||||
print(f"{'='*80}\n")
|
||||
|
||||
def get_context(self, text: str, pii_text: str, window: int = 50) -> str:
|
||||
"""Extrait le contexte autour d'un PII."""
|
||||
pos = text.find(pii_text)
|
||||
if pos == -1:
|
||||
return ""
|
||||
|
||||
start = max(0, pos - window)
|
||||
end = min(len(text), pos + len(pii_text) + window)
|
||||
context = text[start:end]
|
||||
|
||||
# Nettoyer les retours à la ligne multiples
|
||||
context = re.sub(r'\n+', ' ', context)
|
||||
context = re.sub(r'\s+', ' ', context)
|
||||
|
||||
return context.strip()
|
||||
|
||||
def input_with_default(self, prompt: str, default: str = "") -> str:
|
||||
"""Demande une entrée avec valeur par défaut."""
|
||||
if default:
|
||||
user_input = input(f"{prompt} [{default}]: ").strip()
|
||||
return user_input if user_input else default
|
||||
else:
|
||||
return input(f"{prompt}: ").strip()
|
||||
|
||||
def select_from_list(self, prompt: str, options: List[str], default: Optional[str] = None) -> str:
|
||||
"""Sélection dans une liste d'options."""
|
||||
print(f"\n{prompt}")
|
||||
for i, option in enumerate(options, 1):
|
||||
marker = " (défaut)" if option == default else ""
|
||||
print(f" {i}. {option}{marker}")
|
||||
|
||||
while True:
|
||||
choice = input(f"Choix [1-{len(options)}]: ").strip()
|
||||
|
||||
if not choice and default:
|
||||
return default
|
||||
|
||||
try:
|
||||
idx = int(choice) - 1
|
||||
if 0 <= idx < len(options):
|
||||
return options[idx]
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
print(f"✗ Choix invalide. Entrez un nombre entre 1 et {len(options)}")
|
||||
|
||||
def annotate_pii(self, page_num: int, text: str) -> List[Dict]:
|
||||
"""Annotation interactive des PII d'une page."""
|
||||
page_annotations = []
|
||||
|
||||
print(f"\n--- Annotation de la page {page_num + 1} ---")
|
||||
print("Commandes: 'q' pour terminer la page, 's' pour sauter")
|
||||
|
||||
ann_id = len(self.annotations) + 1
|
||||
|
||||
while True:
|
||||
print(f"\n[Annotation #{ann_id}]")
|
||||
|
||||
# Texte du PII
|
||||
pii_text = input("Texte du PII (ou 'q' pour terminer, 's' pour sauter): ").strip()
|
||||
|
||||
if pii_text.lower() == 'q':
|
||||
break
|
||||
if pii_text.lower() == 's':
|
||||
continue
|
||||
if not pii_text:
|
||||
print("✗ Le texte ne peut pas être vide")
|
||||
continue
|
||||
|
||||
# Type de PII
|
||||
pii_type = self.select_from_list(
|
||||
"Type de PII:",
|
||||
self.PII_TYPES,
|
||||
default="NOM"
|
||||
)
|
||||
|
||||
# Contexte
|
||||
context = self.get_context(text, pii_text)
|
||||
if context:
|
||||
print(f"Contexte détecté: {context[:100]}...")
|
||||
use_context = input("Utiliser ce contexte? [O/n]: ").strip().lower()
|
||||
if use_context == 'n':
|
||||
context = input("Contexte manuel: ").strip()
|
||||
else:
|
||||
context = input("Contexte: ").strip()
|
||||
|
||||
# Obligatoire?
|
||||
mandatory_input = input("PII obligatoire (RGPD)? [O/n]: ").strip().lower()
|
||||
mandatory = mandatory_input != 'n'
|
||||
|
||||
# Difficulté
|
||||
difficulty = self.select_from_list(
|
||||
"Difficulté de détection:",
|
||||
["easy", "medium", "hard"],
|
||||
default="medium"
|
||||
)
|
||||
|
||||
# Méthodes de détection attendues
|
||||
print("\nMéthodes de détection attendues (séparées par des virgules):")
|
||||
print(" Options: regex, vlm, ner, contextual, trackare")
|
||||
methods_input = input("Méthodes [regex,ner]: ").strip()
|
||||
if not methods_input:
|
||||
methods = ["regex", "ner"]
|
||||
else:
|
||||
methods = [m.strip() for m in methods_input.split(',')]
|
||||
|
||||
# Créer l'annotation
|
||||
annotation = {
|
||||
"id": f"ann_{ann_id:03d}",
|
||||
"page": page_num,
|
||||
"type": pii_type,
|
||||
"text": pii_text,
|
||||
"bbox": None, # Pas de bbox pour l'instant (annotation manuelle)
|
||||
"context": context,
|
||||
"mandatory": mandatory,
|
||||
"difficulty": difficulty,
|
||||
"detection_method_expected": methods
|
||||
}
|
||||
|
||||
page_annotations.append(annotation)
|
||||
print(f"✓ Annotation ajoutée: {pii_type} = '{pii_text}'")
|
||||
|
||||
ann_id += 1
|
||||
|
||||
return page_annotations
|
||||
|
||||
def annotate_document(self):
|
||||
"""Annotation complète du document."""
|
||||
if not self.load_pdf():
|
||||
return False
|
||||
|
||||
# Métadonnées du document
|
||||
print("\n=== Métadonnées du document ===")
|
||||
|
||||
self.metadata["annotator"] = self.input_with_default(
|
||||
"Nom de l'annotateur",
|
||||
default="annotator_1"
|
||||
)
|
||||
|
||||
self.metadata["document_type"] = self.select_from_list(
|
||||
"Type de document:",
|
||||
["compte_rendu", "trackare", "anapath", "bacterio", "consultation", "autre"],
|
||||
default="compte_rendu"
|
||||
)
|
||||
|
||||
self.metadata["difficulty"] = self.select_from_list(
|
||||
"Difficulté globale du document:",
|
||||
["simple", "moyen", "complexe"],
|
||||
default="moyen"
|
||||
)
|
||||
|
||||
# Annoter chaque page
|
||||
for page_num in range(len(self.doc)):
|
||||
self.display_page(page_num)
|
||||
|
||||
annotate_page = input(f"\nAnnoter cette page? [O/n]: ").strip().lower()
|
||||
if annotate_page == 'n':
|
||||
continue
|
||||
|
||||
text = self.extract_text(page_num)
|
||||
page_annotations = self.annotate_pii(page_num, text)
|
||||
self.annotations.extend(page_annotations)
|
||||
|
||||
# Termes médicaux à préserver
|
||||
print("\n=== Termes médicaux à préserver ===")
|
||||
print("Entrez les termes médicaux qui ne doivent PAS être masqués")
|
||||
print("(un par ligne, ligne vide pour terminer)")
|
||||
|
||||
while True:
|
||||
term = input("Terme médical: ").strip()
|
||||
if not term:
|
||||
break
|
||||
self.medical_terms.append(term)
|
||||
print(f"✓ Ajouté: {term}")
|
||||
|
||||
return True
|
||||
|
||||
def save_annotations(self):
|
||||
"""Sauvegarde les annotations au format JSON."""
|
||||
# Calculer les statistiques
|
||||
stats = {
|
||||
"total_pii": len(self.annotations),
|
||||
"by_type": {}
|
||||
}
|
||||
|
||||
for ann in self.annotations:
|
||||
pii_type = ann["type"]
|
||||
stats["by_type"][pii_type] = stats["by_type"].get(pii_type, 0) + 1
|
||||
|
||||
# Créer la structure finale
|
||||
output = {
|
||||
"pdf_path": str(self.pdf_path),
|
||||
"metadata": self.metadata,
|
||||
"annotations": self.annotations,
|
||||
"medical_terms_to_preserve": self.medical_terms,
|
||||
"statistics": stats
|
||||
}
|
||||
|
||||
# Sauvegarder
|
||||
with open(self.annotations_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(output, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"\n✓ Annotations sauvegardées: {self.annotations_path}")
|
||||
print(f" Total PII: {stats['total_pii']}")
|
||||
print(f" Types: {', '.join(f'{k}={v}' for k, v in stats['by_type'].items())}")
|
||||
print(f" Termes médicaux: {len(self.medical_terms)}")
|
||||
|
||||
def load_existing_annotations(self) -> bool:
|
||||
"""Charge les annotations existantes si disponibles."""
|
||||
if not self.annotations_path.exists():
|
||||
return False
|
||||
|
||||
try:
|
||||
with open(self.annotations_path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
|
||||
self.metadata = data.get("metadata", self.metadata)
|
||||
self.annotations = data.get("annotations", [])
|
||||
self.medical_terms = data.get("medical_terms_to_preserve", [])
|
||||
|
||||
print(f"✓ Annotations existantes chargées: {len(self.annotations)} PII")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"✗ Erreur lors du chargement des annotations: {e}")
|
||||
return False
|
||||
|
||||
def run(self):
|
||||
"""Exécute l'outil d'annotation."""
|
||||
print(f"\n{'='*80}")
|
||||
print(f"OUTIL D'ANNOTATION - {self.pdf_path.name}")
|
||||
print(f"{'='*80}")
|
||||
|
||||
# Vérifier si des annotations existent déjà
|
||||
if self.annotations_path.exists():
|
||||
overwrite = input(f"\n⚠ Des annotations existent déjà. Écraser? [o/N]: ").strip().lower()
|
||||
if overwrite != 'o':
|
||||
print("Annotation annulée.")
|
||||
return False
|
||||
|
||||
# Annoter le document
|
||||
if not self.annotate_document():
|
||||
return False
|
||||
|
||||
# Sauvegarder
|
||||
self.save_annotations()
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def list_documents():
|
||||
"""Liste les documents disponibles pour annotation."""
|
||||
pdfs_dir = Path("tests/ground_truth/pdfs")
|
||||
|
||||
if not pdfs_dir.exists():
|
||||
print(f"✗ Répertoire introuvable: {pdfs_dir}")
|
||||
return
|
||||
|
||||
pdfs = sorted(pdfs_dir.glob("*.pdf"))
|
||||
|
||||
if not pdfs:
|
||||
print(f"✗ Aucun PDF trouvé dans {pdfs_dir}")
|
||||
return
|
||||
|
||||
print(f"\n{'='*80}")
|
||||
print(f"DOCUMENTS DISPONIBLES ({len(pdfs)})")
|
||||
print(f"{'='*80}\n")
|
||||
|
||||
for pdf in pdfs:
|
||||
annotation_file = pdf.parent / f"{pdf.stem}.annotations.json"
|
||||
status = "✓ Annoté" if annotation_file.exists() else "○ À annoter"
|
||||
print(f"{status} {pdf.name}")
|
||||
|
||||
|
||||
def find_next_unannotated() -> Optional[Path]:
|
||||
"""Trouve le prochain document non annoté."""
|
||||
pdfs_dir = Path("tests/ground_truth/pdfs")
|
||||
|
||||
if not pdfs_dir.exists():
|
||||
return None
|
||||
|
||||
for pdf in sorted(pdfs_dir.glob("*.pdf")):
|
||||
annotation_file = pdf.parent / f"{pdf.stem}.annotations.json"
|
||||
if not annotation_file.exists():
|
||||
return pdf
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) > 1:
|
||||
if sys.argv[1] == "--list":
|
||||
list_documents()
|
||||
return
|
||||
elif sys.argv[1] == "--resume":
|
||||
next_pdf = find_next_unannotated()
|
||||
if next_pdf:
|
||||
print(f"Prochain document à annoter: {next_pdf.name}")
|
||||
tool = AnnotationTool(next_pdf)
|
||||
tool.run()
|
||||
else:
|
||||
print("✓ Tous les documents sont annotés!")
|
||||
return
|
||||
else:
|
||||
pdf_path = Path(sys.argv[1])
|
||||
else:
|
||||
print("Usage:")
|
||||
print(" python tools/annotation_tool.py <pdf_path>")
|
||||
print(" python tools/annotation_tool.py --list")
|
||||
print(" python tools/annotation_tool.py --resume")
|
||||
sys.exit(1)
|
||||
|
||||
if not pdf_path.exists():
|
||||
print(f"✗ Fichier introuvable: {pdf_path}")
|
||||
sys.exit(1)
|
||||
|
||||
tool = AnnotationTool(pdf_path)
|
||||
success = tool.run()
|
||||
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
96
tools/copy_selected_docs.py
Normal file
96
tools/copy_selected_docs.py
Normal file
@@ -0,0 +1,96 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script pour copier les documents sélectionnés dans tests/ground_truth/
|
||||
"""
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
def copy_selected_documents():
|
||||
"""Copie les documents sélectionnés dans le répertoire de test."""
|
||||
|
||||
# Charger la liste des documents sélectionnés
|
||||
selected_file = Path("tests/ground_truth/selected_documents.json")
|
||||
with open(selected_file, 'r', encoding='utf-8') as f:
|
||||
documents = json.load(f)
|
||||
|
||||
# Créer le répertoire de destination
|
||||
dest_dir = Path("tests/ground_truth/pdfs")
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Copier chaque document
|
||||
copied = 0
|
||||
errors = []
|
||||
|
||||
for i, doc in enumerate(documents, 1):
|
||||
src_path = Path(doc['path'])
|
||||
|
||||
# Créer un nom de fichier unique et descriptif
|
||||
# Format: {index:03d}_{complexity}_{type}_{original_name}
|
||||
doc_type = doc.get('type', 'unknown')
|
||||
complexity = doc.get('complexity', 'unknown')
|
||||
original_name = doc['filename']
|
||||
|
||||
# Nettoyer le nom de fichier
|
||||
safe_name = original_name.replace(' ', '_')
|
||||
dest_name = f"{i:03d}_{complexity}_{doc_type}_{safe_name}"
|
||||
dest_path = dest_dir / dest_name
|
||||
|
||||
try:
|
||||
if src_path.exists():
|
||||
shutil.copy2(src_path, dest_path)
|
||||
print(f"✓ Copié: {dest_name}")
|
||||
copied += 1
|
||||
else:
|
||||
error_msg = f"✗ Fichier introuvable: {src_path}"
|
||||
print(error_msg)
|
||||
errors.append(error_msg)
|
||||
except Exception as e:
|
||||
error_msg = f"✗ Erreur lors de la copie de {src_path}: {e}"
|
||||
print(error_msg)
|
||||
errors.append(error_msg)
|
||||
|
||||
# Résumé
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Résumé:")
|
||||
print(f" Documents copiés: {copied}/{len(documents)}")
|
||||
print(f" Erreurs: {len(errors)}")
|
||||
print(f" Destination: {dest_dir.absolute()}")
|
||||
|
||||
if errors:
|
||||
print(f"\nErreurs rencontrées:")
|
||||
for error in errors:
|
||||
print(f" {error}")
|
||||
|
||||
# Créer un fichier de mapping
|
||||
mapping = []
|
||||
for i, doc in enumerate(documents, 1):
|
||||
doc_type = doc.get('type', 'unknown')
|
||||
complexity = doc.get('complexity', 'unknown')
|
||||
original_name = doc['filename']
|
||||
safe_name = original_name.replace(' ', '_')
|
||||
dest_name = f"{i:03d}_{complexity}_{doc_type}_{safe_name}"
|
||||
|
||||
mapping.append({
|
||||
"id": i,
|
||||
"dest_filename": dest_name,
|
||||
"original_path": doc['path'],
|
||||
"folder": doc['folder'],
|
||||
"original_filename": doc['filename'],
|
||||
"type": doc_type,
|
||||
"complexity": complexity,
|
||||
"pages": doc.get('pages', 0),
|
||||
"size_mb": doc.get('size_mb', 0)
|
||||
})
|
||||
|
||||
mapping_file = dest_dir / "mapping.json"
|
||||
with open(mapping_file, 'w', encoding='utf-8') as f:
|
||||
json.dump(mapping, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"\nFichier de mapping créé: {mapping_file}")
|
||||
|
||||
return copied, len(errors)
|
||||
|
||||
if __name__ == "__main__":
|
||||
copied, errors = copy_selected_documents()
|
||||
exit(0 if errors == 0 else 1)
|
||||
Reference in New Issue
Block a user