Dom/anonymisation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Phase 1 - Système d'évaluation de la qualité

Browse Source 
- Sélection et copie de 27 documents représentatifs (10 simples, 12 moyens, 5 complexes)
- Outil d'annotation CLI complet (tools/annotation_tool.py)
- Guide d'annotation détaillé (docs/annotation_guide.md)
- Évaluateur de qualité (evaluation/quality_evaluator.py)
  * Calcul Précision, Rappel, F1-Score
  * Identification faux positifs/négatifs
  * Métriques par type de PII
  * Export JSON et rapports texte
- Scanner de fuite (evaluation/leak_scanner.py)
  * Détection PII résiduels (CRITIQUE)
  * Détection nouveaux PII (HAUTE)
  * Scan métadonnées PDF (MOYENNE)
- Benchmark de performance (evaluation/benchmark.py)
  * Mesure temps de traitement
  * Mesure CPU/RAM
  * Export JSON/CSV
- Tests unitaires complets pour tous les composants
- Documentation complète du module d'évaluation

Tâches complétées:
- 1.1.1 Sélection de 27 documents (au lieu de 30)
- 1.1.2 Outil d'annotation CLI
- 1.2.1 Évaluateur de qualité
- 1.2.2 Scanner de fuite
- 1.2.3 Benchmark de performance

Prochaines étapes:
- 1.1.3 Annotation des 27 documents (manuel)
- 1.1.4 Enrichissement stopwords médicaux
- 1.3 Mesure de la baseline
This commit is contained in:
Domi31tls
2026-03-02 10:07:41 +01:00
parent 0067738df6
commit 340348b820
86 changed files with 35587 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

245
tests/ground_truth/selected_documents.json Normal file
View File

@@ -0,0 +1,245 @@
[
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/148_23018396/BACTERIO 23018396.pdf",
"folder": "148_23018396",
"filename": "BACTERIO 23018396.pdf",
"size_mb": 0.04,
"pages": 1,
"type": "unknown",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/476_23159413/bacterio 476_23159413.pdf",
"folder": "476_23159413",
"filename": "bacterio 476_23159413.pdf",
"size_mb": 0.04,
"pages": 2,
"type": "unknown",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/210_23155084/CRO 23155084.pdf",
"folder": "210_23155084",
"filename": "CRO 23155084.pdf",
"size_mb": 0.05,
"pages": 1,
"type": "compte_rendu",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/anapath 53_23224186.redacted_raster.pdf",
"folder": "anonymise",
"filename": "anapath 53_23224186.redacted_raster.pdf",
"size_mb": 0.29,
"pages": 1,
"type": "anapath",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/212_23155836/CRH 23155836.pdf",
"folder": "212_23155836",
"filename": "CRH 23155836.pdf",
"size_mb": 0.14,
"pages": 2,
"type": "compte_rendu",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/204_23142660/ANAPATH 23142660.pdf",
"folder": "204_23142660",
"filename": "ANAPATH 23142660.pdf",
"size_mb": 0.16,
"pages": 0,
"type": "anapath",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/15_23096332/ANAPATH 23096332.pdf",
"folder": "15_23096332",
"filename": "ANAPATH 23096332.pdf",
"size_mb": 0.16,
"pages": 1,
"type": "anapath",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/80_23202435/trackare-14004105-23202435_14004105_23202435.pdf",
"folder": "80_23202435",
"filename": "trackare-14004105-23202435_14004105_23202435.pdf",
"size_mb": 0.11,
"pages": 1,
"type": "trackare",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/138_23051225/CRO 23051225.pdf",
"folder": "138_23051225",
"filename": "CRO 23051225.pdf",
"size_mb": 0.06,
"pages": 2,
"type": "compte_rendu",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/49_23217289/ANAPATH 23217289.pdf",
"folder": "49_23217289",
"filename": "ANAPATH 23217289.pdf",
"size_mb": 0.17,
"pages": 1,
"type": "anapath",
"complexity": "simple"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/132_23080179/CRH 23080179.pdf",
"folder": "132_23080179",
"filename": "CRH 23080179.pdf",
"size_mb": 0.07,
"pages": 4,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/692_23200418/CRH 692_23200418.pdf",
"folder": "692_23200418",
"filename": "CRH 692_23200418.pdf",
"size_mb": 0.59,
"pages": 2,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/363_23085243/363_23085243 CRO.pdf",
"folder": "363_23085243",
"filename": "363_23085243 CRO.pdf",
"size_mb": 0.58,
"pages": 2,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23167029.redacted_raster.pdf",
"folder": "anonymise",
"filename": "CRO 23167029.redacted_raster.pdf",
"size_mb": 0.65,
"pages": 2,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/222_23139653/CONSULTATION ANESTHESISTE 23139653.pdf",
"folder": "222_23139653",
"filename": "CONSULTATION ANESTHESISTE 23139653.pdf",
"size_mb": 0.12,
"pages": 3,
"type": "unknown",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/199_23149905/CRH 23149905.pdf",
"folder": "199_23149905",
"filename": "CRH 23149905.pdf",
"size_mb": 0.15,
"pages": 3,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23222062.redacted_raster.pdf",
"folder": "anonymise",
"filename": "CRO 23222062.redacted_raster.pdf",
"size_mb": 0.57,
"pages": 2,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/1_23042753/CRH 23042753.pdf",
"folder": "1_23042753",
"filename": "CRH 23042753.pdf",
"size_mb": 0.15,
"pages": 4,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/332_23049003/CRO 332_23049003.pdf",
"folder": "332_23049003",
"filename": "CRO 332_23049003.pdf",
"size_mb": 0.43,
"pages": 2,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23084754.redacted_raster.pdf",
"folder": "anonymise",
"filename": "CRO 23084754.redacted_raster.pdf",
"size_mb": 0.46,
"pages": 2,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/anonymise/CRO 23201117.redacted_raster.pdf",
"folder": "anonymise",
"filename": "CRO 23201117.redacted_raster.pdf",
"size_mb": 0.33,
"pages": 1,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/516_23187028/cro2 516_23187028.pdf",
"folder": "516_23187028",
"filename": "cro2 516_23187028.pdf",
"size_mb": 0.3,
"pages": 1,
"type": "compte_rendu",
"complexity": "moyen"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/153_23102610/CRH 23102610.pdf",
"folder": "153_23102610",
"filename": "CRH 23102610.pdf",
"size_mb": 0.14,
"pages": 9,
"type": "compte_rendu",
"complexity": "complexe"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/115_23066188/trackare-17001141-23066188_17001141_23066188.pdf",
"folder": "115_23066188",
"filename": "trackare-17001141-23066188_17001141_23066188.pdf",
"size_mb": 0.21,
"pages": 19,
"type": "trackare",
"complexity": "complexe"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/400_23095226/trackare-02016820-23095226_02016820_23095226.pdf",
"folder": "400_23095226",
"filename": "trackare-02016820-23095226_02016820_23095226.pdf",
"size_mb": 0.29,
"pages": 31,
"type": "trackare",
"complexity": "complexe"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/343_23074384/trackare-15000536-23074384_15000536_23074384.pdf",
"folder": "343_23074384",
"filename": "trackare-15000536-23074384_15000536_23074384.pdf",
"size_mb": 0.25,
"pages": 25,
"type": "trackare",
"complexity": "complexe"
},
{
"path": "/home/dom/Téléchargements/II-1 Ctrl_T2A_2025_CHCB_DocJustificatifs (1)/45_23183041/trackare-10027557-23183041_10027557_23183041.pdf",
"folder": "45_23183041",
"filename": "trackare-10027557-23183041_10027557_23183041.pdf",
"size_mb": 0.25,
"pages": 20,
"type": "trackare",
"complexity": "complexe"
}
]