This commit is contained in:
1 parent
e9532e8878
commit
5225090490
15 files changed
+2553
-17
No files matched your search
@@ -216,6 +216,22 @@ Das Frontend ist eine Streamlit-App, die sich wie eine Single-Page-Application (
|
||||
|
||||
Das Backend ist das Herzstück. Es stellt die Logik via REST-API bereit.
|
||||
|
||||
**Wichtig:** Seit WP-14 ist die Core-Logik in spezialisierte Pakete unterteilt:
|
||||
|
||||
#### Core-Pakete (Modularisierung WP-14)
|
||||
|
||||
| Paket | Zweck | Wichtige Module |
|
||||
| :--- | :--- | :--- |
|
||||
| **`app/core/chunking/`** | Text-Segmentierung | `chunking_strategies.py` (Sliding/Heading), `chunking_processor.py` (Orchestrierung) |
|
||||
| **`app/core/database/`** | Qdrant-Infrastruktur | `qdrant.py` (Client), `qdrant_points.py` (Point-Mapping) |
|
||||
| **`app/core/graph/`** | Graph-Logik | `graph_subgraph.py` (Expansion), `graph_weights.py` (Scoring) |
|
||||
| **`app/core/ingestion/`** | Import-Pipeline | `ingestion_processor.py` (Two-Pass), `ingestion_validation.py` (Mistral-safe Parsing) |
|
||||
| **`app/core/parser/`** | Markdown-Parsing | `parsing_markdown.py` (Frontmatter/Body), `parsing_scanner.py` (File-Scan) |
|
||||
| **`app/core/retrieval/`** | Suche & Scoring | `retriever.py` (Orchestrator), `retriever_scoring.py` (Mathematik) |
|
||||
| **`app/core/registry.py`** | SSOT & Utilities | Text-Bereinigung, Circular-Import-Fix |
|
||||
|
||||
**Legacy-Bridges:** Die alten Dateien (`app/core/retriever.py`, `app/core/qdrant.py`) existieren noch als Proxy-Adapter für Abwärtskompatibilität, delegieren aber an die neuen Pakete.
|
||||
|
||||
| Layer | Datei | Status | Verantwortung |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Entry** | `app/main.py` | 🟢 **Core** | **Entrypoint.** Initialisiert FastAPI, CORS, und bindet alle Router ein. |
|
||||
|
||||
@@ -0,0 +1,392 @@
|
||||
---
|
||||
doc_type: developer_guide
|
||||
audience: developer, tester
|
||||
scope: testing, quality_assurance, test_strategies
|
||||
status: active
|
||||
version: 2.9.1
|
||||
context: "Umfassender Test-Guide für Mindnet: Test-Strategien, Test-Frameworks, Test-Daten und Best Practices."
|
||||
---
|
||||
|
||||
# Testing Guide
|
||||
|
||||
Dieses Dokument beschreibt die Test-Strategien, Test-Frameworks und Best Practices für die Qualitätssicherung von Mindnet.
|
||||
|
||||
## 1. Test-Strategie & Ebenen
|
||||
|
||||
Mindnet nutzt eine **dreistufige Test-Pyramide**:
|
||||
|
||||
```
|
||||
/\
|
||||
/E2E\ ← Wenige, langsame, teure Tests
|
||||
/------\
|
||||
/Integration\ ← Mittlere Anzahl, mittlere Geschwindigkeit
|
||||
/------------\
|
||||
/ Unit Tests \ ← Viele, schnelle, isolierte Tests
|
||||
/----------------\
|
||||
```
|
||||
|
||||
### 1.1 Unit Tests (Pytest)
|
||||
|
||||
**Zweck:** Isolierte Logik-Tests ohne externe Abhängigkeiten.
|
||||
|
||||
**Framework:** `pytest`
|
||||
|
||||
**Beispiele:**
|
||||
- `tests/test_retriever_basic.py` - Scoring-Logik
|
||||
- `tests/test_chunking.py` - Chunking-Strategien
|
||||
- `tests/test_edges_all.py` - Edge-Logik
|
||||
- `tests/test_type_registry.py` - Registry-Funktionen
|
||||
|
||||
**Ausführung:**
|
||||
```bash
|
||||
# Einzelner Test
|
||||
pytest tests/test_retriever_basic.py -v
|
||||
|
||||
# Alle Unit Tests
|
||||
pytest tests/ -k "test_" --ignore=tests/test_*_smoke.py
|
||||
|
||||
# Mit Coverage
|
||||
pytest tests/ --cov=app --cov-report=html
|
||||
```
|
||||
|
||||
**Best Practices:**
|
||||
- Tests sollten isoliert sein (keine Abhängigkeiten zu Qdrant/Ollama)
|
||||
- Mock externe Services (LLM, Qdrant)
|
||||
- Schnelle Ausführung (< 1 Sekunde pro Test)
|
||||
|
||||
### 1.2 Integration Tests
|
||||
|
||||
**Zweck:** Prüfen den Datenfluss von Markdown bis Qdrant.
|
||||
|
||||
**Tools:**
|
||||
- `scripts/payload_dryrun.py` - JSON-Schema-Konformität
|
||||
- `scripts/edges_full_check.py` - Graph-Integrität
|
||||
- `scripts/make_test_vault.py` - Test-Daten-Generierung
|
||||
|
||||
**Ausführung:**
|
||||
```bash
|
||||
# Payload-Validierung
|
||||
python3 -m scripts.payload_dryrun --vault ./test_vault --with-edges
|
||||
|
||||
# Graph-Integrität
|
||||
python3 -m scripts.edges_full_check
|
||||
|
||||
# Test-Vault erstellen
|
||||
python3 -m scripts.make_test_vault --out ./test_vault --force
|
||||
```
|
||||
|
||||
**Was wird geprüft:**
|
||||
- Frontmatter-Parsing
|
||||
- Chunk-Generierung
|
||||
- Edge-Erstellung
|
||||
- Payload-Schema-Konformität
|
||||
- Graph-Invarianten (keine Dangling Edges)
|
||||
|
||||
### 1.3 E2E / Smoke Tests
|
||||
|
||||
**Zweck:** Prüfen das laufende System gegen echte Infrastruktur (Qdrant, Ollama).
|
||||
|
||||
**Framework:** Python `unittest` + Shell-Skripte
|
||||
|
||||
**Beispiele:**
|
||||
- `tests/test_wp06_decision.py` - Decision Engine
|
||||
- `tests/test_feedback_smoke.py` - Feedback-Loop
|
||||
- `tests/test_dialog_full_flow.py` - Vollständiger Dialog-Flow
|
||||
- `tests/run_e2e_roundtrip.sh` - Kompletter Roundtrip (Import → Export)
|
||||
|
||||
**Ausführung:**
|
||||
```bash
|
||||
# Decision Engine Test
|
||||
python tests/test_wp06_decision.py -p 8002 -e DECISION -q "Soll ich X tun?"
|
||||
|
||||
# Feedback Test
|
||||
python tests/test_feedback_smoke.py --url http://localhost:8002/query
|
||||
|
||||
# E2E Roundtrip
|
||||
./tests/run_e2e_roundtrip.sh --vault ./test_vault --prefix mindnet_test
|
||||
```
|
||||
|
||||
**Voraussetzungen:**
|
||||
- Qdrant läuft auf `localhost:6333`
|
||||
- Ollama läuft auf `localhost:11434`
|
||||
- API läuft auf Port 8002 (Dev)
|
||||
|
||||
---
|
||||
|
||||
## 2. Test-Daten & Vaults
|
||||
|
||||
### 2.1 Test-Vault erstellen
|
||||
|
||||
**Tool:** `scripts/make_test_vault.py`
|
||||
|
||||
**Zweck:** Erstellt einen minimalen, nachvollziehbaren Test-Vault.
|
||||
|
||||
**Inhalt:**
|
||||
- Verschiedene Note-Typen (`concept`, `experience`, `project`)
|
||||
- Verschiedene Edge-Szenarien (explicit, implicit, missing links)
|
||||
- Externe Links für Edge-Tests
|
||||
|
||||
**Verwendung:**
|
||||
```bash
|
||||
# Standard
|
||||
python3 -m scripts.make_test_vault --out ./test_vault
|
||||
|
||||
# Mit fehlenden Links (für Link-Auflösungs-Tests)
|
||||
python3 -m scripts.make_test_vault --out ./test_vault --with-missing
|
||||
|
||||
# Überschreiben bestehenden Vault
|
||||
python3 -m scripts.make_test_vault --out ./test_vault --force
|
||||
```
|
||||
|
||||
### 2.2 Test-Collection Prefix
|
||||
|
||||
**Wichtig:** Nutze separate Prefixes für Tests, um Produktionsdaten nicht zu beeinflussen.
|
||||
|
||||
```bash
|
||||
export COLLECTION_PREFIX="mindnet_test"
|
||||
python3 -m scripts.import_markdown --vault ./test_vault --prefix mindnet_test --apply
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Test-Frameworks & Tools
|
||||
|
||||
### 3.1 Pytest (Unit Tests)
|
||||
|
||||
**Installation:**
|
||||
```bash
|
||||
pip install pytest pytest-asyncio pytest-cov
|
||||
```
|
||||
|
||||
**Konfiguration:** `pytest.ini` oder `pyproject.toml`
|
||||
|
||||
**Async Tests:**
|
||||
```python
|
||||
import pytest
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_function():
|
||||
result = await async_function()
|
||||
assert result == expected
|
||||
```
|
||||
|
||||
### 3.2 Unittest (E2E Tests)
|
||||
|
||||
**Framework:** Python `unittest`
|
||||
|
||||
**Beispiel:**
|
||||
```python
|
||||
import unittest
|
||||
from app.routers.ingest import save_note, SaveRequest
|
||||
|
||||
class TestIngest(unittest.IsolatedAsyncioTestCase):
|
||||
async def test_save_note(self):
|
||||
req = SaveRequest(markdown_content="...", filename="test.md")
|
||||
response = await save_note(req)
|
||||
self.assertEqual(response.status, "queued")
|
||||
```
|
||||
|
||||
### 3.3 Shell-Skripte (E2E Roundtrip)
|
||||
|
||||
**Tool:** `tests/run_e2e_roundtrip.sh`
|
||||
|
||||
**Zweck:** Vollständiger Test-Zyklus (Import → Export → Vergleich)
|
||||
|
||||
**Schritte:**
|
||||
1. Qdrant truncate
|
||||
2. Import (Create-Fall)
|
||||
3. Import (Idempotenz-Test)
|
||||
4. Hash-Reporter
|
||||
5. Export
|
||||
6. Vergleich Vault vs. Export
|
||||
7. Sync-Deletes Test
|
||||
|
||||
---
|
||||
|
||||
## 4. Test-Szenarien
|
||||
|
||||
### 4.1 Chunking-Tests
|
||||
|
||||
**Was wird getestet:**
|
||||
- Sliding Window Strategie
|
||||
- Heading-basierte Strategie
|
||||
- Edge-Vererbung
|
||||
- Chunk-Größen-Limits
|
||||
|
||||
**Tests:**
|
||||
- `tests/test_smart_chunking_integration.py`
|
||||
- `scripts/preview_chunks.py` (Manuelle Inspektion)
|
||||
|
||||
### 4.2 Retrieval-Tests
|
||||
|
||||
**Was wird getestet:**
|
||||
- Semantic Search
|
||||
- Hybrid Search (Semantik + Graph)
|
||||
- Scoring-Formel
|
||||
- Explanation Layer
|
||||
|
||||
**Tests:**
|
||||
- `tests/test_retriever_basic.py`
|
||||
- `tests/test_retriever_edges.py`
|
||||
- `tests/test_retriever_weight.py`
|
||||
- `tests/test_explanation_smoke.py`
|
||||
|
||||
### 4.3 Edge-Tests
|
||||
|
||||
**Was wird getestet:**
|
||||
- Edge-Erstellung (explicit, smart, rule)
|
||||
- Edge-Validierung (Registry)
|
||||
- Edge-Vererbung
|
||||
- Unresolved References
|
||||
|
||||
**Tests:**
|
||||
- `tests/test_edges_all.py`
|
||||
- `tests/test_edges_smoke.py`
|
||||
- `tests/test_edges_defaults_smoke.py`
|
||||
- `scripts/edges_full_check.py`
|
||||
|
||||
### 4.4 Chat & Intent-Tests
|
||||
|
||||
**Was wird getestet:**
|
||||
- Intent-Erkennung (FACT, DECISION, EMPATHY, INTERVIEW)
|
||||
- Decision Engine
|
||||
- Interview-Modus
|
||||
- Feedback-Loop
|
||||
|
||||
**Tests:**
|
||||
- `tests/test_wp06_decision.py`
|
||||
- `tests/test_interview_intent.py`
|
||||
- `tests/test_chat_wp05.py`
|
||||
- `tests/test_feedback_smoke.py`
|
||||
|
||||
### 4.5 Ingestion-Tests
|
||||
|
||||
**Was wird getestet:**
|
||||
- Two-Pass Workflow
|
||||
- Change Detection (Hash-basiert)
|
||||
- Background Tasks
|
||||
- Smart Edge Allocation
|
||||
|
||||
**Tests:**
|
||||
- `tests/test_dialog_full_flow.py`
|
||||
- `tests/test_WP22_intelligence.py`
|
||||
- `scripts/import_markdown.py` (mit `--dry-run`)
|
||||
|
||||
---
|
||||
|
||||
## 5. Continuous Integration
|
||||
|
||||
### 5.1 Pre-Commit Checks
|
||||
|
||||
**Empfohlene Checks:**
|
||||
```bash
|
||||
# Linting
|
||||
flake8 app/ scripts/
|
||||
|
||||
# Type Checking (optional)
|
||||
mypy app/
|
||||
|
||||
# Unit Tests
|
||||
pytest tests/ -k "test_" --ignore=tests/test_*_smoke.py
|
||||
```
|
||||
|
||||
### 5.2 CI/CD Pipeline
|
||||
|
||||
**Gitea Actions:** `.gitea/workflows/`
|
||||
|
||||
**Typische Pipeline:**
|
||||
1. Checkout
|
||||
2. Python Setup
|
||||
3. Dependencies installieren
|
||||
4. Unit Tests
|
||||
5. Integration Tests (optional)
|
||||
6. Deployment (bei main branch)
|
||||
|
||||
---
|
||||
|
||||
## 6. Test-Best Practices
|
||||
|
||||
### 6.1 Isolation
|
||||
|
||||
- **Unit Tests:** Keine externen Abhängigkeiten (Mock alles)
|
||||
- **Integration Tests:** Echte Qdrant, aber isolierte Collections
|
||||
- **E2E Tests:** Separate Test-Umgebung (Port 8002)
|
||||
|
||||
### 6.2 Test-Daten
|
||||
|
||||
- Nutze `make_test_vault.py` für konsistente Test-Daten
|
||||
- Separate Collection-Prefixes für Tests
|
||||
- Cleanup nach Tests (oder isolierte Collections)
|
||||
|
||||
### 6.3 Performance
|
||||
|
||||
- Unit Tests sollten < 1 Sekunde dauern
|
||||
- Integration Tests können länger dauern (10-30 Sekunden)
|
||||
- E2E Tests sind langsam (1-5 Minuten)
|
||||
|
||||
### 6.4 Wartbarkeit
|
||||
|
||||
- Klare Test-Namen (`test_<functionality>_<scenario>`)
|
||||
- Dokumentation in Test-Docstrings
|
||||
- Keine Hardcoded-Pfade (nutze `os.path`)
|
||||
|
||||
---
|
||||
|
||||
## 7. Debugging & Diagnose
|
||||
|
||||
### 7.1 Test-Debugging
|
||||
|
||||
**Pytest Debug-Modus:**
|
||||
```bash
|
||||
pytest tests/test_retriever_basic.py -v --pdb
|
||||
```
|
||||
|
||||
**Einzelnen Test inspizieren:**
|
||||
```bash
|
||||
python3 -m scripts.payload_dryrun --vault ./test_vault --note-id "test-note"
|
||||
```
|
||||
|
||||
### 7.2 Qdrant-State prüfen
|
||||
|
||||
```bash
|
||||
# Collection-Status
|
||||
python3 -m scripts.debug_qdrant_state --prefix mindnet_test
|
||||
|
||||
# Payload-Indexe prüfen
|
||||
python3 -m scripts.diag_payload_indexes --prefix mindnet_test
|
||||
```
|
||||
|
||||
### 7.3 Chunk-Inspektion
|
||||
|
||||
```bash
|
||||
# Chunks für eine Note anzeigen
|
||||
python3 -m scripts.dump_note_chunks --note-id "test-note" --prefix mindnet_test
|
||||
|
||||
# Chunk-Text-Verifikation
|
||||
python3 -m scripts.verify_chunk_texts --prefix mindnet_test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Test-Checkliste für Pull Requests
|
||||
|
||||
Vor jedem PR sollten folgende Tests durchlaufen:
|
||||
|
||||
- [ ] Unit Tests: `pytest tests/ -k "test_" --ignore=tests/test_*_smoke.py`
|
||||
- [ ] Payload-Validierung: `python3 -m scripts.payload_dryrun --vault ./test_vault`
|
||||
- [ ] Graph-Integrität: `python3 -m scripts.edges_full_check`
|
||||
- [ ] Smoke Test (wenn API läuft): `python tests/test_wp06_decision.py -p 8002`
|
||||
|
||||
---
|
||||
|
||||
## 9. Weitere Informationen
|
||||
|
||||
- **Developer Guide:** Siehe [Developer Guide](05_developer_guide.md)
|
||||
- **Scripts-Übersicht:** Siehe [Developer Guide - Scripts](05_developer_guide.md#44-scripts--tooling-die-admin-toolbox)
|
||||
- **Troubleshooting:** Siehe [Admin Operations - Troubleshooting](../04_Operations/04_admin_operations.md#33-troubleshooting-guide)
|
||||
|
||||
---
|
||||
|
||||
**Letzte Aktualisierung:** 2025-01-XX
|
||||
**Version:** 2.9.1
|
||||
|
||||
Reference in new issue
Block a user