Dokumentation WP14&WP15b
This commit is contained in:
1 parent
7fa9ce81bd
commit
fa909e2e7d
8 files changed
+267
-246
No files matched your search
@@ -1,10 +1,10 @@
|
||||
---
|
||||
doc_type: developer_guide
|
||||
audience: developer
|
||||
scope: workflow, testing, architecture, modules
|
||||
scope: workflow, testing, architecture, modules, modularization
|
||||
status: active
|
||||
version: 2.6.1
|
||||
context: "Umfassender Guide für Entwickler: Architektur, Modul-Interna (Deep Dive), Setup, Git-Workflow und Erweiterungs-Anleitungen."
|
||||
version: 2.9.1
|
||||
context: "Umfassender Guide für Entwickler: Modularisierte Architektur (WP-14), Two-Pass Ingestion (WP-15b), Modul-Interna, Setup und Git-Workflow."
|
||||
---
|
||||
|
||||
# Mindnet Developer Guide & Workflow
|
||||
@@ -23,8 +23,6 @@ Dieser Guide ist die zentrale technische Referenz für Mindnet v2.6. Er vereint
|
||||
- [Kern-Philosophie](#kern-philosophie)
|
||||
- [2. Architektur](#2-architektur)
|
||||
- [2.1 High-Level Übersicht](#21-high-level-übersicht)
|
||||
- [2.2 Datenfluss-Muster](#22-datenfluss-muster)
|
||||
- [A. Ingestion (Write)](#a-ingestion-write)
|
||||
- [B. Retrieval (Read)](#b-retrieval-read)
|
||||
- [C. Visualisierung (Graph)](#c-visualisierung-graph)
|
||||
- [3. Physische Architektur](#3-physische-architektur)
|
||||
@@ -84,23 +82,28 @@ graph TD
|
||||
API["main.py"]
|
||||
RouterChat["Chat / RAG"]
|
||||
RouterIngest["Ingest / Write"]
|
||||
CoreRet["Retriever Engine"]
|
||||
CoreIngest["Ingestion Pipeline"]
|
||||
|
||||
subgraph "Core Packages (WP-14)"
|
||||
PkgRet["retrieval/ (Search)"]
|
||||
PkgIng["ingestion/ (Import)"]
|
||||
PkgGra["graph/ (Logic)"]
|
||||
PkgDb["database/ (Infrastr.)"]
|
||||
Registry["registry.py (Neutral)"]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph "Infrastructure & Services"
|
||||
LLM["Ollama (Phi3/Nomic)"]
|
||||
LLM["Ollama / Cloud (Hybrid)"]
|
||||
DB[("Qdrant Vector DB")]
|
||||
FS["File System (.md)"]
|
||||
end
|
||||
|
||||
User <--> UI
|
||||
UI -- "REST (Chat, Save, Feedback)" --> API
|
||||
UI -. "Direct Read (Graph Viz Performance)" .-> DB
|
||||
API -- "Embeddings & Completion" --> LLM
|
||||
API -- "Read/Write" --> DB
|
||||
API -- "Read/Write (Source of Truth)" --> FS
|
||||
```
|
||||
UI -- "REST Call" --> API
|
||||
PkgRet -- "Direct Query" --> PkgDb
|
||||
PkgIng -- "Process & Write" --> PkgDb
|
||||
PkgDb -- "API" --> DB
|
||||
API -- "Inference" --> LLM```
|
||||
|
||||
### 2.2 Datenfluss-Muster
|
||||
|
||||
@@ -108,14 +111,12 @@ graph TD
|
||||
Vom Markdown zur Vektor-Datenbank.
|
||||
```mermaid
|
||||
graph LR
|
||||
MD["Markdown File"] --> Parser("Parser")
|
||||
Parser --> Chunker("Chunker")
|
||||
Chunker -- "Text Chunks" --> SemAn{"SemanticAnalyzer<br/>(LLM)"}
|
||||
SemAn -- "Smart Edges" --> Embedder("Embedder")
|
||||
Embedder --> DB[("Qdrant<br/>Points")]
|
||||
|
||||
style DB fill:#f9f,stroke:#333,stroke-width:2px
|
||||
style SemAn fill:#ff9,stroke:#333,stroke-width:2px
|
||||
MD["Markdown File"] --> Pass1["Pass 1: Pre-Scan"]
|
||||
Pass1 --> Cache[("LocalBatchCache<br/>(Titles/Summaries)")]
|
||||
MD --> Pass2["Pass 2: Processing"]
|
||||
Cache -- "Context" --> SmartEdges{"Smart Edge<br/>Validation"}
|
||||
SmartEdges --> Embedder("Embedder")
|
||||
Embedder --> DB[("Qdrant Points")]
|
||||
```
|
||||
|
||||
#### B. Retrieval (Read)
|
||||
@@ -123,17 +124,10 @@ Die hybride Suche für Chat & RAG.
|
||||
```mermaid
|
||||
graph LR
|
||||
Query(["Query"]) --> Embed("Embedding")
|
||||
Embed --> Hybrid{"Hybrid Search"}
|
||||
|
||||
subgraph Search Components
|
||||
Vec["Vector Score"]
|
||||
Graph["Graph/Edge Bonus"]
|
||||
end
|
||||
|
||||
Vec --> Hybrid
|
||||
Graph --> Hybrid
|
||||
|
||||
Hybrid --> Rank("Re-Ranking")
|
||||
Embed --> Seed["Seed Search (Vector)"]
|
||||
Seed --> Expand{"Graph Expansion"}
|
||||
Expand --> Scoring["Scoring Engine (WP-22)"]
|
||||
Scoring --> Rank("Final Ranking")
|
||||
Rank --> Ctx["LLM Context"]
|
||||
```
|
||||
|
||||
@@ -170,6 +164,12 @@ Das System ist modular aufgebaut. Hier ist die detaillierte Analyse aller Kompon
|
||||
mindnet/
|
||||
├── app/
|
||||
│ ├── core/ # Business Logic & Algorithms
|
||||
│ │ ├── database/ # WP-14: Qdrant Client & Point Mapping
|
||||
│ │ ├── ingestion/ # WP-14: Pipeline, Multi-Hash, Validation
|
||||
│ │ ├── retrieval/ # WP-14: Search Orchestrator & Scoring
|
||||
│ │ ├── graph/ # WP-14: Subgraph-Logik & Weights
|
||||
│ │ ├── registry.py # SSOT: Circular Import Fix & Text Cleanup
|
||||
│ │ └── *.py (Proxy) # Legacy Bridges für Abwärtskompatibilität
|
||||
│ ├── routers/ # API Interface (FastAPI)
|
||||
│ ├── services/ # External Integrations (LLM, DB)
|
||||
│ ├── models/ # Pydantic DTOs
|
||||
@@ -285,6 +285,8 @@ Folgende Dateien wurden im Audit v2.6 als veraltet, redundant oder "Zombie-Code"
|
||||
| `app/core/type_registry.py` | **Redundant.** Logik in `ingestion.py` integriert. | 🗑️ Löschen |
|
||||
| `app/core/env_vars.py` | **Veraltet.** Ersetzt durch `config.py`. | 🗑️ Löschen |
|
||||
| `app/services/llm_ollama.py` | **Veraltet.** Ersetzt durch `llm_service.py`. | 🗑️ Löschen |
|
||||
| `app/core/type_registry.py` | **Redundant.** Logik in `app/core/registry.py` integriert. | 🗑️ Löschen |
|
||||
| `app/core/ranking.py` | **Redundant.** Logik in `retrieval/retriever_scoring.py` integriert. | 🗑️ Löschen |
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user