Compare commits
90
Commits
WF_Endnote
..
V1.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b5f745fc3e | ||
|
|
549c31431e | ||
|
|
3fa01dd686 | ||
|
|
c9357d4c0e | ||
|
|
f3a61091c7 | ||
|
|
10d24bbef7 | ||
|
|
ff8104a533 | ||
|
|
3b7f89a214 | ||
|
|
ba773e677b | ||
|
|
2443b5ac3a | ||
|
|
4c9e0e3c98 | ||
|
|
0ce98e8973 | ||
|
|
d803f39de3 | ||
|
|
300d96a9d8 | ||
|
|
28b6fb28d5 | ||
|
|
8b731313df | ||
|
|
3541c416f9 | ||
|
|
8d89b23db1 | ||
|
|
c0525cf2d2 | ||
|
|
88f0b5a0a4 | ||
|
|
aeb0ee6ad9 | ||
|
|
a4c8b4bd9a | ||
|
|
8f6d60681e | ||
|
|
65500c899b | ||
|
|
a1723db387 | ||
|
|
b453ce63c6 | ||
|
|
12893cba78 | ||
|
|
ebca44829e | ||
|
|
0629f88b37 | ||
|
|
6945b748cb | ||
|
|
08a2485f43 | ||
|
|
894ee1dd02 | ||
|
|
a9bd3faabb | ||
|
|
5b96bd4f75 | ||
|
|
c5b0540b11 | ||
|
|
1855f6e57a | ||
|
|
5a0c71dd90 | ||
|
|
e60976e1cc | ||
|
|
b7cd710c32 | ||
|
|
ad7aa2d255 | ||
|
|
a51ee1d304 | ||
|
|
e35d167055 | ||
|
|
c0fcdea1fe | ||
|
|
8b67f7ab55 | ||
|
|
8ee9fb84ba | ||
|
|
fe7a69fb07 | ||
|
|
bb6eefc837 | ||
|
|
0d0ab62674 | ||
|
|
d6d7e738a5 | ||
|
|
41cc0ed2a8 | ||
|
|
26ab11eb7b | ||
|
|
b4cc3cb934 | ||
|
|
c10da55ec6 | ||
|
|
338163ac0b | ||
|
|
5e5f3b4e5a | ||
|
|
7e9da46fe5 | ||
|
|
66979f3f51 | ||
|
|
851018b3b9 | ||
|
|
36417bfdf3 | ||
|
|
4a771f6a83 | ||
|
|
09439ad1a5 | ||
|
|
75736dadec | ||
|
|
8d56c352fc | ||
|
|
ebea9946a7 | ||
|
|
9fb04af7df | ||
|
|
3f5b9b8b58 | ||
|
|
5bbea036c2 | ||
|
|
c762495b6f | ||
|
|
979e734bd9 | ||
|
|
79cb3e0100 | ||
|
|
9d22e7e8af | ||
|
|
f46c367c27 | ||
|
|
ac4c6760d7 | ||
|
|
5a6a140dfd | ||
|
|
931012c16b | ||
|
|
03f4b871a9 | ||
|
|
601fc80178 | ||
|
|
3ff2a1bf45 | ||
|
|
51aa57f304 | ||
|
|
ca4411f30f | ||
|
|
b551365fb5 | ||
|
|
272c123952 | ||
|
|
9387670a7b | ||
|
|
d4a8401a6a | ||
|
|
aaf88a6f12 | ||
|
|
b789c1bd44 | ||
|
|
5062aa8068 | ||
|
|
8a042589e7 | ||
|
|
3898b5ad45 | ||
|
|
9d15336144 |
No files matched your search
@@ -12,6 +12,7 @@ Dieser Ordner ist der **primäre Orientierungspunkt** für Claude Code / Cursor-
|
|||||||
| 2 | **`rules/DOCUMENTATION.md`** – Ablage- und Dokumentationsregeln |
|
| 2 | **`rules/DOCUMENTATION.md`** – Ablage- und Dokumentationsregeln |
|
||||||
| 3 | `rules/ARCHITECTURE.md`, `rules/CODING_RULES.md`, `rules/LESSONS_LEARNED.md` |
|
| 3 | `rules/ARCHITECTURE.md`, `rules/CODING_RULES.md`, `rules/LESSONS_LEARNED.md` |
|
||||||
| 4 | Issue-Landkarte: **`.claude/docs/GITEA_ISSUES_INDEX.md`** |
|
| 4 | Issue-Landkarte: **`.claude/docs/GITEA_ISSUES_INDEX.md`** |
|
||||||
|
| 5 | **Universal CSV Import** (Modul/Executor/Vorlagen): **`docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`** (unter `.claude/`) |
|
||||||
|
|
||||||
Themen mit UI/Nav/PWA: siehe `../docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md` (im **Projekt**-`docs/`, nicht hier).
|
Themen mit UI/Nav/PWA: siehe `../docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md` (im **Projekt**-`docs/`, nicht hier).
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Gitea Issues – Landkarte (Auswertung)
|
# Gitea Issues – Landkarte (Auswertung)
|
||||||
|
|
||||||
**Quelle:** Gitea `Lars/mitai-jinkendo`, Stand **2026-04-08** (Abfrage `state=all`).
|
**Quelle:** Gitea `Lars/mitai-jinkendo`, Stand **2026-04-09** (Abfrage `state=all`, ergänzt: #71).
|
||||||
**URL:** http://192.168.2.144:3000/Lars/mitai-jinkendo/issues
|
**URL:** http://192.168.2.144:3000/Lars/mitai-jinkendo/issues
|
||||||
|
|
||||||
Dieses Dokument ist ein **Orientierungs-Index** für Agenten und Entwickler. Verbindliches Tracking bleibt **in Gitea**; hier: Kategorien, Dubletten-Hinweise, grobe Prioritätseinschätzung.
|
Dieses Dokument ist ein **Orientierungs-Index** für Agenten und Entwickler. Verbindliches Tracking bleibt **in Gitea**; hier: Kategorien, Dubletten-Hinweise, grobe Prioritätseinschätzung.
|
||||||
@@ -88,7 +88,6 @@ Dieses Dokument ist ein **Orientierungs-Index** für Agenten und Entwickler. Ver
|
|||||||
| # | Titel |
|
| # | Titel |
|
||||||
|---|--------|
|
|---|--------|
|
||||||
| 15 | [FEAT-002] Quality-Filter für KI-Auswertungen & Charts integrieren |
|
| 15 | [FEAT-002] Quality-Filter für KI-Auswertungen & Charts integrieren |
|
||||||
| 21 | [FEATURE] Universeller CSV-Parser mit lernbarem Feldmapping |
|
|
||||||
| 36 | BUG-009: Trainingstyp-Erstellung führt zu Internal Server Error |
|
| 36 | BUG-009: Trainingstyp-Erstellung führt zu Internal Server Error |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -54,6 +54,7 @@ _Dieser Ordner `.claude/docs/` ist per `.gitignore`-Ausnahme **versioniert** (Sp
|
|||||||
| Platzhalter / Registry | `technical/PLACEHOLDER_REGISTRY_FRAMEWORK.md`, `technical/PLACEHOLDER_DEVELOPMENT_GUIDE.md` | `backend/placeholder_registrations/`, `backend/placeholder_resolver.py` |
|
| Platzhalter / Registry | `technical/PLACEHOLDER_REGISTRY_FRAMEWORK.md`, `technical/PLACEHOLDER_DEVELOPMENT_GUIDE.md` | `backend/placeholder_registrations/`, `backend/placeholder_resolver.py` |
|
||||||
| Dashboard-Lab-Widgets | `technical/DASHBOARD_WIDGETS_AGENT_GUIDE.md` | Widget-Katalog + Registrierung (siehe Guide) |
|
| Dashboard-Lab-Widgets | `technical/DASHBOARD_WIDGETS_AGENT_GUIDE.md` | Widget-Katalog + Registrierung (siehe Guide) |
|
||||||
| Training Profiler / Resolver | `technical/TRAINING_PROFILE_RESOLVER_LAYER1.md`, `functional/TRAINING_TYPE_PROFILES.md` | Resolver-Module wie im Guide genannt |
|
| Training Profiler / Resolver | `technical/TRAINING_PROFILE_RESOLVER_LAYER1.md`, `functional/TRAINING_TYPE_PROFILES.md` | Resolver-Module wie im Guide genannt |
|
||||||
|
| Universal CSV Import | `technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` | `backend/csv_parser/`, `routers/csv_import.py`, `routers/admin_csv_templates.py` |
|
||||||
| Mitgliedschaft / Features | `technical/MEMBERSHIP_SYSTEM.md`, `architecture/FEATURE_ENFORCEMENT.md` | `backend/auth.py`, Feature-Logging, Router mit Enforcement |
|
| Mitgliedschaft / Features | `technical/MEMBERSHIP_SYSTEM.md`, `architecture/FEATURE_ENFORCEMENT.md` | `backend/auth.py`, Feature-Logging, Router mit Enforcement |
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -111,6 +112,7 @@ _Dieser Ordner `.claude/docs/` ist per `.gitignore`-Ausnahme **versioniert** (Sp
|
|||||||
| `PROFILE_REFERENCE_VALUES.md` | Profil-Referenzwerte |
|
| `PROFILE_REFERENCE_VALUES.md` | Profil-Referenzwerte |
|
||||||
| `TRAINING_PROFILE_RESOLVER_LAYER1.md` | Training-Resolver Schicht 1 |
|
| `TRAINING_PROFILE_RESOLVER_LAYER1.md` | Training-Resolver Schicht 1 |
|
||||||
| `TRAINING_TYPE_PROFILES_TECHNICAL.md` | Trainingsprofile technisch |
|
| `TRAINING_TYPE_PROFILES_TECHNICAL.md` | Trainingsprofile technisch |
|
||||||
|
| `UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` | Universal CSV: Registry, Executor, Vorlagen, Agent-Checkliste |
|
||||||
| `V9D_PHASE2_VITALS_SLEEP.md` | v9d Vitalwerte/Schlaf (Release-Bezug) |
|
| `V9D_PHASE2_VITALS_SLEEP.md` | v9d Vitalwerte/Schlaf (Release-Bezug) |
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -174,4 +176,4 @@ Siehe [`audit/README.md`](./audit/README.md).
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Letzte Aktualisierung:** 8. April 2026 (Struktur-Index, Duplikatbereinigung, Abgleich-Hinweise)
|
**Letzte Aktualisierung:** 9. April 2026 (Universal CSV Agent-Guide, Abgleich-Tabelle)
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Universal CSV Import – Agent-Leitfaden
|
||||||
|
|
||||||
|
**Stand:** 2026-04-09 · **Kontext:** Issue #21 (Universeller CSV-Parser), Prod-Migrationen u. a. 051–053.
|
||||||
|
|
||||||
|
Dieses Dokument ist **normativ für Agenten**, die ein neues Import-Zielmodul anlegen oder bestehende Import-Pfade (Executor, Vorlagen, DB) ändern.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Architektur (Kurz)
|
||||||
|
|
||||||
|
| Komponente | Pfad / Rolle |
|
||||||
|
|------------|----------------|
|
||||||
|
| Modul-Definitionen | `backend/csv_parser/module_registry.py` (`MODULE_DEFINITIONS`) |
|
||||||
|
| Typ-/Einheiten-Konvertierung | `backend/csv_parser/type_converter.py`, `field_units.py` |
|
||||||
|
| Zeilen-Aggregation (z. B. Ernährung pro Tag) | `backend/csv_parser/import_row_processing.py` |
|
||||||
|
| Import-Ausführung | `backend/csv_parser/executor.py` |
|
||||||
|
| Fehlertexte / Transaktions-Hinweise | `backend/csv_parser/import_errors.py` (`enrich_row_error`) |
|
||||||
|
| Admin-Systemvorlagen | `backend/routers/admin_csv_templates.py` |
|
||||||
|
| Nutzer-Import (Profil-Mappings) | `backend/routers/csv_import.py` |
|
||||||
|
| Vorlagen-Validierung (strukturell + Sample) | `backend/csv_parser/template_validator.py` (`validate_csv_template`) |
|
||||||
|
|
||||||
|
**Single Source of Truth** für erlaubte Zielfelder, Typen und Duplikat-Keys ist **`module_registry.py`**. Keine parallele Feldliste in Routern duplizieren.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Checkliste: Neues Zielmodul
|
||||||
|
|
||||||
|
1. **`MODULE_DEFINITIONS`** um Eintrag erweitern: `table`, `fields` (Typen `date` / `datetime` / `float` / `int` / `string`), `duplicate_key`, `duplicate_strategy`, ggf. `derive_date_from_datetime_field`, `import_mode` (Spezialpfade wie Schlaf).
|
||||||
|
2. **DB:** Migration nur nach Projektregel (`backend/migrations/NNN_*.sql`). Spaltenbreiten/Typen so wählen, dass importierte Werte (z. B. kJ→kcal, große Energiebeträge) **keinen NUMERIC-Overflow** verursachen.
|
||||||
|
3. **`source` / CHECK-Constraints:** Wenn die Zieltabelle `source` hat, muss der Wert **`csv`** (oder der vereinbarte Import-Tag) in der DB erlaubt sein (Migration anpassen, nicht nur App-Code).
|
||||||
|
4. **Executor:** Einfügen/Aktualisieren in `executor.py` nur über bestehende Muster (ein Cursor, **kein** verschachteltes `get_db()` im gleichen Request). Bei mehreren Zeilen pro Transaktion: bei **Zeilenfehlern** SAVEPOINT pro Zeile nutzen (siehe Activity-Pattern), damit die Transaktion nicht dauerhaft abgebrochen ist.
|
||||||
|
5. **Trainingstyp / FK-Auflösung:** DB-Zugriffe für abhängige Entitäten (z. B. `get_training_type_for_activity_with_cursor`) **mit dem gleichen Cursor** wie der Import – keine zweite Connection aus dem Importpfad.
|
||||||
|
6. **Vorlagen:** System-Templates in Migration/Seed pflegen (`csv_field_mappings`, `is_system=true`). `type_conversions` und `source_unit` dort setzen, wo Einheiten aus Exporten abweichen (z. B. Apple kJ).
|
||||||
|
7. **Validierung:** Neue/angepasste Admin-Vorlagen müssen **`validate_csv_template`** passieren (Create/Update liefert bei Fehlern **422** mit `validation`). Tests für Randfälle ergänzen (`tests/test_template_validator.py` o. ä.).
|
||||||
|
8. **API / Frontend:** Neue Admin-Endpunkte in `main.py` registrieren; Frontend **nur** über `api.js`. Bei strukturierten FastAPI-Fehlern (`detail` als Objekt/Liste) bestehende Hilfen (`formatFastApiDetail`) nutzen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Checkliste: Bestehendes Modul ändern
|
||||||
|
|
||||||
|
- Schema-Änderung: Migration + ggf. **`module_registry`**-Felder anpassen.
|
||||||
|
- Neue Spalte im Import: Executor-Mapping, optional `type_conversions` / Validator.
|
||||||
|
- Änderung an Duplikatlogik: `duplicate_key` / `ON CONFLICT`-Pfad im Executor prüfen.
|
||||||
|
- Datums-/Zeit-Parsing: **`type_converter`** – ISO-Daten `YYYY-MM-DD` konsistent (**`dayfirst=False`**), Zeiten `HH:MM` ohne Sekunden unterstützen wo nötig.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Bekannte Einschränkungen (Follow-up in Gitea)
|
||||||
|
|
||||||
|
- Admin **„Format prüfen“** kann `import_row_processing` derzeit weglassen; volle Parität mit dem gespeicherten Template erst beim Speichern / echten Import.
|
||||||
|
- Nutzer-Mappings (Copy aus Systemvorlage) laufen nicht automatisch durch **`validate_csv_template`** – Tracking: **Gitea #71** (http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/71).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Verwandte Regeln
|
||||||
|
|
||||||
|
- `.claude/rules/ARCHITECTURE.md` – Router, DB, `source`-Tracking
|
||||||
|
- `.claude/rules/CODING_RULES.md` – Kurzverweis Universal CSV
|
||||||
|
- `.claude/rules/DOCUMENTATION.md` – Ablage technischer Specs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Version:** 1.0
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
Folgende Ergebnisse des Tests:
|
||||||
|
- Valididierung gibt immer noch keine Aufschlüsse was für Fehler und Warning es sind, Es zeigt immer nur noch die Anzahl der entsprechenden Fehler/Warnungen
|
||||||
|
- Speichern als kurzes PopUp -- gut
|
||||||
|
- In der Node selbst wird nun eine Fehlermeldung ausgegeben. Das ist gut. In größen Workflows aber schwierig den Fehler zu lokalisieren.
|
||||||
|
- In der automatischen Zusammenfassung in der Endnode kommt als Überschrift, z.B. Node 10, anstatt den Node-Name auszugeben.
|
||||||
|
- Alle Änderungen an Nodes scheinen automatisch in den Gesamtflow übernommen zu werden. Diese werden dann nach dem Speichern aktiv. Da muss man sehr vorsichtig sein, bei kurzen Änderungen und dem Ausprobieren.
|
||||||
|
- Der Testlauf "Execute" sollte auf dem aktuellen Workflowstand ausgeführt werden, auch wenn dieser vom gespeicherten Abweicht. Ich würde natürlich vor dem Speichern den Workflow testen können. Prüfe und bewerte diesen Punkt, setze ihn aber noch nicht um.
|
||||||
|
- Die Workflows werden aktuell nicht in Analyse und den verfügbaren KI-Asuwertungen angezeigt. ggf. weil wir sie aktuell noch keinem Bereich zuordnen können. Diesen könnten wir ggf. über die Start-Node im Workflow konfigurieren.
|
||||||
|
- Das löschen von Knoten und Kanten funktioniert aktuell nur über Backspace nicht über entfernen
|
||||||
|
- Wir sollten auch dafür sorgen, dass jeweils nur eine Start-Node, End-Node in einem Workflow existiert, Prüfe ob mehrere End-Nodes sinnvoll sind, da wir ja auch Logik-Pfade abbilden und ggf. auch eine route beschreiten, die ein anderes Ende hat. (Prüfe, ob das heute schon möglich wäre!)
|
||||||
|
- Als zukünftige Ausbaustufe sollten wir überlegen, ob wir auch Trigger implementieren, z.B. um Kurzstatements zu generieren, wenn neue Daten hereinkommen und wir diese Bewertungen aktualisieren wollen
|
||||||
@@ -0,0 +1,458 @@
|
|||||||
|
-- Migration XXX: CSV Parser - System Templates Seed Data
|
||||||
|
-- Legt Standard-Import-Konfigurationen für bekannte CSV-Formate an
|
||||||
|
-- Diese Templates sind für alle User verfügbar (is_system = true, profile_id = NULL)
|
||||||
|
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
-- NUTRITION (Ernährung)
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
|
||||||
|
-- 1. FDDB Export (Deutsch)
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'nutrition',
|
||||||
|
'FDDB Export (Standard)',
|
||||||
|
'Standard-Format für FDDB.de CSV-Exporte (Deutsch). Delimiter Semikolon, kJ → kcal Konvertierung.',
|
||||||
|
ARRAY['datum_tag_monat_jahr_stunde_minute', 'fett_g', 'kh_g', 'kj', 'protein_g']::TEXT[],
|
||||||
|
';',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"datum_tag_monat_jahr_stunde_minute": "date",
|
||||||
|
"kj": "kcal",
|
||||||
|
"fett_g": "fat_g",
|
||||||
|
"kh_g": "carbs_g",
|
||||||
|
"protein_g": "protein_g"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "dd.mm.yyyy HH:MM",
|
||||||
|
"extract": "date_only"
|
||||||
|
},
|
||||||
|
"kcal": {
|
||||||
|
"type": "float",
|
||||||
|
"source_unit": "kj",
|
||||||
|
"decimal_separator": ","
|
||||||
|
},
|
||||||
|
"fat_g": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": ","
|
||||||
|
},
|
||||||
|
"carbs_g": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": ","
|
||||||
|
},
|
||||||
|
"protein_g": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": ","
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 2. MyFitnessPal Export (English)
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'nutrition',
|
||||||
|
'MyFitnessPal Export',
|
||||||
|
'Standard CSV export from MyFitnessPal (English)',
|
||||||
|
ARRAY['Carbohydrates (g)', 'Calories', 'Date', 'Fat (g)', 'Protein (g)']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Date": "date",
|
||||||
|
"Calories": "kcal",
|
||||||
|
"Fat (g)": "fat_g",
|
||||||
|
"Carbohydrates (g)": "carbs_g",
|
||||||
|
"Protein (g)": "protein_g"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "yyyy-mm-dd"
|
||||||
|
},
|
||||||
|
"kcal": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"fat_g": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"carbs_g": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"protein_g": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 3. Cronometer Export
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'nutrition',
|
||||||
|
'Cronometer Export',
|
||||||
|
'Cronometer daily nutrition export (English)',
|
||||||
|
ARRAY['Day', 'Energy (kcal)', 'Fat (g)', 'Net Carbs (g)', 'Protein (g)']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Day": "date",
|
||||||
|
"Energy (kcal)": "kcal",
|
||||||
|
"Fat (g)": "fat_g",
|
||||||
|
"Net Carbs (g)": "carbs_g",
|
||||||
|
"Protein (g)": "protein_g"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "yyyy-mm-dd"
|
||||||
|
},
|
||||||
|
"kcal": {"type": "float", "decimal_separator": "."},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": "."},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": "."},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": "."}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
-- ACTIVITY (Aktivität)
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
|
||||||
|
-- 1. Apple Health Workout Export (English)
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'activity',
|
||||||
|
'Apple Health Workout Export (English)',
|
||||||
|
'Apple Health CSV-Export für Workouts (English). Automatisches Training-Type-Mapping.',
|
||||||
|
ARRAY['Active Energy (kcal)', 'Distance (km)', 'Duration', 'End', 'Heart Rate Average (bpm)', 'Start', 'Workout Type']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Workout Type": "activity_type",
|
||||||
|
"Start": "start_time",
|
||||||
|
"End": "end_time",
|
||||||
|
"Duration": "duration_min",
|
||||||
|
"Distance (km)": "distance_km",
|
||||||
|
"Active Energy (kcal)": "kcal_active",
|
||||||
|
"Heart Rate Average (bpm)": "hr_avg"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"start_time": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_and_time"
|
||||||
|
},
|
||||||
|
"end_time": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS"
|
||||||
|
},
|
||||||
|
"duration_min": {
|
||||||
|
"type": "duration",
|
||||||
|
"format": "HH:MM:SS",
|
||||||
|
"target_unit": "minutes"
|
||||||
|
},
|
||||||
|
"distance_km": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"kcal_active": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"hr_avg": {
|
||||||
|
"type": "int"
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 2. Apple Health Workout Export (Deutsch)
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'activity',
|
||||||
|
'Apple Health Workout Export (Deutsch)',
|
||||||
|
'Apple Health CSV-Export für Workouts (Deutsch). Automatisches Training-Type-Mapping.',
|
||||||
|
ARRAY['Aktive Energie (kcal)', 'Dauer', 'Durchschnittliche Herzfrequenz (bpm)', 'Ende', 'Start', 'Strecke (km)', 'Trainingsart']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Trainingsart": "activity_type",
|
||||||
|
"Start": "start_time",
|
||||||
|
"Ende": "end_time",
|
||||||
|
"Dauer": "duration_min",
|
||||||
|
"Strecke (km)": "distance_km",
|
||||||
|
"Aktive Energie (kcal)": "kcal_active",
|
||||||
|
"Durchschnittliche Herzfrequenz (bpm)": "hr_avg"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"start_time": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_and_time"
|
||||||
|
},
|
||||||
|
"end_time": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS"
|
||||||
|
},
|
||||||
|
"duration_min": {
|
||||||
|
"type": "duration",
|
||||||
|
"format": "HH:MM:SS",
|
||||||
|
"target_unit": "minutes"
|
||||||
|
},
|
||||||
|
"distance_km": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": ","
|
||||||
|
},
|
||||||
|
"kcal_active": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": ","
|
||||||
|
},
|
||||||
|
"hr_avg": {
|
||||||
|
"type": "int"
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 3. Garmin Connect Export
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'activity',
|
||||||
|
'Garmin Connect Export',
|
||||||
|
'Garmin Connect activity CSV export (English)',
|
||||||
|
ARRAY['Activity Type', 'Avg HR', 'Calories', 'Date', 'Distance', 'Duration', 'Time']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Activity Type": "activity_type",
|
||||||
|
"Date": "date",
|
||||||
|
"Time": "start_time",
|
||||||
|
"Duration": "duration_min",
|
||||||
|
"Distance": "distance_km",
|
||||||
|
"Calories": "kcal_active",
|
||||||
|
"Avg HR": "hr_avg"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "yyyy-mm-dd"
|
||||||
|
},
|
||||||
|
"start_time": {
|
||||||
|
"type": "time",
|
||||||
|
"format": "HH:MM:SS"
|
||||||
|
},
|
||||||
|
"duration_min": {
|
||||||
|
"type": "duration",
|
||||||
|
"format": "HH:MM:SS",
|
||||||
|
"target_unit": "minutes"
|
||||||
|
},
|
||||||
|
"distance_km": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"kcal_active": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
},
|
||||||
|
"hr_avg": {
|
||||||
|
"type": "int"
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
-- BLOOD PRESSURE (Blutdruck)
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
|
||||||
|
-- 1. Omron Export (Deutsch)
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'blood_pressure',
|
||||||
|
'Omron Export (Deutsch)',
|
||||||
|
'Omron Blutdruckmessgerät CSV-Export (Deutsch)',
|
||||||
|
ARRAY['Datum', 'Diastolisch (mmHg)', 'Puls (bpm)', 'Systolisch (mmHg)', 'Zeit']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Datum": "measured_date",
|
||||||
|
"Zeit": "measured_time",
|
||||||
|
"Systolisch (mmHg)": "systolic",
|
||||||
|
"Diastolisch (mmHg)": "diastolic",
|
||||||
|
"Puls (bpm)": "pulse"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"measured_date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "dd.mm.yyyy"
|
||||||
|
},
|
||||||
|
"measured_time": {
|
||||||
|
"type": "time",
|
||||||
|
"format": "HH:MM"
|
||||||
|
},
|
||||||
|
"systolic": {"type": "int"},
|
||||||
|
"diastolic": {"type": "int"},
|
||||||
|
"pulse": {"type": "int"}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 2. Omron Export (English)
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'blood_pressure',
|
||||||
|
'Omron Export (English)',
|
||||||
|
'Omron blood pressure monitor CSV export (English)',
|
||||||
|
ARRAY['Date', 'Diastolic (mmHg)', 'Pulse (bpm)', 'Systolic (mmHg)', 'Time']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Date": "measured_date",
|
||||||
|
"Time": "measured_time",
|
||||||
|
"Systolic (mmHg)": "systolic",
|
||||||
|
"Diastolic (mmHg)": "diastolic",
|
||||||
|
"Pulse (bpm)": "pulse"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"measured_date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "mm/dd/yyyy"
|
||||||
|
},
|
||||||
|
"measured_time": {
|
||||||
|
"type": "time",
|
||||||
|
"format": "HH:MM"
|
||||||
|
},
|
||||||
|
"systolic": {"type": "int"},
|
||||||
|
"diastolic": {"type": "int"},
|
||||||
|
"pulse": {"type": "int"}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
-- WEIGHT (Gewicht)
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
|
||||||
|
-- 1. Apple Health Weight Export
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'weight',
|
||||||
|
'Apple Health Weight Export',
|
||||||
|
'Apple Health body mass CSV export',
|
||||||
|
ARRAY['Body Mass (kg)', 'Start']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Start": "date",
|
||||||
|
"Body Mass (kg)": "weight"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only"
|
||||||
|
},
|
||||||
|
"weight": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- 2. Withings Export
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
) VALUES (
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'weight',
|
||||||
|
'Withings Export',
|
||||||
|
'Withings smart scale CSV export (weight, body fat, muscle mass)',
|
||||||
|
ARRAY['Body Fat (%)', 'Date', 'Muscle Mass (kg)', 'Weight (kg)']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Date": "date",
|
||||||
|
"Weight (kg)": "weight"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "yyyy-mm-dd"
|
||||||
|
},
|
||||||
|
"weight": {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": "."
|
||||||
|
}
|
||||||
|
}'::JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
-- SUMMARY
|
||||||
|
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
|
||||||
|
DO $$
|
||||||
|
DECLARE
|
||||||
|
template_count INTEGER;
|
||||||
|
BEGIN
|
||||||
|
SELECT COUNT(*) INTO template_count FROM csv_field_mappings WHERE is_system = true;
|
||||||
|
RAISE NOTICE '✓ CSV Parser: % System-Templates created', template_count;
|
||||||
|
RAISE NOTICE ' - Nutrition: 3 (FDDB, MyFitnessPal, Cronometer)';
|
||||||
|
RAISE NOTICE ' - Activity: 3 (Apple Health DE/EN, Garmin)';
|
||||||
|
RAISE NOTICE ' - Blood Pressure: 2 (Omron DE/EN)';
|
||||||
|
RAISE NOTICE ' - Weight: 2 (Apple Health, Withings)';
|
||||||
|
END $$;
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -216,8 +216,11 @@ updated_at TIMESTAMP DEFAULT NOW()
|
|||||||
Tabellen die Daten aus externen Quellen empfangen brauchen:
|
Tabellen die Daten aus externen Quellen empfangen brauchen:
|
||||||
```sql
|
```sql
|
||||||
source VARCHAR(50) DEFAULT 'manual'
|
source VARCHAR(50) DEFAULT 'manual'
|
||||||
-- Werte: 'manual' | 'apple_health' | 'garmin' | 'withings'
|
-- Werte u. a.: 'manual' | 'apple_health' | 'garmin' | 'withings' | 'csv'
|
||||||
```
|
```
|
||||||
|
Importe über den **Universal CSV**-Pfad setzen `source = 'csv'`, sofern die Tabelle ein `source`-Feld hat; CHECK-Constraints und Migrationen müssen diesen Wert erlauben.
|
||||||
|
|
||||||
|
**Agent-Pflicht bei neuen Import-Zielen oder Executor-Änderungen:** `.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`
|
||||||
|
|
||||||
Manuelle Einträge (`source = 'manual'`) haben IMMER Vorrang bei Reimport:
|
Manuelle Einträge (`source = 'manual'`) haben IMMER Vorrang bei Reimport:
|
||||||
```sql
|
```sql
|
||||||
@@ -384,26 +387,55 @@ Dev-URL: dev.mitai.jinkendo.de
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 8. Test-Regeln
|
## 8. CSV-Import vs. Data Layer (Issue #53)
|
||||||
|
|
||||||
### 8.1 Tests schreiben ist Pflicht
|
### 8.1 Leitlinie: Wo Interpretation stattfindet
|
||||||
|
|
||||||
|
| Schicht | Erlaubt | Nicht Sinn der Schicht |
|
||||||
|
|--------|---------|-------------------------|
|
||||||
|
| **Import (Ingest)** | Zuordnung CSV→Speicherfeld, **Typ-/Einheits-Konvertierung** (`type_conversions`), Duplikat-/Constraint-Logik | Fachliche **Interpretation**, Aggregation von „Bedeutung“, Metriken für Auswertung |
|
||||||
|
| **Data Layer (Issue #53, Layer 1+)** | Daten lesen, aufbereiten, ableiten, für Charts/KI/Prompts bereitstellen | — |
|
||||||
|
|
||||||
|
Verbindlich: **Semantik und Auswertung** nicht dauerhaft im Import verstecken; neue Features werden an dieser Grenze geprüft.
|
||||||
|
|
||||||
|
**Detail & Zielbild (Multi-Layer, Single Source of Truth):** `docs/issues/issue-53-phase-0c-multi-layer-architecture.md`
|
||||||
|
|
||||||
|
**Umsetzung Schlaf-Import (Refactoring, Offen):** Gitea http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/69
|
||||||
|
|
||||||
|
### 8.2 Ist-Einordnung Import-Pfade (Übergang)
|
||||||
|
|
||||||
|
Bis sukzessive auf das Zielbild umgestellt ist, gilt:
|
||||||
|
|
||||||
|
| Pfad | Einordnung |
|
||||||
|
|------|----------------|
|
||||||
|
| Universal-CSV (`csv_parser`, `routers/csv_import.py`, Executor für u. a. Gewicht/Ernährung/Blutdruck/Aktivität/Vitals) | **Zielrichtung:** Mapping + Typkonvertierung |
|
||||||
|
| Apple-Schlaf-Aggregat (`csv_parser/sleep_apple_import.py`, `import_mode: apple_sleep_aggregate`) | **Legacy-Adapter** (quellenspezifische Aufbereitung) – Austausch gegen mapping-nah + Layer 1 geplant |
|
||||||
|
| Dedizierte Import-Endpoints (z. B. `/api/activity/import-csv`, Vitals Apple) | **Legacy/Parallel** – neue Quellen bevorzugt über Universal-Pfad + Vorlagen |
|
||||||
|
|
||||||
|
Änderungen an Import-Pfaden: Legacy nur erweitern mit **expliziter** Issue-/Review-Begründung; kein neues „wir rechnen Auswertung beim Insert“ ohne Data-Layer-Bezug.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Test-Regeln
|
||||||
|
|
||||||
|
### 9.1 Tests schreiben ist Pflicht
|
||||||
Jedes neue Feature bekommt mindestens einen Playwright-Test in
|
Jedes neue Feature bekommt mindestens einen Playwright-Test in
|
||||||
`tests/dev-smoke-test.spec.js`.
|
`tests/dev-smoke-test.spec.js`.
|
||||||
|
|
||||||
### 8.2 Reihenfolge: Test vor Commit
|
### 9.2 Reihenfolge: Test vor Commit
|
||||||
```
|
```
|
||||||
Implementieren → Tests schreiben → Tests grün → Committen
|
Implementieren → Tests schreiben → Tests grün → Committen
|
||||||
NIEMALS: Implementieren → Committen → Tests später
|
NIEMALS: Implementieren → Committen → Tests später
|
||||||
```
|
```
|
||||||
|
|
||||||
### 8.3 Claude Code schreibt Tests selbst
|
### 9.3 Claude Code schreibt Tests selbst
|
||||||
Nach jeder Implementierung:
|
Nach jeder Implementierung:
|
||||||
1. Passende Tests in dev-smoke-test.spec.js ergänzen
|
1. Passende Tests in dev-smoke-test.spec.js ergänzen
|
||||||
2. `npx playwright test` ausführen
|
2. `npx playwright test` ausführen
|
||||||
3. Fehler korrigieren bis alle Tests grün
|
3. Fehler korrigieren bis alle Tests grün
|
||||||
4. Erst dann committen
|
4. Erst dann committen
|
||||||
|
|
||||||
### 8.4 Test-Kategorien
|
### 9.4 Test-Kategorien
|
||||||
```javascript
|
```javascript
|
||||||
// UI-Test (Playwright)
|
// UI-Test (Playwright)
|
||||||
test('FEATURE: Beschreibung', async ({ page }) => { ... })
|
test('FEATURE: Beschreibung', async ({ page }) => { ... })
|
||||||
@@ -412,18 +444,18 @@ test('FEATURE: Beschreibung', async ({ page }) => { ... })
|
|||||||
test('API: Endpoint', async ({ request }) => { ... })
|
test('API: Endpoint', async ({ request }) => { ... })
|
||||||
```
|
```
|
||||||
|
|
||||||
### 8.5 Screenshots bei Fehlern
|
### 9.5 Screenshots bei Fehlern
|
||||||
Fehlgeschlagene Tests erzeugen automatisch Screenshots in:
|
Fehlgeschlagene Tests erzeugen automatisch Screenshots in:
|
||||||
`test-results/TESTNAME/test-failed-1.png`
|
`test-results/TESTNAME/test-failed-1.png`
|
||||||
→ Immer ansehen bevor Code geändert wird
|
→ Immer ansehen bevor Code geändert wird
|
||||||
|
|
||||||
### 8.6 Prod nie testen
|
### 9.6 Prod nie testen
|
||||||
Tests laufen IMMER gegen dev.mitai.jinkendo.de
|
Tests laufen IMMER gegen dev.mitai.jinkendo.de
|
||||||
NIEMALS gegen mitai.jinkendo.de
|
NIEMALS gegen mitai.jinkendo.de
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 9. Dashboard-Lab-Widgets und Feature-System
|
## 10. Dashboard-Lab-Widgets und Feature-System
|
||||||
|
|
||||||
**Kontext:** Dashboard-Widgets (`backend/widget_catalog.py`, Lab unter `/api/app/...`) und das **Subscription-/Feature-Modell** (`features`, `tier_limits`, `check_feature_access` in `backend/auth.py`) sind **getrennte Schichten**, müssen aber bei tariffrelevanten Widgets **verknüpft** werden.
|
**Kontext:** Dashboard-Widgets (`backend/widget_catalog.py`, Lab unter `/api/app/...`) und das **Subscription-/Feature-Modell** (`features`, `tier_limits`, `check_feature_access` in `backend/auth.py`) sind **getrennte Schichten**, müssen aber bei tariffrelevanten Widgets **verknüpft** werden.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,339 @@
|
|||||||
|
# Architektur-Regeln – Mitai Jinkendo
|
||||||
|
|
||||||
|
> **PFLICHTLEKTÜRE für Claude Code vor jeder Implementierung.**
|
||||||
|
> Diese Regeln sind verbindlich und dürfen nicht ohne explizite
|
||||||
|
> Genehmigung des Nutzers abgeändert werden.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Router-Architektur
|
||||||
|
|
||||||
|
### 1.1 Ein Modul = Ein Router
|
||||||
|
Jedes fachliche Modul hat genau eine Router-Datei in `backend/routers/`.
|
||||||
|
|
||||||
|
```
|
||||||
|
backend/routers/
|
||||||
|
├── auth.py # Authentifizierung
|
||||||
|
├── profiles.py # Nutzerprofile
|
||||||
|
├── weight.py # Gewichts-Tracking
|
||||||
|
├── sleep.py # Schlaf-Modul
|
||||||
|
├── training_types.py # Trainingstypen + HF
|
||||||
|
└── ... # je neues Modul = neue Datei
|
||||||
|
```
|
||||||
|
|
||||||
|
**Regeln:**
|
||||||
|
- Kein Endpoint darf außerhalb seines thematischen Routers definiert werden
|
||||||
|
- Neue Module immer als neue Router-Datei anlegen, nie in bestehende einfügen
|
||||||
|
- Router in `main.py` registrieren: `app.include_router(modul.router, prefix="/api")`
|
||||||
|
- Router-Datei-Name = Modul-Name in `version.py` MODULE_VERSIONS
|
||||||
|
|
||||||
|
### 1.2 API-First Prinzip
|
||||||
|
Jede Funktion ist zuerst als API-Endpoint implementiert – die UI nutzt ausschließlich
|
||||||
|
diese Endpoints über `api.js`. Keine Business-Logik im Frontend.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# ✅ Richtig: Logik im Backend-Endpoint
|
||||||
|
@router.get("/sleep/stats")
|
||||||
|
def get_sleep_stats(session=Depends(require_auth)):
|
||||||
|
# Berechnung hier
|
||||||
|
return {"avg_duration": ..., "sleep_debt": ...}
|
||||||
|
|
||||||
|
# ❌ Falsch: Berechnung im Frontend
|
||||||
|
const sleepDebt = entries.reduce((sum, e) => sum + (goal - e.duration), 0)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.3 Einheitliche Fehlerbehandlung
|
||||||
|
```python
|
||||||
|
# ✅ Immer dieses Format:
|
||||||
|
raise HTTPException(status_code=404, detail="Eintrag nicht gefunden")
|
||||||
|
# Response: {"detail": "Eintrag nicht gefunden"}
|
||||||
|
|
||||||
|
# ❌ Nie eigene Formate:
|
||||||
|
return {"error": "not found"}
|
||||||
|
return {"message": "Fehler", "success": False}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Versionskontrollsystem
|
||||||
|
|
||||||
|
### 2.1 Versionierungsschema
|
||||||
|
**Semantic Versioning: `MAJOR.MINOR.PATCH`**
|
||||||
|
|
||||||
|
| Typ | Wann | Beispiel |
|
||||||
|
|-----|------|---------|
|
||||||
|
| MAJOR | Breaking Change, DB-Migration inkompatibel | 9.0.0 → 10.0.0 |
|
||||||
|
| MINOR | Neues Feature, neues Modul | 9.2.0 → 9.3.0 |
|
||||||
|
| PATCH | Bugfix, kleine Änderung, Refactor | 9.3.0 → 9.3.1 |
|
||||||
|
|
||||||
|
### 2.2 Versions-Dateien
|
||||||
|
|
||||||
|
**Backend: `backend/version.py`**
|
||||||
|
```python
|
||||||
|
APP_VERSION = "9.3.0"
|
||||||
|
BUILD_DATE = "2026-03-22"
|
||||||
|
|
||||||
|
MODULE_VERSIONS = {
|
||||||
|
"auth": "1.2.0",
|
||||||
|
"profiles": "1.1.0",
|
||||||
|
"weight": "1.0.3",
|
||||||
|
"circumference": "1.0.1",
|
||||||
|
"caliper": "1.0.1",
|
||||||
|
"activity": "1.1.0",
|
||||||
|
"nutrition": "1.0.2",
|
||||||
|
"photos": "1.0.0",
|
||||||
|
"insights": "1.3.0",
|
||||||
|
"prompts": "1.1.0",
|
||||||
|
"admin": "1.2.0",
|
||||||
|
"stats": "1.0.1",
|
||||||
|
"exportdata": "1.1.0",
|
||||||
|
"importdata": "1.0.0",
|
||||||
|
"membership": "2.1.0",
|
||||||
|
}
|
||||||
|
|
||||||
|
CHANGELOG = [
|
||||||
|
{
|
||||||
|
"version": "9.3.0",
|
||||||
|
"date": "2026-03-22",
|
||||||
|
"changes": [
|
||||||
|
"Feature: Sleep Module (sleep_log, JSONB-Segmente)",
|
||||||
|
"Feature: Vitalwerte-Seite in Navigation",
|
||||||
|
"Feature: Trainingstypen-Kategorisierung",
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"version": "9.2.1",
|
||||||
|
"date": "2026-03-20",
|
||||||
|
"changes": [
|
||||||
|
"Fix: Feature-Enforcement Rollback",
|
||||||
|
"Fix: Erholungsstatus-Gewichtung korrigiert",
|
||||||
|
]
|
||||||
|
},
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Frontend: `frontend/src/version.js`**
|
||||||
|
```javascript
|
||||||
|
export const APP_VERSION = "9.3.0"
|
||||||
|
export const BUILD_DATE = "2026-03-22"
|
||||||
|
|
||||||
|
export const PAGE_VERSIONS = {
|
||||||
|
Dashboard: "1.3.0",
|
||||||
|
LoginScreen: "1.1.0",
|
||||||
|
WeightPage: "1.0.3",
|
||||||
|
ActivityPage: "1.2.0",
|
||||||
|
NutritionPage: "1.1.0",
|
||||||
|
AnalysisPage: "1.3.0",
|
||||||
|
SettingsPage: "1.4.0",
|
||||||
|
AdminPanel: "1.2.0",
|
||||||
|
SubscriptionPage: "1.0.0",
|
||||||
|
// Neue Seiten hier eintragen
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 Versions-Endpoint
|
||||||
|
|
||||||
|
**`GET /api/version`** – öffentlich (kein Auth erforderlich)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"app_version": "9.3.0",
|
||||||
|
"build_date": "2026-03-22",
|
||||||
|
"backend_version": "9.3.0",
|
||||||
|
"modules": {
|
||||||
|
"auth": "1.2.0",
|
||||||
|
"sleep": "1.0.0"
|
||||||
|
},
|
||||||
|
"db_schema_version": "20260322",
|
||||||
|
"environment": "production"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Dieser Endpoint wird in `backend/routers/version.py` implementiert und liest
|
||||||
|
direkt aus `version.py`.
|
||||||
|
|
||||||
|
### 2.4 Versions-Anzeige in der App
|
||||||
|
|
||||||
|
**Settings-Seite – Versions-Panel:**
|
||||||
|
```
|
||||||
|
System-Versionen
|
||||||
|
─────────────────────────────────────
|
||||||
|
App (gesamt) 9.3.0
|
||||||
|
Backend 9.3.0 ✓ erreichbar
|
||||||
|
Frontend 9.3.0 ✓ geladen
|
||||||
|
DB-Schema 20260322
|
||||||
|
Umgebung production
|
||||||
|
─────────────────────────────────────
|
||||||
|
Module
|
||||||
|
auth 1.2.0
|
||||||
|
sleep 1.0.0
|
||||||
|
membership 2.1.0
|
||||||
|
[alle Module...]
|
||||||
|
─────────────────────────────────────
|
||||||
|
[Changelog] [Cache leeren]
|
||||||
|
```
|
||||||
|
|
||||||
|
Frontend ruft beim Laden der Settings-Seite `/api/version` ab und vergleicht
|
||||||
|
mit der eigenen `APP_VERSION` aus `version.js`. Bei Abweichung: Warnung anzeigen.
|
||||||
|
|
||||||
|
### 2.5 Pflicht-Regel: Versions-Bump bei jedem Commit
|
||||||
|
|
||||||
|
**Jede Code-Änderung erfordert:**
|
||||||
|
1. Versions-Bump in `backend/version.py` (APP_VERSION + betroffenes MODULE_VERSION)
|
||||||
|
2. Versions-Bump in `frontend/src/version.js` (APP_VERSION + betroffene PAGE_VERSION)
|
||||||
|
3. Changelog-Eintrag in `backend/version.py` CHANGELOG
|
||||||
|
|
||||||
|
**Claude Code prüft das im `/deploy` Command automatisch.**
|
||||||
|
|
||||||
|
Kein Commit ohne Versions-Bump – keine Ausnahme.
|
||||||
|
|
||||||
|
### 2.6 DB-Schema-Version
|
||||||
|
|
||||||
|
Format: `YYYYMMDD` (Datum der letzten Migration)
|
||||||
|
|
||||||
|
Gespeichert in `backend/version.py`:
|
||||||
|
```python
|
||||||
|
DB_SCHEMA_VERSION = "20260322"
|
||||||
|
```
|
||||||
|
|
||||||
|
Bei jeder Schema-Änderung (ALTER TABLE, neue Tabelle) → DB_SCHEMA_VERSION aktualisieren.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Datenbankregeln
|
||||||
|
|
||||||
|
### 3.1 Pflichtfelder für neue Tabellen
|
||||||
|
```sql
|
||||||
|
-- Jede neue Tabelle braucht:
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
created_at TIMESTAMP DEFAULT NOW(),
|
||||||
|
updated_at TIMESTAMP DEFAULT NOW()
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 Source-Tracking bei Import-Daten
|
||||||
|
Tabellen die Daten aus externen Quellen empfangen brauchen:
|
||||||
|
```sql
|
||||||
|
source VARCHAR(50) DEFAULT 'manual'
|
||||||
|
-- Werte: 'manual' | 'apple_health' | 'garmin' | 'withings'
|
||||||
|
```
|
||||||
|
|
||||||
|
Manuelle Einträge (`source = 'manual'`) haben IMMER Vorrang bei Reimport:
|
||||||
|
```sql
|
||||||
|
-- Reimport überschreibt nur nicht-manuelle Einträge:
|
||||||
|
INSERT INTO sleep_log (...) ON CONFLICT (profile_id, date)
|
||||||
|
DO UPDATE SET ... WHERE sleep_log.source != 'manual'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.3 Profile-ID Isolation
|
||||||
|
Jede Tabelle mit Nutzerdaten hat `profile_id` als Foreign Key.
|
||||||
|
Kein Endpoint gibt Daten eines anderen Profils zurück.
|
||||||
|
Profile-ID kommt IMMER aus der Session, nie aus Request-Parametern.
|
||||||
|
|
||||||
|
### 3.4 Boolean-Werte
|
||||||
|
```sql
|
||||||
|
-- PostgreSQL Boolean (nicht SQLite 0/1):
|
||||||
|
WHERE active = true ✓
|
||||||
|
WHERE active = 1 ✗
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Frontend-Regeln
|
||||||
|
|
||||||
|
### 4.1 Alle API-Calls über api.js
|
||||||
|
```javascript
|
||||||
|
// ✅ Richtig:
|
||||||
|
import { api } from '../utils/api'
|
||||||
|
const data = await api.listSleep()
|
||||||
|
|
||||||
|
// ❌ Falsch:
|
||||||
|
const r = await fetch('/api/sleep')
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 Neue Seite = Eintrag in PAGE_VERSIONS
|
||||||
|
Jede neue Seite in `frontend/src/version.js` registrieren.
|
||||||
|
|
||||||
|
### 4.3 CSS-Variablen statt Hardcoded-Farben
|
||||||
|
```javascript
|
||||||
|
// ✅ Richtig:
|
||||||
|
style={{color: 'var(--accent)'}}
|
||||||
|
|
||||||
|
// ❌ Falsch:
|
||||||
|
style={{color: '#1D9E75'}}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.4 Fehlerbehandlung in allen async Funktionen
|
||||||
|
```javascript
|
||||||
|
try {
|
||||||
|
const data = await api.meinEndpoint()
|
||||||
|
setData(data)
|
||||||
|
} catch(e) {
|
||||||
|
setError(e.message)
|
||||||
|
} finally {
|
||||||
|
setLoading(false)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Git & Deployment-Regeln
|
||||||
|
|
||||||
|
### 5.1 Nie direkt auf main pushen
|
||||||
|
Immer über Pull Request in Gitea: develop → main.
|
||||||
|
develop Branch niemals löschen.
|
||||||
|
|
||||||
|
### 5.2 Commit-Message Format
|
||||||
|
```
|
||||||
|
feat: neues Feature oder Modul
|
||||||
|
fix: Bugfix
|
||||||
|
refactor: Umbau ohne Funktionsänderung
|
||||||
|
docs: Dokumentation
|
||||||
|
version: Versions-Bump
|
||||||
|
ci: CI/CD Änderungen
|
||||||
|
chore: Maintenance
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 Versions-Bump im Commit
|
||||||
|
```
|
||||||
|
feat: Sleep Module v1.0.0
|
||||||
|
|
||||||
|
- sleep_log Tabelle mit JSONB-Segmenten
|
||||||
|
- Import aus Apple Health CSV
|
||||||
|
- Korrelationen Schlaf <-> Ruhepuls
|
||||||
|
|
||||||
|
version: 9.3.0 (backend + frontend)
|
||||||
|
module: sleep 1.0.0
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Dokumentations-Regeln
|
||||||
|
|
||||||
|
### 6.1 Neue Module dokumentieren
|
||||||
|
Bei jedem neuen Modul:
|
||||||
|
1. Fachliche Spec: `.claude/docs/functional/MODUL_NAME.md`
|
||||||
|
2. Technische Spec: `.claude/docs/technical/MODUL_NAME.md`
|
||||||
|
3. Nach Fertigstellung: `.claude/library/` aktualisieren
|
||||||
|
|
||||||
|
### 6.2 CLAUDE.md aktuell halten
|
||||||
|
Nach größeren Änderungen CLAUDE.md Versions-Tabelle aktualisieren.
|
||||||
|
|
||||||
|
### 6.3 Lessons Learned dokumentieren
|
||||||
|
Jeder Rollback oder schwerer Bug → Eintrag in `.claude/rules/LESSONS_LEARNED.md`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Zusammenfassung: Checkliste vor jedem Commit
|
||||||
|
|
||||||
|
```
|
||||||
|
[ ] Versions-Bump in backend/version.py (APP_VERSION + MODULE)
|
||||||
|
[ ] Versions-Bump in frontend/src/version.js (APP_VERSION + PAGE)
|
||||||
|
[ ] Changelog-Eintrag in backend/version.py
|
||||||
|
[ ] DB_SCHEMA_VERSION aktualisiert (wenn Schema geändert)
|
||||||
|
[ ] Neues Modul in PAGE_VERSIONS / MODULE_VERSIONS eingetragen
|
||||||
|
[ ] Auth auf alle neuen Endpoints (require_auth)
|
||||||
|
[ ] Fehlerformat einheitlich (HTTPException mit detail)
|
||||||
|
[ ] Neue Tabellen haben created_at + updated_at
|
||||||
|
[ ] Import-Tabellen haben source-Feld
|
||||||
|
[ ] api.js für alle Frontend API-Calls
|
||||||
|
```
|
||||||
@@ -39,6 +39,13 @@ from slowapi import Limiter
|
|||||||
def sensitive(request: Request, ...):
|
def sensitive(request: Request, ...):
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 6. Universal CSV Import / Admin-Vorlagen
|
||||||
|
Neues **Import-Zielmodul**, Änderungen an **`csv_parser`**, Executor, DB-`source`/`CHECK`, oder System-CSV-Vorlagen:
|
||||||
|
|
||||||
|
- Pflichtlektüre und Checkliste: **`.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`**
|
||||||
|
- Keine zweite DB-Connection im Importpfad; Zeilenfehler ohne „aborted transaction“ (SAVEPOINT-Muster wo nötig)
|
||||||
|
- Admin Create/Update von Systemvorlagen: Validierung über `validate_csv_template` nicht umgehen
|
||||||
|
|
||||||
## Frontend
|
## Frontend
|
||||||
|
|
||||||
### 1. api.js für alle API-Calls
|
### 1. api.js für alle API-Calls
|
||||||
|
|||||||
@@ -3,22 +3,82 @@ name: Build Test
|
|||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [main, develop]
|
branches: [main, develop]
|
||||||
|
workflow_run:
|
||||||
|
workflows: ["Deploy Development", "Deploy Production"]
|
||||||
|
types: [completed]
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
lint-backend:
|
pytest-backend:
|
||||||
|
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- name: Check backend syntax
|
- name: Run backend pytest suite in deployed container
|
||||||
run: |
|
run: |
|
||||||
python3 -m py_compile /home/lars/docker/bodytrack/backend/main.py
|
EVENT_NAME="${{ github.event_name }}"
|
||||||
|
REF_NAME="${{ github.ref_name }}"
|
||||||
|
RUN_WORKFLOW="${{ github.event.workflow_run.name }}"
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack"
|
||||||
|
COMPOSE_FILE="docker-compose.yml"
|
||||||
|
|
||||||
|
if [ "$EVENT_NAME" = "workflow_run" ]; then
|
||||||
|
if [ "$RUN_WORKFLOW" = "Deploy Development" ]; then
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack-dev"
|
||||||
|
COMPOSE_FILE="docker-compose.dev-env.yml"
|
||||||
|
fi
|
||||||
|
elif [ "$REF_NAME" = "develop" ]; then
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack-dev"
|
||||||
|
COMPOSE_FILE="docker-compose.dev-env.yml"
|
||||||
|
fi
|
||||||
|
|
||||||
|
cd "$APP_DIR"
|
||||||
|
docker compose -f "$COMPOSE_FILE" exec -T backend sh -lc "
|
||||||
|
pip install -r /app/requirements-dev.txt &&
|
||||||
|
cd /app &&
|
||||||
|
python -m pytest tests -m 'not slow' -ra -vv --tb=short
|
||||||
|
"
|
||||||
|
|
||||||
|
lint-backend:
|
||||||
|
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Check backend syntax on deployed app
|
||||||
|
run: |
|
||||||
|
EVENT_NAME="${{ github.event_name }}"
|
||||||
|
REF_NAME="${{ github.ref_name }}"
|
||||||
|
RUN_WORKFLOW="${{ github.event.workflow_run.name }}"
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack"
|
||||||
|
|
||||||
|
if [ "$EVENT_NAME" = "workflow_run" ]; then
|
||||||
|
if [ "$RUN_WORKFLOW" = "Deploy Development" ]; then
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack-dev"
|
||||||
|
fi
|
||||||
|
elif [ "$REF_NAME" = "develop" ]; then
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack-dev"
|
||||||
|
fi
|
||||||
|
|
||||||
|
python3 -m py_compile "$APP_DIR/backend/main.py"
|
||||||
echo "✓ Backend syntax OK"
|
echo "✓ Backend syntax OK"
|
||||||
|
|
||||||
build-frontend:
|
build-frontend:
|
||||||
|
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- name: Build frontend
|
- name: Build frontend on deployed app
|
||||||
run: |
|
run: |
|
||||||
cd /home/lars/docker/bodytrack/frontend
|
EVENT_NAME="${{ github.event_name }}"
|
||||||
|
REF_NAME="${{ github.ref_name }}"
|
||||||
|
RUN_WORKFLOW="${{ github.event.workflow_run.name }}"
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack"
|
||||||
|
|
||||||
|
if [ "$EVENT_NAME" = "workflow_run" ]; then
|
||||||
|
if [ "$RUN_WORKFLOW" = "Deploy Development" ]; then
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack-dev"
|
||||||
|
fi
|
||||||
|
elif [ "$REF_NAME" = "develop" ]; then
|
||||||
|
APP_DIR="/home/lars/docker/bodytrack-dev"
|
||||||
|
fi
|
||||||
|
|
||||||
|
cd "$APP_DIR/frontend"
|
||||||
npm install
|
npm install
|
||||||
npm run build
|
npm run build
|
||||||
echo "✓ Frontend build OK"
|
echo "✓ Frontend build OK"
|
||||||
@@ -8,6 +8,7 @@
|
|||||||
> | Coding-Regeln | `.claude/rules/CODING_RULES.md` |
|
> | Coding-Regeln | `.claude/rules/CODING_RULES.md` |
|
||||||
> | Lessons Learned | `.claude/rules/LESSONS_LEARNED.md` |
|
> | Lessons Learned | `.claude/rules/LESSONS_LEARNED.md` |
|
||||||
> | **Gitea-Landkarte (lokal gepflegt)** | **`.claude/docs/GITEA_ISSUES_INDEX.md`** |
|
> | **Gitea-Landkarte (lokal gepflegt)** | **`.claude/docs/GITEA_ISSUES_INDEX.md`** |
|
||||||
|
> | **Universal CSV Import** (neues Modul / Executor / Vorlagen) | **`.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`** |
|
||||||
> | **GUI / IA / Admin / Nav / PWA-Leiste** | **`docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md`** |
|
> | **GUI / IA / Admin / Nav / PWA-Leiste** | **`docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md`** |
|
||||||
> | **Dashboard-Lab-Widgets** (Katalog, Registrierung, `config`) | **`.claude/docs/technical/DASHBOARD_WIDGETS_AGENT_GUIDE.md`** |
|
> | **Dashboard-Lab-Widgets** (Katalog, Registrierung, `config`) | **`.claude/docs/technical/DASHBOARD_WIDGETS_AGENT_GUIDE.md`** |
|
||||||
> | **Agent-Einstieg** | **`.claude/README.md`** |
|
> | **Agent-Einstieg** | **`.claude/README.md`** |
|
||||||
@@ -98,6 +99,12 @@ frontend/src/
|
|||||||
**Branch:** develop
|
**Branch:** develop
|
||||||
**Nächster Schritt:** Frontend Chart Integration → Testing → Prod Deploy v0.9i
|
**Nächster Schritt:** Frontend Chart Integration → Testing → Prod Deploy v0.9i
|
||||||
|
|
||||||
|
### Updates (09.04.2026 - Universal CSV Import, Prod-Migration abgeschlossen)
|
||||||
|
|
||||||
|
- **Agent-Leitfaden:** `.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` (Checkliste für neue Import-Module, Executor, Vorlagen, `source=csv`, SAVEPOINT-/Cursor-Regeln)
|
||||||
|
- **Regeln:** Verweise in `.claude/rules/ARCHITECTURE.md` (§3.2 `source`), `.claude/rules/CODING_RULES.md` (§6)
|
||||||
|
- **Follow-ups:** **Gitea #71** – Dry-Run inkl. `import_row_processing`, Nutzer-Mapping-Validierung, Fehler-Hints in der Import-UI ([Issue](http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/71))
|
||||||
|
|
||||||
### GUI / Informationsarchitektur (Abnahme dieser Iteration, 2026-04-05)
|
### GUI / Informationsarchitektur (Abnahme dieser Iteration, 2026-04-05)
|
||||||
|
|
||||||
Admin-Bereich (`AdminShell`, Hub-Routen), Hauptnavigation inkl. **Ziele** (`config/appNav.js`), Einstellungen nur aktives Profil + E-Mail, KI-Analyse Ergebnis in rechter Spalte, **PWA** Bottom-Nav inkl. iOS Safe Area. Zentrale Agent-Doku: **`docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md`**. Responsive-Epic **Gitea #30:** Phasenplan `docs/issues/PHASE_PLAN_RESPONSIVE_UI.md` — **P7 Kern erledigt**, **P8** (Regression/Abnahme) ausstehend; Issue bewusst **nicht** geschlossen.
|
Admin-Bereich (`AdminShell`, Hub-Routen), Hauptnavigation inkl. **Ziele** (`config/appNav.js`), Einstellungen nur aktives Profil + E-Mail, KI-Analyse Ergebnis in rechter Spalte, **PWA** Bottom-Nav inkl. iOS Safe Area. Zentrale Agent-Doku: **`docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md`**. Responsive-Epic **Gitea #30:** Phasenplan `docs/issues/PHASE_PLAN_RESPONSIVE_UI.md` — **P7 Kern erledigt**, **P8** (Regression/Abnahme) ausstehend; Issue bewusst **nicht** geschlossen.
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
"""Universal CSV import foundation (Issue #21)."""
|
||||||
|
|
||||||
|
from csv_parser.core import (
|
||||||
|
decode_raw_bytes,
|
||||||
|
sniff_delimiter,
|
||||||
|
parse_csv_sample,
|
||||||
|
column_signature,
|
||||||
|
normalize_header_for_signature,
|
||||||
|
)
|
||||||
|
from csv_parser.module_registry import MODULE_DEFINITIONS, get_module_definition, list_modules
|
||||||
|
from csv_parser.type_converter import convert_value, build_row_after_mapping
|
||||||
|
from csv_parser.permissions import user_may_delete_mapping, user_may_edit_mapping_row
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"decode_raw_bytes",
|
||||||
|
"sniff_delimiter",
|
||||||
|
"parse_csv_sample",
|
||||||
|
"column_signature",
|
||||||
|
"normalize_header_for_signature",
|
||||||
|
"MODULE_DEFINITIONS",
|
||||||
|
"get_module_definition",
|
||||||
|
"list_modules",
|
||||||
|
"convert_value",
|
||||||
|
"build_row_after_mapping",
|
||||||
|
"user_may_delete_mapping",
|
||||||
|
"user_may_edit_mapping_row",
|
||||||
|
]
|
||||||
@@ -0,0 +1,208 @@
|
|||||||
|
"""
|
||||||
|
CSV bytes → text, delimiter sniffing, strukturierte Erstzeilen für Analyse (Issue #21).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
import io
|
||||||
|
import re
|
||||||
|
from typing import Any, Dict, Iterator, List, Sequence, Tuple
|
||||||
|
|
||||||
|
_DEFAULT_DELIMS = [",", ";", "\t"]
|
||||||
|
|
||||||
|
|
||||||
|
def decode_raw_bytes(raw: bytes) -> str:
|
||||||
|
"""UTF-8 bevorzugt, Fallback Latin-1; BOM entfernen."""
|
||||||
|
if not raw:
|
||||||
|
return ""
|
||||||
|
for enc in ("utf-8-sig", "utf-8", "latin-1"):
|
||||||
|
try:
|
||||||
|
text = raw.decode(enc)
|
||||||
|
break
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
text = ""
|
||||||
|
continue
|
||||||
|
else:
|
||||||
|
text = raw.decode("utf-8", errors="replace")
|
||||||
|
if text.startswith("\ufeff"):
|
||||||
|
text = text[1:]
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def sniff_delimiter(sample_line: str) -> str:
|
||||||
|
"""
|
||||||
|
Heuristik: Zähle Vorkommen der Kandidaten in der ersten Datenzeile.
|
||||||
|
Kein csv.Sniffer (robuster gegen kurze Zeilen).
|
||||||
|
"""
|
||||||
|
if not sample_line or not sample_line.strip():
|
||||||
|
return ","
|
||||||
|
best = ","
|
||||||
|
best_count = -1
|
||||||
|
for d in _DEFAULT_DELIMS:
|
||||||
|
c = sample_line.count(d)
|
||||||
|
if c > best_count:
|
||||||
|
best_count = c
|
||||||
|
best = d
|
||||||
|
return best
|
||||||
|
|
||||||
|
|
||||||
|
def _split_first_lines(text: str, max_lines: int = 5) -> List[str]:
|
||||||
|
lines: List[str] = []
|
||||||
|
for line in text.splitlines():
|
||||||
|
if line.strip():
|
||||||
|
lines.append(line)
|
||||||
|
if len(lines) >= max_lines:
|
||||||
|
break
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
|
def parse_csv_sample(
|
||||||
|
text: str,
|
||||||
|
delimiter: str | None = None,
|
||||||
|
has_header: bool = True,
|
||||||
|
max_data_rows: int = 5,
|
||||||
|
) -> Tuple[List[str], List[dict[str, str]], str]:
|
||||||
|
"""
|
||||||
|
Gibt (headers, rows_as_dicts, verwendetes_delimiter) zurück.
|
||||||
|
rows sind Rohstrings pro Zelle.
|
||||||
|
"""
|
||||||
|
lines = _split_first_lines(text, max_lines=50)
|
||||||
|
if not lines:
|
||||||
|
return [], [], ","
|
||||||
|
|
||||||
|
delim = delimiter if delimiter is not None else sniff_delimiter(lines[0])
|
||||||
|
reader = csv.reader(io.StringIO(text.replace("\r\n", "\n").replace("\r", "\n")), delimiter=delim)
|
||||||
|
rows_raw: List[List[str]] = []
|
||||||
|
for i, row in enumerate(reader):
|
||||||
|
if i >= 1 + max_data_rows + (1 if has_header else 0):
|
||||||
|
break
|
||||||
|
if not any(c.strip() for c in row):
|
||||||
|
continue
|
||||||
|
rows_raw.append(row)
|
||||||
|
|
||||||
|
if not rows_raw:
|
||||||
|
return [], [], delim
|
||||||
|
|
||||||
|
if has_header:
|
||||||
|
headers = [h.strip() for h in rows_raw[0]]
|
||||||
|
data = rows_raw[1 : 1 + max_data_rows]
|
||||||
|
else:
|
||||||
|
n = len(rows_raw[0])
|
||||||
|
headers = [f"col_{i}" for i in range(n)]
|
||||||
|
data = rows_raw[:max_data_rows]
|
||||||
|
|
||||||
|
dict_rows: List[dict[str, str]] = []
|
||||||
|
for r in data:
|
||||||
|
row_dict: dict[str, str] = {}
|
||||||
|
for j, h in enumerate(headers):
|
||||||
|
row_dict[h] = r[j].strip() if j < len(r) else ""
|
||||||
|
dict_rows.append(row_dict)
|
||||||
|
|
||||||
|
return headers, dict_rows, delim
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_header_for_signature(name: str) -> str:
|
||||||
|
s = name.strip().lower()
|
||||||
|
s = re.sub(r"\s+", "_", s)
|
||||||
|
s = re.sub(r"[^a-z0-9_äöüß().%-]+", "_", s)
|
||||||
|
return s.strip("_")
|
||||||
|
|
||||||
|
|
||||||
|
def column_signature(headers: List[str]) -> List[str]:
|
||||||
|
"""Sortierte normalisierte Spaltennamen für Signatur-Vergleich."""
|
||||||
|
return sorted({normalize_header_for_signature(h) for h in headers if h is not None and str(h).strip()})
|
||||||
|
|
||||||
|
|
||||||
|
def headers_signature_match_score(sig_csv: List[str], sig_template: List[str]) -> float:
|
||||||
|
"""Jaccard-Überlappung 0..1 (|A∩B|/|A∪B|). Fällt stark, wenn die CSV viele Zusatzspalten hat."""
|
||||||
|
a, b = set(sig_csv), set(sig_template)
|
||||||
|
if not a and not b:
|
||||||
|
return 1.0
|
||||||
|
if not a or not b:
|
||||||
|
return 0.0
|
||||||
|
inter = len(a & b)
|
||||||
|
union = len(a | b)
|
||||||
|
return inter / union if union else 0.0
|
||||||
|
|
||||||
|
|
||||||
|
def headers_signature_template_recall(sig_csv: Sequence[str], sig_template: Sequence[str]) -> float:
|
||||||
|
"""
|
||||||
|
Anteil der Template-Spalten (Signatur), die in der CSV vorkommen: |A∩B|/|B|.
|
||||||
|
100 %, sobald alle für die Vorlage relevanten Spalten in der Datei sind — unabhängig von
|
||||||
|
Zusatzspalten (Gewicht + Ernährung in einer Datei erzeugt keinen „Abzug“ für die jeweilige Vorlage).
|
||||||
|
"""
|
||||||
|
a = set(sig_csv)
|
||||||
|
b = {normalize_header_for_signature(str(x)) for x in sig_template}
|
||||||
|
b.discard("")
|
||||||
|
if not b:
|
||||||
|
return 1.0 if not a else 0.0
|
||||||
|
inter = len(a & b)
|
||||||
|
return inter / len(b)
|
||||||
|
|
||||||
|
|
||||||
|
def headers_signature_rank_metrics(sig_csv: List[str], sig_template: List[str]) -> dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Einheitliche Kennzahlen für Vorlagen-Ranking und UI.
|
||||||
|
confidence = template_recall (empfohlen für Anzeige / Sortierung primär).
|
||||||
|
"""
|
||||||
|
a = set(sig_csv)
|
||||||
|
b = {normalize_header_for_signature(str(x)) for x in sig_template}
|
||||||
|
b.discard("")
|
||||||
|
inter = a & b
|
||||||
|
n_inter = len(inter)
|
||||||
|
n_b = len(b)
|
||||||
|
n_a = len(a)
|
||||||
|
union = len(a | b)
|
||||||
|
template_recall = n_inter / n_b if n_b else (1.0 if not n_a else 0.0)
|
||||||
|
jaccard = n_inter / union if union else 0.0
|
||||||
|
return {
|
||||||
|
"confidence": round(template_recall, 4),
|
||||||
|
"template_recall": round(template_recall, 4),
|
||||||
|
"jaccard": round(jaccard, 4),
|
||||||
|
"columns_matched": n_inter,
|
||||||
|
"columns_in_template": n_b,
|
||||||
|
"columns_in_csv": n_a,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_csv_import_limits(conn_row: dict | None) -> dict[str, int]:
|
||||||
|
"""Liest Limits aus system_config.csv_import; Fallback bei fehlendem Key."""
|
||||||
|
defaults = {"max_rows_per_file": 50_000, "max_file_bytes": 52_428_800}
|
||||||
|
if not conn_row or "value" not in conn_row:
|
||||||
|
return defaults
|
||||||
|
val = conn_row["value"]
|
||||||
|
if isinstance(val, dict):
|
||||||
|
out = {**defaults, **{k: int(v) for k, v in val.items() if k in defaults}}
|
||||||
|
return out
|
||||||
|
return defaults
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_header_key(k: str | None) -> str:
|
||||||
|
if k is None:
|
||||||
|
return ""
|
||||||
|
return str(k).strip().removeprefix("\ufeff")
|
||||||
|
|
||||||
|
|
||||||
|
def iter_csv_dict_rows(
|
||||||
|
text: str,
|
||||||
|
delimiter: str,
|
||||||
|
*,
|
||||||
|
has_header: bool = True,
|
||||||
|
) -> Iterator[Dict[str, str]]:
|
||||||
|
"""
|
||||||
|
Vollständige Datei zeilenweise als Dict (Header = Keys).
|
||||||
|
Spaltenreihenfolge ist egal; zusätzliche Spalten werden ignoriert, wenn sie nicht
|
||||||
|
in field_mappings vorkommen. Keine Obergrenze für die Spaltenanzahl (nur Zeilenlimits
|
||||||
|
kommen aus system_config / Import-Router).
|
||||||
|
"""
|
||||||
|
if not has_header:
|
||||||
|
raise ValueError("CSV ohne Kopfzeile wird für Import noch nicht unterstützt")
|
||||||
|
normalized = text.replace("\r\n", "\n").replace("\r", "\n")
|
||||||
|
reader = csv.DictReader(io.StringIO(normalized), delimiter=delimiter)
|
||||||
|
for row in reader:
|
||||||
|
if row is None:
|
||||||
|
continue
|
||||||
|
if not any(v and str(v).strip() for v in row.values()):
|
||||||
|
continue
|
||||||
|
yield {_strip_header_key(k): (v or "").strip() for k, v in row.items() if _strip_header_key(k)}
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,115 @@
|
|||||||
|
"""
|
||||||
|
Kanonische Speichereinheiten pro CSV-Zielfeld (module_registry: field.unit) und
|
||||||
|
abwählbare Quelleinheiten → Faktor für type_conversions.source_unit.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from csv_parser.module_registry import get_module_definition
|
||||||
|
|
||||||
|
# 1 kcal = 4.184 kJ (IEC/ISO 80000)
|
||||||
|
_KJ_TO_KCAL = 1.0 / 4.184
|
||||||
|
|
||||||
|
# — Energie (Ziel kcal) —
|
||||||
|
_ENERGY: list[dict[str, Any]] = [
|
||||||
|
{"id": "kcal", "label": "Kilokalorien (kcal), wie in DB", "factor": 1.0},
|
||||||
|
{"id": "kj", "label": "Kilojoule (kJ) → kcal", "factor": _KJ_TO_KCAL},
|
||||||
|
{"id": "j", "label": "Joule (J) → kcal", "factor": _KJ_TO_KCAL / 1000.0},
|
||||||
|
]
|
||||||
|
|
||||||
|
# — Masse klein (Ziel g): Makronährstoffe —
|
||||||
|
_GRAM: list[dict[str, Any]] = [
|
||||||
|
{"id": "g", "label": "Gramm (g), wie in DB", "factor": 1.0},
|
||||||
|
{"id": "kg", "label": "Kilogramm (kg) → g", "factor": 1000.0},
|
||||||
|
{"id": "mg", "label": "Milligramm (mg) → g", "factor": 0.001},
|
||||||
|
]
|
||||||
|
|
||||||
|
# — Körpergewicht (Ziel kg) —
|
||||||
|
_KG: list[dict[str, Any]] = [
|
||||||
|
{"id": "kg", "label": "Kilogramm (kg), wie in DB", "factor": 1.0},
|
||||||
|
{"id": "g", "label": "Gramm (g) → kg", "factor": 0.001},
|
||||||
|
{"id": "lb", "label": "Pfund / lb → kg", "factor": 0.45359237},
|
||||||
|
{"id": "oz", "label": "Unze / oz → kg", "factor": 0.028349523125},
|
||||||
|
]
|
||||||
|
|
||||||
|
# — Strecke (Ziel km) —
|
||||||
|
_KM: list[dict[str, Any]] = [
|
||||||
|
{"id": "km", "label": "Kilometer (km), wie in DB", "factor": 1.0},
|
||||||
|
{"id": "m", "label": "Meter (m) → km", "factor": 0.001},
|
||||||
|
{"id": "mi", "label": "Meilen (mi) → km", "factor": 1.609344},
|
||||||
|
]
|
||||||
|
|
||||||
|
_UNIT_FAMILY: dict[str, list[dict[str, Any]]] = {
|
||||||
|
"kcal": _ENERGY,
|
||||||
|
"g": _GRAM,
|
||||||
|
"kg": _KG,
|
||||||
|
"km": _KM,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_canonical_unit(module: str, db_field: str) -> str | None:
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
return None
|
||||||
|
finfo: dict[str, Any] | None = mod.get("fields", {}).get(db_field)
|
||||||
|
if not finfo:
|
||||||
|
return None
|
||||||
|
u = finfo.get("unit")
|
||||||
|
return str(u) if u else None
|
||||||
|
|
||||||
|
|
||||||
|
def source_unit_choices_for_field(module: str, db_field: str) -> list[dict[str, Any]]:
|
||||||
|
"""Optionen für GUI: id, label, canonical_unit, is_canonical (Umrechnung serverseitig)."""
|
||||||
|
cu = get_canonical_unit(module, db_field)
|
||||||
|
if not cu:
|
||||||
|
return []
|
||||||
|
choices = _UNIT_FAMILY.get(cu)
|
||||||
|
if not choices:
|
||||||
|
return []
|
||||||
|
out: list[dict[str, Any]] = [
|
||||||
|
{
|
||||||
|
"id": c["id"],
|
||||||
|
"label": c["label"],
|
||||||
|
"canonical_unit": cu,
|
||||||
|
"is_canonical": c["id"] == cu,
|
||||||
|
}
|
||||||
|
for c in choices
|
||||||
|
]
|
||||||
|
out.append(
|
||||||
|
{
|
||||||
|
"id": "custom",
|
||||||
|
"label": "Benutzerdefiniert (Konvertierungsfaktor, z. B. ml→g je nach Dichte)",
|
||||||
|
"canonical_unit": cu,
|
||||||
|
"is_canonical": False,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def factor_source_to_canonical(module: str, db_field: str, source_unit: str | None) -> float:
|
||||||
|
"""
|
||||||
|
Multiplikator: CSV-Zahl * Faktor → Wert in kanonischer DB-Einheit.
|
||||||
|
Unbekannte/None/leer/Passthrough → 1.0
|
||||||
|
|
||||||
|
``source_unit`` ``custom`` / ``none``: kein Registry-Faktor (1.0); freie Skalierung nur über
|
||||||
|
``conversion_factor`` in type_conversions (JSON).
|
||||||
|
"""
|
||||||
|
if source_unit is None:
|
||||||
|
return 1.0
|
||||||
|
su = str(source_unit).strip().lower()
|
||||||
|
if not su:
|
||||||
|
return 1.0
|
||||||
|
if su in ("custom", "none"):
|
||||||
|
return 1.0
|
||||||
|
cu = get_canonical_unit(module, db_field)
|
||||||
|
if not cu:
|
||||||
|
return 1.0
|
||||||
|
choices = _UNIT_FAMILY.get(cu)
|
||||||
|
if not choices:
|
||||||
|
return 1.0
|
||||||
|
for c in choices:
|
||||||
|
if str(c["id"]).lower() == su:
|
||||||
|
return float(c["factor"])
|
||||||
|
return 1.0
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
"""
|
||||||
|
Menschenlesbare Hinweise zu typischen Import-/DB-Fehlern (Universal-CSV).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def enrich_row_error(message: str, module: str | None = None) -> dict[str, str | None]:
|
||||||
|
"""
|
||||||
|
Ergänzt eine Rohexception-Zeichenkette um ``code`` und ``hint`` für die Fehlerliste im Import.
|
||||||
|
"""
|
||||||
|
low = (message or "").lower()
|
||||||
|
out: dict[str, str | None] = {"error": message, "code": None, "hint": None}
|
||||||
|
|
||||||
|
if "numeric field overflow" in low or "numeric value out of range" in low:
|
||||||
|
out["code"] = "db_numeric_overflow"
|
||||||
|
out["hint"] = (
|
||||||
|
"Wert passt nicht in die Datenbank-Spalte (z. B. NUMERIC mit begrenzter Größe). "
|
||||||
|
"Häufig: Kilojoule aus dem Export landen im Kalorien-Feld – in der Vorlage für kcal_active/kcal_resting "
|
||||||
|
'"source_unit": "kj" setzen. Oder eine falsche CSV-Spalte ist einem kleinen Zielfeld zugeordnet '
|
||||||
|
"(z. B. große Zahl in einem HF-Feld)."
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
if "violates check constraint" in low and "source" in low:
|
||||||
|
out["code"] = "db_check_constraint_source"
|
||||||
|
out["hint"] = (
|
||||||
|
"Die Tabelle erlaubt den gesetzten «source»-Wert nicht. "
|
||||||
|
"System-Vorlage / Migration zur erlaubten Quelle prüfen (z. B. csv für Universal-Import)."
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
if "current transaction is aborted" in low:
|
||||||
|
out["code"] = "transaction_aborted"
|
||||||
|
out["hint"] = (
|
||||||
|
"Eine frühere Zeile hat einen Datenbankfehler ausgelöst. "
|
||||||
|
"Zuerst die niedrigste Zeilennummer in error_details beheben (Vorlage/Daten prüfen)."
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
if "invalid input syntax" in low and "time" in low:
|
||||||
|
out["code"] = "db_time_cast"
|
||||||
|
out["hint"] = (
|
||||||
|
"start_time/end_time passen nicht zum erwarteten Zeitformat in der Datenbank. "
|
||||||
|
"Vorlage: Datums- und Zeitanteil konsistent (oft nur Uhrzeit, wenn date separat)."
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
if module == "activity" and "foreign key" in low:
|
||||||
|
out["code"] = "db_foreign_key"
|
||||||
|
out["hint"] = "Verknüpfung zur Datenbank verletzt (z. B. training_type). Support kontaktieren."
|
||||||
|
|
||||||
|
return out
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
"""
|
||||||
|
Zeilenaggregation nach CSV-Mapping (group_by + aggregates), vor dem DB-Upsert.
|
||||||
|
|
||||||
|
Spezifikation in der Vorlage (import_row_processing JSONB). Optional: Modul-Default
|
||||||
|
(import_row_processing_default in module_registry) nur als **Legacy-Fallback**, wenn
|
||||||
|
die Vorlage nichts speichert — mittelfristig sollen Vorlagen explizit sein.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import statistics
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from csv_parser.module_registry import get_module_definition
|
||||||
|
|
||||||
|
ALLOWED_AGGREGATES = frozenset({"sum", "mean", "min", "max", "median", "first", "last"})
|
||||||
|
# Mehr als eine CSV-Zeile pro group_by-Schlüssel
|
||||||
|
ALLOWED_MULTI_ROW_POLICIES = frozenset({"aggregate", "reject", "first_row", "last_row"})
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_import_row_processing(module: str, mapping_row: Mapping[str, Any]) -> dict[str, Any] | None:
|
||||||
|
"""Explizite Vorlage hat Vorrang; sonst Modul-Default; leeres Dict zählt wie „nicht gesetzt“."""
|
||||||
|
raw = mapping_row.get("import_row_processing")
|
||||||
|
if isinstance(raw, dict) and raw:
|
||||||
|
return dict(raw)
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
return None
|
||||||
|
default = mod.get("import_row_processing_default")
|
||||||
|
if isinstance(default, dict) and default:
|
||||||
|
return dict(default)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def validate_import_row_processing(
|
||||||
|
module: str,
|
||||||
|
spec: Mapping[str, Any],
|
||||||
|
field_mappings: Mapping[str, Any],
|
||||||
|
) -> None:
|
||||||
|
"""Wirft ValueError bei ungültiger Konfiguration."""
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
raise ValueError(f"Unbekanntes Modul: {module}")
|
||||||
|
allowed = set(mod.get("fields") or [])
|
||||||
|
fm_targets = {str(v) for v in field_mappings.values() if v and v not in ("-", "_skip")}
|
||||||
|
|
||||||
|
group_by = spec.get("group_by") or []
|
||||||
|
if not isinstance(group_by, list) or not all(isinstance(x, str) for x in group_by):
|
||||||
|
raise ValueError("import_row_processing.group_by muss eine Liste von Feldnamen sein")
|
||||||
|
aggregates = spec.get("aggregates") or {}
|
||||||
|
if not isinstance(aggregates, dict):
|
||||||
|
raise ValueError("import_row_processing.aggregates muss ein Objekt sein")
|
||||||
|
|
||||||
|
for g in group_by:
|
||||||
|
if g not in allowed:
|
||||||
|
raise ValueError(f"group_by: unbekanntes Feld '{g}' für Modul '{module}'")
|
||||||
|
if g not in fm_targets:
|
||||||
|
raise ValueError(
|
||||||
|
f"group_by: Zielfeld '{g}' ist keiner CSV-Spalte zugeordnet — Aggregation nicht möglich."
|
||||||
|
)
|
||||||
|
|
||||||
|
for field, op in aggregates.items():
|
||||||
|
if field not in allowed:
|
||||||
|
raise ValueError(f"aggregates: unbekanntes Feld '{field}' für Modul '{module}'")
|
||||||
|
if str(op) not in ALLOWED_AGGREGATES:
|
||||||
|
raise ValueError(
|
||||||
|
f"aggregates['{field}']: ungültige Operation '{op}'. "
|
||||||
|
f"Erlaubt: {', '.join(sorted(ALLOWED_AGGREGATES))}"
|
||||||
|
)
|
||||||
|
|
||||||
|
mrp = spec.get("multi_row_policy")
|
||||||
|
if mrp is not None and str(mrp) not in ALLOWED_MULTI_ROW_POLICIES:
|
||||||
|
raise ValueError(
|
||||||
|
f"multi_row_policy: ungültiger Wert '{mrp}'. "
|
||||||
|
f"Erlaubt: {', '.join(sorted(ALLOWED_MULTI_ROW_POLICIES))}"
|
||||||
|
)
|
||||||
|
|
||||||
|
dedupe = spec.get("dedupe_identical_rows")
|
||||||
|
if dedupe is not None and not isinstance(dedupe, bool):
|
||||||
|
raise ValueError("dedupe_identical_rows muss ein Boolean sein")
|
||||||
|
|
||||||
|
|
||||||
|
def _sort_key_for_group(v: Any) -> Any:
|
||||||
|
if isinstance(v, dt.datetime):
|
||||||
|
return v.isoformat()
|
||||||
|
if isinstance(v, dt.date):
|
||||||
|
return v.isoformat()
|
||||||
|
if isinstance(v, dt.time):
|
||||||
|
return v.isoformat()
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_aggregate(op: str, values: list[Any]) -> Any:
|
||||||
|
nums: list[float] = []
|
||||||
|
for x in values:
|
||||||
|
if x is None or x == "":
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
nums.append(float(x))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
|
||||||
|
if op == "sum":
|
||||||
|
return sum(nums) if nums else None
|
||||||
|
if op == "mean":
|
||||||
|
return statistics.mean(nums) if nums else None
|
||||||
|
if op == "median":
|
||||||
|
return float(statistics.median(nums)) if nums else None
|
||||||
|
if op == "min":
|
||||||
|
return min(nums) if nums else None
|
||||||
|
if op == "max":
|
||||||
|
return max(nums) if nums else None
|
||||||
|
if op == "first":
|
||||||
|
for x in values:
|
||||||
|
if x is not None and x != "":
|
||||||
|
return x
|
||||||
|
return None
|
||||||
|
if op == "last":
|
||||||
|
for x in reversed(values):
|
||||||
|
if x is not None and x != "":
|
||||||
|
return x
|
||||||
|
return None
|
||||||
|
raise ValueError(f"Unbekannte Aggregations-Operation: {op}")
|
||||||
|
|
||||||
|
|
||||||
|
def _row_identity_signature(r: dict[str, Any]) -> tuple[Any, ...]:
|
||||||
|
return tuple(sorted((k, _sort_key_for_group(r.get(k))) for k in sorted(r.keys())))
|
||||||
|
|
||||||
|
|
||||||
|
def _dedupe_identical_mapped_rows(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||||
|
"""Exakt gleiche gemappte Zeilen (alle Keys/Werte) — erste behalten."""
|
||||||
|
seen: set[tuple[Any, ...]] = set()
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
for r in rows:
|
||||||
|
sig = _row_identity_signature(r)
|
||||||
|
if sig in seen:
|
||||||
|
continue
|
||||||
|
seen.add(sig)
|
||||||
|
out.append(r)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def aggregate_mapped_rows(
|
||||||
|
rows: list[dict[str, Any]],
|
||||||
|
spec: Mapping[str, Any],
|
||||||
|
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
||||||
|
"""
|
||||||
|
Gruppiert gemappte Zeilen-Dicts nach group_by und wendet aggregates an.
|
||||||
|
Felder, die weder in group_by noch in aggregates vorkommen: Wert aus der ersten Zeile der Gruppe.
|
||||||
|
|
||||||
|
Rückgabe: (merged_rows, strukturelle Fehler / Hinweise, z. B. abgelehnte Schlüsselgruppen).
|
||||||
|
"""
|
||||||
|
errors: list[dict[str, Any]] = []
|
||||||
|
rows = list(rows)
|
||||||
|
if spec.get("dedupe_identical_rows"):
|
||||||
|
rows = _dedupe_identical_mapped_rows(rows)
|
||||||
|
|
||||||
|
group_by = spec.get("group_by") or []
|
||||||
|
aggregates = spec.get("aggregates") or {}
|
||||||
|
policy = str(spec.get("multi_row_policy") or "aggregate")
|
||||||
|
if policy not in ALLOWED_MULTI_ROW_POLICIES:
|
||||||
|
policy = "aggregate"
|
||||||
|
|
||||||
|
if not group_by:
|
||||||
|
return rows, errors
|
||||||
|
|
||||||
|
buckets: dict[tuple[Any, ...], list[dict[str, Any]]] = {}
|
||||||
|
order: list[tuple[Any, ...]] = []
|
||||||
|
for r in rows:
|
||||||
|
key = tuple(_sort_key_for_group(r.get(g)) for g in group_by)
|
||||||
|
if key not in buckets:
|
||||||
|
buckets[key] = []
|
||||||
|
order.append(key)
|
||||||
|
buckets[key].append(r)
|
||||||
|
|
||||||
|
gb_label = ", ".join(group_by)
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
for key in order:
|
||||||
|
group_rows = buckets[key]
|
||||||
|
if len(group_rows) > 1:
|
||||||
|
if policy == "reject":
|
||||||
|
errors.append(
|
||||||
|
{
|
||||||
|
"error": "mehrere_zeilen_pro_schluessel",
|
||||||
|
"message": (
|
||||||
|
f"{len(group_rows)} CSV-Zeilen mit gleichem Schlüssel ({gb_label}); "
|
||||||
|
"laut Vorlage abgelehnt (multi_row_policy=reject)."
|
||||||
|
),
|
||||||
|
"rows_in_group": len(group_rows),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if policy == "first_row":
|
||||||
|
group_rows = [group_rows[0]]
|
||||||
|
elif policy == "last_row":
|
||||||
|
group_rows = [group_rows[-1]]
|
||||||
|
|
||||||
|
first = group_rows[0]
|
||||||
|
merged: dict[str, Any] = {}
|
||||||
|
for g in group_by:
|
||||||
|
merged[g] = first.get(g)
|
||||||
|
for field, op in aggregates.items():
|
||||||
|
merged[field] = _apply_aggregate(str(op), [row.get(field) for row in group_rows])
|
||||||
|
for row in group_rows:
|
||||||
|
for k, v in row.items():
|
||||||
|
if k in merged:
|
||||||
|
continue
|
||||||
|
if k in group_by or k in aggregates:
|
||||||
|
continue
|
||||||
|
merged[k] = v
|
||||||
|
out.append(merged)
|
||||||
|
return out, errors
|
||||||
@@ -0,0 +1,240 @@
|
|||||||
|
"""
|
||||||
|
Heuristische Vorschläge für CSV field_mappings / type_conversions (Admin-Editor, Issue #21).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from copy import deepcopy
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from csv_parser.core import normalize_header_for_signature
|
||||||
|
from csv_parser.module_registry import get_module_definition
|
||||||
|
|
||||||
|
# Normalisierte Header-Fragmente → DB-Feld (Substring- oder exakter Norm-Vergleich)
|
||||||
|
_MODULE_HEADER_ALIASES: dict[str, dict[str, frozenset[str]]] = {
|
||||||
|
"nutrition": {
|
||||||
|
"date": frozenset(
|
||||||
|
{"datum", "date", "tag", "day", "zeit", "timestamp", "uhrzeit", "monat", "jahr"}
|
||||||
|
),
|
||||||
|
"kcal": frozenset({"kcal", "kalorie", "calorie", "energie", "energy", "kj", "joule"}),
|
||||||
|
"protein_g": frozenset({"protein", "eiwei", "eiweiss"}),
|
||||||
|
"fat_g": frozenset({"fett", "fat", "lipid"}),
|
||||||
|
"carbs_g": frozenset({"kh", "carb", "kohlenhydr", "carbs", "sugar", "zucker"}),
|
||||||
|
},
|
||||||
|
"weight": {
|
||||||
|
"date": frozenset({"datum", "date", "tag", "day", "zeit"}),
|
||||||
|
"weight": frozenset({"gewicht", "weight", "masse", "kg", "kilo"}),
|
||||||
|
"note": frozenset({"notiz", "note", "comment", "kommentar"}),
|
||||||
|
},
|
||||||
|
"blood_pressure": {
|
||||||
|
"measured_date": frozenset({"datum", "date", "tag", "day", "messdatum"}),
|
||||||
|
"measured_time": frozenset({"zeit", "time", "uhr", "uhrzeit"}),
|
||||||
|
"systolic": frozenset({"systol", "sys", "sbp", "oberdruck"}),
|
||||||
|
"diastolic": frozenset({"diastol", "dia", "dbp", "unterdruck"}),
|
||||||
|
"pulse": frozenset({"puls", "pulse", "hr", "herz", "bpm"}),
|
||||||
|
},
|
||||||
|
"activity": {
|
||||||
|
"date": frozenset({"datum", "date", "tag", "day"}),
|
||||||
|
"start_time": frozenset({"start", "beginn", "von"}),
|
||||||
|
"end_time": frozenset({"end", "ende", "bis", "stop"}),
|
||||||
|
"activity_type": frozenset({"workout", "training", "typ", "type", "art", "aktiv"}),
|
||||||
|
"duration_min": frozenset({"dauer", "duration", "min"}),
|
||||||
|
"distance_km": frozenset({"strecke", "distance", "km", "distanz"}),
|
||||||
|
"kcal_active": frozenset({"kcal", "kalorie", "energie", "active"}),
|
||||||
|
"kcal_resting": frozenset({"ruhe", "resting"}),
|
||||||
|
"hr_avg": frozenset({"puls", "heart", "hr", "bpm", "herzfrequenz", "durchschn"}),
|
||||||
|
"hr_max": frozenset({"max", "peak"}),
|
||||||
|
},
|
||||||
|
"vitals_baseline": {
|
||||||
|
"date": frozenset({"datum", "date", "tag", "start", "zeit"}),
|
||||||
|
"resting_hr": frozenset({"ruhepuls", "resting", "rhr"}),
|
||||||
|
"hrv": frozenset({"hrv", "variabilit", "vfc"}),
|
||||||
|
"vo2_max": frozenset({"vo2"}),
|
||||||
|
"spo2": frozenset({"sauerstoff", "spo2", "oxygen"}),
|
||||||
|
"respiratory_rate": frozenset({"atem", "respiratory"}),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
_DEFAULT_TYPE_CONVERSIONS: dict[str, dict[str, dict[str, Any]]] = {
|
||||||
|
"nutrition": {
|
||||||
|
"date": {"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only", "flexible": True},
|
||||||
|
"kcal": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
},
|
||||||
|
"weight": {
|
||||||
|
"date": {"type": "date", "format": "dd.mm.yyyy", "flexible": True},
|
||||||
|
"weight": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"note": {"type": "string"},
|
||||||
|
},
|
||||||
|
"blood_pressure": {
|
||||||
|
"measured_date": {"type": "date", "format": "dd.mm.yyyy", "flexible": True},
|
||||||
|
"measured_time": {"type": "time", "format": "HH:MM", "flexible": True},
|
||||||
|
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
|
||||||
|
"systolic": {"type": "int", "flexible": True},
|
||||||
|
"diastolic": {"type": "int", "flexible": True},
|
||||||
|
"pulse": {"type": "int", "flexible": True},
|
||||||
|
},
|
||||||
|
"activity": {
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
|
||||||
|
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
|
||||||
|
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
|
||||||
|
"activity_type": {"type": "string"},
|
||||||
|
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes", "flexible": True},
|
||||||
|
"distance_km": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"kcal_active": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"kcal_resting": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"hr_avg": {"type": "int", "flexible": True},
|
||||||
|
"hr_max": {"type": "int", "flexible": True},
|
||||||
|
},
|
||||||
|
"vitals_baseline": {
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": True},
|
||||||
|
"hrv": {"type": "int", "flexible": True},
|
||||||
|
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"spo2": {"type": "int", "flexible": True},
|
||||||
|
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _norm_key(header: str) -> str:
|
||||||
|
return normalize_header_for_signature(header)
|
||||||
|
|
||||||
|
|
||||||
|
def _match_seed_to_db_field(header: str, seed_fm: Mapping[str, str]) -> str | None:
|
||||||
|
"""Findet Ziel-Feld, wenn Seed-Key zu diesem Header passt (exakt oder normalisiert)."""
|
||||||
|
if header in seed_fm:
|
||||||
|
v = seed_fm[header]
|
||||||
|
if v and v not in ("-", "_skip"):
|
||||||
|
return v
|
||||||
|
nh = _norm_key(header)
|
||||||
|
if nh in seed_fm:
|
||||||
|
v = seed_fm[nh]
|
||||||
|
if v and v not in ("-", "_skip"):
|
||||||
|
return v
|
||||||
|
for sk, sv in seed_fm.items():
|
||||||
|
if not sv or sv in ("-", "_skip"):
|
||||||
|
continue
|
||||||
|
if _norm_key(str(sk)) == nh:
|
||||||
|
return sv
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _alias_suggest(norm: str, module: str, used: set[str]) -> str | None:
|
||||||
|
aliases = _MODULE_HEADER_ALIASES.get(module, {})
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
return None
|
||||||
|
field_order = list(mod["fields"].keys())
|
||||||
|
for db_field in field_order:
|
||||||
|
if db_field in used:
|
||||||
|
continue
|
||||||
|
tokens = aliases.get(db_field, frozenset())
|
||||||
|
nlow = norm.lower()
|
||||||
|
if nlow == db_field or nlow.replace("_", "") == db_field.replace("_", ""):
|
||||||
|
return db_field
|
||||||
|
for tok in tokens:
|
||||||
|
if len(tok) >= 2 and tok in nlow:
|
||||||
|
return db_field
|
||||||
|
if len(tok) >= 4 and tok in norm:
|
||||||
|
return db_field
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def suggest_field_mappings(
|
||||||
|
headers: list[str],
|
||||||
|
module: str,
|
||||||
|
seed_fm: Mapping[str, str] | None = None,
|
||||||
|
) -> dict[str, str]:
|
||||||
|
"""
|
||||||
|
Mappt jede CSV-Spalte (Roh-Header als Key) auf DB-Feld oder '-'.
|
||||||
|
Nutzt zuerst eine passende Seed-Vorlage, dann Alias-Heuristik.
|
||||||
|
"""
|
||||||
|
if module == "sleep":
|
||||||
|
return {h: "-" for h in headers}
|
||||||
|
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
return {h: "-" for h in headers}
|
||||||
|
|
||||||
|
fm: dict[str, str] = {h: "-" for h in headers}
|
||||||
|
used: set[str] = set()
|
||||||
|
|
||||||
|
if seed_fm:
|
||||||
|
for h in headers:
|
||||||
|
db = _match_seed_to_db_field(h, seed_fm)
|
||||||
|
if db and db not in used:
|
||||||
|
fm[h] = db
|
||||||
|
used.add(db)
|
||||||
|
|
||||||
|
for h in headers:
|
||||||
|
if fm[h] != "-":
|
||||||
|
continue
|
||||||
|
norm = _norm_key(h)
|
||||||
|
db = _alias_suggest(norm, module, used)
|
||||||
|
if db:
|
||||||
|
fm[h] = db
|
||||||
|
used.add(db)
|
||||||
|
|
||||||
|
return fm
|
||||||
|
|
||||||
|
|
||||||
|
def build_type_conversions_for_mapping(
|
||||||
|
module: str,
|
||||||
|
field_mappings: Mapping[str, str],
|
||||||
|
seed_tc: Mapping[str, Any] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""type_conversions nur für zugewiesene Zielfelder; Seed überschreibt Defaults."""
|
||||||
|
if module == "sleep":
|
||||||
|
return {}
|
||||||
|
|
||||||
|
defaults = _DEFAULT_TYPE_CONVERSIONS.get(module, {})
|
||||||
|
out: dict[str, Any] = {}
|
||||||
|
targets = {v for v in field_mappings.values() if v and v not in ("-", "_skip")}
|
||||||
|
|
||||||
|
if seed_tc:
|
||||||
|
for k, v in seed_tc.items():
|
||||||
|
if k in targets and isinstance(v, dict):
|
||||||
|
out[k] = deepcopy(v)
|
||||||
|
|
||||||
|
for t in targets:
|
||||||
|
if t not in out and t in defaults:
|
||||||
|
out[t] = deepcopy(defaults[t])
|
||||||
|
|
||||||
|
_apply_energy_kj_hint_from_headers(module, field_mappings, out)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
_ENERGY_FIELDS = frozenset({"kcal", "kcal_active", "kcal_resting"})
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_energy_kj_hint_from_headers(
|
||||||
|
module: str,
|
||||||
|
field_mappings: Mapping[str, str],
|
||||||
|
out: dict[str, Any],
|
||||||
|
) -> None:
|
||||||
|
"""Wenn Überschrift kJ/Kilojoule nahelegt (nicht kcal), source_unit kj setzen (FDDB & Co.)."""
|
||||||
|
if module not in ("nutrition", "activity"):
|
||||||
|
return
|
||||||
|
for csv_col, db_field in field_mappings.items():
|
||||||
|
if db_field not in _ENERGY_FIELDS:
|
||||||
|
continue
|
||||||
|
spec = out.get(db_field)
|
||||||
|
if not isinstance(spec, dict):
|
||||||
|
continue
|
||||||
|
if spec.get("source_unit"):
|
||||||
|
continue
|
||||||
|
norm = normalize_header_for_signature(str(csv_col)).lower()
|
||||||
|
if "kcal" in norm:
|
||||||
|
continue
|
||||||
|
if "kj" in norm or "kilojoule" in norm:
|
||||||
|
spec2 = deepcopy(spec)
|
||||||
|
spec2["source_unit"] = "kj"
|
||||||
|
out[db_field] = spec2
|
||||||
@@ -0,0 +1,161 @@
|
|||||||
|
"""
|
||||||
|
Ziel-Module für CSV-Import: Tabellen-Felder, Pflichtfelder, Duplikat-Strategie (Issue #21).
|
||||||
|
|
||||||
|
Hinweis: blood_pressure nutzt in der DB measured_at; Logik-Felder measured_date + measured_time
|
||||||
|
werden im Executor zu measured_at zusammengefügt (Phase Import-Executor).
|
||||||
|
|
||||||
|
Activity: date kann aus start_time (ISO-Datetime) abgeleitet werden, wenn nur start_time gesetzt ist.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Dict, cast
|
||||||
|
|
||||||
|
MODULE_DEFINITIONS: Dict[str, Dict[str, Any]] = {
|
||||||
|
"nutrition": {
|
||||||
|
"table": "nutrition_log",
|
||||||
|
"fields": {
|
||||||
|
"date": {"type": "date", "required": True},
|
||||||
|
"kcal": {"type": "float", "required": False, "unit": "kcal"},
|
||||||
|
"protein_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
|
||||||
|
"fat_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
|
||||||
|
"carbs_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
|
||||||
|
},
|
||||||
|
"duplicate_key": ["profile_id", "date"],
|
||||||
|
"duplicate_strategy": "update",
|
||||||
|
# Legacy-Fallback wenn die Vorlage kein import_row_processing speichert — Vorlagen mittelfristig explizit.
|
||||||
|
"import_row_processing_default": {
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {
|
||||||
|
"kcal": "sum",
|
||||||
|
"protein_g": "sum",
|
||||||
|
"fat_g": "sum",
|
||||||
|
"carbs_g": "sum",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"activity": {
|
||||||
|
"table": "activity_log",
|
||||||
|
"fields": {
|
||||||
|
"date": {"type": "date", "required": False},
|
||||||
|
"start_time": {"type": "datetime", "required": False},
|
||||||
|
"end_time": {"type": "datetime", "required": False},
|
||||||
|
"activity_type": {"type": "string", "required": True},
|
||||||
|
"duration_min": {"type": "float", "required": False, "min": 0},
|
||||||
|
"kcal_active": {"type": "float", "required": False, "unit": "kcal"},
|
||||||
|
"kcal_resting": {"type": "float", "required": False, "unit": "kcal"},
|
||||||
|
"distance_km": {"type": "float", "required": False, "unit": "km"},
|
||||||
|
"hr_avg": {"type": "float", "required": False, "min": 30, "max": 220},
|
||||||
|
"hr_max": {"type": "float", "required": False, "min": 30, "max": 220},
|
||||||
|
},
|
||||||
|
"derive_date_from_datetime_field": "start_time",
|
||||||
|
"duplicate_key": ["profile_id", "date", "start_time"],
|
||||||
|
"duplicate_strategy": "update",
|
||||||
|
},
|
||||||
|
"sleep": {
|
||||||
|
"table": "sleep_log",
|
||||||
|
"fields": {},
|
||||||
|
"import_mode": "apple_sleep_aggregate",
|
||||||
|
},
|
||||||
|
"vitals_baseline": {
|
||||||
|
"table": "vitals_baseline",
|
||||||
|
"fields": {
|
||||||
|
"date": {"type": "date", "required": True},
|
||||||
|
"resting_hr": {"type": "int", "required": False},
|
||||||
|
"hrv": {"type": "int", "required": False},
|
||||||
|
"vo2_max": {"type": "float", "required": False},
|
||||||
|
"spo2": {"type": "int", "required": False},
|
||||||
|
"respiratory_rate": {"type": "float", "required": False},
|
||||||
|
},
|
||||||
|
"duplicate_key": ["profile_id", "date"],
|
||||||
|
"duplicate_strategy": "update",
|
||||||
|
# Legacy-Fallback — Vorlagen mittelfristig explizit setzen.
|
||||||
|
"import_row_processing_default": {
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {
|
||||||
|
"resting_hr": "mean",
|
||||||
|
"hrv": "mean",
|
||||||
|
"vo2_max": "mean",
|
||||||
|
"spo2": "mean",
|
||||||
|
"respiratory_rate": "mean",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"blood_pressure": {
|
||||||
|
"table": "blood_pressure_log",
|
||||||
|
"fields": {
|
||||||
|
"measured_date": {"type": "date", "required": True},
|
||||||
|
"measured_time": {"type": "time", "required": True},
|
||||||
|
# Apple Health: eine Spalte „Start“ / „Datum/Uhrzeit“ (Datetime); Executor splittet.
|
||||||
|
"start_time": {"type": "datetime", "required": False},
|
||||||
|
"systolic": {"type": "int", "required": True},
|
||||||
|
"diastolic": {"type": "int", "required": True},
|
||||||
|
"pulse": {"type": "int", "required": False},
|
||||||
|
},
|
||||||
|
"logical_to_db": "blood_pressure_composite_measured_at",
|
||||||
|
"duplicate_key": ["profile_id", "measured_at"],
|
||||||
|
"duplicate_strategy": "update",
|
||||||
|
},
|
||||||
|
"weight": {
|
||||||
|
"table": "weight_log",
|
||||||
|
"fields": {
|
||||||
|
"date": {"type": "date", "required": True},
|
||||||
|
"weight": {"type": "float", "required": True, "min": 20, "max": 400, "unit": "kg"},
|
||||||
|
"note": {"type": "string", "required": False, "max_length": 2000},
|
||||||
|
},
|
||||||
|
"duplicate_key": ["profile_id", "date"],
|
||||||
|
"duplicate_strategy": "update",
|
||||||
|
# Legacy-Fallback — Vorlagen mittelfristig explizit setzen.
|
||||||
|
"import_row_processing_default": {
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {
|
||||||
|
"weight": "last",
|
||||||
|
"note": "last",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_module_definition(module: str) -> Dict[str, Any] | None:
|
||||||
|
return MODULE_DEFINITIONS.get(module)
|
||||||
|
|
||||||
|
|
||||||
|
def list_modules() -> list[str]:
|
||||||
|
return sorted(MODULE_DEFINITIONS.keys())
|
||||||
|
|
||||||
|
|
||||||
|
def validate_field_mappings(module: str, field_mappings: dict) -> None:
|
||||||
|
"""Wirft ValueError bei unbekanntem Modul oder unbekanntem DB-Feld."""
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
raise ValueError(f"Unbekanntes Modul: {module}")
|
||||||
|
fields = cast(dict, mod["fields"])
|
||||||
|
allowed = set(fields.keys())
|
||||||
|
if not allowed:
|
||||||
|
for _csv_col, db_field in field_mappings.items():
|
||||||
|
if db_field not in ("", None, "-", "_skip"):
|
||||||
|
raise ValueError(
|
||||||
|
f"Modul '{module}' nutzt einen Aggregat-Import ohne Spalten-Mapping; "
|
||||||
|
f"alle Spalten müssen „ignorieren“ sein."
|
||||||
|
)
|
||||||
|
return
|
||||||
|
for _csv_col, db_field in field_mappings.items():
|
||||||
|
if db_field in ("", None, "-", "_skip"):
|
||||||
|
continue
|
||||||
|
if db_field not in allowed:
|
||||||
|
raise ValueError(f"Ungültiges Zielfeld '{db_field}' für Modul '{module}'")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_required_field_targets(module: str, field_mappings: dict) -> None:
|
||||||
|
"""Stellt sicher, dass jedes als required markierte Zielfeld mindestens einer Spalte zugeordnet ist."""
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
raise ValueError(f"Unbekanntes Modul: {module}")
|
||||||
|
field_defs = cast(dict, mod["fields"])
|
||||||
|
targets = {v for v in field_mappings.values() if v and v not in ("-", "_skip")}
|
||||||
|
if module == "blood_pressure" and "start_time" in targets:
|
||||||
|
targets = set(targets) | {"measured_date", "measured_time"}
|
||||||
|
for fname, finfo in field_defs.items():
|
||||||
|
if finfo.get("required") and fname not in targets:
|
||||||
|
raise ValueError(f"Pflicht-Zielfeld nicht zugeordnet: {fname}")
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
"""Zugriffsregeln für csv_field_mappings (Issue #21)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
|
||||||
|
def user_may_edit_mapping_row(row: Mapping[str, Any], session: Mapping[str, Any]) -> bool:
|
||||||
|
if session.get("role") == "admin":
|
||||||
|
return True
|
||||||
|
if row.get("is_system"):
|
||||||
|
return False
|
||||||
|
return str(row.get("profile_id")) == str(session.get("profile_id"))
|
||||||
|
|
||||||
|
|
||||||
|
def user_may_delete_mapping(row: Mapping[str, Any], session: Mapping[str, Any]) -> bool:
|
||||||
|
if row.get("is_system"):
|
||||||
|
return False
|
||||||
|
return str(row.get("profile_id")) == str(session.get("profile_id"))
|
||||||
@@ -0,0 +1,350 @@
|
|||||||
|
"""
|
||||||
|
Apple-Health-Schlaf-CSV → sleep_log (für Universal-Import und /api/sleep/import).
|
||||||
|
Nutzt dieselbe Logik wie der Sleep-Router, ohne HTTPException — arbeitet auf einem übergebenen Cursor.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import csv
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from datetime import date, datetime
|
||||||
|
from decimal import Decimal, InvalidOperation
|
||||||
|
from typing import Any, Literal
|
||||||
|
|
||||||
|
from dateutil import parser as dateutil_parser
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_row_keys(row: dict) -> dict:
|
||||||
|
return {(k or "").strip(): (v.strip() if isinstance(v, str) else v) for k, v in row.items()}
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_float(value: Any) -> float | None:
|
||||||
|
if value is None or value == "":
|
||||||
|
return None
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
return float(value)
|
||||||
|
if isinstance(value, Decimal):
|
||||||
|
return float(value)
|
||||||
|
try:
|
||||||
|
s = str(value).strip().replace(",", ".")
|
||||||
|
return float(s)
|
||||||
|
except (ValueError, InvalidOperation):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_apple_sleep_datetime(value: str) -> datetime:
|
||||||
|
raw = (value or "").strip()
|
||||||
|
if not raw:
|
||||||
|
raise ValueError("empty datetime")
|
||||||
|
fmts = (
|
||||||
|
"%Y-%m-%d %H:%M:%S %z",
|
||||||
|
"%d.%m.%y %H:%M:%S",
|
||||||
|
"%d.%m.%Y %H:%M:%S",
|
||||||
|
"%Y-%m-%d %H:%M:%S",
|
||||||
|
)
|
||||||
|
for fmt in fmts:
|
||||||
|
try:
|
||||||
|
return datetime.strptime(raw, fmt)
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
return dateutil_parser.parse(raw, dayfirst=False)
|
||||||
|
except (ValueError, TypeError, OverflowError) as e:
|
||||||
|
raise ValueError(f"Unbekanntes Datumsformat: {raw!r}") from e
|
||||||
|
|
||||||
|
|
||||||
|
def _hr_to_minutes(hours: float | None) -> int:
|
||||||
|
if hours is None:
|
||||||
|
return 0
|
||||||
|
return int(round(float(hours) * 60))
|
||||||
|
|
||||||
|
|
||||||
|
def detect_apple_sleep_csv_format(fieldnames: list[str] | None) -> Literal["segments", "summary"]:
|
||||||
|
if not fieldnames:
|
||||||
|
raise ValueError("CSV enthält keine Spaltenüberschriften.")
|
||||||
|
fn = {(f or "").strip() for f in fieldnames}
|
||||||
|
if {"Start", "End", "Duration (hr)", "Value"}.issubset(fn):
|
||||||
|
return "segments"
|
||||||
|
if "Start" in fn and "End" in fn and "Total Sleep (hr)" in fn:
|
||||||
|
return "summary"
|
||||||
|
raise ValueError(
|
||||||
|
"Unbekanntes Apple-Health-Schlaf-CSV. Erwartet: Segment-Export oder Schlafanalyse-Zusammenfassung."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_nights_from_apple_summary(reader: csv.DictReader) -> dict[date, dict]:
|
||||||
|
nights_dict: dict[date, dict] = {}
|
||||||
|
for raw in reader:
|
||||||
|
row = _strip_row_keys(raw)
|
||||||
|
start_s = row.get("Start") or ""
|
||||||
|
end_s = row.get("End") or ""
|
||||||
|
if not start_s or not end_s:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
start_dt = _parse_apple_sleep_datetime(start_s)
|
||||||
|
end_dt = _parse_apple_sleep_datetime(end_s)
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
dt_key = (row.get("Date/Time") or row.get("Datum/Uhrzeit") or "").strip()
|
||||||
|
if dt_key:
|
||||||
|
try:
|
||||||
|
wake_d = datetime.strptime(dt_key[:10], "%Y-%m-%d").date()
|
||||||
|
except ValueError:
|
||||||
|
wake_d = end_dt.date()
|
||||||
|
else:
|
||||||
|
wake_d = end_dt.date()
|
||||||
|
|
||||||
|
core_hr = _safe_float(row.get("Core (hr)"))
|
||||||
|
if core_hr is None:
|
||||||
|
core_hr = _safe_float(row.get("Light (hr)")) or 0.0
|
||||||
|
deep_min = _hr_to_minutes(_safe_float(row.get("Deep (hr)")))
|
||||||
|
rem_min = _hr_to_minutes(_safe_float(row.get("REM (hr)")))
|
||||||
|
light_min = _hr_to_minutes(core_hr)
|
||||||
|
awake_min = _hr_to_minutes(_safe_float(row.get("Awake (hr)")))
|
||||||
|
total_sleep_hr = _safe_float(row.get("Total Sleep (hr)"))
|
||||||
|
nights_dict[wake_d] = {
|
||||||
|
"bedtime": start_dt,
|
||||||
|
"wake_time": end_dt,
|
||||||
|
"segments": [],
|
||||||
|
"deep_minutes": deep_min,
|
||||||
|
"rem_minutes": rem_min,
|
||||||
|
"light_minutes": light_min,
|
||||||
|
"awake_minutes": awake_min,
|
||||||
|
"total_sleep_hr": total_sleep_hr,
|
||||||
|
}
|
||||||
|
return nights_dict
|
||||||
|
|
||||||
|
|
||||||
|
def _build_nights_from_apple_segments(reader: csv.DictReader, phase_map: dict) -> dict[date, dict]:
|
||||||
|
segments = []
|
||||||
|
for raw in reader:
|
||||||
|
row = _strip_row_keys(raw)
|
||||||
|
phase_key = (row.get("Value") or "").strip()
|
||||||
|
phase_en = phase_map.get(phase_key)
|
||||||
|
if phase_en is None:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
start_dt = _parse_apple_sleep_datetime(row.get("Start") or "")
|
||||||
|
end_dt = _parse_apple_sleep_datetime(row.get("End") or "")
|
||||||
|
duration_hr = _safe_float(row.get("Duration (hr)"))
|
||||||
|
if duration_hr is None:
|
||||||
|
continue
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
continue
|
||||||
|
duration_min = int(duration_hr * 60)
|
||||||
|
segments.append({
|
||||||
|
"start": start_dt,
|
||||||
|
"end": end_dt,
|
||||||
|
"duration_min": duration_min,
|
||||||
|
"phase": phase_en,
|
||||||
|
})
|
||||||
|
|
||||||
|
segments.sort(key=lambda s: s["start"])
|
||||||
|
nights = []
|
||||||
|
current_night = None
|
||||||
|
|
||||||
|
for seg in segments:
|
||||||
|
if current_night is None or (seg["start"] - current_night["wake_time"]).total_seconds() > 7200:
|
||||||
|
current_night = {
|
||||||
|
"bedtime": seg["start"],
|
||||||
|
"wake_time": seg["end"],
|
||||||
|
"segments": [],
|
||||||
|
"deep_minutes": 0,
|
||||||
|
"rem_minutes": 0,
|
||||||
|
"light_minutes": 0,
|
||||||
|
"awake_minutes": 0,
|
||||||
|
}
|
||||||
|
nights.append(current_night)
|
||||||
|
|
||||||
|
current_night["segments"].append(seg)
|
||||||
|
current_night["wake_time"] = max(current_night["wake_time"], seg["end"])
|
||||||
|
current_night["bedtime"] = min(current_night["bedtime"], seg["start"])
|
||||||
|
|
||||||
|
if seg["phase"] == "deep":
|
||||||
|
current_night["deep_minutes"] += seg["duration_min"]
|
||||||
|
elif seg["phase"] == "rem":
|
||||||
|
current_night["rem_minutes"] += seg["duration_min"]
|
||||||
|
elif seg["phase"] == "light":
|
||||||
|
current_night["light_minutes"] += seg["duration_min"]
|
||||||
|
elif seg["phase"] == "awake":
|
||||||
|
current_night["awake_minutes"] += seg["duration_min"]
|
||||||
|
|
||||||
|
nights_dict: dict[date, dict] = {}
|
||||||
|
for night in nights:
|
||||||
|
wake_date = night["wake_time"].date()
|
||||||
|
nights_dict[wake_date] = night
|
||||||
|
return nights_dict
|
||||||
|
|
||||||
|
|
||||||
|
def import_apple_sleep_nights(cur, profile_id: str, text: str) -> dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Schreibt in sleep_log. Kein conn.commit — Aufrufer rollt Transaktion.
|
||||||
|
Gibt Statistik im Executor-Format zurück.
|
||||||
|
"""
|
||||||
|
csv_text = text.replace("\r\n", "\n").replace("\r", "\n")
|
||||||
|
if csv_text.startswith("\ufeff"):
|
||||||
|
csv_text = csv_text[1:]
|
||||||
|
reader = csv.DictReader(io.StringIO(csv_text))
|
||||||
|
fmt = detect_apple_sleep_csv_format(reader.fieldnames)
|
||||||
|
|
||||||
|
phase_map = {
|
||||||
|
"Kern": "light",
|
||||||
|
"Core": "light",
|
||||||
|
"Light": "light",
|
||||||
|
"REM": "rem",
|
||||||
|
"Tief": "deep",
|
||||||
|
"Deep": "deep",
|
||||||
|
"Wach": "awake",
|
||||||
|
"Awake": "awake",
|
||||||
|
"Schlafend": None,
|
||||||
|
"Asleep": None,
|
||||||
|
"In Bed": None,
|
||||||
|
}
|
||||||
|
|
||||||
|
if fmt == "summary":
|
||||||
|
nights_dict = _build_nights_from_apple_summary(reader)
|
||||||
|
else:
|
||||||
|
nights_dict = _build_nights_from_apple_segments(reader, phase_map)
|
||||||
|
|
||||||
|
if not nights_dict:
|
||||||
|
raise ValueError(
|
||||||
|
"Keine importierbaren Schlafzeilen gefunden (prüfe Start/Ende und Format)."
|
||||||
|
)
|
||||||
|
|
||||||
|
inserted = 0
|
||||||
|
updated = 0
|
||||||
|
skipped = 0
|
||||||
|
error_details: list[dict[str, Any]] = []
|
||||||
|
affected_ids: list[str] = []
|
||||||
|
|
||||||
|
row_hint = 0
|
||||||
|
for wake_date, night in nights_dict.items():
|
||||||
|
row_hint += 1
|
||||||
|
phase_sum = (
|
||||||
|
night["deep_minutes"] + night["rem_minutes"] + night["light_minutes"]
|
||||||
|
)
|
||||||
|
total_hr = night.get("total_sleep_hr")
|
||||||
|
fallback_min = int(round(float(total_hr) * 60)) if total_hr is not None else 0
|
||||||
|
duration_minutes = phase_sum if phase_sum > 0 else fallback_min
|
||||||
|
if duration_minutes <= 0:
|
||||||
|
logger.warning(
|
||||||
|
"Sleep import: überspringe %s — Dauer 0 (Phasen-Summe und Total Sleep (hr) leer/0).",
|
||||||
|
wake_date,
|
||||||
|
)
|
||||||
|
error_details.append(
|
||||||
|
{
|
||||||
|
"row": row_hint,
|
||||||
|
"error": f"Schlafdauer für {wake_date} ist 0 — Phasen oder Total Sleep (hr) fehlen.",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
wake_count = sum(1 for seg in night["segments"] if seg["phase"] == "awake")
|
||||||
|
|
||||||
|
sleep_segments = [
|
||||||
|
{
|
||||||
|
"phase": seg["phase"],
|
||||||
|
"start": seg["start"].isoformat(),
|
||||||
|
"end": seg["end"].isoformat(),
|
||||||
|
"duration_min": seg["duration_min"],
|
||||||
|
}
|
||||||
|
for seg in night["segments"]
|
||||||
|
]
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT id, source FROM sleep_log
|
||||||
|
WHERE profile_id = %s AND date = %s
|
||||||
|
""",
|
||||||
|
(profile_id, wake_date),
|
||||||
|
)
|
||||||
|
existing = cur.fetchone()
|
||||||
|
|
||||||
|
if existing and existing["source"] == "manual":
|
||||||
|
skipped += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
if existing:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
UPDATE sleep_log SET
|
||||||
|
bedtime = %s,
|
||||||
|
wake_time = %s,
|
||||||
|
duration_minutes = %s,
|
||||||
|
wake_count = %s,
|
||||||
|
deep_minutes = %s,
|
||||||
|
rem_minutes = %s,
|
||||||
|
light_minutes = %s,
|
||||||
|
awake_minutes = %s,
|
||||||
|
sleep_segments = %s,
|
||||||
|
source = 'apple_health',
|
||||||
|
updated_at = CURRENT_TIMESTAMP
|
||||||
|
WHERE id = %s AND profile_id = %s
|
||||||
|
RETURNING id
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
night["bedtime"].time(),
|
||||||
|
night["wake_time"].time(),
|
||||||
|
duration_minutes,
|
||||||
|
wake_count,
|
||||||
|
night["deep_minutes"],
|
||||||
|
night["rem_minutes"],
|
||||||
|
night["light_minutes"],
|
||||||
|
night["awake_minutes"],
|
||||||
|
json.dumps(sleep_segments),
|
||||||
|
existing["id"],
|
||||||
|
profile_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
row = cur.fetchone()
|
||||||
|
updated += 1
|
||||||
|
if row and row.get("id"):
|
||||||
|
affected_ids.append(str(row["id"]))
|
||||||
|
else:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO sleep_log (
|
||||||
|
profile_id, date, bedtime, wake_time, duration_minutes,
|
||||||
|
wake_count, deep_minutes, rem_minutes, light_minutes, awake_minutes,
|
||||||
|
sleep_segments, source, created_at, updated_at
|
||||||
|
) VALUES (
|
||||||
|
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, 'apple_health', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
RETURNING id
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
profile_id,
|
||||||
|
wake_date,
|
||||||
|
night["bedtime"].time(),
|
||||||
|
night["wake_time"].time(),
|
||||||
|
duration_minutes,
|
||||||
|
wake_count,
|
||||||
|
night["deep_minutes"],
|
||||||
|
night["rem_minutes"],
|
||||||
|
night["light_minutes"],
|
||||||
|
night["awake_minutes"],
|
||||||
|
json.dumps(sleep_segments),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
row = cur.fetchone()
|
||||||
|
inserted += 1
|
||||||
|
if row and row.get("id"):
|
||||||
|
affected_ids.append(str(row["id"]))
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("Sleep import row failed: %s", e)
|
||||||
|
error_details.append({"row": row_hint, "error": str(e)})
|
||||||
|
|
||||||
|
return {
|
||||||
|
"rows_total": len(nights_dict),
|
||||||
|
"inserted": inserted,
|
||||||
|
"updated": updated,
|
||||||
|
"skipped": skipped,
|
||||||
|
"new_entries": inserted,
|
||||||
|
"error_details": error_details,
|
||||||
|
"affected_ids": affected_ids,
|
||||||
|
}
|
||||||
@@ -0,0 +1,235 @@
|
|||||||
|
"""
|
||||||
|
Formatprüfung für CSV-Import-Vorlagen (field_mappings, type_conversions).
|
||||||
|
|
||||||
|
Liefert strukturierte Fehler/Warnungen für Admin-UI und Speicher-Guards.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from csv_parser.core import normalize_header_for_signature
|
||||||
|
from csv_parser.import_row_processing import validate_import_row_processing as validate_import_row_processing_spec
|
||||||
|
from csv_parser.module_registry import (
|
||||||
|
get_module_definition,
|
||||||
|
validate_field_mappings,
|
||||||
|
validate_required_field_targets,
|
||||||
|
)
|
||||||
|
|
||||||
|
ALLOWED_SPEC_TYPES = frozenset(
|
||||||
|
{"string", "float", "number", "int", "date", "time", "datetime", "duration"}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _issue(
|
||||||
|
severity: str,
|
||||||
|
code: str,
|
||||||
|
message: str,
|
||||||
|
*,
|
||||||
|
hint: str | None = None,
|
||||||
|
field: str | None = None,
|
||||||
|
csv_columns: list[str] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
out: dict[str, Any] = {
|
||||||
|
"severity": severity,
|
||||||
|
"code": code,
|
||||||
|
"message": message,
|
||||||
|
}
|
||||||
|
if hint:
|
||||||
|
out["hint"] = hint
|
||||||
|
if field:
|
||||||
|
out["field"] = field
|
||||||
|
if csv_columns:
|
||||||
|
out["csv_columns"] = csv_columns
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def validate_csv_template(
|
||||||
|
module: str,
|
||||||
|
field_mappings: Mapping[str, Any] | None,
|
||||||
|
type_conversions: Mapping[str, Any] | None = None,
|
||||||
|
import_row_processing: Mapping[str, Any] | None = None,
|
||||||
|
column_signature: list[str] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Prüft eine Vorlage ohne Datei-Upload.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
``{"valid": bool, "errors": [...], "warnings": [...]}``
|
||||||
|
"""
|
||||||
|
errors: list[dict[str, Any]] = []
|
||||||
|
warnings: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
fm = dict(field_mappings or {})
|
||||||
|
tc: dict[str, Any] = dict(type_conversions or {}) if type_conversions else {}
|
||||||
|
mod = get_module_definition(module)
|
||||||
|
if not mod:
|
||||||
|
errors.append(
|
||||||
|
_issue(
|
||||||
|
"error",
|
||||||
|
"unknown_module",
|
||||||
|
f"Unbekanntes Modul «{module}».",
|
||||||
|
hint="Nur registrierte Module in module_registry sind erlaubt.",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return {"valid": False, "errors": errors, "warnings": warnings}
|
||||||
|
|
||||||
|
try:
|
||||||
|
validate_field_mappings(module, fm)
|
||||||
|
except ValueError as e:
|
||||||
|
errors.append(
|
||||||
|
_issue(
|
||||||
|
"error",
|
||||||
|
"invalid_field_mapping",
|
||||||
|
str(e),
|
||||||
|
hint="Jede Zuordnung muss auf ein bekanntes Zielfeld des Moduls zeigen (oder „–“ / ignorieren).",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
validate_required_field_targets(module, fm)
|
||||||
|
except ValueError as e:
|
||||||
|
errors.append(
|
||||||
|
_issue(
|
||||||
|
"error",
|
||||||
|
"missing_required_target",
|
||||||
|
str(e),
|
||||||
|
hint="Pflichtfelder des Moduls müssen mindestens einer CSV-Spalte zugeordnet sein.",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
if import_row_processing:
|
||||||
|
try:
|
||||||
|
validate_import_row_processing_spec(module, import_row_processing, fm)
|
||||||
|
except ValueError as e:
|
||||||
|
errors.append(
|
||||||
|
_issue(
|
||||||
|
"error",
|
||||||
|
"invalid_import_row_processing",
|
||||||
|
str(e),
|
||||||
|
hint="import_row_processing: group_by und aggregates prüfen (siehe Doku Issue #21).",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
field_defs = mod.get("fields") or {}
|
||||||
|
for db_field, spec in tc.items():
|
||||||
|
if db_field not in field_defs:
|
||||||
|
errors.append(
|
||||||
|
_issue(
|
||||||
|
"error",
|
||||||
|
"unknown_type_conversion_field",
|
||||||
|
f"type_conversions enthält unbekanntes Zielfeld «{db_field}».",
|
||||||
|
hint="Nur Felder aus der Moduldefinition sind erlaubt.",
|
||||||
|
field=db_field,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
if not isinstance(spec, Mapping):
|
||||||
|
errors.append(
|
||||||
|
_issue(
|
||||||
|
"error",
|
||||||
|
"type_conversion_not_object",
|
||||||
|
f"type_conversions[\"{db_field}\"] muss ein JSON-Objekt sein.",
|
||||||
|
field=db_field,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
stype = spec.get("type", "string")
|
||||||
|
if stype not in ALLOWED_SPEC_TYPES:
|
||||||
|
warnings.append(
|
||||||
|
_issue(
|
||||||
|
"warning",
|
||||||
|
"unusual_conversion_type",
|
||||||
|
f"Ungewöhnlicher Typ «{stype}» für «{db_field}» (erwartet u. a. string, float, date, datetime).",
|
||||||
|
field=db_field,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
finfo = field_defs.get(db_field) or {}
|
||||||
|
expected = finfo.get("type")
|
||||||
|
if expected == "date" and stype not in ("date", "datetime"):
|
||||||
|
warnings.append(
|
||||||
|
_issue(
|
||||||
|
"warning",
|
||||||
|
"date_field_conversion",
|
||||||
|
f"Zielfeld «{db_field}» ist ein Datum; der Konvertierungstyp ist «{stype}».",
|
||||||
|
hint="Meist «date» oder «datetime» mit passendem format.",
|
||||||
|
field=db_field,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if expected == "float" and stype == "int" and db_field in ("hr_avg", "hr_max"):
|
||||||
|
warnings.append(
|
||||||
|
_issue(
|
||||||
|
"warning",
|
||||||
|
"hr_as_int",
|
||||||
|
"Herzfrequenz als «int» konvertiert; Nachkommastellen aus Apple-Export gehen verloren.",
|
||||||
|
hint="Optional «float» mit flexible: true verwenden.",
|
||||||
|
field=db_field,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Mehrere CSV-Spalten → dasselbe Zielfeld
|
||||||
|
by_target: dict[str, list[str]] = {}
|
||||||
|
for csv_col, dbf in fm.items():
|
||||||
|
if dbf in (None, "", "-", "_skip"):
|
||||||
|
continue
|
||||||
|
by_target.setdefault(str(dbf), []).append(str(csv_col))
|
||||||
|
for dbf, cols in by_target.items():
|
||||||
|
if len(cols) > 1:
|
||||||
|
warnings.append(
|
||||||
|
_issue(
|
||||||
|
"warning",
|
||||||
|
"duplicate_target_columns",
|
||||||
|
f"Mehrere Spalten mappen auf «{dbf}»: {', '.join(cols)}.",
|
||||||
|
hint="Beim Import gewinnt die letzte Spalte in der CSV-Kopfzeilen-Reihenfolge.",
|
||||||
|
field=dbf,
|
||||||
|
csv_columns=cols,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Kilojoule in kcal-Feldern (häufiger Apple-DE-Fehler)
|
||||||
|
for csv_col, dbf in fm.items():
|
||||||
|
if dbf not in ("kcal_active", "kcal_resting"):
|
||||||
|
continue
|
||||||
|
col_l = str(csv_col).lower()
|
||||||
|
if "kj" in col_l or "kilojoule" in col_l:
|
||||||
|
sub = tc.get(dbf)
|
||||||
|
su = (sub or {}).get("source_unit") if isinstance(sub, Mapping) else None
|
||||||
|
if str(su or "").strip().lower() != "kj":
|
||||||
|
warnings.append(
|
||||||
|
_issue(
|
||||||
|
"warning",
|
||||||
|
"energy_kj_without_source_unit",
|
||||||
|
f"Spalte «{csv_col}» deutet auf Kilojoule, Zielfeld «{dbf}» speichert kcal.",
|
||||||
|
hint='In type_conversions für dieses Feld "source_unit": "kj" setzen (Faktor 1/4.184).',
|
||||||
|
field=str(dbf),
|
||||||
|
csv_columns=[str(csv_col)],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Signatur vs. gemappte Spalten: beide Seiten wie beim Import normalisieren
|
||||||
|
# (column_signature kann sortierte Normalform aus Analyse sein, field_mappings rohe Header).
|
||||||
|
if column_signature:
|
||||||
|
sig_forms = {
|
||||||
|
normalize_header_for_signature(str(c))
|
||||||
|
for c in column_signature
|
||||||
|
if str(c).strip()
|
||||||
|
}
|
||||||
|
sig_forms.discard("")
|
||||||
|
mapped_forms = {
|
||||||
|
normalize_header_for_signature(str(k))
|
||||||
|
for k in fm.keys()
|
||||||
|
if str(k).strip()
|
||||||
|
}
|
||||||
|
mapped_forms.discard("")
|
||||||
|
if sig_forms and mapped_forms and not sig_forms.intersection(mapped_forms):
|
||||||
|
warnings.append(
|
||||||
|
_issue(
|
||||||
|
"warning",
|
||||||
|
"signature_vs_mappings_mismatch",
|
||||||
|
"column_signature und field_mappings (Schlüssel) haben nach Normalisierung keine gemeinsame Spalte.",
|
||||||
|
hint="Prüfen Sie, ob die gespeicherte Signatur zur CSV passt; Zuordnungen nutzen rohe Kopfzeilen, die Signatur oft die gleiche Normalform wie in der Analyse.",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return {"valid": len(errors) == 0, "errors": errors, "warnings": warnings}
|
||||||
@@ -0,0 +1,675 @@
|
|||||||
|
"""
|
||||||
|
Typkonvertierung für CSV-Zellen gemäß type_conversions-JSON (Issue #21).
|
||||||
|
|
||||||
|
Locale-robust: dieselbe Vorlage kann Exporte mit wechselndem Datumsformat oder
|
||||||
|
Dezimaltrenner verarbeiten, wenn flexible oder auto-Optionen gesetzt sind.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import re
|
||||||
|
from decimal import Decimal, InvalidOperation
|
||||||
|
from typing import Any, Mapping, Sequence
|
||||||
|
|
||||||
|
from dateutil import parser as dateutil_parser
|
||||||
|
|
||||||
|
from csv_parser.core import normalize_header_for_signature
|
||||||
|
from csv_parser.field_units import factor_source_to_canonical
|
||||||
|
|
||||||
|
# Alias → strptime (JSON in Kleinbuchstaben)
|
||||||
|
DATE_FORMAT_STRPTIME: dict[str, str] = {
|
||||||
|
"yyyy-mm-dd": "%Y-%m-%d",
|
||||||
|
"mm/dd/yyyy": "%m/%d/%Y",
|
||||||
|
"dd/mm/yyyy": "%d/%m/%Y",
|
||||||
|
"dd.mm.yyyy": "%d.%m.%Y",
|
||||||
|
"dd.mm.yyyy hh:mm": "%d.%m.%Y %H:%M",
|
||||||
|
"dd.mm.yyyy HH:MM": "%d.%m.%Y %H:%M",
|
||||||
|
"yyyy-mm-dd hh:mm:ss": "%Y-%m-%d %H:%M:%S",
|
||||||
|
"yyyy-mm-dd HH:MM:SS": "%Y-%m-%d %H:%M:%S",
|
||||||
|
}
|
||||||
|
|
||||||
|
TIME_FORMAT_STRPTIME: dict[str, str] = {
|
||||||
|
"HH:MM": "%H:%M",
|
||||||
|
"HH:MM:SS": "%H:%M:%S",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Wenn flexible: zusätzliche strptime-Versuche (ungefähr häufig → seltener)
|
||||||
|
_STRPTIME_FALLBACK_DATES: list[str] = [
|
||||||
|
"%Y-%m-%d",
|
||||||
|
"%d.%m.%Y",
|
||||||
|
"%d.%m.%y",
|
||||||
|
"%d/%m/%Y",
|
||||||
|
"%m/%d/%Y",
|
||||||
|
"%Y/%m/%d",
|
||||||
|
"%Y%m%d",
|
||||||
|
]
|
||||||
|
_STRPTIME_FALLBACK_DATETIME: list[str] = [
|
||||||
|
"%Y-%m-%d",
|
||||||
|
"%Y-%m-%d %H:%M:%S",
|
||||||
|
"%Y-%m-%d %H:%M",
|
||||||
|
"%d.%m.%Y %H:%M:%S",
|
||||||
|
"%d.%m.%Y %H:%M",
|
||||||
|
"%d.%m.%y %H:%M:%S",
|
||||||
|
"%d.%m.%y %H:%M",
|
||||||
|
"%Y-%m-%dT%H:%M:%S",
|
||||||
|
"%Y-%m-%dT%H:%M:%SZ",
|
||||||
|
"%Y-%m-%dT%H:%M:%S%z",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_num_token(raw: str) -> str:
|
||||||
|
return re.sub(r"[\s\u00a0\u202f]", "", raw.strip())
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_float_auto(s: str) -> float:
|
||||||
|
"""
|
||||||
|
Heuristik ohne festes Locale: Punkt/Komma als Tausender vs. Dezimal,
|
||||||
|
basierend auf der letzten erkannten Trennstelle und Gruppierung.
|
||||||
|
"""
|
||||||
|
s = _normalize_num_token(s)
|
||||||
|
if not s or s in ("-", "—", "–"):
|
||||||
|
raise ValueError("leer")
|
||||||
|
neg = False
|
||||||
|
if s.startswith("(") and s.endswith(")"):
|
||||||
|
neg = True
|
||||||
|
s = s[1:-1].strip()
|
||||||
|
if s.startswith("-"):
|
||||||
|
neg = not neg
|
||||||
|
s = s[1:]
|
||||||
|
elif s.startswith("+"):
|
||||||
|
s = s[1:]
|
||||||
|
|
||||||
|
last_comma = s.rfind(",")
|
||||||
|
last_dot = s.rfind(".")
|
||||||
|
|
||||||
|
if last_comma >= 0 and last_dot >= 0:
|
||||||
|
if last_comma > last_dot:
|
||||||
|
s = s.replace(".", "").replace(",", ".")
|
||||||
|
else:
|
||||||
|
s = s.replace(",", "")
|
||||||
|
elif last_comma >= 0:
|
||||||
|
parts = s.split(",")
|
||||||
|
if len(parts) == 2 and len(parts[1]) <= 2:
|
||||||
|
s = parts[0].replace(".", "") + "." + parts[1]
|
||||||
|
elif len(parts) == 2 and len(parts[1]) == 3 and len(parts[0]) <= 3:
|
||||||
|
s = parts[0] + parts[1]
|
||||||
|
else:
|
||||||
|
s = s.replace(",", "")
|
||||||
|
elif last_dot >= 0:
|
||||||
|
parts = s.split(".")
|
||||||
|
if len(parts) == 2 and len(parts[1]) <= 2:
|
||||||
|
s = parts[0].replace(",", "") + "." + parts[1]
|
||||||
|
elif len(parts) == 2 and len(parts[1]) == 3 and len(parts[0]) <= 3:
|
||||||
|
s = parts[0] + parts[1]
|
||||||
|
elif len(parts) > 2:
|
||||||
|
if len(parts[-1]) <= 2:
|
||||||
|
s = "".join(parts[:-1]) + "." + parts[-1]
|
||||||
|
else:
|
||||||
|
s = "".join(parts)
|
||||||
|
else:
|
||||||
|
s = s.replace(".", "")
|
||||||
|
|
||||||
|
try:
|
||||||
|
v = float(Decimal(s))
|
||||||
|
except (InvalidOperation, ValueError) as e:
|
||||||
|
raise ValueError(f"Zahl nicht parsbar: {raw!r}") from e
|
||||||
|
return -v if neg else v
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_float(raw: str, decimal_sep: str) -> float:
|
||||||
|
s = _normalize_num_token(raw)
|
||||||
|
if not s:
|
||||||
|
raise ValueError("leer")
|
||||||
|
if "." in s and "," in s:
|
||||||
|
return _parse_float_auto(s)
|
||||||
|
if decimal_sep == ",":
|
||||||
|
if "," not in s and "." in s:
|
||||||
|
return _parse_float_auto(s)
|
||||||
|
s = s.replace(".", "").replace(",", ".")
|
||||||
|
else:
|
||||||
|
if "," in s and "." not in s:
|
||||||
|
return _parse_float_auto(s)
|
||||||
|
s = s.replace(",", "")
|
||||||
|
return float(Decimal(s))
|
||||||
|
|
||||||
|
|
||||||
|
def _float_from_spec(raw: str, spec: Mapping[str, Any]) -> float:
|
||||||
|
dec = spec.get("decimal_separator", ".")
|
||||||
|
flexible = bool(spec.get("flexible"))
|
||||||
|
if dec in (None, "auto"):
|
||||||
|
return _parse_float_auto(raw)
|
||||||
|
try:
|
||||||
|
return _parse_float(raw, str(dec))
|
||||||
|
except (InvalidOperation, ValueError):
|
||||||
|
if flexible:
|
||||||
|
return _parse_float_auto(raw)
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_strptime_pattern(fmt_key: str) -> str | None:
|
||||||
|
k = fmt_key.strip()
|
||||||
|
if k.startswith("%"):
|
||||||
|
return k
|
||||||
|
return DATE_FORMAT_STRPTIME.get(k.lower())
|
||||||
|
|
||||||
|
|
||||||
|
def _collect_strptime_date_formats(spec: Mapping[str, Any], *, for_datetime: bool) -> list[str]:
|
||||||
|
seen: set[str] = set()
|
||||||
|
out: list[str] = []
|
||||||
|
|
||||||
|
def add(fmt_key: str) -> None:
|
||||||
|
p = _resolve_strptime_pattern(fmt_key)
|
||||||
|
if p and p not in seen:
|
||||||
|
seen.add(p)
|
||||||
|
out.append(p)
|
||||||
|
if not for_datetime and p.endswith(" %H:%M") and "%H:%M:%S" not in p:
|
||||||
|
p2 = p + ":%S"
|
||||||
|
if p2 not in seen:
|
||||||
|
seen.add(p2)
|
||||||
|
out.append(p2)
|
||||||
|
elif for_datetime and p.endswith(":%S"):
|
||||||
|
# z. B. Apple Health „2026-04-09 16:48“ ohne Sekunden
|
||||||
|
p_short = p[:-3]
|
||||||
|
if p_short not in seen:
|
||||||
|
seen.add(p_short)
|
||||||
|
out.append(p_short)
|
||||||
|
|
||||||
|
primary = spec.get("format")
|
||||||
|
if primary:
|
||||||
|
add(str(primary))
|
||||||
|
extra = spec.get("formats")
|
||||||
|
if isinstance(extra, Sequence) and not isinstance(extra, (str, bytes)):
|
||||||
|
for item in extra:
|
||||||
|
if item:
|
||||||
|
add(str(item))
|
||||||
|
|
||||||
|
if bool(spec.get("flexible")):
|
||||||
|
for p in _STRPTIME_FALLBACK_DATETIME if for_datetime else _STRPTIME_FALLBACK_DATES:
|
||||||
|
if p not in seen:
|
||||||
|
seen.add(p)
|
||||||
|
out.append(p)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _try_strptime(s: str, patterns: Sequence[str]) -> dt.datetime | None:
|
||||||
|
for pat in patterns:
|
||||||
|
try:
|
||||||
|
return dt.datetime.strptime(s, pat)
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _try_strptime_trim_time(s: str, patterns: Sequence[str]) -> dt.datetime | None:
|
||||||
|
head = s.split(maxsplit=1)[0].strip() if s else ""
|
||||||
|
if head and head != s:
|
||||||
|
hit = _try_strptime(head, patterns)
|
||||||
|
if hit:
|
||||||
|
return hit
|
||||||
|
return _try_strptime(s, patterns)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_locale_date_months(s: str) -> str:
|
||||||
|
"""
|
||||||
|
Omron Connect / Berichte: «10 Apr. 2026», «31 März 2026» — ohne DE→EN scheitert dateutil.
|
||||||
|
"""
|
||||||
|
if not s:
|
||||||
|
return s
|
||||||
|
out = s
|
||||||
|
for pat, rep in (
|
||||||
|
(r"März", "March"),
|
||||||
|
(r"Maerz", "March"),
|
||||||
|
(r"Januar", "January"),
|
||||||
|
(r"Februar", "February"),
|
||||||
|
(r"Oktober", "October"),
|
||||||
|
(r"Dezember", "December"),
|
||||||
|
(r"Juni", "June"),
|
||||||
|
(r"Juli", "July"),
|
||||||
|
(r"\bMai\b", "May"),
|
||||||
|
):
|
||||||
|
out = re.sub(pat, rep, out, flags=re.IGNORECASE)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _dateutil_parse(s: str, spec: Mapping[str, Any]) -> dt.datetime | None:
|
||||||
|
s_trim = s.strip()
|
||||||
|
dayfirst_opt = spec.get("dayfirst")
|
||||||
|
# ISO YYYY-MM-DD: dayfirst=True vertauscht Monat/Tag (09.04. → 04.09.)
|
||||||
|
iso_ymd_prefix = bool(re.match(r"^\d{4}-\d{2}-\d{2}(\D|$)", s_trim))
|
||||||
|
tries: list[bool | None]
|
||||||
|
if dayfirst_opt is True:
|
||||||
|
tries = [True]
|
||||||
|
elif dayfirst_opt is False:
|
||||||
|
tries = [False]
|
||||||
|
else:
|
||||||
|
tries = [False, True] if iso_ymd_prefix else [True, False]
|
||||||
|
for df in tries:
|
||||||
|
try:
|
||||||
|
return dateutil_parser.parse(s_trim, dayfirst=df)
|
||||||
|
except (ValueError, TypeError, OverflowError):
|
||||||
|
continue
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_date_typed(s: str, spec: Mapping[str, Any]) -> dt.date | dt.datetime:
|
||||||
|
extract = spec.get("extract", "date_only")
|
||||||
|
s0 = _normalize_locale_date_months(s.strip())
|
||||||
|
patterns = _collect_strptime_date_formats(spec, for_datetime=False)
|
||||||
|
part = _try_strptime_trim_time(s0, patterns) if patterns else None
|
||||||
|
if part is None:
|
||||||
|
part = _try_strptime(s0, _collect_strptime_date_formats(spec, for_datetime=True))
|
||||||
|
if part is None and (bool(spec.get("flexible")) or spec.get("formats")):
|
||||||
|
part = _dateutil_parse(s0, spec)
|
||||||
|
if part is None:
|
||||||
|
merged: dict[str, Any] = {**dict(spec), "flexible": True}
|
||||||
|
if "dayfirst" not in merged:
|
||||||
|
merged["dayfirst"] = True
|
||||||
|
part = _dateutil_parse(s0, merged)
|
||||||
|
if part is None:
|
||||||
|
fmt_key = str(spec.get("format", ""))
|
||||||
|
raise ValueError(f"Datum nicht parsbar: {fmt_key} / {s!r}")
|
||||||
|
if extract == "date_only":
|
||||||
|
return part.date()
|
||||||
|
return part
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_datetime_typed(s: str, spec: Mapping[str, Any]) -> dt.datetime:
|
||||||
|
s0 = _normalize_locale_date_months(s.strip())
|
||||||
|
patterns = _collect_strptime_date_formats(spec, for_datetime=True)
|
||||||
|
part = _try_strptime(s0, patterns)
|
||||||
|
if part is None and (bool(spec.get("flexible")) or spec.get("formats")):
|
||||||
|
du = _dateutil_parse(s0, spec)
|
||||||
|
if du:
|
||||||
|
part = du
|
||||||
|
if part is None:
|
||||||
|
merged: dict[str, Any] = {**dict(spec), "flexible": True}
|
||||||
|
if "dayfirst" not in merged:
|
||||||
|
merged["dayfirst"] = True
|
||||||
|
du = _dateutil_parse(s0, merged)
|
||||||
|
if du:
|
||||||
|
part = du
|
||||||
|
if part is None:
|
||||||
|
fmt_key = str(spec.get("format", ""))
|
||||||
|
raise ValueError(f"Datetime nicht parsbar: {fmt_key} / {s!r}")
|
||||||
|
return part
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_time_typed(s: str, spec: Mapping[str, Any]) -> dt.time:
|
||||||
|
patterns: list[str] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
primary = spec.get("format")
|
||||||
|
if primary:
|
||||||
|
fk = str(primary)
|
||||||
|
p = TIME_FORMAT_STRPTIME.get(fk, _resolve_strptime_pattern(fk) or fk)
|
||||||
|
if p not in seen:
|
||||||
|
seen.add(p)
|
||||||
|
patterns.append(p)
|
||||||
|
extra = spec.get("formats")
|
||||||
|
if isinstance(extra, Sequence) and not isinstance(extra, (str, bytes)):
|
||||||
|
for item in extra:
|
||||||
|
if not item:
|
||||||
|
continue
|
||||||
|
p = TIME_FORMAT_STRPTIME.get(str(item), str(item))
|
||||||
|
if p not in seen:
|
||||||
|
seen.add(p)
|
||||||
|
patterns.append(p)
|
||||||
|
if bool(spec.get("flexible")):
|
||||||
|
for p in ("%H:%M:%S", "%H:%M"):
|
||||||
|
if p not in seen:
|
||||||
|
seen.add(p)
|
||||||
|
patterns.append(p)
|
||||||
|
part = _try_strptime(s.strip(), patterns)
|
||||||
|
if part is None:
|
||||||
|
raise ValueError(f"Zeit nicht parsbar: {s!r}")
|
||||||
|
return part.time()
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_int(raw: str, spec: Mapping[str, Any]) -> int:
|
||||||
|
s = raw.strip()
|
||||||
|
if bool(spec.get("flexible")) or spec.get("thousands_separator") == "auto":
|
||||||
|
s2 = _normalize_num_token(s)
|
||||||
|
if not s2 or s2 in ("-", "—", "–"):
|
||||||
|
raise ValueError("leer")
|
||||||
|
# EU-Dezimal (z. B. Apple DE «37,26» für HRV) — nicht alle Ziffern konkatenieren (würde 3726 → CHECK).
|
||||||
|
if "," in s2 or "." in s2:
|
||||||
|
try:
|
||||||
|
fv = _parse_float_auto(s2)
|
||||||
|
return int(round(fv))
|
||||||
|
except (ValueError, InvalidOperation):
|
||||||
|
pass
|
||||||
|
neg = s2.startswith("-")
|
||||||
|
body = s2[1:] if neg else s2
|
||||||
|
digits = re.sub(r"\D", "", body)
|
||||||
|
if not digits:
|
||||||
|
raise ValueError("leer")
|
||||||
|
v = int(digits)
|
||||||
|
return -v if neg else v
|
||||||
|
s = re.sub(r"[^\d-]", "", s)
|
||||||
|
if not s:
|
||||||
|
raise ValueError("leer")
|
||||||
|
return int(s)
|
||||||
|
|
||||||
|
|
||||||
|
def convert_value(
|
||||||
|
raw: str,
|
||||||
|
db_field: str,
|
||||||
|
spec: Mapping[str, Any] | None,
|
||||||
|
module: str | None = None,
|
||||||
|
) -> Any:
|
||||||
|
"""
|
||||||
|
Konvertiert eine Roh-Zelle in einen Python-Wert.
|
||||||
|
spec kommt aus type_conversions[db_field].
|
||||||
|
|
||||||
|
Optionen (JSON):
|
||||||
|
- flexible: true — nach Primärformat Fallbacks (Datum/Zahl/Zeit/Duration).
|
||||||
|
- decimal_separator: ".", ",", "auto" — bei auto Heuristik EU/US-Mischformen.
|
||||||
|
- formats: [ "yyyy-mm-dd", "%d.%m.%y", ... ] — weitere strptime-/Alias-Ketten.
|
||||||
|
- dayfirst: true|false — nur für dateutil-Fallback; Standard: true dann false.
|
||||||
|
- source_unit: Registry-IDs (z. B. "kj", "kg") oder "custom"/"none" — letztere ohne
|
||||||
|
vordefinierten Faktor; beliebige Skalierung dann nur über conversion_factor.
|
||||||
|
- conversion_factor: Zusätzlicher Multiplikator nach dem Parsen (und nach source_unit);
|
||||||
|
für nicht vordefinierte Umrechnungen source_unit weglassen oder "custom" setzen und
|
||||||
|
hier den Faktor angeben.
|
||||||
|
"""
|
||||||
|
if spec is None:
|
||||||
|
return raw.strip() if raw else None
|
||||||
|
if raw is None:
|
||||||
|
return None
|
||||||
|
s = raw.strip()
|
||||||
|
if s == "":
|
||||||
|
return None
|
||||||
|
|
||||||
|
t = spec.get("type", "string")
|
||||||
|
if t == "string":
|
||||||
|
return s
|
||||||
|
|
||||||
|
if t in ("float", "number"):
|
||||||
|
v = _float_from_spec(raw, spec)
|
||||||
|
if module:
|
||||||
|
su = spec.get("source_unit")
|
||||||
|
if su is not None and str(su).strip() != "":
|
||||||
|
v = float(v) * factor_source_to_canonical(module, db_field, str(su))
|
||||||
|
factor = spec.get("conversion_factor")
|
||||||
|
if factor is not None:
|
||||||
|
v = float(v) * float(factor)
|
||||||
|
return v
|
||||||
|
|
||||||
|
if t == "int":
|
||||||
|
return _parse_int(raw, spec)
|
||||||
|
|
||||||
|
if t == "date":
|
||||||
|
return _parse_date_typed(s, spec)
|
||||||
|
|
||||||
|
if t == "time":
|
||||||
|
return _parse_time_typed(s, spec)
|
||||||
|
|
||||||
|
if t == "datetime":
|
||||||
|
return _parse_datetime_typed(s, spec)
|
||||||
|
|
||||||
|
if t == "duration":
|
||||||
|
target = spec.get("target_unit", "minutes")
|
||||||
|
parts = [p.strip() for p in s.split(":")]
|
||||||
|
flexible = bool(spec.get("flexible"))
|
||||||
|
if len(parts) == 3:
|
||||||
|
try:
|
||||||
|
h, m, sec = int(parts[0]), int(parts[1]), int(parts[2])
|
||||||
|
if target == "minutes":
|
||||||
|
return round(h * 60 + m + sec / 60.0, 4)
|
||||||
|
except ValueError:
|
||||||
|
if not flexible:
|
||||||
|
raise ValueError(f"Duration nicht parsbar: {s!r}") from None
|
||||||
|
if flexible:
|
||||||
|
try:
|
||||||
|
h, m, sec = int(parts[0]), int(parts[1]), float(parts[2])
|
||||||
|
if target == "minutes":
|
||||||
|
return round(h * 60 + m + sec / 60.0, 4)
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
if len(parts) == 2:
|
||||||
|
try:
|
||||||
|
h, m = int(parts[0]), int(parts[1])
|
||||||
|
return h * 60 + m
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
raise ValueError(f"Duration nicht parsbar: {s!r}")
|
||||||
|
|
||||||
|
return s
|
||||||
|
|
||||||
|
|
||||||
|
def _lookup_db_field(csv_col: str, field_mappings: Mapping[str, str]) -> str | None:
|
||||||
|
"""
|
||||||
|
CSV-Spaltennamen können Roh-Header sein; Vorlagen-Schlüssel oft normalisiert
|
||||||
|
(wie column_signature). Exakter Treffer, dann Schlüssel nach Normalisierung,
|
||||||
|
dann Abgleich aller Vorlagen-Keys über deren Normalform.
|
||||||
|
"""
|
||||||
|
v = field_mappings.get(csv_col)
|
||||||
|
if v:
|
||||||
|
return v if v not in ("-", "_skip") else None
|
||||||
|
norm = normalize_header_for_signature(csv_col)
|
||||||
|
v = field_mappings.get(norm)
|
||||||
|
if v:
|
||||||
|
return v if v not in ("-", "_skip") else None
|
||||||
|
for k, fv in field_mappings.items():
|
||||||
|
if normalize_header_for_signature(str(k)) == norm:
|
||||||
|
return fv if fv not in ("-", "_skip") else None
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _vitals_baseline_alias_db_field(csv_col: str) -> str | None:
|
||||||
|
"""
|
||||||
|
Apple Health: deutsch „Vitalwerte.csv“ (Breitexport) vs. schmale Vorlage
|
||||||
|
(Start / Resting Heart Rate …). Ohne Alias wählt die Analyse oft die
|
||||||
|
englische Vorlage → jede Zeile „Datum fehlt“.
|
||||||
|
Abgleich über normalisierten Header (normalize_header_for_signature).
|
||||||
|
"""
|
||||||
|
n = normalize_header_for_signature(str(csv_col))
|
||||||
|
if n in ("datum_uhrzeit", "start", "date_time", "datetime"):
|
||||||
|
return "date"
|
||||||
|
if "ruhepuls" in n or n.startswith("resting_heart_rate"):
|
||||||
|
return "resting_hr"
|
||||||
|
if "herzfrequenzvariabilit" in n or "heart_rate_variability" in n:
|
||||||
|
return "hrv"
|
||||||
|
if "vo2" in n and "max" in n:
|
||||||
|
return "vo2_max"
|
||||||
|
if "blutsauerstoff" in n or "oxygen_saturation" in n:
|
||||||
|
return "spo2"
|
||||||
|
if "atemfrequenz" in n or "respiratory_rate" in n:
|
||||||
|
return "respiratory_rate"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _blood_pressure_alias_db_field(csv_col: str) -> str | None:
|
||||||
|
"""
|
||||||
|
Omron (schmal) vs. Apple-Gesundheit (Breitexport): unterschiedliche Spaltennamen;
|
||||||
|
kombinierte Messzeit oft als „Start“ oder „Datum/Uhrzeit“.
|
||||||
|
"""
|
||||||
|
n = normalize_header_for_signature(str(csv_col))
|
||||||
|
low = str(csv_col).lower()
|
||||||
|
if n in ("datum_uhrzeit", "datetime", "date_time", "messzeitpunkt"):
|
||||||
|
return "start_time"
|
||||||
|
if n in ("start", "beginn"):
|
||||||
|
return "start_time"
|
||||||
|
if n in ("datum", "date", "messdatum"):
|
||||||
|
return "measured_date"
|
||||||
|
if n in ("zeit", "time", "uhrzeit"):
|
||||||
|
return "measured_time"
|
||||||
|
if "systolisch" in n or ("blutdruck" in n and "systol" in low) or n.startswith("systolic"):
|
||||||
|
return "systolic"
|
||||||
|
if "diastolisch" in n or ("blutdruck" in n and "diastol" in low) or n.startswith("diastolic"):
|
||||||
|
return "diastolic"
|
||||||
|
if n.startswith("puls") or n.startswith("pulse") or "puls_" in n:
|
||||||
|
return "pulse"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _activity_alias_db_field(csv_col: str) -> str | None:
|
||||||
|
"""
|
||||||
|
Apple-Workout schmal vs. Breitexport (viele Spalten): Trainingsart/Dauer/Strecke
|
||||||
|
trotzdem zuverlässig erkennen.
|
||||||
|
"""
|
||||||
|
n = normalize_header_for_signature(str(csv_col))
|
||||||
|
low = str(csv_col).lower()
|
||||||
|
if n in ("trainingsart", "workout_type", "activity_type", "workouttype"):
|
||||||
|
return "activity_type"
|
||||||
|
if ("trainings" in n and "art" in n) or ("workout" in low and "type" in low):
|
||||||
|
return "activity_type"
|
||||||
|
if n in ("datum_uhrzeit", "start", "beginn", "startzeit", "von"):
|
||||||
|
return "start_time"
|
||||||
|
if n in ("ende", "end", "endzeit", "bis"):
|
||||||
|
return "end_time"
|
||||||
|
if n in ("date", "datum"):
|
||||||
|
return "date"
|
||||||
|
if "dauer" in n or n == "duration" or n.startswith("duration_"):
|
||||||
|
return "duration_min"
|
||||||
|
if ("strecke" in n or "distance" in low) and ("km" in low or "(km" in low or " km" in low):
|
||||||
|
return "distance_km"
|
||||||
|
if "aktive_energie" in n or "active_energy" in n:
|
||||||
|
return "kcal_active"
|
||||||
|
if "ruheeintr" in n or "ruheenergie" in n or "resting_energy" in n:
|
||||||
|
return "kcal_resting"
|
||||||
|
if ("herzfrequenz" in n or "heart_rate" in n) and ("max" in low or "max" in n):
|
||||||
|
return "hr_max"
|
||||||
|
if (
|
||||||
|
"durchschnittliche_herzfrequenz" in n
|
||||||
|
or "heart_rate_average" in n
|
||||||
|
or ("herzfrequenz" in n and ("durchschn" in n or "avg" in low or "average" in low))
|
||||||
|
or ("heart_rate" in n and ("avg" in low or "average" in low))
|
||||||
|
):
|
||||||
|
return "hr_avg"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _effective_conversion_spec(
|
||||||
|
db_field: str,
|
||||||
|
spec: Mapping[str, Any] | None,
|
||||||
|
module: str | None,
|
||||||
|
) -> Mapping[str, Any] | None:
|
||||||
|
if spec is not None:
|
||||||
|
return spec
|
||||||
|
if module == "blood_pressure" and db_field == "start_time":
|
||||||
|
return {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True}
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def build_row_after_mapping(
|
||||||
|
csv_row: Mapping[str, str],
|
||||||
|
field_mappings: Mapping[str, str],
|
||||||
|
type_conversions: Mapping[str, Any] | None,
|
||||||
|
module: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Wendet Zuordnung csv_spalte → db_feld und Typkonvertierung an.
|
||||||
|
Unzugeordnete oder „—“ werden übersprungen.
|
||||||
|
Die Reihenfolge der Spalten in der CSV spielt keine Rolle (Dict-Zugriff nach Name).
|
||||||
|
Falls mehrere Spalten auf dasselbe db_field abbilden, gewinnt die zuletzt verarbeitete
|
||||||
|
(iterreihenfolge = Kopfzeilen-Reihenfolge in der Datei) — in der Praxis selten.
|
||||||
|
"""
|
||||||
|
out: dict[str, Any] = {}
|
||||||
|
tc = type_conversions or {}
|
||||||
|
for csv_col, raw in csv_row.items():
|
||||||
|
db_field = _lookup_db_field(str(csv_col), field_mappings)
|
||||||
|
if not db_field and module == "vitals_baseline":
|
||||||
|
db_field = _vitals_baseline_alias_db_field(csv_col)
|
||||||
|
elif not db_field and module == "blood_pressure":
|
||||||
|
db_field = _blood_pressure_alias_db_field(csv_col)
|
||||||
|
elif not db_field and module == "activity":
|
||||||
|
db_field = _activity_alias_db_field(csv_col)
|
||||||
|
if not db_field:
|
||||||
|
continue
|
||||||
|
raw_spec = tc.get(db_field) if isinstance(tc, dict) else None
|
||||||
|
if not isinstance(raw_spec, dict):
|
||||||
|
raw_spec = None
|
||||||
|
spec = _effective_conversion_spec(db_field, raw_spec, module)
|
||||||
|
try:
|
||||||
|
out[db_field] = convert_value(
|
||||||
|
raw, db_field, spec if isinstance(spec, dict) else None, module=module
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
out[db_field] = None
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def diagnose_row_mapping(
|
||||||
|
csv_row: Mapping[str, str],
|
||||||
|
field_mappings: Mapping[str, str],
|
||||||
|
type_conversions: Mapping[str, Any] | None,
|
||||||
|
module: str | None = None,
|
||||||
|
*,
|
||||||
|
mapped_typed: Mapping[str, Any] | None = None,
|
||||||
|
max_columns: int = 512,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Nur für Diagnose-Endpunkt: Quelle (Vorlage vs. Alias), Konvertierung pro Spalte,
|
||||||
|
Ergebnis wie build_row_after_mapping (json-freundliche Vorschau).
|
||||||
|
max_columns begrenzt nur die Länge der Liste „per_column“ in der Antwort — der echte
|
||||||
|
Import verarbeitet alle Spalten (siehe iter_csv_dict_rows / build_row_after_mapping).
|
||||||
|
"""
|
||||||
|
tc = type_conversions or {}
|
||||||
|
per_column: list[dict[str, Any]] = []
|
||||||
|
n = 0
|
||||||
|
for csv_col, raw in csv_row.items():
|
||||||
|
if n >= max_columns:
|
||||||
|
break
|
||||||
|
n += 1
|
||||||
|
sc = str(csv_col)
|
||||||
|
via_t = _lookup_db_field(sc, field_mappings)
|
||||||
|
via_a = None
|
||||||
|
if not via_t and module == "vitals_baseline":
|
||||||
|
via_a = _vitals_baseline_alias_db_field(sc)
|
||||||
|
elif not via_t and module == "blood_pressure":
|
||||||
|
via_a = _blood_pressure_alias_db_field(sc)
|
||||||
|
elif not via_t and module == "activity":
|
||||||
|
via_a = _activity_alias_db_field(sc)
|
||||||
|
target = via_t or via_a
|
||||||
|
src = "template" if via_t else ("alias" if via_a else "none")
|
||||||
|
raw_spec = tc.get(target) if isinstance(tc, dict) and target else None
|
||||||
|
if not isinstance(raw_spec, dict):
|
||||||
|
raw_spec = None
|
||||||
|
spec = _effective_conversion_spec(target, raw_spec, module) if target else None
|
||||||
|
conv_err: str | None = None
|
||||||
|
conv_preview: Any = None
|
||||||
|
if target:
|
||||||
|
try:
|
||||||
|
conv_val = convert_value(
|
||||||
|
(raw or "").strip(),
|
||||||
|
target,
|
||||||
|
spec if isinstance(spec, dict) else None,
|
||||||
|
module=module,
|
||||||
|
)
|
||||||
|
conv_preview = conv_val.isoformat() if hasattr(conv_val, "isoformat") else conv_val
|
||||||
|
except Exception as e:
|
||||||
|
conv_err = str(e)
|
||||||
|
per_column.append(
|
||||||
|
{
|
||||||
|
"csv_column": sc,
|
||||||
|
"raw_preview": ((raw or "")[:120]),
|
||||||
|
"db_field": target,
|
||||||
|
"source": src,
|
||||||
|
"convert_error": conv_err,
|
||||||
|
"converted_preview": conv_preview,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
src_map = (
|
||||||
|
build_row_after_mapping(csv_row, field_mappings, type_conversions, module=module)
|
||||||
|
if mapped_typed is None
|
||||||
|
else mapped_typed
|
||||||
|
)
|
||||||
|
mapped_preview: dict[str, Any] = {}
|
||||||
|
for k, v in src_map.items():
|
||||||
|
mapped_preview[k] = v.isoformat() if hasattr(v, "isoformat") else v
|
||||||
|
|
||||||
|
tmpl_keys = [
|
||||||
|
str(k)
|
||||||
|
for k, v in field_mappings.items()
|
||||||
|
if v not in (None, "-", "_skip")
|
||||||
|
]
|
||||||
|
|
||||||
|
return {
|
||||||
|
"per_column": per_column,
|
||||||
|
"columns_truncated": len(csv_row) > max_columns,
|
||||||
|
"template_mapped_keys": tmpl_keys[:40],
|
||||||
|
"template_mapped_keys_truncated": len(tmpl_keys) > 40,
|
||||||
|
"mapped": mapped_preview,
|
||||||
|
}
|
||||||
@@ -31,6 +31,7 @@ from routers import workflows # Phase 2 Workflow Engine - Execution
|
|||||||
from routers import reference_values # Persönliche Referenzwerte (Profil)
|
from routers import reference_values # Persönliche Referenzwerte (Profil)
|
||||||
from routers import admin_reference_value_types # Admin: Referenzwert-Typen
|
from routers import admin_reference_value_types # Admin: Referenzwert-Typen
|
||||||
from routers import app_dashboard # Geschützter App-Bereich: Dashboard-Lab Layout
|
from routers import app_dashboard # Geschützter App-Bereich: Dashboard-Lab Layout
|
||||||
|
from routers import csv_import, admin_csv_templates # Issue #21 Universal CSV Parser
|
||||||
|
|
||||||
# ── App Configuration ─────────────────────────────────────────────────────────
|
# ── App Configuration ─────────────────────────────────────────────────────────
|
||||||
DATA_DIR = Path(os.getenv("DATA_DIR", "./data"))
|
DATA_DIR = Path(os.getenv("DATA_DIR", "./data"))
|
||||||
@@ -121,6 +122,8 @@ app.include_router(workflows.router) # /api/workflows/* (Phase 2 Exec
|
|||||||
app.include_router(reference_values.router) # /api/reference-value-types, /api/profile-reference-values
|
app.include_router(reference_values.router) # /api/reference-value-types, /api/profile-reference-values
|
||||||
app.include_router(admin_reference_value_types.router) # /api/admin/reference-value-types
|
app.include_router(admin_reference_value_types.router) # /api/admin/reference-value-types
|
||||||
app.include_router(app_dashboard.router) # /api/app/dashboard-layout
|
app.include_router(app_dashboard.router) # /api/app/dashboard-layout
|
||||||
|
app.include_router(csv_import.router) # /api/csv/* (Issue #21)
|
||||||
|
app.include_router(admin_csv_templates.router) # /api/admin/csv-templates/* (Issue #21)
|
||||||
|
|
||||||
# ── Health Check ──────────────────────────────────────────────────────────────
|
# ── Health Check ──────────────────────────────────────────────────────────────
|
||||||
@app.get("/")
|
@app.get("/")
|
||||||
|
|||||||
@@ -0,0 +1,75 @@
|
|||||||
|
-- Migration 042: Universal CSV Parser – Mapping-Registry & Import-Log (Issue #21)
|
||||||
|
-- Tabellen für System-Templates (profile_id NULL, is_system true) und User-Mappings.
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS csv_field_mappings (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
profile_id UUID REFERENCES profiles(id) ON DELETE CASCADE,
|
||||||
|
is_system BOOLEAN NOT NULL DEFAULT false,
|
||||||
|
module VARCHAR(50) NOT NULL,
|
||||||
|
mapping_name VARCHAR(100) NOT NULL,
|
||||||
|
description TEXT,
|
||||||
|
column_signature TEXT[] NOT NULL DEFAULT '{}',
|
||||||
|
delimiter VARCHAR(10) NOT NULL DEFAULT ',',
|
||||||
|
encoding VARCHAR(20) NOT NULL DEFAULT 'utf-8',
|
||||||
|
has_header BOOLEAN NOT NULL DEFAULT true,
|
||||||
|
field_mappings JSONB NOT NULL DEFAULT '{}',
|
||||||
|
type_conversions JSONB,
|
||||||
|
usage_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
last_used_at TIMESTAMPTZ,
|
||||||
|
success_rate REAL NOT NULL DEFAULT 1.0,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
CONSTRAINT csv_field_mappings_system_profile CHECK (
|
||||||
|
(is_system = true AND profile_id IS NULL)
|
||||||
|
OR (is_system = false AND profile_id IS NOT NULL)
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
COMMENT ON TABLE csv_field_mappings IS 'CSV Import: System-Templates + User-Mappings (Issue #21)';
|
||||||
|
COMMENT ON COLUMN csv_field_mappings.is_system IS 'true = globales Template (nur Admin pflegbar), false = User-Mapping';
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_csv_field_mappings_system_module_name
|
||||||
|
ON csv_field_mappings (module, mapping_name)
|
||||||
|
WHERE is_system = true AND profile_id IS NULL;
|
||||||
|
|
||||||
|
CREATE UNIQUE INDEX IF NOT EXISTS idx_csv_field_mappings_user_module_name
|
||||||
|
ON csv_field_mappings (profile_id, module, mapping_name)
|
||||||
|
WHERE is_system = false;
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_csv_field_mappings_module_profile
|
||||||
|
ON csv_field_mappings (module, profile_id);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_csv_field_mappings_system_module
|
||||||
|
ON csv_field_mappings (module)
|
||||||
|
WHERE is_system = true;
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS csv_import_log (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
profile_id UUID NOT NULL REFERENCES profiles(id) ON DELETE CASCADE,
|
||||||
|
mapping_id INTEGER REFERENCES csv_field_mappings(id) ON DELETE SET NULL,
|
||||||
|
module VARCHAR(50) NOT NULL,
|
||||||
|
filename VARCHAR(255),
|
||||||
|
rows_total INTEGER,
|
||||||
|
rows_imported INTEGER,
|
||||||
|
rows_updated INTEGER,
|
||||||
|
rows_skipped INTEGER,
|
||||||
|
rows_errors INTEGER,
|
||||||
|
error_details JSONB,
|
||||||
|
started_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
finished_at TIMESTAMPTZ,
|
||||||
|
status VARCHAR(20) NOT NULL DEFAULT 'running',
|
||||||
|
affected_ids JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_csv_import_log_profile_module
|
||||||
|
ON csv_import_log (profile_id, module DESC, started_at DESC);
|
||||||
|
|
||||||
|
COMMENT ON COLUMN csv_import_log.affected_ids IS 'Pro Import gesammelte Primärschlüssel je Tabelle (Rollback / Bereinigung)';
|
||||||
|
|
||||||
|
INSERT INTO system_config (key, value, updated_at)
|
||||||
|
VALUES (
|
||||||
|
'csv_import',
|
||||||
|
'{"max_rows_per_file": 50000, "max_file_bytes": 52428800}'::jsonb,
|
||||||
|
CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
ON CONFLICT (key) DO NOTHING;
|
||||||
@@ -0,0 +1,315 @@
|
|||||||
|
-- Migration 043: CSV Parser – System-Templates (Issue #21)
|
||||||
|
-- Idempotent: pro Template nur einfügen, wenn noch kein System-Eintrag für module+mapping_name existiert.
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'nutrition',
|
||||||
|
'FDDB Export (Standard)',
|
||||||
|
'Standard-Format für FDDB.de CSV-Exporte (Deutsch). Delimiter Semikolon, kJ → kcal Konvertierung.',
|
||||||
|
ARRAY['datum_tag_monat_jahr_stunde_minute', 'fett_g', 'kh_g', 'kj', 'protein_g']::TEXT[],
|
||||||
|
';',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"datum_tag_monat_jahr_stunde_minute": "date",
|
||||||
|
"kj": "kcal",
|
||||||
|
"fett_g": "fat_g",
|
||||||
|
"kh_g": "carbs_g",
|
||||||
|
"protein_g": "protein_g"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "date",
|
||||||
|
"format": "dd.mm.yyyy HH:MM",
|
||||||
|
"extract": "date_only"
|
||||||
|
},
|
||||||
|
"kcal": {
|
||||||
|
"type": "float",
|
||||||
|
"source_unit": "kj",
|
||||||
|
"decimal_separator": ","
|
||||||
|
},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": ","},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": ","},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": ","}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'nutrition' AND f.mapping_name = 'FDDB Export (Standard)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'nutrition', 'MyFitnessPal Export',
|
||||||
|
'Standard CSV export from MyFitnessPal (English)',
|
||||||
|
ARRAY['Carbohydrates (g)', 'Calories', 'Date', 'Fat (g)', 'Protein (g)']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Date": "date",
|
||||||
|
"Calories": "kcal",
|
||||||
|
"Fat (g)": "fat_g",
|
||||||
|
"Carbohydrates (g)": "carbs_g",
|
||||||
|
"Protein (g)": "protein_g"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd"},
|
||||||
|
"kcal": {"type": "float", "decimal_separator": "."},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": "."},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": "."},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": "."}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'nutrition' AND f.mapping_name = 'MyFitnessPal Export'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'nutrition', 'Cronometer Export',
|
||||||
|
'Cronometer daily nutrition export (English)',
|
||||||
|
ARRAY['Day', 'Energy (kcal)', 'Fat (g)', 'Net Carbs (g)', 'Protein (g)']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Day": "date",
|
||||||
|
"Energy (kcal)": "kcal",
|
||||||
|
"Fat (g)": "fat_g",
|
||||||
|
"Net Carbs (g)": "carbs_g",
|
||||||
|
"Protein (g)": "protein_g"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd"},
|
||||||
|
"kcal": {"type": "float", "decimal_separator": "."},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": "."},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": "."},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": "."}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'nutrition' AND f.mapping_name = 'Cronometer Export'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'activity', 'Apple Health Workout Export (English)',
|
||||||
|
'Apple Health CSV-Export für Workouts (English). Automatisches Training-Type-Mapping.',
|
||||||
|
ARRAY['Active Energy (kcal)', 'Distance (km)', 'Duration', 'End', 'Heart Rate Average (bpm)', 'Start', 'Workout Type']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Workout Type": "activity_type",
|
||||||
|
"Start": "start_time",
|
||||||
|
"End": "end_time",
|
||||||
|
"Duration": "duration_min",
|
||||||
|
"Distance (km)": "distance_km",
|
||||||
|
"Active Energy (kcal)": "kcal_active",
|
||||||
|
"Heart Rate Average (bpm)": "hr_avg"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "extract": "date_and_time", "flexible": true},
|
||||||
|
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": true},
|
||||||
|
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes"},
|
||||||
|
"distance_km": {"type": "float", "decimal_separator": "."},
|
||||||
|
"kcal_active": {"type": "float", "decimal_separator": "."},
|
||||||
|
"hr_avg": {"type": "int"}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'activity' AND f.mapping_name = 'Apple Health Workout Export (English)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'activity', 'Apple Health Workout Export (Deutsch)',
|
||||||
|
'Apple Health CSV-Export für Workouts (Deutsch). Automatisches Training-Type-Mapping.',
|
||||||
|
ARRAY['Aktive Energie (kJ)', 'Aktive Energie (kcal)', 'Dauer', 'Durchschnittliche Herzfrequenz (bpm)', 'Ende', 'Ruheeinträge (kJ)', 'Start', 'Strecke (km)', 'Trainingsart']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Trainingsart": "activity_type",
|
||||||
|
"Start": "start_time",
|
||||||
|
"Ende": "end_time",
|
||||||
|
"Dauer": "duration_min",
|
||||||
|
"Strecke (km)": "distance_km",
|
||||||
|
"Aktive Energie (kcal)": "kcal_active",
|
||||||
|
"Aktive Energie (kJ)": "kcal_active",
|
||||||
|
"Ruheeinträge (kJ)": "kcal_resting",
|
||||||
|
"Durchschnittliche Herzfrequenz (bpm)": "hr_avg"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "extract": "date_and_time", "flexible": true},
|
||||||
|
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": true},
|
||||||
|
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes"},
|
||||||
|
"distance_km": {"type": "float", "decimal_separator": ",", "flexible": true},
|
||||||
|
"kcal_active": {"type": "float", "decimal_separator": ".", "flexible": true, "source_unit": "kj"},
|
||||||
|
"kcal_resting": {"type": "float", "decimal_separator": ".", "flexible": true, "source_unit": "kj"},
|
||||||
|
"hr_avg": {"type": "int", "flexible": true}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'activity' AND f.mapping_name = 'Apple Health Workout Export (Deutsch)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'activity', 'Garmin Connect Export',
|
||||||
|
'Garmin Connect activity CSV export (English)',
|
||||||
|
ARRAY['Activity Type', 'Avg HR', 'Calories', 'Date', 'Distance', 'Duration', 'Time']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Activity Type": "activity_type",
|
||||||
|
"Date": "date",
|
||||||
|
"Time": "start_time",
|
||||||
|
"Duration": "duration_min",
|
||||||
|
"Distance": "distance_km",
|
||||||
|
"Calories": "kcal_active",
|
||||||
|
"Avg HR": "hr_avg"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd"},
|
||||||
|
"start_time": {"type": "time", "format": "HH:MM:SS"},
|
||||||
|
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes"},
|
||||||
|
"distance_km": {"type": "float", "decimal_separator": "."},
|
||||||
|
"kcal_active": {"type": "float", "decimal_separator": "."},
|
||||||
|
"hr_avg": {"type": "int"}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'activity' AND f.mapping_name = 'Garmin Connect Export'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'blood_pressure', 'Omron Export (Deutsch)',
|
||||||
|
'Omron Blutdruckmessgerät CSV-Export (Deutsch)',
|
||||||
|
ARRAY['Datum', 'Diastolisch (mmHg)', 'Puls (bpm)', 'Systolisch (mmHg)', 'Zeit']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Datum": "measured_date",
|
||||||
|
"Zeit": "measured_time",
|
||||||
|
"Systolisch (mmHg)": "systolic",
|
||||||
|
"Diastolisch (mmHg)": "diastolic",
|
||||||
|
"Puls (bpm)": "pulse"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"measured_date": {"type": "date", "format": "dd.mm.yyyy"},
|
||||||
|
"measured_time": {"type": "time", "format": "HH:MM"},
|
||||||
|
"systolic": {"type": "int"},
|
||||||
|
"diastolic": {"type": "int"},
|
||||||
|
"pulse": {"type": "int"}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'blood_pressure' AND f.mapping_name = 'Omron Export (Deutsch)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'blood_pressure', 'Omron Export (English)',
|
||||||
|
'Omron blood pressure monitor CSV export (English)',
|
||||||
|
ARRAY['Date', 'Diastolic (mmHg)', 'Pulse (bpm)', 'Systolic (mmHg)', 'Time']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Date": "measured_date",
|
||||||
|
"Time": "measured_time",
|
||||||
|
"Systolic (mmHg)": "systolic",
|
||||||
|
"Diastolic (mmHg)": "diastolic",
|
||||||
|
"Pulse (bpm)": "pulse"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"measured_date": {"type": "date", "format": "mm/dd/yyyy"},
|
||||||
|
"measured_time": {"type": "time", "format": "HH:MM"},
|
||||||
|
"systolic": {"type": "int"},
|
||||||
|
"diastolic": {"type": "int"},
|
||||||
|
"pulse": {"type": "int"}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'blood_pressure' AND f.mapping_name = 'Omron Export (English)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'weight', 'Apple Health Weight Export',
|
||||||
|
'Apple Health body mass CSV export',
|
||||||
|
ARRAY['Body Mass (kg)', 'Start']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Start": "date",
|
||||||
|
"Body Mass (kg)": "weight"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "extract": "date_only"},
|
||||||
|
"weight": {"type": "float", "decimal_separator": "."}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'weight' AND f.mapping_name = 'Apple Health Weight Export'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL, true, 'weight', 'Withings Export',
|
||||||
|
'Withings smart scale CSV export (weight, body fat, muscle mass)',
|
||||||
|
ARRAY['Body Fat (%)', 'Date', 'Muscle Mass (kg)', 'Weight (kg)']::TEXT[],
|
||||||
|
',', 'utf-8', true,
|
||||||
|
'{
|
||||||
|
"Date": "date",
|
||||||
|
"Weight (kg)": "weight"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd"},
|
||||||
|
"weight": {"type": "float", "decimal_separator": "."}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'weight' AND f.mapping_name = 'Withings Export'
|
||||||
|
);
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
-- Migration 044: CSV Parser — System-Vorlagen Schlaf (Apple) + Vitalwerte Baseline (Issue #21)
|
||||||
|
-- Idempotent wie 043.
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'sleep',
|
||||||
|
'Apple Health Schlaf (Segment-Export)',
|
||||||
|
'Apple-Health-Schlaf-CSV mit Phasen-Zeilen (Start, End, Duration (hr), Value). Import läuft aggregiert pro Nacht.',
|
||||||
|
ARRAY['Start', 'End', 'Duration (hr)', 'Value']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Start": "-",
|
||||||
|
"End": "-",
|
||||||
|
"Duration (hr)": "-",
|
||||||
|
"Value": "-"
|
||||||
|
}'::JSONB,
|
||||||
|
'{}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'sleep' AND f.mapping_name = 'Apple Health Schlaf (Segment-Export)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'sleep',
|
||||||
|
'Apple Health Schlaf (Schlafanalyse / Nacht)',
|
||||||
|
'Apple-Health-Schlafanalyse mit Nacht-Zusammenfassung (u. a. Total Sleep (hr), Start, End).',
|
||||||
|
ARRAY['Start', 'End', 'Total Sleep (hr)']::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Start": "-",
|
||||||
|
"End": "-",
|
||||||
|
"Total Sleep (hr)": "-"
|
||||||
|
}'::JSONB,
|
||||||
|
'{}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'sleep' AND f.mapping_name = 'Apple Health Schlaf (Schlafanalyse / Nacht)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'vitals_baseline',
|
||||||
|
'Apple Health Baseline Vitals (English)',
|
||||||
|
'Tägliche Baseline-Messungen aus Apple Health (Ruhepuls, HRV, VO2, SpO2, Atemfrequenz).',
|
||||||
|
ARRAY[
|
||||||
|
'Start',
|
||||||
|
'Resting Heart Rate',
|
||||||
|
'Heart Rate Variability',
|
||||||
|
'VO2 Max',
|
||||||
|
'Oxygen Saturation',
|
||||||
|
'Respiratory Rate'
|
||||||
|
]::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Start": "date",
|
||||||
|
"Resting Heart Rate": "resting_hr",
|
||||||
|
"Heart Rate Variability": "hrv",
|
||||||
|
"VO2 Max": "vo2_max",
|
||||||
|
"Oxygen Saturation": "spo2",
|
||||||
|
"Respiratory Rate": "respiratory_rate"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": true
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": true},
|
||||||
|
"hrv": {"type": "int", "flexible": true},
|
||||||
|
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": true},
|
||||||
|
"spo2": {"type": "int", "flexible": true},
|
||||||
|
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": true}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'vitals_baseline' AND f.mapping_name = 'Apple Health Baseline Vitals (English)'
|
||||||
|
);
|
||||||
|
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
NULL,
|
||||||
|
true,
|
||||||
|
'vitals_baseline',
|
||||||
|
'Apple Health Baseline Vitals (Deutsch)',
|
||||||
|
'Tägliche Baseline-Messungen aus Apple Health (deutsche Spaltenbezeichnungen).',
|
||||||
|
ARRAY[
|
||||||
|
'Datum/Uhrzeit',
|
||||||
|
'Ruhepuls (count/min)',
|
||||||
|
'Herzfrequenzvariabilität (ms)',
|
||||||
|
'VO2 max (ml/(kg·min))',
|
||||||
|
'Blutsauerstoffsättigung (%)',
|
||||||
|
'Atemfrequenz (count/min)'
|
||||||
|
]::TEXT[],
|
||||||
|
',',
|
||||||
|
'utf-8',
|
||||||
|
true,
|
||||||
|
'{
|
||||||
|
"Datum/Uhrzeit": "date",
|
||||||
|
"Ruhepuls (count/min)": "resting_hr",
|
||||||
|
"Herzfrequenzvariabilität (ms)": "hrv",
|
||||||
|
"VO2 max (ml/(kg·min))": "vo2_max",
|
||||||
|
"Blutsauerstoffsättigung (%)": "spo2",
|
||||||
|
"Atemfrequenz (count/min)": "respiratory_rate"
|
||||||
|
}'::JSONB,
|
||||||
|
'{
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": true
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": true},
|
||||||
|
"hrv": {"type": "int", "flexible": true},
|
||||||
|
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": true},
|
||||||
|
"spo2": {"type": "int", "flexible": true},
|
||||||
|
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": true}
|
||||||
|
}'::JSONB
|
||||||
|
WHERE NOT EXISTS (
|
||||||
|
SELECT 1 FROM csv_field_mappings f
|
||||||
|
WHERE f.is_system AND f.profile_id IS NULL
|
||||||
|
AND f.module = 'vitals_baseline' AND f.mapping_name = 'Apple Health Baseline Vitals (Deutsch)'
|
||||||
|
);
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
-- Migration 045: Schlaf-Systemvorlage — Signatur wie „Schlafanalyse“-Export (Date/Time, Sources, …)
|
||||||
|
-- Ergänzt die Vorlage aus 044 für besseres Matching; Import-Logik unverändert (Apple-Aggregat-Parser).
|
||||||
|
|
||||||
|
UPDATE csv_field_mappings
|
||||||
|
SET
|
||||||
|
column_signature = ARRAY[
|
||||||
|
'Date/Time',
|
||||||
|
'Start',
|
||||||
|
'End',
|
||||||
|
'Total Sleep (hr)',
|
||||||
|
'Asleep (Unspecified) (hr)',
|
||||||
|
'In Bed (hr)',
|
||||||
|
'Core (hr)',
|
||||||
|
'Deep (hr)',
|
||||||
|
'REM (hr)',
|
||||||
|
'Awake (hr)',
|
||||||
|
'Sources'
|
||||||
|
]::TEXT[],
|
||||||
|
description = COALESCE(
|
||||||
|
description,
|
||||||
|
'Apple-Health-Schlafanalyse (Nacht-Zusammenfassung): Spalten wie Date/Time, Start/End mit Zeitzone, Kern/Tief/REM etc.'
|
||||||
|
),
|
||||||
|
updated_at = CURRENT_TIMESTAMP
|
||||||
|
WHERE is_system = true
|
||||||
|
AND profile_id IS NULL
|
||||||
|
AND module = 'sleep'
|
||||||
|
AND mapping_name = 'Apple Health Schlaf (Schlafanalyse / Nacht)';
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
-- Migration 046: FDDB-Systemvorlage — kcal ohne legacy conversion_factor/target_unit (Issue #21, source_unit-Registry)
|
||||||
|
-- Doppelte Umrechnung (kj-Faktor * 0.239) wurde fälschlich gespeichert; target_unit ist nur für duration vorgesehen.
|
||||||
|
|
||||||
|
UPDATE csv_field_mappings
|
||||||
|
SET type_conversions = jsonb_set(
|
||||||
|
type_conversions,
|
||||||
|
'{kcal}',
|
||||||
|
((type_conversions->'kcal') - 'conversion_factor' - 'target_unit')
|
||||||
|
|| jsonb_build_object('source_unit', 'kj')
|
||||||
|
)
|
||||||
|
WHERE is_system = true
|
||||||
|
AND profile_id IS NULL
|
||||||
|
AND module = 'nutrition'
|
||||||
|
AND mapping_name = 'FDDB Export (Standard)'
|
||||||
|
AND (type_conversions->'kcal') IS NOT NULL
|
||||||
|
AND (
|
||||||
|
(type_conversions->'kcal') ? 'conversion_factor'
|
||||||
|
OR (type_conversions->'kcal') ? 'target_unit'
|
||||||
|
);
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
-- Migration 047: CSV-Vorlagen — optionale Zeilenaggregation (group_by + aggregates) vor DB-Schreiben
|
||||||
|
|
||||||
|
ALTER TABLE csv_field_mappings
|
||||||
|
ADD COLUMN IF NOT EXISTS import_row_processing JSONB;
|
||||||
|
|
||||||
|
COMMENT ON COLUMN csv_field_mappings.import_row_processing IS
|
||||||
|
'Optional: { "group_by": ["date"], "aggregates": { "kcal": "sum" } } — siehe csv_parser/import_row_processing.py';
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
-- Universal-CSV-Import setzt source = 'csv'. Alte CHECK-Constraints kennen das nicht.
|
||||||
|
-- Alle passenden CHECKs zu `source` droppen (PG speichert IN oder = ANY (ARRAY[...])).
|
||||||
|
DO $$
|
||||||
|
DECLARE
|
||||||
|
r RECORD;
|
||||||
|
BEGIN
|
||||||
|
FOR r IN
|
||||||
|
SELECT c.conname
|
||||||
|
FROM pg_constraint c
|
||||||
|
JOIN pg_class t ON c.conrelid = t.oid
|
||||||
|
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||||
|
WHERE n.nspname = 'public'
|
||||||
|
AND t.relname = 'vitals_baseline'
|
||||||
|
AND c.contype = 'c'
|
||||||
|
AND (
|
||||||
|
c.conname = 'vitals_baseline_source_check'
|
||||||
|
OR COALESCE(pg_get_constraintdef(c.oid), '') ILIKE '%source%'
|
||||||
|
)
|
||||||
|
LOOP
|
||||||
|
EXECUTE format('ALTER TABLE public.vitals_baseline DROP CONSTRAINT %I', r.conname);
|
||||||
|
END LOOP;
|
||||||
|
END $$;
|
||||||
|
|
||||||
|
ALTER TABLE public.vitals_baseline DROP CONSTRAINT IF EXISTS vitals_baseline_source_check;
|
||||||
|
|
||||||
|
ALTER TABLE public.vitals_baseline ADD CONSTRAINT vitals_baseline_source_check
|
||||||
|
CHECK (source IN ('manual', 'apple_health', 'garmin', 'withings', 'csv'));
|
||||||
|
|
||||||
|
COMMENT ON COLUMN vitals_baseline.source IS 'manual | apple_health | garmin | withings | csv (Universal-Import)';
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
-- Idempotent: gleicher Zielzustand wie 048, auch wenn die CHECK schon existiert (z. B. 048 DB-seitig
|
||||||
|
-- erfolgreich, aber nicht in schema_migrations). Ein einziges DO vermeidet Mehrfach-Statement-Probleme.
|
||||||
|
DO $$
|
||||||
|
DECLARE
|
||||||
|
r RECORD;
|
||||||
|
BEGIN
|
||||||
|
FOR r IN
|
||||||
|
SELECT c.conname
|
||||||
|
FROM pg_constraint c
|
||||||
|
JOIN pg_class t ON c.conrelid = t.oid
|
||||||
|
JOIN pg_namespace n ON n.oid = t.relnamespace
|
||||||
|
WHERE n.nspname = 'public'
|
||||||
|
AND t.relname = 'vitals_baseline'
|
||||||
|
AND c.contype = 'c'
|
||||||
|
AND (
|
||||||
|
c.conname = 'vitals_baseline_source_check'
|
||||||
|
OR COALESCE(pg_get_constraintdef(c.oid), '') ILIKE '%source%'
|
||||||
|
)
|
||||||
|
LOOP
|
||||||
|
EXECUTE format('ALTER TABLE public.vitals_baseline DROP CONSTRAINT %I', r.conname);
|
||||||
|
END LOOP;
|
||||||
|
|
||||||
|
EXECUTE $ddl$
|
||||||
|
ALTER TABLE public.vitals_baseline ADD CONSTRAINT vitals_baseline_source_check
|
||||||
|
CHECK (source IN ('manual', 'apple_health', 'garmin', 'withings', 'csv'))
|
||||||
|
$ddl$;
|
||||||
|
EXCEPTION
|
||||||
|
WHEN duplicate_object THEN
|
||||||
|
NULL;
|
||||||
|
END $$;
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
-- Apple Health Workout-CSV: Zeit oft ohne Sekunden (HH:MM); dateutil dayfirst bricht ISO YYYY-MM-DD.
|
||||||
|
-- type_converter: zusätzliche Patterns + ISO-reihenfolge in _dateutil_parse.
|
||||||
|
-- Bestehende System-Vorlagen: flexible für Start/End (idempotent).
|
||||||
|
|
||||||
|
UPDATE csv_field_mappings
|
||||||
|
SET type_conversions = jsonb_set(
|
||||||
|
jsonb_set(
|
||||||
|
COALESCE(type_conversions, '{}'::jsonb),
|
||||||
|
'{start_time}',
|
||||||
|
COALESCE(type_conversions->'start_time', '{}'::jsonb) || '{"flexible": true}'::jsonb,
|
||||||
|
true
|
||||||
|
),
|
||||||
|
'{end_time}',
|
||||||
|
COALESCE(type_conversions->'end_time', '{}'::jsonb) || '{"flexible": true}'::jsonb,
|
||||||
|
true
|
||||||
|
)
|
||||||
|
WHERE is_system = true
|
||||||
|
AND profile_id IS NULL
|
||||||
|
AND module = 'activity'
|
||||||
|
AND mapping_name IN (
|
||||||
|
'Apple Health Workout Export (English)',
|
||||||
|
'Apple Health Workout Export (Deutsch)'
|
||||||
|
)
|
||||||
|
AND type_conversions IS NOT NULL
|
||||||
|
AND type_conversions ? 'start_time'
|
||||||
|
AND type_conversions ? 'end_time';
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
-- Universal-CSV-Import schreibt source = 'csv' (siehe csv_parser/executor _import_blood_pressure).
|
||||||
|
|
||||||
|
ALTER TABLE blood_pressure_log DROP CONSTRAINT IF EXISTS blood_pressure_log_source_check;
|
||||||
|
|
||||||
|
ALTER TABLE blood_pressure_log ADD CONSTRAINT blood_pressure_log_source_check
|
||||||
|
CHECK (source IN ('manual', 'omron', 'apple_health', 'withings', 'csv'));
|
||||||
|
|
||||||
|
COMMENT ON COLUMN blood_pressure_log.source IS
|
||||||
|
'manual | omron | apple_health | withings | csv (Universal-CSV-Import)';
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
-- Apple-/Import-Werte: Energie oft >1000 (kJ oder kcal), HR-Felder NUMERIC(5,2) zu eng für Fehlzuordnungen.
|
||||||
|
-- Idempotent: Typ nur anheben, nie verkleinern.
|
||||||
|
|
||||||
|
ALTER TABLE activity_log
|
||||||
|
ALTER COLUMN kcal_active TYPE NUMERIC(12, 2),
|
||||||
|
ALTER COLUMN kcal_resting TYPE NUMERIC(12, 2),
|
||||||
|
ALTER COLUMN hr_avg TYPE NUMERIC(6, 2),
|
||||||
|
ALTER COLUMN hr_max TYPE NUMERIC(6, 2);
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
-- Apple Health Workout (Deutsch): „Aktive Energie (kJ)“ / „Ruheeinträge (kJ)“ → DB speichert kcal.
|
||||||
|
-- type_conversions.source_unit „kj“ nutzt field_units (1/4.184).
|
||||||
|
|
||||||
|
UPDATE csv_field_mappings
|
||||||
|
SET
|
||||||
|
field_mappings = COALESCE(field_mappings, '{}'::jsonb)
|
||||||
|
|| '{"Aktive Energie (kJ)": "kcal_active", "Ruheeinträge (kJ)": "kcal_resting"}'::jsonb,
|
||||||
|
type_conversions = jsonb_set(
|
||||||
|
jsonb_set(
|
||||||
|
COALESCE(type_conversions, '{}'::jsonb),
|
||||||
|
'{kcal_active}',
|
||||||
|
COALESCE(type_conversions->'kcal_active', '{}'::jsonb) || '{"source_unit": "kj"}'::jsonb,
|
||||||
|
true
|
||||||
|
),
|
||||||
|
'{kcal_resting}',
|
||||||
|
COALESCE(
|
||||||
|
type_conversions->'kcal_resting',
|
||||||
|
'{"type": "float", "decimal_separator": ",", "flexible": true}'::jsonb
|
||||||
|
) || '{"source_unit": "kj"}'::jsonb,
|
||||||
|
true
|
||||||
|
)
|
||||||
|
WHERE is_system = true
|
||||||
|
AND profile_id IS NULL
|
||||||
|
AND module = 'activity'
|
||||||
|
AND mapping_name = 'Apple Health Workout Export (Deutsch)'
|
||||||
|
AND type_conversions IS NOT NULL;
|
||||||
@@ -12,30 +12,48 @@
|
|||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- 1. Rename csv_import to data_import
|
-- 1. csv_import → data_import (FK-sicher)
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
UPDATE features
|
-- Zuerst Ziel-Feature-Zeile sichern, dann alle FKs umhängen, dann csv_import
|
||||||
SET
|
-- entfernen. PK direkt per UPDATE ändern scheitert, solange tier_limits noch
|
||||||
id = 'data_import',
|
-- feature_id = 'csv_import' referenziert (tier_limits_feature_id_fkey).
|
||||||
name = 'Daten importieren',
|
|
||||||
description = 'CSV-Import (FDDB, Apple Health) + ZIP-Backup-Import'
|
INSERT INTO features (id, name, description, category, limit_type, reset_period, default_limit, active)
|
||||||
WHERE id = 'csv_import';
|
VALUES ('data_import', 'Daten importieren', 'CSV-Import (FDDB, Apple Health) + ZIP-Backup-Import', 'import', 'count', 'monthly', 0, true)
|
||||||
|
ON CONFLICT (id) DO UPDATE SET
|
||||||
|
name = EXCLUDED.name,
|
||||||
|
description = EXCLUDED.description,
|
||||||
|
category = EXCLUDED.category,
|
||||||
|
limit_type = EXCLUDED.limit_type,
|
||||||
|
reset_period = EXCLUDED.reset_period;
|
||||||
|
|
||||||
-- Update tier_limits references
|
|
||||||
UPDATE tier_limits
|
UPDATE tier_limits
|
||||||
SET feature_id = 'data_import'
|
SET feature_id = 'data_import'
|
||||||
WHERE feature_id = 'csv_import';
|
WHERE feature_id = 'csv_import';
|
||||||
|
|
||||||
-- Update user_feature_restrictions references
|
|
||||||
UPDATE user_feature_restrictions
|
UPDATE user_feature_restrictions
|
||||||
SET feature_id = 'data_import'
|
SET feature_id = 'data_import'
|
||||||
WHERE feature_id = 'csv_import';
|
WHERE feature_id = 'csv_import';
|
||||||
|
|
||||||
-- Update user_feature_usage references
|
|
||||||
UPDATE user_feature_usage
|
UPDATE user_feature_usage
|
||||||
SET feature_id = 'data_import'
|
SET feature_id = 'data_import'
|
||||||
WHERE feature_id = 'csv_import';
|
WHERE feature_id = 'csv_import';
|
||||||
|
|
||||||
|
-- Widget-Gateway (Migration 041): sonst blockiert FK beim Löschen von csv_import
|
||||||
|
DO $$
|
||||||
|
BEGIN
|
||||||
|
IF EXISTS (
|
||||||
|
SELECT 1 FROM information_schema.tables
|
||||||
|
WHERE table_schema = 'public' AND table_name = 'widget_feature_requirements'
|
||||||
|
) THEN
|
||||||
|
UPDATE widget_feature_requirements
|
||||||
|
SET feature_id = 'data_import'
|
||||||
|
WHERE feature_id = 'csv_import';
|
||||||
|
END IF;
|
||||||
|
END $$;
|
||||||
|
|
||||||
|
DELETE FROM features WHERE id = 'csv_import';
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- 2. Remove old export_csv/json/zip features
|
-- 2. Remove old export_csv/json/zip features
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
@@ -69,16 +87,8 @@ ON CONFLICT (id) DO UPDATE SET
|
|||||||
reset_period = EXCLUDED.reset_period;
|
reset_period = EXCLUDED.reset_period;
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- 4. Ensure data_import exists and is properly configured
|
-- 4. data_import: in Schritt 1 angelegt; kein zweites INSERT nötig
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
INSERT INTO features (id, name, description, category, limit_type, reset_period, default_limit, active)
|
|
||||||
VALUES ('data_import', 'Daten importieren', 'CSV-Import (FDDB, Apple Health) + ZIP-Backup-Import', 'import', 'count', 'monthly', 0, true)
|
|
||||||
ON CONFLICT (id) DO UPDATE SET
|
|
||||||
name = EXCLUDED.name,
|
|
||||||
description = EXCLUDED.description,
|
|
||||||
category = EXCLUDED.category,
|
|
||||||
limit_type = EXCLUDED.limit_type,
|
|
||||||
reset_period = EXCLUDED.reset_period;
|
|
||||||
|
|
||||||
-- ============================================================================
|
-- ============================================================================
|
||||||
-- 5. Update tier_limits for data_export (consolidate from old features)
|
-- 5. Update tier_limits for data_export (consolidate from old features)
|
||||||
|
|||||||
@@ -30,7 +30,8 @@ def extract_value_raw(value_display: str, output_type: OutputType, placeholder_t
|
|||||||
Returns: (raw_value, success)
|
Returns: (raw_value, success)
|
||||||
"""
|
"""
|
||||||
if not value_display or value_display in ['nicht verfügbar', 'nicht genug Daten']:
|
if not value_display or value_display in ['nicht verfügbar', 'nicht genug Daten']:
|
||||||
return None, True
|
# V2 strict mode: missing/unavailable value is not a successful extraction
|
||||||
|
return None, False
|
||||||
|
|
||||||
# JSON output type
|
# JSON output type
|
||||||
if output_type == OutputType.JSON:
|
if output_type == OutputType.JSON:
|
||||||
@@ -111,7 +112,8 @@ def infer_unit_strict(key: str, description: str, output_type: OutputType, place
|
|||||||
return 'kg'
|
return 'kg'
|
||||||
|
|
||||||
# Circumferences/lengths
|
# Circumferences/lengths
|
||||||
if any(x in key_lower for x in ['umfang', 'waist', 'hip', 'chest', 'arm', 'leg', 'delta']) and 'circumference' in desc_lower:
|
circumference_terms = ['umfang', 'waist', 'hip', 'chest', 'arm', 'leg', 'taill', 'hueft', 'brust', 'oberarm', 'oberschenkel']
|
||||||
|
if any(x in key_lower for x in circumference_terms) or any(x in desc_lower for x in ['circumference', 'umfang', 'taill', 'hüft', 'hueft', 'brust', 'oberarm', 'oberschenkel']):
|
||||||
return 'cm'
|
return 'cm'
|
||||||
|
|
||||||
# Time durations
|
# Time durations
|
||||||
|
|||||||
@@ -0,0 +1,9 @@
|
|||||||
|
[pytest]
|
||||||
|
testpaths = tests
|
||||||
|
python_files = test_*.py
|
||||||
|
python_functions = test_*
|
||||||
|
addopts = -q --tb=short
|
||||||
|
markers =
|
||||||
|
smoke: Fast smoke tests for core regression checks.
|
||||||
|
integration: Integration tests across modules/db/api behavior.
|
||||||
|
slow: Long-running or heavy tests.
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# Entwicklung / CI: nach requirements.txt installieren
|
||||||
|
# pip install -r requirements.txt -r requirements-dev.txt
|
||||||
|
#
|
||||||
|
# Empfohlene Smoke-Suite (CSV-Import + Parser, ohne DB/Workflow-Integration):
|
||||||
|
# python -m pytest tests/test_csv_parser_core.py tests/test_csv_import_executor.py tests/test_mapping_suggest.py
|
||||||
|
pytest>=8.0
|
||||||
+38
-27
@@ -198,6 +198,43 @@ def activity_stats(x_profile_id: Optional[str]=Header(default=None), session: di
|
|||||||
return {"count":len(rows),"total_kcal":round(total_kcal),"total_min":round(total_min),"by_type":by_type}
|
return {"count":len(rows),"total_kcal":round(total_kcal),"total_min":round(total_min),"by_type":by_type}
|
||||||
|
|
||||||
|
|
||||||
|
def get_training_type_for_activity_with_cursor(cur, activity_type: str, profile_id: str | None = None):
|
||||||
|
"""
|
||||||
|
Wie get_training_type_for_activity, aber mit bestehendem Cursor (z. B. Universal-CSV-Import).
|
||||||
|
Vermeidet verschachteltes get_db() — bei maxconn=1 sonst Deadlock auf dem Connection-Pool.
|
||||||
|
"""
|
||||||
|
if profile_id:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT m.training_type_id, t.category, t.subcategory
|
||||||
|
FROM activity_type_mappings m
|
||||||
|
JOIN training_types t ON m.training_type_id = t.id
|
||||||
|
WHERE m.activity_type = %s AND m.profile_id = %s
|
||||||
|
LIMIT 1
|
||||||
|
""",
|
||||||
|
(activity_type, profile_id),
|
||||||
|
)
|
||||||
|
row = cur.fetchone()
|
||||||
|
if row:
|
||||||
|
return (row["training_type_id"], row["category"], row["subcategory"])
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT m.training_type_id, t.category, t.subcategory
|
||||||
|
FROM activity_type_mappings m
|
||||||
|
JOIN training_types t ON m.training_type_id = t.id
|
||||||
|
WHERE m.activity_type = %s AND m.profile_id IS NULL
|
||||||
|
LIMIT 1
|
||||||
|
""",
|
||||||
|
(activity_type,),
|
||||||
|
)
|
||||||
|
row = cur.fetchone()
|
||||||
|
if row:
|
||||||
|
return (row["training_type_id"], row["category"], row["subcategory"])
|
||||||
|
|
||||||
|
return (None, None, None)
|
||||||
|
|
||||||
|
|
||||||
def get_training_type_for_activity(activity_type: str, profile_id: str = None):
|
def get_training_type_for_activity(activity_type: str, profile_id: str = None):
|
||||||
"""
|
"""
|
||||||
Map activity_type to training_type_id using database mappings.
|
Map activity_type to training_type_id using database mappings.
|
||||||
@@ -211,33 +248,7 @@ def get_training_type_for_activity(activity_type: str, profile_id: str = None):
|
|||||||
"""
|
"""
|
||||||
with get_db() as conn:
|
with get_db() as conn:
|
||||||
cur = get_cursor(conn)
|
cur = get_cursor(conn)
|
||||||
|
return get_training_type_for_activity_with_cursor(cur, activity_type, profile_id)
|
||||||
# Try user-specific mapping first
|
|
||||||
if profile_id:
|
|
||||||
cur.execute("""
|
|
||||||
SELECT m.training_type_id, t.category, t.subcategory
|
|
||||||
FROM activity_type_mappings m
|
|
||||||
JOIN training_types t ON m.training_type_id = t.id
|
|
||||||
WHERE m.activity_type = %s AND m.profile_id = %s
|
|
||||||
LIMIT 1
|
|
||||||
""", (activity_type, profile_id))
|
|
||||||
row = cur.fetchone()
|
|
||||||
if row:
|
|
||||||
return (row['training_type_id'], row['category'], row['subcategory'])
|
|
||||||
|
|
||||||
# Try global mapping
|
|
||||||
cur.execute("""
|
|
||||||
SELECT m.training_type_id, t.category, t.subcategory
|
|
||||||
FROM activity_type_mappings m
|
|
||||||
JOIN training_types t ON m.training_type_id = t.id
|
|
||||||
WHERE m.activity_type = %s AND m.profile_id IS NULL
|
|
||||||
LIMIT 1
|
|
||||||
""", (activity_type,))
|
|
||||||
row = cur.fetchone()
|
|
||||||
if row:
|
|
||||||
return (row['training_type_id'], row['category'], row['subcategory'])
|
|
||||||
|
|
||||||
return (None, None, None)
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/uncategorized")
|
@router.get("/uncategorized")
|
||||||
|
|||||||
@@ -0,0 +1,430 @@
|
|||||||
|
"""
|
||||||
|
Admin: System-CSV-Templates (csv_field_mappings, is_system=true) pflegen (Issue #21).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, List, Optional
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
from psycopg2.extras import Json
|
||||||
|
|
||||||
|
from auth import require_admin
|
||||||
|
from db import get_db, get_cursor, r2d
|
||||||
|
from csv_parser.core import (
|
||||||
|
column_signature,
|
||||||
|
decode_raw_bytes,
|
||||||
|
get_csv_import_limits,
|
||||||
|
headers_signature_rank_metrics,
|
||||||
|
normalize_header_for_signature,
|
||||||
|
parse_csv_sample,
|
||||||
|
)
|
||||||
|
from csv_parser.mapping_suggest import build_type_conversions_for_mapping, suggest_field_mappings
|
||||||
|
from csv_parser.import_row_processing import (
|
||||||
|
validate_import_row_processing as validate_import_row_processing_spec,
|
||||||
|
)
|
||||||
|
from csv_parser.module_registry import get_module_definition
|
||||||
|
from csv_parser.template_validator import validate_csv_template
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/admin/csv-templates", tags=["admin", "csv-import"])
|
||||||
|
|
||||||
|
|
||||||
|
class CsvSystemTemplateCreate(BaseModel):
|
||||||
|
module: str
|
||||||
|
mapping_name: str = Field(..., min_length=1, max_length=100)
|
||||||
|
description: Optional[str] = None
|
||||||
|
column_signature: List[str] = Field(default_factory=list)
|
||||||
|
delimiter: str = ","
|
||||||
|
encoding: str = "utf-8"
|
||||||
|
has_header: bool = True
|
||||||
|
field_mappings: dict = Field(default_factory=dict)
|
||||||
|
type_conversions: Optional[dict] = None
|
||||||
|
import_row_processing: Optional[dict] = None
|
||||||
|
|
||||||
|
|
||||||
|
class CsvSystemTemplateUpdate(BaseModel):
|
||||||
|
mapping_name: Optional[str] = Field(default=None, min_length=1, max_length=100)
|
||||||
|
description: Optional[str] = None
|
||||||
|
column_signature: Optional[List[str]] = None
|
||||||
|
delimiter: Optional[str] = None
|
||||||
|
encoding: Optional[str] = None
|
||||||
|
has_header: Optional[bool] = None
|
||||||
|
field_mappings: Optional[dict] = None
|
||||||
|
type_conversions: Optional[dict] = None
|
||||||
|
import_row_processing: Optional[dict] = None
|
||||||
|
|
||||||
|
|
||||||
|
class CsvImportLimitsBody(BaseModel):
|
||||||
|
max_rows_per_file: int = Field(..., ge=100, le=2_000_000)
|
||||||
|
max_file_bytes: int = Field(..., ge=10_000, le=2_147_483_648)
|
||||||
|
|
||||||
|
|
||||||
|
class CsvTemplateValidateBody(BaseModel):
|
||||||
|
"""Formatprüfung ohne Speichern (field_mappings + type_conversions + optional row_processing)."""
|
||||||
|
|
||||||
|
module: str
|
||||||
|
field_mappings: dict = Field(default_factory=dict)
|
||||||
|
type_conversions: Optional[dict] = None
|
||||||
|
import_row_processing: Optional[dict] = None
|
||||||
|
column_signature: Optional[List[str]] = None
|
||||||
|
|
||||||
|
|
||||||
|
def _row_full(m: dict) -> dict:
|
||||||
|
return {
|
||||||
|
"id": m["id"],
|
||||||
|
"module": m["module"],
|
||||||
|
"mapping_name": m["mapping_name"],
|
||||||
|
"description": m.get("description"),
|
||||||
|
"column_signature": list(m["column_signature"]) if m.get("column_signature") else [],
|
||||||
|
"delimiter": m["delimiter"],
|
||||||
|
"encoding": m["encoding"],
|
||||||
|
"has_header": m["has_header"],
|
||||||
|
"field_mappings": m["field_mappings"],
|
||||||
|
"type_conversions": m.get("type_conversions"),
|
||||||
|
"import_row_processing": m.get("import_row_processing"),
|
||||||
|
"usage_count": m.get("usage_count"),
|
||||||
|
"success_rate": m.get("success_rate"),
|
||||||
|
"last_used_at": m.get("last_used_at"),
|
||||||
|
"created_at": m.get("created_at"),
|
||||||
|
"updated_at": m.get("updated_at"),
|
||||||
|
"is_system": m["is_system"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/import-limits")
|
||||||
|
def admin_get_csv_import_limits(session: dict = Depends(require_admin)):
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute("SELECT value FROM system_config WHERE key = %s", ("csv_import",))
|
||||||
|
row = cur.fetchone()
|
||||||
|
return get_csv_import_limits(r2d(row) if row else None)
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/import-limits")
|
||||||
|
def admin_put_csv_import_limits(body: CsvImportLimitsBody, session: dict = Depends(require_admin)):
|
||||||
|
payload = {"max_rows_per_file": body.max_rows_per_file, "max_file_bytes": body.max_file_bytes}
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO system_config (key, value, updated_at)
|
||||||
|
VALUES ('csv_import', %s, CURRENT_TIMESTAMP)
|
||||||
|
ON CONFLICT (key) DO UPDATE
|
||||||
|
SET value = EXCLUDED.value, updated_at = CURRENT_TIMESTAMP
|
||||||
|
""",
|
||||||
|
(Json(payload),),
|
||||||
|
)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("")
|
||||||
|
def list_system_templates(
|
||||||
|
module: Optional[str] = None,
|
||||||
|
session: dict = Depends(require_admin),
|
||||||
|
):
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT * FROM csv_field_mappings
|
||||||
|
WHERE is_system = true AND profile_id IS NULL
|
||||||
|
AND (%s::text IS NULL OR module = %s)
|
||||||
|
ORDER BY module, mapping_name
|
||||||
|
""",
|
||||||
|
(module, module),
|
||||||
|
)
|
||||||
|
rows = [r2d(r) for r in cur.fetchall()]
|
||||||
|
return {"templates": [_row_full(m) for m in rows]}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/analyze-upload")
|
||||||
|
async def admin_analyze_csv_for_template(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
module: str = Form(...),
|
||||||
|
delimiter: Optional[str] = Form(default=None),
|
||||||
|
seed_template_id: Optional[int] = Form(default=None),
|
||||||
|
session: dict = Depends(require_admin),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
CSV hochladen wie im Nutzer-Import: Spalten + Vorschau + Vorschläge für field_mappings
|
||||||
|
und type_conversions. Optional Seed-Vorlage (ID) oder beste Systemvorlage (Abdeckung, dann Jaccard).
|
||||||
|
"""
|
||||||
|
_ = session
|
||||||
|
if not get_module_definition(module):
|
||||||
|
raise HTTPException(400, f"Unbekanntes Modul: {module}")
|
||||||
|
|
||||||
|
raw = await file.read()
|
||||||
|
limits = _admin_csv_limits()
|
||||||
|
max_bytes = limits.get("max_file_bytes", 52_428_800)
|
||||||
|
if len(raw) > max_bytes:
|
||||||
|
raise HTTPException(
|
||||||
|
413,
|
||||||
|
f"Datei zu groß (max. {max_bytes} Bytes laut Systemkonfiguration)",
|
||||||
|
)
|
||||||
|
|
||||||
|
text = decode_raw_bytes(raw)
|
||||||
|
if not text.strip():
|
||||||
|
raise HTTPException(400, "Leere Datei")
|
||||||
|
|
||||||
|
max_rows = limits.get("max_rows_per_file", 50_000)
|
||||||
|
if text.count("\n") > max_rows + 5:
|
||||||
|
raise HTTPException(
|
||||||
|
413,
|
||||||
|
f"Zu viele Zeilen (>{max_rows}) laut Systemkonfiguration",
|
||||||
|
)
|
||||||
|
|
||||||
|
delim = delimiter if delimiter in (",", ";", "\t") else None
|
||||||
|
headers, sample_rows, used_delim = parse_csv_sample(text, delimiter=delim, max_data_rows=5)
|
||||||
|
if not headers:
|
||||||
|
raise HTTPException(400, "Keine Kopfzeile oder leeres CSV")
|
||||||
|
|
||||||
|
sig = column_signature(headers)
|
||||||
|
|
||||||
|
seed_row: dict | None = None
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
if seed_template_id is not None:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT * FROM csv_field_mappings
|
||||||
|
WHERE id = %s AND is_system = true AND profile_id IS NULL AND module = %s
|
||||||
|
""",
|
||||||
|
(seed_template_id, module),
|
||||||
|
)
|
||||||
|
seed_row = r2d(cur.fetchone())
|
||||||
|
if not seed_row:
|
||||||
|
raise HTTPException(404, "Seed-Vorlage nicht gefunden oder falsches Modul")
|
||||||
|
else:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT * FROM csv_field_mappings
|
||||||
|
WHERE is_system = true AND profile_id IS NULL AND module = %s
|
||||||
|
""",
|
||||||
|
(module,),
|
||||||
|
)
|
||||||
|
rows = [r2d(r) for r in cur.fetchall()]
|
||||||
|
best: dict | None = None
|
||||||
|
best_key: tuple[float, int, float] = (-1.0, -1, -1.0)
|
||||||
|
for t in rows:
|
||||||
|
t_sig = list(t.get("column_signature") or [])
|
||||||
|
m = headers_signature_rank_metrics(sig, t_sig)
|
||||||
|
key = (m["confidence"], m["columns_matched"], m["jaccard"])
|
||||||
|
if key > best_key:
|
||||||
|
best_key = key
|
||||||
|
best = t
|
||||||
|
if best and best_key[0] > 0:
|
||||||
|
seed_row = best
|
||||||
|
|
||||||
|
seed_fm = (seed_row or {}).get("field_mappings") or {}
|
||||||
|
if isinstance(seed_fm, str):
|
||||||
|
seed_fm = {}
|
||||||
|
seed_tc = (seed_row or {}).get("type_conversions")
|
||||||
|
if not isinstance(seed_tc, dict):
|
||||||
|
seed_tc = {}
|
||||||
|
|
||||||
|
field_mappings = suggest_field_mappings(headers, module, seed_fm if seed_fm else None)
|
||||||
|
type_conversions = build_type_conversions_for_mapping(module, field_mappings, seed_tc if seed_tc else None)
|
||||||
|
|
||||||
|
seed_meta = None
|
||||||
|
if seed_row:
|
||||||
|
t_sig = list(seed_row.get("column_signature") or [])
|
||||||
|
sm = headers_signature_rank_metrics(sig, t_sig)
|
||||||
|
seed_meta = {
|
||||||
|
"id": seed_row["id"],
|
||||||
|
"mapping_name": seed_row.get("mapping_name"),
|
||||||
|
"confidence": sm["confidence"],
|
||||||
|
"template_recall": sm["template_recall"],
|
||||||
|
"jaccard": sm["jaccard"],
|
||||||
|
"columns_matched": sm["columns_matched"],
|
||||||
|
"columns_in_template": sm["columns_in_template"],
|
||||||
|
"columns_in_csv": sm["columns_in_csv"],
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
"filename": file.filename,
|
||||||
|
"module": module,
|
||||||
|
"delimiter": used_delim,
|
||||||
|
"encoding": "utf-8",
|
||||||
|
"columns": headers,
|
||||||
|
"column_signature_normalized": sig,
|
||||||
|
"sample_rows": sample_rows,
|
||||||
|
"seed_template": seed_meta,
|
||||||
|
"field_mappings": field_mappings,
|
||||||
|
"type_conversions": type_conversions,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _admin_csv_limits() -> dict[str, int]:
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute("SELECT value FROM system_config WHERE key = %s", ("csv_import",))
|
||||||
|
row = cur.fetchone()
|
||||||
|
return get_csv_import_limits(r2d(row) if row else None)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/validate")
|
||||||
|
def validate_system_template_dry_run(body: CsvTemplateValidateBody, session: dict = Depends(require_admin)):
|
||||||
|
"""
|
||||||
|
Validatorlauf für eine Vorlagen-Konfiguration (ohne DB-Schreiben).
|
||||||
|
Nutzbar aus dem Admin-Editor vor dem Speichern.
|
||||||
|
"""
|
||||||
|
if not get_module_definition(body.module):
|
||||||
|
raise HTTPException(400, f"Unbekanntes Modul: {body.module}")
|
||||||
|
return validate_csv_template(
|
||||||
|
body.module,
|
||||||
|
body.field_mappings,
|
||||||
|
body.type_conversions,
|
||||||
|
body.import_row_processing,
|
||||||
|
body.column_signature,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{template_id}")
|
||||||
|
def get_system_template(template_id: int, session: dict = Depends(require_admin)):
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"SELECT * FROM csv_field_mappings WHERE id = %s AND is_system = true AND profile_id IS NULL",
|
||||||
|
(template_id,),
|
||||||
|
)
|
||||||
|
m = r2d(cur.fetchone())
|
||||||
|
if not m:
|
||||||
|
raise HTTPException(404, "System-Template nicht gefunden")
|
||||||
|
return _row_full(m)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("")
|
||||||
|
def create_system_template(body: CsvSystemTemplateCreate, session: dict = Depends(require_admin)):
|
||||||
|
if not get_module_definition(body.module):
|
||||||
|
raise HTTPException(400, f"Unbekanntes Modul: {body.module}")
|
||||||
|
report = validate_csv_template(
|
||||||
|
body.module,
|
||||||
|
body.field_mappings,
|
||||||
|
body.type_conversions,
|
||||||
|
body.import_row_processing,
|
||||||
|
body.column_signature,
|
||||||
|
)
|
||||||
|
if not report["valid"]:
|
||||||
|
raise HTTPException(status_code=422, detail=report)
|
||||||
|
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions, import_row_processing
|
||||||
|
) VALUES (
|
||||||
|
NULL, true, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s
|
||||||
|
) RETURNING id
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
body.module,
|
||||||
|
body.mapping_name,
|
||||||
|
body.description,
|
||||||
|
body.column_signature,
|
||||||
|
body.delimiter,
|
||||||
|
body.encoding,
|
||||||
|
body.has_header,
|
||||||
|
Json(body.field_mappings),
|
||||||
|
Json(body.type_conversions) if body.type_conversions is not None else None,
|
||||||
|
Json(body.import_row_processing) if body.import_row_processing is not None else None,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
new_id = cur.fetchone()["id"]
|
||||||
|
return {"id": new_id, "validation": report}
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/{template_id}")
|
||||||
|
def update_system_template(
|
||||||
|
template_id: int,
|
||||||
|
body: CsvSystemTemplateUpdate,
|
||||||
|
session: dict = Depends(require_admin),
|
||||||
|
):
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"SELECT * FROM csv_field_mappings WHERE id = %s AND is_system = true AND profile_id IS NULL",
|
||||||
|
(template_id,),
|
||||||
|
)
|
||||||
|
existing = r2d(cur.fetchone())
|
||||||
|
if not existing:
|
||||||
|
raise HTTPException(404, "System-Template nicht gefunden")
|
||||||
|
|
||||||
|
patch: dict[str, Any] = body.model_dump(exclude_unset=True)
|
||||||
|
if not patch:
|
||||||
|
return _row_full(existing)
|
||||||
|
|
||||||
|
fm = patch.get("field_mappings", existing["field_mappings"])
|
||||||
|
tc_eff = patch.get("type_conversions", existing.get("type_conversions"))
|
||||||
|
irp_eff = patch.get("import_row_processing", existing.get("import_row_processing"))
|
||||||
|
col_eff = patch.get("column_signature", existing.get("column_signature"))
|
||||||
|
|
||||||
|
report = validate_csv_template(
|
||||||
|
existing["module"],
|
||||||
|
fm,
|
||||||
|
tc_eff,
|
||||||
|
irp_eff,
|
||||||
|
col_eff if isinstance(col_eff, list) else None,
|
||||||
|
)
|
||||||
|
if not report["valid"]:
|
||||||
|
raise HTTPException(status_code=422, detail=report)
|
||||||
|
|
||||||
|
fields_sql = []
|
||||||
|
vals: list = []
|
||||||
|
if "mapping_name" in patch:
|
||||||
|
fields_sql.append("mapping_name = %s")
|
||||||
|
vals.append(patch["mapping_name"])
|
||||||
|
if "description" in patch:
|
||||||
|
fields_sql.append("description = %s")
|
||||||
|
vals.append(patch["description"])
|
||||||
|
if "column_signature" in patch:
|
||||||
|
fields_sql.append("column_signature = %s")
|
||||||
|
vals.append(patch["column_signature"])
|
||||||
|
if "delimiter" in patch:
|
||||||
|
fields_sql.append("delimiter = %s")
|
||||||
|
vals.append(patch["delimiter"])
|
||||||
|
if "encoding" in patch:
|
||||||
|
fields_sql.append("encoding = %s")
|
||||||
|
vals.append(patch["encoding"])
|
||||||
|
if "has_header" in patch:
|
||||||
|
fields_sql.append("has_header = %s")
|
||||||
|
vals.append(patch["has_header"])
|
||||||
|
if "field_mappings" in patch:
|
||||||
|
fields_sql.append("field_mappings = %s")
|
||||||
|
vals.append(Json(patch["field_mappings"]))
|
||||||
|
if "type_conversions" in patch:
|
||||||
|
fields_sql.append("type_conversions = %s")
|
||||||
|
tc = patch["type_conversions"]
|
||||||
|
vals.append(Json(tc) if tc is not None else None)
|
||||||
|
if "import_row_processing" in patch:
|
||||||
|
irp = patch["import_row_processing"]
|
||||||
|
fields_sql.append("import_row_processing = %s")
|
||||||
|
vals.append(Json(irp) if irp is not None else None)
|
||||||
|
|
||||||
|
fields_sql.append("updated_at = CURRENT_TIMESTAMP")
|
||||||
|
vals.append(template_id)
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
f"UPDATE csv_field_mappings SET {', '.join(fields_sql)} WHERE id = %s",
|
||||||
|
tuple(vals),
|
||||||
|
)
|
||||||
|
|
||||||
|
cur.execute("SELECT * FROM csv_field_mappings WHERE id = %s", (template_id,))
|
||||||
|
m = r2d(cur.fetchone())
|
||||||
|
return {**_row_full(m), "validation": report}
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/{template_id}")
|
||||||
|
def delete_system_template(template_id: int, session: dict = Depends(require_admin)):
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"DELETE FROM csv_field_mappings WHERE id = %s AND is_system = true AND profile_id IS NULL RETURNING id",
|
||||||
|
(template_id,),
|
||||||
|
)
|
||||||
|
row = cur.fetchone()
|
||||||
|
if not row:
|
||||||
|
raise HTTPException(404, "System-Template nicht gefunden")
|
||||||
|
return {"deleted": template_id}
|
||||||
@@ -0,0 +1,646 @@
|
|||||||
|
"""
|
||||||
|
CSV-Import: Nutzer-Endpunkte für Analyse, Mappings, Limits (Issue #21).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, File, Form, Header, HTTPException, UploadFile
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from psycopg2.extras import Json
|
||||||
|
|
||||||
|
from auth import require_auth, check_feature_access, increment_feature_usage
|
||||||
|
from feature_logger import log_feature_usage
|
||||||
|
from db import get_db, get_cursor, r2d
|
||||||
|
from routers.profiles import get_pid
|
||||||
|
from csv_parser.executor import (
|
||||||
|
diagnose_activity_row,
|
||||||
|
diagnose_blood_pressure_row,
|
||||||
|
diagnose_vitals_row,
|
||||||
|
run_universal_csv_import,
|
||||||
|
)
|
||||||
|
from csv_parser.core import (
|
||||||
|
decode_raw_bytes,
|
||||||
|
column_signature,
|
||||||
|
get_csv_import_limits,
|
||||||
|
headers_signature_rank_metrics,
|
||||||
|
iter_csv_dict_rows,
|
||||||
|
normalize_header_for_signature,
|
||||||
|
parse_csv_sample,
|
||||||
|
)
|
||||||
|
from csv_parser.type_converter import build_row_after_mapping, diagnose_row_mapping
|
||||||
|
from csv_parser.field_units import source_unit_choices_for_field
|
||||||
|
from csv_parser.import_errors import enrich_row_error
|
||||||
|
from csv_parser.module_registry import get_module_definition, list_modules, validate_field_mappings
|
||||||
|
from csv_parser.sleep_apple_import import detect_apple_sleep_csv_format
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/csv", tags=["csv-import"])
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_import_limits() -> dict[str, int]:
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute("SELECT value FROM system_config WHERE key = %s", ("csv_import",))
|
||||||
|
row = cur.fetchone()
|
||||||
|
return get_csv_import_limits(r2d(row) if row else None)
|
||||||
|
|
||||||
|
|
||||||
|
def _mapping_to_summary(m: dict) -> dict:
|
||||||
|
return {
|
||||||
|
"id": m["id"],
|
||||||
|
"module": m["module"],
|
||||||
|
"name": m["mapping_name"],
|
||||||
|
"description": m.get("description"),
|
||||||
|
"is_system": m["is_system"],
|
||||||
|
"usage_count": m.get("usage_count"),
|
||||||
|
"success_rate": m.get("success_rate"),
|
||||||
|
"last_used_at": m.get("last_used_at"),
|
||||||
|
"created_at": m.get("created_at"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/modules")
|
||||||
|
def csv_modules(session: dict = Depends(require_auth)):
|
||||||
|
"""Unterstützte Import-Module und Felddefinitionen."""
|
||||||
|
out = []
|
||||||
|
for mid in list_modules():
|
||||||
|
d = get_module_definition(mid)
|
||||||
|
if d:
|
||||||
|
fields_out = {}
|
||||||
|
for fname, finfo in (d.get("fields") or {}).items():
|
||||||
|
fd = dict(finfo)
|
||||||
|
opts = source_unit_choices_for_field(mid, fname)
|
||||||
|
if opts:
|
||||||
|
fd["source_unit_options"] = opts
|
||||||
|
fields_out[fname] = fd
|
||||||
|
out.append(
|
||||||
|
{
|
||||||
|
"id": mid,
|
||||||
|
"table": d["table"],
|
||||||
|
"fields": fields_out,
|
||||||
|
"import_mode": d.get("import_mode"),
|
||||||
|
"import_row_processing_default": d.get("import_row_processing_default"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {"modules": out}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/limits")
|
||||||
|
def csv_limits(session: dict = Depends(require_auth)):
|
||||||
|
"""Admin-konfigurierbare Import-Limits (system_config.csv_import)."""
|
||||||
|
return _load_import_limits()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/mappings")
|
||||||
|
def list_csv_mappings(
|
||||||
|
module: Optional[str] = None,
|
||||||
|
session: dict = Depends(require_auth),
|
||||||
|
):
|
||||||
|
"""System-Templates + eigene User-Mappings."""
|
||||||
|
pid = str(session["profile_id"])
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT id, module, mapping_name, description, is_system, profile_id,
|
||||||
|
usage_count, success_rate, last_used_at, created_at
|
||||||
|
FROM csv_field_mappings
|
||||||
|
WHERE is_system = true
|
||||||
|
AND (%s::text IS NULL OR module = %s)
|
||||||
|
ORDER BY usage_count DESC NULLS LAST, mapping_name
|
||||||
|
""",
|
||||||
|
(module, module),
|
||||||
|
)
|
||||||
|
system_rows = [r2d(r) for r in cur.fetchall()]
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT id, module, mapping_name, description, is_system, profile_id,
|
||||||
|
usage_count, success_rate, last_used_at, created_at
|
||||||
|
FROM csv_field_mappings
|
||||||
|
WHERE is_system = false AND profile_id = %s::uuid
|
||||||
|
AND (%s::text IS NULL OR module = %s)
|
||||||
|
ORDER BY last_used_at DESC NULLS LAST, mapping_name
|
||||||
|
""",
|
||||||
|
(pid, module, module),
|
||||||
|
)
|
||||||
|
user_rows = [r2d(r) for r in cur.fetchall()]
|
||||||
|
|
||||||
|
return {
|
||||||
|
"system_templates": [_mapping_to_summary(m) for m in system_rows],
|
||||||
|
"user_mappings": [_mapping_to_summary(m) for m in user_rows],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class CopyMappingBody(BaseModel):
|
||||||
|
name: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/mappings/{mapping_id}/copy")
|
||||||
|
def copy_csv_mapping(
|
||||||
|
mapping_id: int,
|
||||||
|
body: CopyMappingBody | None = None,
|
||||||
|
session: dict = Depends(require_auth),
|
||||||
|
):
|
||||||
|
"""System- oder eigenes Mapping als neues User-Mapping kopieren."""
|
||||||
|
pid = str(session["profile_id"])
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT * FROM csv_field_mappings WHERE id = %s
|
||||||
|
""",
|
||||||
|
(mapping_id,),
|
||||||
|
)
|
||||||
|
src = r2d(cur.fetchone())
|
||||||
|
if not src:
|
||||||
|
raise HTTPException(404, "Mapping nicht gefunden")
|
||||||
|
if not src["is_system"] and str(src.get("profile_id")) != pid:
|
||||||
|
raise HTTPException(403, "Kein Zugriff auf dieses Mapping")
|
||||||
|
|
||||||
|
base_name = (body.name if body and body.name else None) or f"{src['mapping_name']} (Kopie)"
|
||||||
|
name = base_name
|
||||||
|
n = 1
|
||||||
|
while True:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT 1 FROM csv_field_mappings
|
||||||
|
WHERE profile_id = %s::uuid AND module = %s AND mapping_name = %s
|
||||||
|
""",
|
||||||
|
(pid, src["module"], name),
|
||||||
|
)
|
||||||
|
if not cur.fetchone():
|
||||||
|
break
|
||||||
|
n += 1
|
||||||
|
name = f"{base_name} {n}"
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO csv_field_mappings (
|
||||||
|
profile_id, is_system, module, mapping_name, description,
|
||||||
|
column_signature, delimiter, encoding, has_header,
|
||||||
|
field_mappings, type_conversions, import_row_processing,
|
||||||
|
usage_count, success_rate
|
||||||
|
) VALUES (
|
||||||
|
%s::uuid, false, %s, %s, %s,
|
||||||
|
%s, %s, %s, %s, %s, %s, %s, 0, 1.0
|
||||||
|
) RETURNING id
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
pid,
|
||||||
|
src["module"],
|
||||||
|
name,
|
||||||
|
src.get("description"),
|
||||||
|
src["column_signature"],
|
||||||
|
src["delimiter"],
|
||||||
|
src["encoding"],
|
||||||
|
src["has_header"],
|
||||||
|
Json(src["field_mappings"]),
|
||||||
|
Json(src["type_conversions"]) if src.get("type_conversions") is not None else None,
|
||||||
|
Json(src["import_row_processing"])
|
||||||
|
if src.get("import_row_processing") is not None
|
||||||
|
else None,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
new_id = cur.fetchone()["id"]
|
||||||
|
return {"new_mapping_id": new_id, "mapping_name": name}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/analyze")
|
||||||
|
async def analyze_csv(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
module: Optional[str] = Form(default=None),
|
||||||
|
delimiter: Optional[str] = Form(default=None),
|
||||||
|
session: dict = Depends(require_auth),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Erste Zeilen parsen, Signatur bilden, Vorlagen nach Ähnlichkeit ranken.
|
||||||
|
Ohne `module`: alle System- + eigene User-Vorlagen (echte Format-Erkennung über Spalten-Signatur).
|
||||||
|
Mit `module`: nur Vorlagen dieses Moduls (Abwärtskompatibilität).
|
||||||
|
"""
|
||||||
|
if module and not get_module_definition(module):
|
||||||
|
raise HTTPException(400, f"Unbekanntes Modul: {module}")
|
||||||
|
|
||||||
|
pid = str(session["profile_id"])
|
||||||
|
raw = await file.read()
|
||||||
|
limits = _load_import_limits()
|
||||||
|
max_bytes = limits.get("max_file_bytes", 52_428_800)
|
||||||
|
if len(raw) > max_bytes:
|
||||||
|
raise HTTPException(
|
||||||
|
413,
|
||||||
|
f"Datei zu groß (max. {max_bytes} Bytes laut Systemkonfiguration)",
|
||||||
|
)
|
||||||
|
|
||||||
|
text = decode_raw_bytes(raw)
|
||||||
|
max_rows = limits.get("max_rows_per_file", 50_000)
|
||||||
|
if text.count("\n") > max_rows + 5:
|
||||||
|
raise HTTPException(
|
||||||
|
413,
|
||||||
|
f"Zu viele Zeilen (>{max_rows}) laut Systemkonfiguration csv_import.max_rows_per_file",
|
||||||
|
)
|
||||||
|
delim = delimiter if delimiter in (",", ";", "\t") else None
|
||||||
|
headers, sample_rows, used_delim = parse_csv_sample(text, delimiter=delim, max_data_rows=5)
|
||||||
|
sig = column_signature(headers)
|
||||||
|
|
||||||
|
apple_sleep_csv = False
|
||||||
|
try:
|
||||||
|
detect_apple_sleep_csv_format(headers)
|
||||||
|
apple_sleep_csv = True
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
mod_def = get_module_definition(module) if module else None
|
||||||
|
available_fields = mod_def["fields"] if mod_def else None
|
||||||
|
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
if module:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT id, module, mapping_name, description, column_signature,
|
||||||
|
delimiter, encoding, has_header, field_mappings, type_conversions, is_system
|
||||||
|
FROM csv_field_mappings
|
||||||
|
WHERE module = %s
|
||||||
|
AND (is_system = true OR (is_system = false AND profile_id = %s::uuid))
|
||||||
|
""",
|
||||||
|
(module, pid),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT id, module, mapping_name, description, column_signature,
|
||||||
|
delimiter, encoding, has_header, field_mappings, type_conversions, is_system
|
||||||
|
FROM csv_field_mappings
|
||||||
|
WHERE is_system = true OR (is_system = false AND profile_id = %s::uuid)
|
||||||
|
""",
|
||||||
|
(pid,),
|
||||||
|
)
|
||||||
|
templates = [r2d(r) for r in cur.fetchall()]
|
||||||
|
|
||||||
|
ranked = []
|
||||||
|
for t in templates:
|
||||||
|
t_sig = list(t["column_signature"]) if t["column_signature"] else []
|
||||||
|
metrics = headers_signature_rank_metrics(sig, t_sig)
|
||||||
|
conf = float(metrics["confidence"] or 0)
|
||||||
|
if apple_sleep_csv and t.get("module") == "sleep":
|
||||||
|
conf = max(conf, 1.0)
|
||||||
|
ranked.append(
|
||||||
|
{
|
||||||
|
"mapping_id": t["id"],
|
||||||
|
"module": t["module"],
|
||||||
|
"mapping_name": t["mapping_name"],
|
||||||
|
"is_system": bool(t.get("is_system")),
|
||||||
|
"confidence": round(conf, 4),
|
||||||
|
"template_recall": metrics["template_recall"],
|
||||||
|
"jaccard": metrics["jaccard"],
|
||||||
|
"columns_matched": metrics["columns_matched"],
|
||||||
|
"columns_in_template": metrics["columns_in_template"],
|
||||||
|
"columns_in_csv": metrics["columns_in_csv"],
|
||||||
|
"match_type": "template_recall",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
ranked.sort(
|
||||||
|
key=lambda x: (
|
||||||
|
-(x.get("confidence") or 0),
|
||||||
|
-(x.get("columns_matched") or 0),
|
||||||
|
-(x.get("jaccard") or 0),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
top = ranked[:25]
|
||||||
|
recommended = top[0] if top and (top[0]["confidence"] or 0) > 0 else None
|
||||||
|
|
||||||
|
warnings: list[str] = []
|
||||||
|
if apple_sleep_csv and not any(t.get("module") == "sleep" for t in templates):
|
||||||
|
warnings.append(
|
||||||
|
"Diese Datei ist ein Apple-Schlafexport, aber es fehlt eine Vorlage für das Modul «Schlaf» "
|
||||||
|
"(z. B. Migration 044 / Admin → CSV-Vorlagen)."
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
apple_sleep_csv
|
||||||
|
and recommended
|
||||||
|
and recommended.get("module") != "sleep"
|
||||||
|
and any(t.get("module") == "sleep" for t in templates)
|
||||||
|
):
|
||||||
|
warnings.append(
|
||||||
|
"Apple-Schlaf-CSV erkannt — bitte eine Vorlage unter Modul «Schlaf» wählen, nicht «"
|
||||||
|
+ str(recommended.get("module") or "")
|
||||||
|
+ "».",
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"module_filter": module,
|
||||||
|
"filename": file.filename,
|
||||||
|
"encoding_note": "utf-8/latin-1 mit BOM-Strip",
|
||||||
|
"delimiter": used_delim,
|
||||||
|
"columns": headers,
|
||||||
|
"column_signature_normalized": sig,
|
||||||
|
"sample_rows": sample_rows,
|
||||||
|
"detected_mappings": top,
|
||||||
|
"recommended": recommended,
|
||||||
|
"available_fields": available_fields,
|
||||||
|
"format_detection": {"apple_sleep": apple_sleep_csv},
|
||||||
|
"warnings": warnings,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/import-diagnose")
|
||||||
|
async def csv_import_diagnose(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
mapping_id: int = Form(...),
|
||||||
|
module: Optional[str] = Form(default=None),
|
||||||
|
x_profile_id: Optional[str] = Header(default=None),
|
||||||
|
session: dict = Depends(require_auth),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Kein Import, keine Quota: gleiche Datei + gewählte Vorlage wie beim Import,
|
||||||
|
aber nur die ersten Zeilen durch Mapping/Typkonvertierung (und Vital-Checks).
|
||||||
|
Zum Debugging wenn «Datum fehlt» o. Ä. ohne DB-Zugriff.
|
||||||
|
"""
|
||||||
|
pid = get_pid(x_profile_id)
|
||||||
|
raw = await file.read()
|
||||||
|
limits = _load_import_limits()
|
||||||
|
max_bytes = limits.get("max_file_bytes", 52_428_800)
|
||||||
|
if len(raw) > max_bytes:
|
||||||
|
raise HTTPException(413, f"Datei zu groß (max. {max_bytes} Bytes)")
|
||||||
|
text = decode_raw_bytes(raw)
|
||||||
|
if not text.strip():
|
||||||
|
raise HTTPException(400, "Leere Datei")
|
||||||
|
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
m = _fetch_mapping_row(cur, mapping_id, pid, module)
|
||||||
|
|
||||||
|
if not get_module_definition(m["module"]):
|
||||||
|
raise HTTPException(400, f"Modul nicht unterstützt: {m['module']}")
|
||||||
|
|
||||||
|
fm = m.get("field_mappings") or {}
|
||||||
|
if isinstance(fm, str):
|
||||||
|
fm = {}
|
||||||
|
tc = m.get("type_conversions")
|
||||||
|
if not isinstance(tc, dict):
|
||||||
|
tc = {}
|
||||||
|
delim = str(m.get("delimiter") or ",")
|
||||||
|
exec_module = str(m["module"])
|
||||||
|
|
||||||
|
rows_out: list[dict[str, Any]] = []
|
||||||
|
for i, row in enumerate(iter_csv_dict_rows(text, delim, has_header=True)):
|
||||||
|
if i >= 5:
|
||||||
|
break
|
||||||
|
typed = build_row_after_mapping(row, fm, tc, module=exec_module)
|
||||||
|
dm = diagnose_row_mapping(row, fm, tc, module=exec_module, mapped_typed=typed)
|
||||||
|
entry: dict[str, Any] = {
|
||||||
|
"row_index": i + 1,
|
||||||
|
**dm,
|
||||||
|
}
|
||||||
|
if exec_module == "vitals_baseline":
|
||||||
|
entry["vitals"] = diagnose_vitals_row(typed)
|
||||||
|
elif exec_module == "blood_pressure":
|
||||||
|
entry["blood_pressure"] = diagnose_blood_pressure_row(typed)
|
||||||
|
elif exec_module == "activity":
|
||||||
|
entry["activity"] = diagnose_activity_row(typed)
|
||||||
|
rows_out.append(entry)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"mapping_id": mapping_id,
|
||||||
|
"mapping_name": m.get("mapping_name"),
|
||||||
|
"module": exec_module,
|
||||||
|
"delimiter_used": delim,
|
||||||
|
"has_header": bool(m.get("has_header", True)),
|
||||||
|
"rows_diagnosed": len(rows_out),
|
||||||
|
"rows": rows_out,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_mapping_row(
|
||||||
|
cur,
|
||||||
|
mapping_id: int,
|
||||||
|
profile_id: str,
|
||||||
|
module: Optional[str] = None,
|
||||||
|
) -> dict:
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
SELECT * FROM csv_field_mappings WHERE id = %s
|
||||||
|
""",
|
||||||
|
(mapping_id,),
|
||||||
|
)
|
||||||
|
m = r2d(cur.fetchone())
|
||||||
|
if not m:
|
||||||
|
raise HTTPException(404, "Mapping nicht gefunden")
|
||||||
|
if module is not None and m.get("module") != module:
|
||||||
|
raise HTTPException(400, "Mapping gehört zu einem anderen Modul")
|
||||||
|
if not m.get("is_system"):
|
||||||
|
if str(m.get("profile_id") or "") != profile_id:
|
||||||
|
raise HTTPException(403, "Kein Zugriff auf dieses Mapping")
|
||||||
|
return m
|
||||||
|
|
||||||
|
|
||||||
|
def _check_module_feature_access(pid: str, module: str) -> None:
|
||||||
|
if module == "nutrition":
|
||||||
|
access = check_feature_access(pid, "nutrition_entries")
|
||||||
|
log_feature_usage(pid, "nutrition_entries", access, "csv_universal_import")
|
||||||
|
if not access["allowed"]:
|
||||||
|
raise HTTPException(
|
||||||
|
403,
|
||||||
|
f"Limit erreicht (Ernährungseinträge): {access.get('used')}/{access.get('limit')}",
|
||||||
|
)
|
||||||
|
elif module == "weight":
|
||||||
|
access = check_feature_access(pid, "weight_entries")
|
||||||
|
log_feature_usage(pid, "weight_entries", access, "csv_universal_import")
|
||||||
|
if not access["allowed"]:
|
||||||
|
raise HTTPException(
|
||||||
|
403,
|
||||||
|
f"Limit erreicht (Gewichtseinträge): {access.get('used')}/{access.get('limit')}",
|
||||||
|
)
|
||||||
|
elif module == "activity":
|
||||||
|
access = check_feature_access(pid, "activity_entries")
|
||||||
|
log_feature_usage(pid, "activity_entries", access, "csv_universal_import")
|
||||||
|
if not access["allowed"]:
|
||||||
|
raise HTTPException(
|
||||||
|
403,
|
||||||
|
f"Limit erreicht (Aktivitätseinträge): {access.get('used')}/{access.get('limit')}",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/import")
|
||||||
|
async def csv_import_execute(
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
mapping_id: int = Form(...),
|
||||||
|
module: Optional[str] = Form(default=None),
|
||||||
|
x_profile_id: Optional[str] = Header(default=None),
|
||||||
|
session: dict = Depends(require_auth),
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Universal-CSV-Import: Zielmodul kommt aus der gewählten Vorlage (`mapping_id`).
|
||||||
|
Optional `module` dient nur der Absicherung (muss mit Vorlage übereinstimmen).
|
||||||
|
"""
|
||||||
|
pid = get_pid(x_profile_id)
|
||||||
|
|
||||||
|
access_di = check_feature_access(pid, "data_import")
|
||||||
|
log_feature_usage(pid, "data_import", access_di, "csv_universal_import")
|
||||||
|
if not access_di["allowed"]:
|
||||||
|
raise HTTPException(
|
||||||
|
403,
|
||||||
|
"Limit erreicht (Daten importieren): "
|
||||||
|
f"{access_di.get('used')}/{access_di.get('limit')}",
|
||||||
|
)
|
||||||
|
|
||||||
|
raw = await file.read()
|
||||||
|
limits = _load_import_limits()
|
||||||
|
max_bytes = limits.get("max_file_bytes", 52_428_800)
|
||||||
|
if len(raw) > max_bytes:
|
||||||
|
raise HTTPException(
|
||||||
|
413,
|
||||||
|
f"Datei zu groß (max. {max_bytes} Bytes laut Systemkonfiguration)",
|
||||||
|
)
|
||||||
|
text = decode_raw_bytes(raw)
|
||||||
|
if not text.strip():
|
||||||
|
raise HTTPException(400, "Leere Datei")
|
||||||
|
max_rows = limits.get("max_rows_per_file", 50_000)
|
||||||
|
if text.count("\n") > max_rows + 5:
|
||||||
|
raise HTTPException(
|
||||||
|
413,
|
||||||
|
f"Zu viele Zeilen (>{max_rows}) laut Systemkonfiguration",
|
||||||
|
)
|
||||||
|
|
||||||
|
log_id: int | None = None
|
||||||
|
err_response: HTTPException | None = None
|
||||||
|
result: dict | None = None
|
||||||
|
resolved_module: str | None = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
m = _fetch_mapping_row(cur, mapping_id, pid, module)
|
||||||
|
exec_module = m["module"]
|
||||||
|
resolved_module = exec_module
|
||||||
|
|
||||||
|
if not get_module_definition(exec_module):
|
||||||
|
raise HTTPException(
|
||||||
|
400,
|
||||||
|
f"Modul der Vorlage wird vom Importer noch nicht unterstützt: {exec_module}",
|
||||||
|
)
|
||||||
|
|
||||||
|
_check_module_feature_access(pid, exec_module)
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO csv_import_log (
|
||||||
|
profile_id, mapping_id, module, filename,
|
||||||
|
rows_total, rows_imported, rows_updated, rows_skipped, rows_errors,
|
||||||
|
status, error_details, affected_ids
|
||||||
|
) VALUES (
|
||||||
|
%s::uuid, %s, %s, %s,
|
||||||
|
0, 0, 0, 0, 0,
|
||||||
|
'running', NULL, NULL
|
||||||
|
) RETURNING id
|
||||||
|
""",
|
||||||
|
(pid, mapping_id, exec_module, file.filename or "upload.csv"),
|
||||||
|
)
|
||||||
|
log_id = cur.fetchone()["id"]
|
||||||
|
|
||||||
|
cur.execute("SAVEPOINT csv_import_exec")
|
||||||
|
try:
|
||||||
|
result = run_universal_csv_import(
|
||||||
|
cur,
|
||||||
|
pid,
|
||||||
|
exec_module,
|
||||||
|
text,
|
||||||
|
file.filename or "upload.csv",
|
||||||
|
m,
|
||||||
|
)
|
||||||
|
except Exception as exec_err:
|
||||||
|
logger.exception("Universal-CSV-Import fehlgeschlagen: %s", exec_err)
|
||||||
|
cur.execute("ROLLBACK TO SAVEPOINT csv_import_exec")
|
||||||
|
err_payload = {"error": str(exec_err), **enrich_row_error(str(exec_err), exec_module)}
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
UPDATE csv_import_log SET
|
||||||
|
finished_at = CURRENT_TIMESTAMP,
|
||||||
|
status = 'failed',
|
||||||
|
error_details = %s
|
||||||
|
WHERE id = %s
|
||||||
|
""",
|
||||||
|
(Json([err_payload]), log_id),
|
||||||
|
)
|
||||||
|
hint = err_payload.get("hint")
|
||||||
|
msg = f"Import fehlgeschlagen: {exec_err}"
|
||||||
|
if hint:
|
||||||
|
msg = f"{msg} ({hint})"
|
||||||
|
err_response = HTTPException(500, msg)
|
||||||
|
else:
|
||||||
|
cur.execute("RELEASE SAVEPOINT csv_import_exec")
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
UPDATE csv_import_log SET
|
||||||
|
finished_at = CURRENT_TIMESTAMP,
|
||||||
|
status = 'success',
|
||||||
|
rows_total = %s,
|
||||||
|
rows_imported = %s,
|
||||||
|
rows_updated = %s,
|
||||||
|
rows_skipped = %s,
|
||||||
|
rows_errors = %s,
|
||||||
|
error_details = %s,
|
||||||
|
affected_ids = %s
|
||||||
|
WHERE id = %s
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
result["rows_total"],
|
||||||
|
result["rows_imported"],
|
||||||
|
result["rows_updated"],
|
||||||
|
result["rows_skipped"],
|
||||||
|
result["rows_errors"],
|
||||||
|
Json(result["error_details"]),
|
||||||
|
Json(result["affected_ids"]),
|
||||||
|
log_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
cur.execute(
|
||||||
|
"""
|
||||||
|
UPDATE csv_field_mappings SET
|
||||||
|
usage_count = usage_count + 1,
|
||||||
|
last_used_at = CURRENT_TIMESTAMP,
|
||||||
|
updated_at = CURRENT_TIMESTAMP
|
||||||
|
WHERE id = %s
|
||||||
|
""",
|
||||||
|
(mapping_id,),
|
||||||
|
)
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
|
||||||
|
if err_response:
|
||||||
|
raise err_response
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
|
||||||
|
increment_feature_usage(pid, "data_import")
|
||||||
|
|
||||||
|
ne = result.get("new_entries", result["rows_imported"])
|
||||||
|
if resolved_module == "nutrition":
|
||||||
|
for _ in range(ne):
|
||||||
|
increment_feature_usage(pid, "nutrition_entries")
|
||||||
|
elif resolved_module == "weight":
|
||||||
|
for _ in range(ne):
|
||||||
|
increment_feature_usage(pid, "weight_entries")
|
||||||
|
elif resolved_module == "activity":
|
||||||
|
for _ in range(ne):
|
||||||
|
increment_feature_usage(pid, "activity_entries")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"success": True,
|
||||||
|
"import_log_id": log_id,
|
||||||
|
"module": resolved_module,
|
||||||
|
"stats": {
|
||||||
|
"total_rows": result["rows_total"],
|
||||||
|
"imported": result["rows_imported"],
|
||||||
|
"updated": result["rows_updated"],
|
||||||
|
"skipped": result["rows_skipped"],
|
||||||
|
"errors": result["rows_errors"],
|
||||||
|
},
|
||||||
|
"error_details": result["error_details"],
|
||||||
|
}
|
||||||
+212
-133
@@ -31,6 +31,42 @@ OPENROUTER_MODEL = os.getenv("OPENROUTER_MODEL", "anthropic/claude-sonnet-4")
|
|||||||
|
|
||||||
router = APIRouter(prefix="/api/prompts", tags=["prompts"])
|
router = APIRouter(prefix="/api/prompts", tags=["prompts"])
|
||||||
|
|
||||||
|
# Metadaten-Schlüssel in workflow aggregate_results (nicht als „einziger“ Nutzer-Output)
|
||||||
|
_WORKFLOW_AGG_META_KEYS = frozenset({
|
||||||
|
"combined_analysis",
|
||||||
|
"all_signals",
|
||||||
|
"total_nodes",
|
||||||
|
"executed_nodes",
|
||||||
|
"failed_nodes",
|
||||||
|
"skipped_nodes",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
def _workflow_user_facing_content(agg: object) -> str:
|
||||||
|
"""
|
||||||
|
Nutzer-sichtbarer Text wie im Admin WorkflowResultViewer („Final Output“):
|
||||||
|
primär aggregated_result['analysis_core'], nicht das gesamte JSON.
|
||||||
|
"""
|
||||||
|
if agg is None:
|
||||||
|
return ""
|
||||||
|
if isinstance(agg, str):
|
||||||
|
return agg
|
||||||
|
if not isinstance(agg, dict):
|
||||||
|
return json.dumps(agg, ensure_ascii=False)
|
||||||
|
core = agg.get("analysis_core")
|
||||||
|
if isinstance(core, str) and core.strip():
|
||||||
|
return core
|
||||||
|
combined = agg.get("combined_analysis")
|
||||||
|
if isinstance(combined, str) and combined.strip():
|
||||||
|
return combined
|
||||||
|
non_meta = [k for k in agg.keys() if k not in _WORKFLOW_AGG_META_KEYS]
|
||||||
|
if len(non_meta) == 1:
|
||||||
|
v = agg[non_meta[0]]
|
||||||
|
if isinstance(v, str):
|
||||||
|
return v
|
||||||
|
return json.dumps(v, ensure_ascii=False)
|
||||||
|
return json.dumps(agg, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
@router.get("")
|
@router.get("")
|
||||||
def list_prompts(session: dict=Depends(require_auth)):
|
def list_prompts(session: dict=Depends(require_auth)):
|
||||||
@@ -67,6 +103,156 @@ def list_placeholders_endpoint(session: dict=Depends(require_auth)):
|
|||||||
return get_placeholder_catalog(profile_id)
|
return get_placeholder_catalog(profile_id)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/export-all")
|
||||||
|
def export_all_prompts(session: dict = Depends(require_admin)):
|
||||||
|
"""
|
||||||
|
Export all prompts as JSON array.
|
||||||
|
Admin only. Used for backup and dev→prod sync.
|
||||||
|
|
||||||
|
IMPORTANT: Must be defined BEFORE /{prompt_id} route to avoid routing conflict.
|
||||||
|
"""
|
||||||
|
from datetime import datetime
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute("SELECT * FROM ai_prompts ORDER BY sort_order, slug")
|
||||||
|
prompts = [r2d(row) for row in cur.fetchall()]
|
||||||
|
|
||||||
|
# Convert to export format (clean up DB-specific fields)
|
||||||
|
export_data = []
|
||||||
|
for p in prompts:
|
||||||
|
export_item = {
|
||||||
|
'slug': p['slug'],
|
||||||
|
'name': p['name'],
|
||||||
|
'display_name': p.get('display_name'),
|
||||||
|
'description': p.get('description'),
|
||||||
|
'type': p.get('type', 'pipeline'),
|
||||||
|
'category': p.get('category', 'ganzheitlich'),
|
||||||
|
'template': p.get('template'),
|
||||||
|
'stages': p.get('stages'),
|
||||||
|
'output_format': p.get('output_format', 'text'),
|
||||||
|
'output_schema': p.get('output_schema'),
|
||||||
|
'question_augmentations': p.get('question_augmentations'),
|
||||||
|
'graph_data': p.get('graph_data'),
|
||||||
|
'active': p.get('active', True),
|
||||||
|
'sort_order': p.get('sort_order', 0)
|
||||||
|
}
|
||||||
|
export_data.append(export_item)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'export_date': datetime.now().isoformat(),
|
||||||
|
'count': len(export_data),
|
||||||
|
'prompts': export_data
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/import")
|
||||||
|
def import_prompts(
|
||||||
|
data: dict,
|
||||||
|
overwrite: bool = False,
|
||||||
|
session: dict = Depends(require_admin)
|
||||||
|
):
|
||||||
|
"""
|
||||||
|
Import prompts from JSON export.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
data: Export data from /export-all endpoint
|
||||||
|
overwrite: If true, update existing prompts. If false, skip duplicates.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Summary of import results (created, updated, skipped)
|
||||||
|
|
||||||
|
IMPORTANT: Must be defined BEFORE /{prompt_id} route to avoid routing conflict.
|
||||||
|
"""
|
||||||
|
if 'prompts' not in data:
|
||||||
|
raise HTTPException(400, "Invalid import data: missing 'prompts' key")
|
||||||
|
|
||||||
|
prompts = data['prompts']
|
||||||
|
created = 0
|
||||||
|
updated = 0
|
||||||
|
skipped = 0
|
||||||
|
errors = []
|
||||||
|
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
|
||||||
|
for p in prompts:
|
||||||
|
slug = p.get('slug')
|
||||||
|
if not slug:
|
||||||
|
errors.append('Prompt without slug skipped')
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Check if exists
|
||||||
|
cur.execute("SELECT id FROM ai_prompts WHERE slug=%s", (slug,))
|
||||||
|
existing = cur.fetchone()
|
||||||
|
|
||||||
|
if existing and not overwrite:
|
||||||
|
skipped += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Prepare JSON fields if present
|
||||||
|
stages_json = None
|
||||||
|
if p.get('stages'):
|
||||||
|
stages_json = json.dumps(p['stages']) if isinstance(p['stages'], list) else p['stages']
|
||||||
|
|
||||||
|
output_schema_json = None
|
||||||
|
if p.get('output_schema'):
|
||||||
|
output_schema_json = json.dumps(p['output_schema']) if isinstance(p['output_schema'], dict) else p['output_schema']
|
||||||
|
|
||||||
|
question_aug_json = None
|
||||||
|
if p.get('question_augmentations'):
|
||||||
|
question_aug_json = json.dumps(p['question_augmentations']) if isinstance(p['question_augmentations'], (dict, list)) else p['question_augmentations']
|
||||||
|
|
||||||
|
graph_data_json = None
|
||||||
|
if p.get('graph_data'):
|
||||||
|
graph_data_json = json.dumps(p['graph_data']) if isinstance(p['graph_data'], dict) else p['graph_data']
|
||||||
|
|
||||||
|
if existing:
|
||||||
|
# Update existing
|
||||||
|
cur.execute("""
|
||||||
|
UPDATE ai_prompts SET
|
||||||
|
name=%s, display_name=%s, description=%s, type=%s,
|
||||||
|
category=%s, template=%s, stages=%s, output_format=%s,
|
||||||
|
output_schema=%s, question_augmentations=%s, graph_data=%s,
|
||||||
|
active=%s, sort_order=%s,
|
||||||
|
updated=CURRENT_TIMESTAMP
|
||||||
|
WHERE slug=%s
|
||||||
|
""", (
|
||||||
|
p.get('name'), p.get('display_name'), p.get('description'),
|
||||||
|
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
|
||||||
|
p.get('template'), stages_json, p.get('output_format', 'text'),
|
||||||
|
output_schema_json, question_aug_json, graph_data_json,
|
||||||
|
p.get('active', True), p.get('sort_order', 0), slug
|
||||||
|
))
|
||||||
|
updated += 1
|
||||||
|
else:
|
||||||
|
# Create new
|
||||||
|
cur.execute("""
|
||||||
|
INSERT INTO ai_prompts (
|
||||||
|
slug, name, display_name, description, type, category,
|
||||||
|
template, stages, output_format, output_schema,
|
||||||
|
question_augmentations, graph_data,
|
||||||
|
active, sort_order, created, updated
|
||||||
|
) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP)
|
||||||
|
""", (
|
||||||
|
slug, p.get('name'), p.get('display_name'), p.get('description'),
|
||||||
|
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
|
||||||
|
p.get('template'), stages_json, p.get('output_format', 'text'),
|
||||||
|
output_schema_json, question_aug_json, graph_data_json,
|
||||||
|
p.get('active', True), p.get('sort_order', 0)
|
||||||
|
))
|
||||||
|
created += 1
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
return {
|
||||||
|
'success': True,
|
||||||
|
'created': created,
|
||||||
|
'updated': updated,
|
||||||
|
'skipped': skipped,
|
||||||
|
'errors': errors if errors else None
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
@router.get("/{prompt_id}")
|
@router.get("/{prompt_id}")
|
||||||
def get_prompt(prompt_id: str, session: dict=Depends(require_auth)):
|
def get_prompt(prompt_id: str, session: dict=Depends(require_auth)):
|
||||||
"""Get single AI prompt by ID (UUID)."""
|
"""Get single AI prompt by ID (UUID)."""
|
||||||
@@ -178,11 +364,31 @@ def duplicate_prompt(prompt_id: str, session: dict=Depends(require_admin)):
|
|||||||
|
|
||||||
new_display_name = f"{original.get('display_name') or original['name']} (Kopie)"
|
new_display_name = f"{original.get('display_name') or original['name']} (Kopie)"
|
||||||
|
|
||||||
|
# Prepare JSONB fields (convert dict/list to JSON string if needed)
|
||||||
|
stages_json = json.dumps(original['stages']) if original.get('stages') else None
|
||||||
|
output_schema_json = json.dumps(original['output_schema']) if original.get('output_schema') else None
|
||||||
|
question_aug_json = json.dumps(original['question_augmentations']) if original.get('question_augmentations') else None
|
||||||
|
graph_data_json = json.dumps(original['graph_data']) if original.get('graph_data') else None
|
||||||
|
|
||||||
cur.execute(
|
cur.execute(
|
||||||
"""INSERT INTO ai_prompts (id, name, slug, display_name, description, template, category, active, sort_order, created, updated)
|
"""INSERT INTO ai_prompts (
|
||||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)""",
|
id, name, slug, display_name, description, template, category,
|
||||||
(new_id, new_name, new_slug, new_display_name, original['description'], original['template'],
|
type, stages, output_format, output_schema,
|
||||||
original.get('category', 'ganzheitlich'), original['active'], original['sort_order'])
|
question_augmentations, graph_data,
|
||||||
|
active, sort_order, created, updated
|
||||||
|
) VALUES (
|
||||||
|
%s, %s, %s, %s, %s, %s, %s,
|
||||||
|
%s, %s, %s, %s,
|
||||||
|
%s, %s,
|
||||||
|
%s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
|
||||||
|
)""",
|
||||||
|
(new_id, new_name, new_slug, new_display_name,
|
||||||
|
original.get('description'), original.get('template'),
|
||||||
|
original.get('category', 'ganzheitlich'),
|
||||||
|
original.get('type', 'pipeline'), stages_json,
|
||||||
|
original.get('output_format', 'text'), output_schema_json,
|
||||||
|
question_aug_json, graph_data_json,
|
||||||
|
original.get('active', True), original.get('sort_order', 0))
|
||||||
)
|
)
|
||||||
|
|
||||||
return {"id": new_id, "slug": new_slug, "name": new_name}
|
return {"id": new_id, "slug": new_slug, "name": new_name}
|
||||||
@@ -1253,6 +1459,8 @@ async def execute_unified_prompt(
|
|||||||
content = list(final_output.values())[0]
|
content = list(final_output.values())[0]
|
||||||
else:
|
else:
|
||||||
content = json.dumps(final_output, ensure_ascii=False)
|
content = json.dumps(final_output, ensure_ascii=False)
|
||||||
|
elif result['type'] == 'workflow':
|
||||||
|
content = _workflow_user_facing_content(result.get('aggregated_result'))
|
||||||
else:
|
else:
|
||||||
# For base prompts, use output directly
|
# For base prompts, use output directly
|
||||||
content = result.get('output', '')
|
content = result.get('output', '')
|
||||||
@@ -1569,132 +1777,3 @@ def update_unified_prompt(prompt_id: str, p: UnifiedPromptUpdate, session: dict
|
|||||||
)
|
)
|
||||||
|
|
||||||
return {"ok": True}
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
@router.get("/export-all")
|
|
||||||
def export_all_prompts(session: dict = Depends(require_admin)):
|
|
||||||
"""
|
|
||||||
Export all prompts as JSON array.
|
|
||||||
Admin only. Used for backup and dev→prod sync.
|
|
||||||
"""
|
|
||||||
from datetime import datetime
|
|
||||||
with get_db() as conn:
|
|
||||||
cur = get_cursor(conn)
|
|
||||||
cur.execute("SELECT * FROM ai_prompts ORDER BY sort_order, slug")
|
|
||||||
prompts = [r2d(row) for row in cur.fetchall()]
|
|
||||||
|
|
||||||
# Convert to export format (clean up DB-specific fields)
|
|
||||||
export_data = []
|
|
||||||
for p in prompts:
|
|
||||||
export_item = {
|
|
||||||
'slug': p['slug'],
|
|
||||||
'name': p['name'],
|
|
||||||
'display_name': p.get('display_name'),
|
|
||||||
'description': p.get('description'),
|
|
||||||
'type': p.get('type', 'pipeline'),
|
|
||||||
'category': p.get('category', 'ganzheitlich'),
|
|
||||||
'template': p.get('template'),
|
|
||||||
'stages': p.get('stages'),
|
|
||||||
'output_format': p.get('output_format', 'text'),
|
|
||||||
'output_schema': p.get('output_schema'),
|
|
||||||
'active': p.get('active', True),
|
|
||||||
'sort_order': p.get('sort_order', 0)
|
|
||||||
}
|
|
||||||
export_data.append(export_item)
|
|
||||||
|
|
||||||
return {
|
|
||||||
'export_date': datetime.now().isoformat(),
|
|
||||||
'count': len(export_data),
|
|
||||||
'prompts': export_data
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/import")
|
|
||||||
def import_prompts(
|
|
||||||
data: dict,
|
|
||||||
overwrite: bool = False,
|
|
||||||
session: dict = Depends(require_admin)
|
|
||||||
):
|
|
||||||
"""
|
|
||||||
Import prompts from JSON export.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
data: Export data from /export-all endpoint
|
|
||||||
overwrite: If true, update existing prompts. If false, skip duplicates.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Summary of import results (created, updated, skipped)
|
|
||||||
"""
|
|
||||||
if 'prompts' not in data:
|
|
||||||
raise HTTPException(400, "Invalid import data: missing 'prompts' key")
|
|
||||||
|
|
||||||
prompts = data['prompts']
|
|
||||||
created = 0
|
|
||||||
updated = 0
|
|
||||||
skipped = 0
|
|
||||||
errors = []
|
|
||||||
|
|
||||||
with get_db() as conn:
|
|
||||||
cur = get_cursor(conn)
|
|
||||||
|
|
||||||
for p in prompts:
|
|
||||||
slug = p.get('slug')
|
|
||||||
if not slug:
|
|
||||||
errors.append('Prompt without slug skipped')
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Check if exists
|
|
||||||
cur.execute("SELECT id FROM ai_prompts WHERE slug=%s", (slug,))
|
|
||||||
existing = cur.fetchone()
|
|
||||||
|
|
||||||
if existing and not overwrite:
|
|
||||||
skipped += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Prepare stages JSON if present
|
|
||||||
stages_json = None
|
|
||||||
if p.get('stages'):
|
|
||||||
stages_json = json.dumps(p['stages']) if isinstance(p['stages'], list) else p['stages']
|
|
||||||
|
|
||||||
if existing:
|
|
||||||
# Update existing
|
|
||||||
cur.execute("""
|
|
||||||
UPDATE ai_prompts SET
|
|
||||||
name=%s, display_name=%s, description=%s, type=%s,
|
|
||||||
category=%s, template=%s, stages=%s, output_format=%s,
|
|
||||||
output_schema=%s, active=%s, sort_order=%s,
|
|
||||||
updated=CURRENT_TIMESTAMP
|
|
||||||
WHERE slug=%s
|
|
||||||
""", (
|
|
||||||
p.get('name'), p.get('display_name'), p.get('description'),
|
|
||||||
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
|
|
||||||
p.get('template'), stages_json, p.get('output_format', 'text'),
|
|
||||||
p.get('output_schema'), p.get('active', True),
|
|
||||||
p.get('sort_order', 0), slug
|
|
||||||
))
|
|
||||||
updated += 1
|
|
||||||
else:
|
|
||||||
# Create new
|
|
||||||
cur.execute("""
|
|
||||||
INSERT INTO ai_prompts (
|
|
||||||
slug, name, display_name, description, type, category,
|
|
||||||
template, stages, output_format, output_schema,
|
|
||||||
active, sort_order, created, updated
|
|
||||||
) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP)
|
|
||||||
""", (
|
|
||||||
slug, p.get('name'), p.get('display_name'), p.get('description'),
|
|
||||||
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
|
|
||||||
p.get('template'), stages_json, p.get('output_format', 'text'),
|
|
||||||
p.get('output_schema'), p.get('active', True), p.get('sort_order', 0)
|
|
||||||
))
|
|
||||||
created += 1
|
|
||||||
|
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
return {
|
|
||||||
'success': True,
|
|
||||||
'created': created,
|
|
||||||
'updated': updated,
|
|
||||||
'skipped': skipped,
|
|
||||||
'errors': errors if errors else None
|
|
||||||
}
|
|
||||||
+15
-299
@@ -8,186 +8,16 @@ Endpoints:
|
|||||||
"""
|
"""
|
||||||
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File
|
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
from typing import Literal
|
|
||||||
from datetime import datetime, timedelta, date
|
from datetime import datetime, timedelta, date
|
||||||
from decimal import Decimal, InvalidOperation
|
from typing import Literal
|
||||||
import csv
|
|
||||||
import io
|
|
||||||
import json
|
|
||||||
|
|
||||||
from auth import require_auth
|
from auth import require_auth
|
||||||
|
from csv_parser.sleep_apple_import import import_apple_sleep_nights
|
||||||
from db import get_db, get_cursor
|
from db import get_db, get_cursor
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/sleep", tags=["sleep"])
|
router = APIRouter(prefix="/api/sleep", tags=["sleep"])
|
||||||
|
|
||||||
|
|
||||||
def _strip_row_keys(row: dict) -> dict:
|
|
||||||
return {(k or "").strip(): (v.strip() if isinstance(v, str) else v) for k, v in row.items()}
|
|
||||||
|
|
||||||
|
|
||||||
def _safe_float(value) -> float | None:
|
|
||||||
if value is None or value == "":
|
|
||||||
return None
|
|
||||||
if isinstance(value, (int, float)):
|
|
||||||
return float(value)
|
|
||||||
if isinstance(value, Decimal):
|
|
||||||
return float(value)
|
|
||||||
try:
|
|
||||||
s = str(value).strip().replace(",", ".")
|
|
||||||
return float(s)
|
|
||||||
except (ValueError, InvalidOperation):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_apple_sleep_datetime(value: str) -> datetime:
|
|
||||||
raw = (value or "").strip()
|
|
||||||
if not raw:
|
|
||||||
raise ValueError("empty datetime")
|
|
||||||
fmts = (
|
|
||||||
"%Y-%m-%d %H:%M:%S %z",
|
|
||||||
"%d.%m.%y %H:%M:%S",
|
|
||||||
"%d.%m.%Y %H:%M:%S",
|
|
||||||
"%Y-%m-%d %H:%M:%S",
|
|
||||||
)
|
|
||||||
last_err = None
|
|
||||||
for fmt in fmts:
|
|
||||||
try:
|
|
||||||
return datetime.strptime(raw, fmt)
|
|
||||||
except ValueError as e:
|
|
||||||
last_err = e
|
|
||||||
raise ValueError(f"Unbekanntes Datumsformat: {raw!r}") from last_err
|
|
||||||
|
|
||||||
|
|
||||||
def _hr_to_minutes(hours: float | None) -> int:
|
|
||||||
if hours is None:
|
|
||||||
return 0
|
|
||||||
return int(round(float(hours) * 60))
|
|
||||||
|
|
||||||
|
|
||||||
def _detect_apple_sleep_csv_format(fieldnames: list[str] | None) -> Literal["segments", "summary"]:
|
|
||||||
if not fieldnames:
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=400,
|
|
||||||
detail="CSV enthält keine Spaltenüberschriften.",
|
|
||||||
)
|
|
||||||
fn = {(f or "").strip() for f in fieldnames}
|
|
||||||
if {"Start", "End", "Duration (hr)", "Value"}.issubset(fn):
|
|
||||||
return "segments"
|
|
||||||
if "Start" in fn and "End" in fn and "Total Sleep (hr)" in fn:
|
|
||||||
return "summary"
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=400,
|
|
||||||
detail=(
|
|
||||||
"Unbekanntes Apple-Health-Schlaf-CSV. "
|
|
||||||
"Erwartet wird entweder der Segment-Export (Spalten Start, End, Duration (hr), Value) "
|
|
||||||
"oder die Schlafanalyse mit Nacht-Zusammenfassung (u. a. Total Sleep (hr), Core/Light, Deep, REM)."
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _build_nights_from_apple_summary(reader: csv.DictReader) -> dict[date, dict]:
|
|
||||||
nights_dict: dict[date, dict] = {}
|
|
||||||
for raw in reader:
|
|
||||||
row = _strip_row_keys(raw)
|
|
||||||
start_s = row.get("Start") or ""
|
|
||||||
end_s = row.get("End") or ""
|
|
||||||
if not start_s or not end_s:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
start_dt = _parse_apple_sleep_datetime(start_s)
|
|
||||||
end_dt = _parse_apple_sleep_datetime(end_s)
|
|
||||||
except ValueError:
|
|
||||||
continue
|
|
||||||
|
|
||||||
dt_key = (row.get("Date/Time") or row.get("Datum/Uhrzeit") or "").strip()
|
|
||||||
if dt_key:
|
|
||||||
try:
|
|
||||||
wake_d = datetime.strptime(dt_key[:10], "%Y-%m-%d").date()
|
|
||||||
except ValueError:
|
|
||||||
wake_d = end_dt.date()
|
|
||||||
else:
|
|
||||||
wake_d = end_dt.date()
|
|
||||||
|
|
||||||
core_hr = _safe_float(row.get("Core (hr)"))
|
|
||||||
if core_hr is None:
|
|
||||||
core_hr = _safe_float(row.get("Light (hr)")) or 0.0
|
|
||||||
deep_min = _hr_to_minutes(_safe_float(row.get("Deep (hr)")))
|
|
||||||
rem_min = _hr_to_minutes(_safe_float(row.get("REM (hr)")))
|
|
||||||
light_min = _hr_to_minutes(core_hr)
|
|
||||||
awake_min = _hr_to_minutes(_safe_float(row.get("Awake (hr)")))
|
|
||||||
nights_dict[wake_d] = {
|
|
||||||
"bedtime": start_dt,
|
|
||||||
"wake_time": end_dt,
|
|
||||||
"segments": [],
|
|
||||||
"deep_minutes": deep_min,
|
|
||||||
"rem_minutes": rem_min,
|
|
||||||
"light_minutes": light_min,
|
|
||||||
"awake_minutes": awake_min,
|
|
||||||
}
|
|
||||||
return nights_dict
|
|
||||||
|
|
||||||
|
|
||||||
def _build_nights_from_apple_segments(reader: csv.DictReader, phase_map: dict) -> dict[date, dict]:
|
|
||||||
segments = []
|
|
||||||
for raw in reader:
|
|
||||||
row = _strip_row_keys(raw)
|
|
||||||
phase_key = (row.get("Value") or "").strip()
|
|
||||||
phase_en = phase_map.get(phase_key)
|
|
||||||
if phase_en is None:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
start_dt = _parse_apple_sleep_datetime(row.get("Start") or "")
|
|
||||||
end_dt = _parse_apple_sleep_datetime(row.get("End") or "")
|
|
||||||
duration_hr = _safe_float(row.get("Duration (hr)"))
|
|
||||||
if duration_hr is None:
|
|
||||||
continue
|
|
||||||
except (ValueError, TypeError):
|
|
||||||
continue
|
|
||||||
duration_min = int(duration_hr * 60)
|
|
||||||
segments.append({
|
|
||||||
"start": start_dt,
|
|
||||||
"end": end_dt,
|
|
||||||
"duration_min": duration_min,
|
|
||||||
"phase": phase_en,
|
|
||||||
})
|
|
||||||
|
|
||||||
segments.sort(key=lambda s: s["start"])
|
|
||||||
nights = []
|
|
||||||
current_night = None
|
|
||||||
|
|
||||||
for seg in segments:
|
|
||||||
if current_night is None or (seg["start"] - current_night["wake_time"]).total_seconds() > 7200:
|
|
||||||
current_night = {
|
|
||||||
"bedtime": seg["start"],
|
|
||||||
"wake_time": seg["end"],
|
|
||||||
"segments": [],
|
|
||||||
"deep_minutes": 0,
|
|
||||||
"rem_minutes": 0,
|
|
||||||
"light_minutes": 0,
|
|
||||||
"awake_minutes": 0,
|
|
||||||
}
|
|
||||||
nights.append(current_night)
|
|
||||||
|
|
||||||
current_night["segments"].append(seg)
|
|
||||||
current_night["wake_time"] = max(current_night["wake_time"], seg["end"])
|
|
||||||
current_night["bedtime"] = min(current_night["bedtime"], seg["start"])
|
|
||||||
|
|
||||||
if seg["phase"] == "deep":
|
|
||||||
current_night["deep_minutes"] += seg["duration_min"]
|
|
||||||
elif seg["phase"] == "rem":
|
|
||||||
current_night["rem_minutes"] += seg["duration_min"]
|
|
||||||
elif seg["phase"] == "light":
|
|
||||||
current_night["light_minutes"] += seg["duration_min"]
|
|
||||||
elif seg["phase"] == "awake":
|
|
||||||
current_night["awake_minutes"] += seg["duration_min"]
|
|
||||||
|
|
||||||
nights_dict: dict[date, dict] = {}
|
|
||||||
for night in nights:
|
|
||||||
wake_date = night["wake_time"].date()
|
|
||||||
nights_dict[wake_date] = night
|
|
||||||
return nights_dict
|
|
||||||
|
|
||||||
|
|
||||||
# ── Models ────────────────────────────────────────────────────────────────────
|
# ── Models ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
class SleepCreate(BaseModel):
|
class SleepCreate(BaseModel):
|
||||||
@@ -641,139 +471,25 @@ async def import_apple_health_sleep(
|
|||||||
pid = session['profile_id']
|
pid = session['profile_id']
|
||||||
|
|
||||||
content = await file.read()
|
content = await file.read()
|
||||||
csv_text = content.decode('utf-8-sig')
|
try:
|
||||||
reader = csv.DictReader(io.StringIO(csv_text))
|
csv_text = content.decode("utf-8-sig")
|
||||||
fmt = _detect_apple_sleep_csv_format(reader.fieldnames)
|
except UnicodeDecodeError:
|
||||||
|
csv_text = content.decode("latin-1")
|
||||||
phase_map = {
|
|
||||||
"Kern": "light",
|
|
||||||
"Core": "light",
|
|
||||||
"Light": "light",
|
|
||||||
"REM": "rem",
|
|
||||||
"Tief": "deep",
|
|
||||||
"Deep": "deep",
|
|
||||||
"Wach": "awake",
|
|
||||||
"Awake": "awake",
|
|
||||||
"Schlafend": None,
|
|
||||||
"Asleep": None,
|
|
||||||
"In Bed": None,
|
|
||||||
}
|
|
||||||
|
|
||||||
if fmt == "summary":
|
|
||||||
nights_dict = _build_nights_from_apple_summary(reader)
|
|
||||||
else:
|
|
||||||
nights_dict = _build_nights_from_apple_segments(reader, phase_map)
|
|
||||||
|
|
||||||
if not nights_dict:
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=400,
|
|
||||||
detail="Keine importierbaren Schlafzeilen gefunden (prüfe Start/Ende und Format).",
|
|
||||||
)
|
|
||||||
|
|
||||||
# Insert nights
|
|
||||||
imported = 0
|
|
||||||
skipped = 0
|
|
||||||
|
|
||||||
with get_db() as conn:
|
with get_db() as conn:
|
||||||
cur = get_cursor(conn)
|
cur = get_cursor(conn)
|
||||||
|
try:
|
||||||
for date, night in nights_dict.items():
|
r = import_apple_sleep_nights(cur, pid, csv_text)
|
||||||
# Calculate sleep duration (deep + rem + light, WITHOUT awake)
|
except ValueError as e:
|
||||||
# Note: awake_minutes tracked separately, not part of sleep duration
|
raise HTTPException(status_code=400, detail=str(e))
|
||||||
duration_minutes = (
|
|
||||||
night['deep_minutes'] +
|
|
||||||
night['rem_minutes'] +
|
|
||||||
night['light_minutes']
|
|
||||||
)
|
|
||||||
|
|
||||||
# Calculate wake_count (number of awake segments)
|
|
||||||
wake_count = sum(1 for seg in night['segments'] if seg['phase'] == 'awake')
|
|
||||||
|
|
||||||
# Prepare JSONB segments with full datetime
|
|
||||||
sleep_segments = [
|
|
||||||
{
|
|
||||||
'phase': seg['phase'],
|
|
||||||
'start': seg['start'].isoformat(), # Full datetime: 2026-03-21T22:30:00
|
|
||||||
'end': seg['end'].isoformat(), # Full datetime: 2026-03-21T23:15:00
|
|
||||||
'duration_min': seg['duration_min']
|
|
||||||
}
|
|
||||||
for seg in night['segments']
|
|
||||||
]
|
|
||||||
|
|
||||||
# Check if manual entry exists - do NOT overwrite
|
|
||||||
cur.execute("""
|
|
||||||
SELECT id, source FROM sleep_log
|
|
||||||
WHERE profile_id = %s AND date = %s
|
|
||||||
""", (pid, date))
|
|
||||||
existing = cur.fetchone()
|
|
||||||
|
|
||||||
if existing and existing['source'] == 'manual':
|
|
||||||
skipped += 1
|
|
||||||
continue # Skip - don't overwrite manual entries
|
|
||||||
|
|
||||||
# Upsert (only if not manual)
|
|
||||||
# If entry exists and is NOT manual → update
|
|
||||||
# If entry doesn't exist → insert
|
|
||||||
if existing:
|
|
||||||
# Update existing non-manual entry
|
|
||||||
cur.execute("""
|
|
||||||
UPDATE sleep_log SET
|
|
||||||
bedtime = %s,
|
|
||||||
wake_time = %s,
|
|
||||||
duration_minutes = %s,
|
|
||||||
wake_count = %s,
|
|
||||||
deep_minutes = %s,
|
|
||||||
rem_minutes = %s,
|
|
||||||
light_minutes = %s,
|
|
||||||
awake_minutes = %s,
|
|
||||||
sleep_segments = %s,
|
|
||||||
source = 'apple_health',
|
|
||||||
updated_at = CURRENT_TIMESTAMP
|
|
||||||
WHERE id = %s AND profile_id = %s
|
|
||||||
""", (
|
|
||||||
night['bedtime'].time(),
|
|
||||||
night['wake_time'].time(),
|
|
||||||
duration_minutes,
|
|
||||||
wake_count,
|
|
||||||
night['deep_minutes'],
|
|
||||||
night['rem_minutes'],
|
|
||||||
night['light_minutes'],
|
|
||||||
night['awake_minutes'],
|
|
||||||
json.dumps(sleep_segments),
|
|
||||||
existing['id'],
|
|
||||||
pid
|
|
||||||
))
|
|
||||||
else:
|
|
||||||
# Insert new entry
|
|
||||||
cur.execute("""
|
|
||||||
INSERT INTO sleep_log (
|
|
||||||
profile_id, date, bedtime, wake_time, duration_minutes,
|
|
||||||
wake_count, deep_minutes, rem_minutes, light_minutes, awake_minutes,
|
|
||||||
sleep_segments, source, created_at, updated_at
|
|
||||||
) VALUES (
|
|
||||||
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, 'apple_health', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
|
|
||||||
)
|
|
||||||
""", (
|
|
||||||
pid,
|
|
||||||
date,
|
|
||||||
night['bedtime'].time(),
|
|
||||||
night['wake_time'].time(),
|
|
||||||
duration_minutes,
|
|
||||||
wake_count,
|
|
||||||
night['deep_minutes'],
|
|
||||||
night['rem_minutes'],
|
|
||||||
night['light_minutes'],
|
|
||||||
night['awake_minutes'],
|
|
||||||
json.dumps(sleep_segments)
|
|
||||||
))
|
|
||||||
|
|
||||||
imported += 1
|
|
||||||
|
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
||||||
|
imported = r["inserted"] + r["updated"]
|
||||||
|
skipped = r["skipped"]
|
||||||
|
total = r["rows_total"]
|
||||||
return {
|
return {
|
||||||
"imported": imported,
|
"imported": imported,
|
||||||
"skipped": skipped,
|
"skipped": skipped,
|
||||||
"total_nights": len(nights_dict),
|
"total_nights": total,
|
||||||
"message": f"{imported} Nächte importiert, {skipped} übersprungen (manuelle Einträge)"
|
"message": f"{imported} Nächte importiert, {skipped} übersprungen (manuelle Einträge)",
|
||||||
}
|
}
|
||||||
+4
-4
@@ -160,10 +160,10 @@ CREATE TABLE IF NOT EXISTS activity_log (
|
|||||||
end_time TIME,
|
end_time TIME,
|
||||||
activity_type VARCHAR(50) NOT NULL,
|
activity_type VARCHAR(50) NOT NULL,
|
||||||
duration_min NUMERIC(6,2),
|
duration_min NUMERIC(6,2),
|
||||||
kcal_active NUMERIC(7,2),
|
kcal_active NUMERIC(12,2),
|
||||||
kcal_resting NUMERIC(7,2),
|
kcal_resting NUMERIC(12,2),
|
||||||
hr_avg NUMERIC(5,2),
|
hr_avg NUMERIC(6,2),
|
||||||
hr_max NUMERIC(5,2),
|
hr_max NUMERIC(6,2),
|
||||||
distance_km NUMERIC(7,2),
|
distance_km NUMERIC(7,2),
|
||||||
rpe INTEGER CHECK (rpe >= 1 AND rpe <= 10),
|
rpe INTEGER CHECK (rpe >= 1 AND rpe <= 10),
|
||||||
source VARCHAR(20) DEFAULT 'manual',
|
source VARCHAR(20) DEFAULT 'manual',
|
||||||
|
|||||||
@@ -0,0 +1,385 @@
|
|||||||
|
"""
|
||||||
|
Smoke-Tests für Universal-CSV-Import (Executor + Apple-Schlaf-Parser).
|
||||||
|
|
||||||
|
Nutzt einen minimalen Fake-Cursor (kein PostgreSQL), damit die Pipelines bei jedem
|
||||||
|
pytest-Lauf mitlaufen.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from csv_parser.executor import diagnose_vitals_row, run_universal_csv_import
|
||||||
|
from csv_parser.sleep_apple_import import detect_apple_sleep_csv_format
|
||||||
|
from csv_parser.type_converter import build_row_after_mapping, diagnose_row_mapping
|
||||||
|
|
||||||
|
|
||||||
|
class _SeqCursor:
|
||||||
|
"""Minimaler Cursor: execute protokolliert; fetchone liefert vorgegebene Sequenz."""
|
||||||
|
|
||||||
|
def __init__(self, fetch_sequence: list) -> None:
|
||||||
|
self.executes: list[tuple[str, tuple | None]] = []
|
||||||
|
self._fetch = list(fetch_sequence)
|
||||||
|
|
||||||
|
def execute(self, sql: str, params=None) -> None:
|
||||||
|
self.executes.append((sql, params))
|
||||||
|
|
||||||
|
def fetchone(self):
|
||||||
|
if self._fetch:
|
||||||
|
return self._fetch.pop(0)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
PID = "00000000-0000-0000-0000-000000000001"
|
||||||
|
|
||||||
|
|
||||||
|
def test_detect_apple_sleep_summary_vs_segment():
|
||||||
|
assert detect_apple_sleep_csv_format(["Start", "End", "Total Sleep (hr)", "Core (hr)"]) == "summary"
|
||||||
|
assert detect_apple_sleep_csv_format(["Start", "End", "Duration (hr)", "Value"]) == "segments"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_sleep_one_night_inserts(monkeypatch):
|
||||||
|
"""Eine Summary-Zeile → INSERT; SELECT vorher ohne Treffer."""
|
||||||
|
text = (
|
||||||
|
"Start,End,Total Sleep (hr),Core (hr),Deep (hr),REM (hr),Awake (hr)\n"
|
||||||
|
"2024-01-15 22:00:00,2024-01-16 06:00:00,8.0,5.0,1.0,1.5,0.5\n"
|
||||||
|
)
|
||||||
|
cur = _SeqCursor(
|
||||||
|
[
|
||||||
|
None,
|
||||||
|
{"id": 101},
|
||||||
|
]
|
||||||
|
)
|
||||||
|
out = run_universal_csv_import(cur, PID, "sleep", text, "sleep.csv", {})
|
||||||
|
assert out["rows_total"] >= 1
|
||||||
|
assert out["rows_imported"] >= 1
|
||||||
|
assert any("INSERT INTO sleep_log" in q[0] for q in cur.executes)
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_activity_insert(monkeypatch):
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"csv_parser.executor._resolve_training_type_for_activity",
|
||||||
|
lambda *_a, **_k: (None, None, None),
|
||||||
|
)
|
||||||
|
text = (
|
||||||
|
"Workout Type,Start,End,Duration,Distance (km),Active Energy (kcal)\n"
|
||||||
|
"Running,2024-01-15 08:00:00,2024-01-15 09:00:00,1:00:00,10.0,500\n"
|
||||||
|
)
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {
|
||||||
|
"Workout Type": "activity_type",
|
||||||
|
"Start": "start_time",
|
||||||
|
"End": "end_time",
|
||||||
|
"Duration": "duration_min",
|
||||||
|
"Distance (km)": "distance_km",
|
||||||
|
"Active Energy (kcal)": "kcal_active",
|
||||||
|
},
|
||||||
|
"type_conversions": {
|
||||||
|
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
|
||||||
|
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
|
||||||
|
"duration_min": {
|
||||||
|
"type": "duration",
|
||||||
|
"format": "HH:MM:SS",
|
||||||
|
"target_unit": "minutes",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"distance_km": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
"kcal_active": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
new_id = str(uuid.uuid4())
|
||||||
|
cur = _SeqCursor([None, {"id": new_id}])
|
||||||
|
out = run_universal_csv_import(cur, PID, "activity", text, "act.csv", mapping)
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
assert out["new_entries"] == 1
|
||||||
|
assert any("INSERT INTO activity_log" in q[0] for q in cur.executes)
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_vitals_baseline_upsert_insert_path():
|
||||||
|
text = (
|
||||||
|
"Start,Resting Heart Rate,Heart Rate Variability,VO2 Max\n"
|
||||||
|
"2024-01-15 07:00:00,55,45,42.5\n"
|
||||||
|
)
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {
|
||||||
|
"Start": "date",
|
||||||
|
"Resting Heart Rate": "resting_hr",
|
||||||
|
"Heart Rate Variability": "hrv",
|
||||||
|
"VO2 Max": "vo2_max",
|
||||||
|
},
|
||||||
|
"type_conversions": {
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": True},
|
||||||
|
"hrv": {"type": "int", "flexible": True},
|
||||||
|
"vo2_max": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
cur = _SeqCursor([{"inserted": True, "id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"}])
|
||||||
|
out = run_universal_csv_import(cur, PID, "vitals_baseline", text, "v.csv", mapping)
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
assert any("INSERT INTO vitals_baseline" in q[0] for q in cur.executes)
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_wide_german_vitals_with_english_template_slots():
|
||||||
|
"""Breiter Apple-DE-Export, aber nur englische Vorlagen-Spalten → über Aliase erkennbar."""
|
||||||
|
text = (
|
||||||
|
"Datum/Uhrzeit,Aktive Energie (kJ),Ruhepuls (count/min),Atemfrequenz (count/min)"
|
||||||
|
",Blutsauerstoffsättigung (%),Herzfrequenzvariabilität (ms),VO2 max (ml/(kg·min))\n"
|
||||||
|
"2026-04-03 00:00:00,,53,15.61,95.22,37.26,\n"
|
||||||
|
)
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {
|
||||||
|
"Start": "date",
|
||||||
|
"Resting Heart Rate": "resting_hr",
|
||||||
|
"Heart Rate Variability": "hrv",
|
||||||
|
"VO2 Max": "vo2_max",
|
||||||
|
"Oxygen Saturation": "spo2",
|
||||||
|
"Respiratory Rate": "respiratory_rate",
|
||||||
|
},
|
||||||
|
"type_conversions": {
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": True},
|
||||||
|
"hrv": {"type": "int", "flexible": True},
|
||||||
|
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
"spo2": {"type": "int", "flexible": True},
|
||||||
|
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
cur = _SeqCursor([{"inserted": True, "id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"}])
|
||||||
|
out = run_universal_csv_import(cur, PID, "vitals_baseline", text, "wide.csv", mapping)
|
||||||
|
assert out["rows_errors"] == 0
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_vitals_baseline_two_rows_same_day_averages():
|
||||||
|
text = (
|
||||||
|
"Start,Resting Heart Rate,Heart Rate Variability,VO2 Max\n"
|
||||||
|
"2024-01-15 07:00:00,50,40,42.0\n"
|
||||||
|
"2024-01-15 18:00:00,60,50,43.0\n"
|
||||||
|
)
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {
|
||||||
|
"Start": "date",
|
||||||
|
"Resting Heart Rate": "resting_hr",
|
||||||
|
"Heart Rate Variability": "hrv",
|
||||||
|
"VO2 Max": "vo2_max",
|
||||||
|
},
|
||||||
|
"type_conversions": {
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": True},
|
||||||
|
"hrv": {"type": "int", "flexible": True},
|
||||||
|
"vo2_max": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
cur = _SeqCursor([{"inserted": True, "id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"}])
|
||||||
|
out = run_universal_csv_import(cur, PID, "vitals_baseline", text, "v2.csv", mapping)
|
||||||
|
assert out["rows_total"] == 2
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
upsert = [q for q in cur.executes if "INSERT INTO vitals_baseline" in q[0]]
|
||||||
|
assert len(upsert) == 1
|
||||||
|
params = upsert[0][1]
|
||||||
|
assert params[2] == 55
|
||||||
|
assert params[3] == 45
|
||||||
|
assert params[4] == 42.5
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_activity_garmin_time_plus_date_columns(monkeypatch):
|
||||||
|
"""Datum in eigener Spalte, Uhrzeit wie bei Garmin nur als Uhrzeit."""
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"csv_parser.executor._resolve_training_type_for_activity",
|
||||||
|
lambda *_a, **_k: (None, None, None),
|
||||||
|
)
|
||||||
|
text = (
|
||||||
|
"Activity Type,Date,Time,Duration,Distance,Calories,Avg HR\n"
|
||||||
|
"Run,2024-01-20,08:30:00,0:45:00,8.0,400,140\n"
|
||||||
|
)
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {
|
||||||
|
"Activity Type": "activity_type",
|
||||||
|
"Date": "date",
|
||||||
|
"Time": "start_time",
|
||||||
|
"Duration": "duration_min",
|
||||||
|
"Distance": "distance_km",
|
||||||
|
"Calories": "kcal_active",
|
||||||
|
"Avg HR": "hr_avg",
|
||||||
|
},
|
||||||
|
"type_conversions": {
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
|
||||||
|
"start_time": {"type": "time", "format": "HH:MM:SS", "flexible": True},
|
||||||
|
"duration_min": {
|
||||||
|
"type": "duration",
|
||||||
|
"format": "HH:MM:SS",
|
||||||
|
"target_unit": "minutes",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"distance_km": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
"kcal_active": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
"hr_avg": {"type": "int", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
new_id = str(uuid.uuid4())
|
||||||
|
cur = _SeqCursor([None, {"id": new_id}])
|
||||||
|
out = run_universal_csv_import(cur, PID, "activity", text, "garmin.csv", mapping)
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
# Duplicate-Key muss Datum + kombinierte Startzeit enthalten
|
||||||
|
assert any(
|
||||||
|
params and "2024-01-20 08:30:00" in str(params)
|
||||||
|
for _sql, params in cur.executes
|
||||||
|
if params
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_nutrition_two_rows_same_day_aggregates_to_one_row():
|
||||||
|
"""Modul-Default: mehrere CSV-Zeilen pro Tag → Summe, ein nutrition_log-Eintrag."""
|
||||||
|
text = (
|
||||||
|
"Date,Kalorien,Protein,Fett,KH\n"
|
||||||
|
"2024-01-15,500,10,20,30\n"
|
||||||
|
"2024-01-15,300,5,10,15\n"
|
||||||
|
)
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {
|
||||||
|
"Date": "date",
|
||||||
|
"Kalorien": "kcal",
|
||||||
|
"Protein": "protein_g",
|
||||||
|
"Fett": "fat_g",
|
||||||
|
"KH": "carbs_g",
|
||||||
|
},
|
||||||
|
"type_conversions": {
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
|
||||||
|
"kcal": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
cur = _SeqCursor([None])
|
||||||
|
out = run_universal_csv_import(cur, PID, "nutrition", text, "n.csv", mapping)
|
||||||
|
assert out["rows_total"] == 2
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
insert_sqls = [q for q in cur.executes if "INSERT INTO nutrition_log" in q[0]]
|
||||||
|
assert len(insert_sqls) == 1
|
||||||
|
params = insert_sqls[0][1]
|
||||||
|
# (eid, profile_id, iso, kcal, prot, fat, carbs)
|
||||||
|
assert params[3] == 800.0
|
||||||
|
assert params[4] == 15.0
|
||||||
|
assert params[5] == 30.0
|
||||||
|
assert params[6] == 45.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_universal_import_weight_two_rows_same_day_last_value():
|
||||||
|
"""Mehrere Gewichtszeilen pro Tag → Standard: letzter Wert in der Datei."""
|
||||||
|
text = "Date,Weight\n2024-01-15,85.0\n2024-01-15,83.5\n"
|
||||||
|
mapping = {
|
||||||
|
"delimiter": ",",
|
||||||
|
"has_header": True,
|
||||||
|
"field_mappings": {"Date": "date", "Weight": "weight"},
|
||||||
|
"type_conversions": {
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
|
||||||
|
"weight": {"type": "float", "decimal_separator": ".", "flexible": True},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
cur = _SeqCursor([None])
|
||||||
|
out = run_universal_csv_import(cur, PID, "weight", text, "w.csv", mapping)
|
||||||
|
assert out["rows_total"] == 2
|
||||||
|
assert out["rows_imported"] == 1
|
||||||
|
insert_sqls = [q for q in cur.executes if "INSERT INTO weight_log" in q[0]]
|
||||||
|
assert len(insert_sqls) == 1
|
||||||
|
params = insert_sqls[0][1]
|
||||||
|
assert params[3] == 83.5
|
||||||
|
|
||||||
|
|
||||||
|
def test_activity_alias_maps_german_workout_wide_columns():
|
||||||
|
from csv_parser.type_converter import build_row_after_mapping
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"Datum/Uhrzeit": "2026-04-03 08:00:00",
|
||||||
|
"Trainingsart": "Laufen",
|
||||||
|
"Dauer": "0:45:00",
|
||||||
|
}
|
||||||
|
fm = {"Workout Type": "activity_type"}
|
||||||
|
tc = {
|
||||||
|
"activity_type": {"type": "string"},
|
||||||
|
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
|
||||||
|
"duration_min": {
|
||||||
|
"type": "duration",
|
||||||
|
"format": "HH:MM:SS",
|
||||||
|
"target_unit": "minutes",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
out = build_row_after_mapping(row, fm, tc, module="activity")
|
||||||
|
assert str(out.get("activity_type")) == "Laufen"
|
||||||
|
assert isinstance(out.get("start_time"), dt.datetime)
|
||||||
|
assert out.get("duration_min") == 45.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_blood_pressure_alias_combined_datetime_column():
|
||||||
|
from csv_parser.type_converter import build_row_after_mapping
|
||||||
|
|
||||||
|
row = {
|
||||||
|
"Datum/Uhrzeit": "2026-04-03 10:30:00",
|
||||||
|
"Systolisch (mmHg)": "120",
|
||||||
|
"Diastolisch (mmHg)": "80",
|
||||||
|
}
|
||||||
|
fm = {"Date": "measured_date"}
|
||||||
|
tc = {
|
||||||
|
"systolic": {"type": "int", "flexible": True},
|
||||||
|
"diastolic": {"type": "int", "flexible": True},
|
||||||
|
}
|
||||||
|
out = build_row_after_mapping(row, fm, tc, module="blood_pressure")
|
||||||
|
assert isinstance(out.get("start_time"), dt.datetime)
|
||||||
|
assert int(out.get("systolic")) == 120
|
||||||
|
|
||||||
|
|
||||||
|
def test_diagnose_vitals_row_and_mapping_smoke():
|
||||||
|
fm = {
|
||||||
|
"Datum/Uhrzeit": "date",
|
||||||
|
"Ruhepuls (count/min)": "resting_hr",
|
||||||
|
}
|
||||||
|
tc = {
|
||||||
|
"date": {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_only",
|
||||||
|
"flexible": True,
|
||||||
|
},
|
||||||
|
"resting_hr": {"type": "int", "flexible": True},
|
||||||
|
}
|
||||||
|
row = {"Datum/Uhrzeit": "2026-04-03 00:00:00", "Ruhepuls (count/min)": "53"}
|
||||||
|
typed = build_row_after_mapping(row, fm, tc, module="vitals_baseline")
|
||||||
|
d = diagnose_vitals_row(typed)
|
||||||
|
assert d["date_coerced_iso"] == "2026-04-03"
|
||||||
|
assert d["would_pass_prefilter"] is True
|
||||||
|
dm = diagnose_row_mapping(row, fm, tc, module="vitals_baseline", mapped_typed=typed)
|
||||||
|
assert str(dm["mapped"]["date"]).startswith("2026-04-03")
|
||||||
|
assert any(c["csv_column"] == "Datum/Uhrzeit" and c["source"] == "template" for c in dm["per_column"])
|
||||||
@@ -0,0 +1,284 @@
|
|||||||
|
"""Tests für CSV-Parser Foundation (Issue #21)."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from csv_parser.core import (
|
||||||
|
decode_raw_bytes,
|
||||||
|
sniff_delimiter,
|
||||||
|
parse_csv_sample,
|
||||||
|
column_signature,
|
||||||
|
headers_signature_match_score,
|
||||||
|
headers_signature_rank_metrics,
|
||||||
|
get_csv_import_limits,
|
||||||
|
iter_csv_dict_rows,
|
||||||
|
)
|
||||||
|
from csv_parser.field_units import source_unit_choices_for_field
|
||||||
|
from csv_parser.mapping_suggest import build_type_conversions_for_mapping
|
||||||
|
from csv_parser.type_converter import convert_value, build_row_after_mapping
|
||||||
|
|
||||||
|
|
||||||
|
def test_decode_bom_utf8():
|
||||||
|
raw = "\ufeffa;b;c\n1;2;3".encode("utf-8-sig")
|
||||||
|
t = decode_raw_bytes(raw)
|
||||||
|
assert not t.startswith("\ufeff")
|
||||||
|
assert "a;b;c" in t
|
||||||
|
|
||||||
|
|
||||||
|
def test_sniff_delimiter():
|
||||||
|
assert sniff_delimiter("a;b;c;d") == ";"
|
||||||
|
assert sniff_delimiter("a,b,c") == ","
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_csv_sample_header():
|
||||||
|
text = "Date;kcal\n2024-01-01;2000\n"
|
||||||
|
headers, rows, delim = parse_csv_sample(text, delimiter=";", max_data_rows=3)
|
||||||
|
assert headers == ["Date", "kcal"]
|
||||||
|
assert delim == ";"
|
||||||
|
assert rows[0]["Date"] == "2024-01-01"
|
||||||
|
assert rows[0]["kcal"] == "2000"
|
||||||
|
|
||||||
|
|
||||||
|
def test_column_signature_sorted_unique():
|
||||||
|
sig = column_signature(["B", "a", "a"])
|
||||||
|
assert sig == ["a", "b"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_jaccard():
|
||||||
|
s1 = column_signature(["Date", "Calories"])
|
||||||
|
s2 = column_signature(["Date", "Calories", "Fat"])
|
||||||
|
assert headers_signature_match_score(s1, s2) == pytest.approx(2 / 3)
|
||||||
|
|
||||||
|
|
||||||
|
def test_template_recall_full_when_csv_has_extra_columns():
|
||||||
|
"""Alle Template-Spalten in der CSV → Recall 1.0; Jaccard niedriger bei vielen Zusatzspalten."""
|
||||||
|
csv_sig = column_signature(
|
||||||
|
["D", "E", "F", "Extra1", "Extra2", "Extra3", "Extra4", "Extra5"]
|
||||||
|
)
|
||||||
|
tmpl_sig = column_signature(["d", "e", "f"])
|
||||||
|
m = headers_signature_rank_metrics(csv_sig, tmpl_sig)
|
||||||
|
assert m["confidence"] == 1.0
|
||||||
|
assert m["template_recall"] == 1.0
|
||||||
|
assert m["columns_matched"] == 3
|
||||||
|
assert m["columns_in_template"] == 3
|
||||||
|
assert m["jaccard"] == pytest.approx(3 / 8)
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_csv_import_limits_default():
|
||||||
|
assert get_csv_import_limits(None)["max_rows_per_file"] == 50_000
|
||||||
|
|
||||||
|
|
||||||
|
def test_convert_date_and_kcal_factor():
|
||||||
|
d = convert_value("15.01.2024", "date", {"type": "date", "format": "dd.mm.yyyy"})
|
||||||
|
assert d.year == 2024 and d.month == 1 and d.day == 15
|
||||||
|
|
||||||
|
k = convert_value("8000", "kcal", {"type": "float", "conversion_factor": 0.239, "decimal_separator": "."})
|
||||||
|
assert abs(k - 8000 * 0.239) < 0.01
|
||||||
|
|
||||||
|
|
||||||
|
def test_convert_kcal_via_source_unit_kj():
|
||||||
|
spec = {"type": "float", "source_unit": "kj", "decimal_separator": "."}
|
||||||
|
k = convert_value("4184", "kcal", spec, module="nutrition")
|
||||||
|
assert abs(k - 1000.0) < 0.05
|
||||||
|
|
||||||
|
|
||||||
|
def test_convert_protein_kg_to_g():
|
||||||
|
spec = {"type": "float", "source_unit": "kg", "decimal_separator": "."}
|
||||||
|
g = convert_value("0.1", "protein_g", spec, module="nutrition")
|
||||||
|
assert abs(g - 100.0) < 0.001
|
||||||
|
|
||||||
|
|
||||||
|
def test_convert_custom_source_unit_only_conversion_factor():
|
||||||
|
"""Nicht vordefinierte Umrechnung: conversion_factor (optional mit source_unit: custom)."""
|
||||||
|
spec = {"type": "float", "source_unit": "custom", "conversion_factor": 2.5, "decimal_separator": "."}
|
||||||
|
k = convert_value("100", "kcal", spec, module="nutrition")
|
||||||
|
assert abs(k - 250.0) < 0.001
|
||||||
|
|
||||||
|
|
||||||
|
def test_convert_unknown_source_unit_uses_conversion_factor_only():
|
||||||
|
spec = {"type": "float", "source_unit": "exotic_unit", "conversion_factor": 0.5, "decimal_separator": "."}
|
||||||
|
k = convert_value("200", "kcal", spec, module="nutrition")
|
||||||
|
assert abs(k - 100.0) < 0.001
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_row_source_unit_without_module_no_factor():
|
||||||
|
"""Ohne module bleibt source_unit wirkungslos (Abwärtskompatibilität)."""
|
||||||
|
spec = {"type": "float", "source_unit": "kj", "decimal_separator": "."}
|
||||||
|
k = convert_value("4184", "kcal", spec, module=None)
|
||||||
|
assert abs(k - 4184.0) < 0.01
|
||||||
|
|
||||||
|
|
||||||
|
def test_iter_csv_dict_rows_full_file():
|
||||||
|
text = "a;b\n1;2\n3;4\n"
|
||||||
|
rows = list(iter_csv_dict_rows(text, ";", has_header=True))
|
||||||
|
assert rows == [{"a": "1", "b": "2"}, {"a": "3", "b": "4"}]
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_row_after_mapping():
|
||||||
|
csv_row = {"Datum": "01.01.2024", "kj": "4200"}
|
||||||
|
fm = {"Datum": "date", "kj": "kcal"}
|
||||||
|
tc = {
|
||||||
|
"date": {"type": "date", "format": "dd.mm.yyyy"},
|
||||||
|
"kcal": {"type": "float", "conversion_factor": 0.239, "decimal_separator": "."},
|
||||||
|
}
|
||||||
|
out = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
|
||||||
|
assert out["date"].month == 1
|
||||||
|
assert out["kcal"] is not None
|
||||||
|
assert abs(float(out["kcal"]) - 4200 * 0.239) < 0.02
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_type_conversions_kj_header_sets_source_unit():
|
||||||
|
fm = {"kJ": "kcal", "Datum": "date"}
|
||||||
|
tc = build_type_conversions_for_mapping("nutrition", fm, None)
|
||||||
|
assert tc["kcal"].get("source_unit") == "kj"
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_row_fddb_raw_header_keys_match_normalized_template():
|
||||||
|
"""FDDB: DictReader liefert deutsche Überschrift, Seed nutzt normalisierten Key."""
|
||||||
|
csv_row = {
|
||||||
|
"Datum Tag Monat Jahr Stunde Minute": "01.01.2024 8:30",
|
||||||
|
"kJ": "42000",
|
||||||
|
"Fett (g)": "50",
|
||||||
|
"KH (g)": "200",
|
||||||
|
"Protein (g)": "100",
|
||||||
|
}
|
||||||
|
fm = {
|
||||||
|
"datum_tag_monat_jahr_stunde_minute": "date",
|
||||||
|
"kj": "kcal",
|
||||||
|
"fett_g": "fat_g",
|
||||||
|
"kh_g": "carbs_g",
|
||||||
|
"protein_g": "protein_g",
|
||||||
|
}
|
||||||
|
tc = {
|
||||||
|
"date": {"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only"},
|
||||||
|
"kcal": {
|
||||||
|
"type": "float",
|
||||||
|
"source_unit": "kj",
|
||||||
|
"decimal_separator": ",",
|
||||||
|
},
|
||||||
|
"fat_g": {"type": "float", "decimal_separator": ","},
|
||||||
|
"carbs_g": {"type": "float", "decimal_separator": ","},
|
||||||
|
"protein_g": {"type": "float", "decimal_separator": ","},
|
||||||
|
}
|
||||||
|
out = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
|
||||||
|
assert out["date"].year == 2024 and out["date"].month == 1 and out["date"].day == 1
|
||||||
|
assert out["kcal"] is not None and abs(float(out["kcal"]) - (42000 / 4.184)) < 0.1
|
||||||
|
|
||||||
|
|
||||||
|
def test_convert_date_ddmm_with_seconds():
|
||||||
|
d = convert_value(
|
||||||
|
"15.01.2024 14:30:00",
|
||||||
|
"date",
|
||||||
|
{"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only"},
|
||||||
|
)
|
||||||
|
assert d.month == 1 and d.day == 15
|
||||||
|
|
||||||
|
|
||||||
|
def test_float_decimal_separator_auto_eu_us():
|
||||||
|
assert abs(convert_value("1.234,56", "x", {"type": "float", "decimal_separator": "auto"}) - 1234.56) < 1e-9
|
||||||
|
assert abs(convert_value("1,234.56", "x", {"type": "float", "decimal_separator": "auto"}) - 1234.56) < 1e-9
|
||||||
|
assert abs(convert_value("1234,5", "x", {"type": "float", "decimal_separator": "auto"}) - 1234.5) < 1e-9
|
||||||
|
|
||||||
|
|
||||||
|
def test_float_flexible_falls_back_to_auto():
|
||||||
|
spec = {"type": "float", "decimal_separator": ",", "flexible": True}
|
||||||
|
assert abs(convert_value("1234.56", "x", spec) - 1234.56) < 1e-9
|
||||||
|
|
||||||
|
|
||||||
|
def test_date_flexible_iso_while_primary_ddmm():
|
||||||
|
spec = {
|
||||||
|
"type": "date",
|
||||||
|
"format": "dd.mm.yyyy",
|
||||||
|
"flexible": True,
|
||||||
|
"extract": "date_only",
|
||||||
|
}
|
||||||
|
d1 = convert_value("2024-03-15", "d", spec)
|
||||||
|
d2 = convert_value("15.03.2024", "d", spec)
|
||||||
|
assert d1 == d2
|
||||||
|
|
||||||
|
|
||||||
|
def test_date_extra_formats_without_days_in_name():
|
||||||
|
spec = {
|
||||||
|
"type": "date",
|
||||||
|
"format": "yyyy-mm-dd",
|
||||||
|
"formats": ["%d.%m.%Y"],
|
||||||
|
"extract": "date_only",
|
||||||
|
}
|
||||||
|
assert convert_value("08.04.2026", "d", spec).day == 8
|
||||||
|
|
||||||
|
|
||||||
|
def test_int_flexible_thousands():
|
||||||
|
assert convert_value("1.234", "n", {"type": "int", "flexible": True}) == 1234
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_row_after_mapping_column_order_independent():
|
||||||
|
fm = {"Spalte B": "resting_hr", "Spalte A": "date"}
|
||||||
|
tc = {
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
|
||||||
|
"resting_hr": {"type": "int", "flexible": True},
|
||||||
|
}
|
||||||
|
r1 = build_row_after_mapping(
|
||||||
|
{"Spalte A": "2026-01-15", "Spalte B": "58"}, fm, tc, module="vitals_baseline"
|
||||||
|
)
|
||||||
|
r2 = build_row_after_mapping(
|
||||||
|
{"Spalte B": "58", "Spalte A": "2026-01-15"}, fm, tc, module="vitals_baseline"
|
||||||
|
)
|
||||||
|
assert r1 == r2
|
||||||
|
assert r1["resting_hr"] == 58
|
||||||
|
|
||||||
|
|
||||||
|
def test_omron_report_date_formats_without_flexible_flag():
|
||||||
|
"""Omron «Bericht»-Export: engl. Month abbrev + deutscher Monatsname; Vorlage oft nur dd.mm.yyyy."""
|
||||||
|
spec = {"type": "date", "format": "dd.mm.yyyy"}
|
||||||
|
assert convert_value("10 Apr. 2026", "measured_date", spec).isoformat() == "2026-04-10"
|
||||||
|
assert convert_value("31 März 2026", "measured_date", spec).isoformat() == "2026-03-31"
|
||||||
|
assert convert_value("11 März 2026", "measured_date", spec).isoformat() == "2026-03-11"
|
||||||
|
|
||||||
|
|
||||||
|
def test_int_flexible_german_decimal_rounds():
|
||||||
|
"""Apple-DE: HRV/SpO2 als «37,26» / «95,22» — nicht 3726 aus Ziffern konkatenieren."""
|
||||||
|
spec = {"type": "int", "flexible": True}
|
||||||
|
assert convert_value("37,26", "hrv", spec) == 37
|
||||||
|
assert convert_value("95,22", "spo2", spec) == 95
|
||||||
|
|
||||||
|
|
||||||
|
def test_datetime_flexible():
|
||||||
|
spec = {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True}
|
||||||
|
dtv = convert_value("15.01.2024 14:30:00", "t", spec)
|
||||||
|
assert dtv.month == 1 and dtv.day == 15 and dtv.hour == 14
|
||||||
|
|
||||||
|
|
||||||
|
def test_apple_workout_datetime_without_seconds_iso_not_swapped():
|
||||||
|
"""Apple Export: 2026-04-09 16:48 — ohne :SS; kein dayfirst-Fehlparser (09↔04)."""
|
||||||
|
spec = {
|
||||||
|
"type": "datetime",
|
||||||
|
"format": "yyyy-mm-dd HH:MM:SS",
|
||||||
|
"extract": "date_and_time",
|
||||||
|
}
|
||||||
|
dtv = convert_value("2026-04-09 16:48", "start_time", spec, module="activity")
|
||||||
|
assert dtv.year == 2026 and dtv.month == 4 and dtv.day == 9
|
||||||
|
assert dtv.hour == 16 and dtv.minute == 48
|
||||||
|
|
||||||
|
|
||||||
|
def test_iso_yyyy_mm_dd_dateutil_fallback_not_dayfirst_swapped():
|
||||||
|
"""Nur dateutil: ISO-Datum darf mit Default-dayfirst nicht vertauscht werden."""
|
||||||
|
spec = {"type": "date", "format": "dd.mm.yyyy", "flexible": True}
|
||||||
|
d = convert_value("2026-04-09", "d", spec)
|
||||||
|
assert d.month == 4 and d.day == 9
|
||||||
|
|
||||||
|
|
||||||
|
def test_activity_kcal_active_kj_source_unit_to_kcal():
|
||||||
|
"""Apple-DE: Aktive Energie als kJ; DB-Feld ist kcal."""
|
||||||
|
spec = {
|
||||||
|
"type": "float",
|
||||||
|
"decimal_separator": ".",
|
||||||
|
"flexible": True,
|
||||||
|
"source_unit": "kj",
|
||||||
|
}
|
||||||
|
v = convert_value("4184", "kcal_active", spec, module="activity")
|
||||||
|
assert abs(v - 1000.0) < 0.02
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_unit_choices_include_custom_at_end():
|
||||||
|
opts = source_unit_choices_for_field("nutrition", "protein_g")
|
||||||
|
assert opts[-1]["id"] == "custom"
|
||||||
|
assert any(o["id"] == "mg" for o in opts)
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
from csv_parser.import_errors import enrich_row_error
|
||||||
|
|
||||||
|
|
||||||
|
def test_enrich_numeric_overflow():
|
||||||
|
d = enrich_row_error(
|
||||||
|
"numeric field overflow\nDETAIL: A field with precision 5, scale 2\n",
|
||||||
|
module="activity",
|
||||||
|
)
|
||||||
|
assert d["code"] == "db_numeric_overflow"
|
||||||
|
assert d["hint"] and "kj" in d["hint"].lower()
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
"""Tests für CSV-Zeilenaggregation (import_row_processing)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import datetime as dt
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from csv_parser.import_row_processing import (
|
||||||
|
aggregate_mapped_rows,
|
||||||
|
resolve_import_row_processing,
|
||||||
|
validate_import_row_processing,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_rejects_unknown_aggregate():
|
||||||
|
with pytest.raises(ValueError, match="ungültige Operation"):
|
||||||
|
validate_import_row_processing(
|
||||||
|
"nutrition",
|
||||||
|
{"group_by": ["date"], "aggregates": {"kcal": "bogus"}},
|
||||||
|
{"Kal": "date", "E": "kcal"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_group_by_must_be_mapped():
|
||||||
|
with pytest.raises(ValueError, match="keiner CSV-Spalte zugeordnet"):
|
||||||
|
validate_import_row_processing(
|
||||||
|
"nutrition",
|
||||||
|
{"group_by": ["date"], "aggregates": {"kcal": "sum"}},
|
||||||
|
{"Kal": "kcal"}, # date nicht gemappt
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_mapped_rows_sums_same_group():
|
||||||
|
d = dt.date(2024, 1, 15)
|
||||||
|
rows = [
|
||||||
|
{"date": d, "kcal": 500.0, "protein_g": 20},
|
||||||
|
{"date": d, "kcal": 300.0, "protein_g": 15},
|
||||||
|
]
|
||||||
|
spec = {"group_by": ["date"], "aggregates": {"kcal": "sum", "protein_g": "sum"}}
|
||||||
|
out, err = aggregate_mapped_rows(rows, spec)
|
||||||
|
assert err == []
|
||||||
|
assert len(out) == 1
|
||||||
|
assert out[0]["kcal"] == 800.0
|
||||||
|
assert out[0]["protein_g"] == 35
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_mapped_rows_reject_second_group():
|
||||||
|
d = dt.date(2024, 1, 15)
|
||||||
|
rows = [
|
||||||
|
{"date": d, "kcal": 100.0},
|
||||||
|
{"date": d, "kcal": 200.0},
|
||||||
|
]
|
||||||
|
spec = {
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {"kcal": "sum"},
|
||||||
|
"multi_row_policy": "reject",
|
||||||
|
}
|
||||||
|
out, err = aggregate_mapped_rows(rows, spec)
|
||||||
|
assert out == []
|
||||||
|
assert len(err) == 1
|
||||||
|
assert err[0].get("error") == "mehrere_zeilen_pro_schluessel"
|
||||||
|
assert err[0].get("rows_in_group") == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_mapped_rows_first_row_no_merge():
|
||||||
|
d = dt.date(2024, 1, 15)
|
||||||
|
rows = [
|
||||||
|
{"date": d, "kcal": 100.0},
|
||||||
|
{"date": d, "kcal": 999.0},
|
||||||
|
]
|
||||||
|
spec = {
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {"kcal": "sum"},
|
||||||
|
"multi_row_policy": "first_row",
|
||||||
|
}
|
||||||
|
out, err = aggregate_mapped_rows(rows, spec)
|
||||||
|
assert err == []
|
||||||
|
assert len(out) == 1
|
||||||
|
assert out[0]["kcal"] == 100.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_dedupe_identical_rows_before_group():
|
||||||
|
d = dt.date(2024, 1, 15)
|
||||||
|
rows = [
|
||||||
|
{"date": d, "kcal": 50.0},
|
||||||
|
{"date": d, "kcal": 50.0},
|
||||||
|
{"date": d, "kcal": 50.0},
|
||||||
|
]
|
||||||
|
spec = {
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {"kcal": "sum"},
|
||||||
|
"dedupe_identical_rows": True,
|
||||||
|
}
|
||||||
|
out, err = aggregate_mapped_rows(rows, spec)
|
||||||
|
assert err == []
|
||||||
|
assert len(out) == 1
|
||||||
|
assert out[0]["kcal"] == 50.0
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_multi_row_policy():
|
||||||
|
with pytest.raises(ValueError, match="multi_row_policy"):
|
||||||
|
validate_import_row_processing(
|
||||||
|
"nutrition",
|
||||||
|
{
|
||||||
|
"group_by": ["date"],
|
||||||
|
"aggregates": {"kcal": "sum"},
|
||||||
|
"multi_row_policy": "nope",
|
||||||
|
},
|
||||||
|
{"D": "date", "K": "kcal"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_explicit_overrides_default():
|
||||||
|
m = {
|
||||||
|
"import_row_processing": {"group_by": ["date"], "aggregates": {"kcal": "mean"}},
|
||||||
|
}
|
||||||
|
spec = resolve_import_row_processing("nutrition", m)
|
||||||
|
assert spec is not None
|
||||||
|
assert spec["aggregates"]["kcal"] == "mean"
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_empty_dict_falls_back_to_module_default():
|
||||||
|
m: dict = {"import_row_processing": {}}
|
||||||
|
spec = resolve_import_row_processing("nutrition", m)
|
||||||
|
assert spec is not None
|
||||||
|
assert spec["group_by"] == ["date"]
|
||||||
|
assert spec["aggregates"]["kcal"] == "sum"
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_none_uses_nutrition_default():
|
||||||
|
spec = resolve_import_row_processing("nutrition", {})
|
||||||
|
assert spec is not None
|
||||||
|
assert "date" in (spec.get("group_by") or [])
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_weight_default_uses_last():
|
||||||
|
spec = resolve_import_row_processing("weight", {})
|
||||||
|
assert spec is not None
|
||||||
|
assert spec.get("aggregates", {}).get("weight") == "last"
|
||||||
|
assert spec.get("aggregates", {}).get("note") == "last"
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_weight_import_row_processing_ok():
|
||||||
|
validate_import_row_processing(
|
||||||
|
"weight",
|
||||||
|
{"group_by": ["date"], "aggregates": {"weight": "mean"}},
|
||||||
|
{"D": "date", "W": "weight"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_vitals_baseline_default_uses_mean():
|
||||||
|
spec = resolve_import_row_processing("vitals_baseline", {})
|
||||||
|
assert spec is not None
|
||||||
|
assert spec.get("aggregates", {}).get("resting_hr") == "mean"
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""Tests für CSV mapping_suggest (Issue #21)."""
|
||||||
|
|
||||||
|
from csv_parser.mapping_suggest import (
|
||||||
|
build_type_conversions_for_mapping,
|
||||||
|
suggest_field_mappings,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_suggest_from_fddb_seed():
|
||||||
|
headers = ["Datum Tag Monat Jahr Stunde Minute", "kJ", "Fett (g)", "KH (g)", "Protein (g)"]
|
||||||
|
seed = {
|
||||||
|
"datum_tag_monat_jahr_stunde_minute": "date",
|
||||||
|
"kj": "kcal",
|
||||||
|
"fett_g": "fat_g",
|
||||||
|
"kh_g": "carbs_g",
|
||||||
|
"protein_g": "protein_g",
|
||||||
|
}
|
||||||
|
fm = suggest_field_mappings(headers, "nutrition", seed)
|
||||||
|
assert fm["Datum Tag Monat Jahr Stunde Minute"] == "date"
|
||||||
|
assert fm["kJ"] == "kcal"
|
||||||
|
|
||||||
|
|
||||||
|
def test_suggest_aliases_without_seed():
|
||||||
|
fm = suggest_field_mappings(
|
||||||
|
["Date", "Calories", "Protein (g)", "Fat (g)", "Carbs"],
|
||||||
|
"nutrition",
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
assert fm["Date"] == "date"
|
||||||
|
assert fm["Calories"] == "kcal"
|
||||||
|
assert fm["Protein (g)"] == "protein_g"
|
||||||
|
|
||||||
|
|
||||||
|
def test_type_conversions_merge():
|
||||||
|
fm = {"Col A": "date", "Col B": "kcal"}
|
||||||
|
seed_tc = {
|
||||||
|
"kcal": {"type": "float", "conversion_factor": 0.239, "decimal_separator": "auto"},
|
||||||
|
}
|
||||||
|
tc = build_type_conversions_for_mapping("nutrition", fm, seed_tc)
|
||||||
|
assert "kcal" in tc
|
||||||
|
assert tc["kcal"].get("conversion_factor") == 0.239
|
||||||
|
assert "date" in tc
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
"""Formatprüfung CSV-Vorlagen (template_validator)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from csv_parser.template_validator import validate_csv_template
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_kj_column_warns_without_source_unit():
|
||||||
|
r = validate_csv_template(
|
||||||
|
"activity",
|
||||||
|
{"Aktive Energie (kJ)": "kcal_active", "Start": "start_time", "Trainingsart": "activity_type"},
|
||||||
|
{"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True}},
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
assert r["valid"] is True
|
||||||
|
codes = {w["code"] for w in r["warnings"]}
|
||||||
|
assert "energy_kj_without_source_unit" in codes
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_invalid_target_error():
|
||||||
|
r = validate_csv_template(
|
||||||
|
"activity",
|
||||||
|
{"X": "not_a_field"},
|
||||||
|
{},
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
assert r["valid"] is False
|
||||||
|
assert any(e["code"] == "invalid_field_mapping" for e in r["errors"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_signature_normalized_vs_raw_mapping_no_false_warning():
|
||||||
|
"""Analyse liefert normalisierte Signatur; field_mappings nutzt Original-Header wie im Seed."""
|
||||||
|
r = validate_csv_template(
|
||||||
|
"blood_pressure",
|
||||||
|
{
|
||||||
|
"Datum": "measured_date",
|
||||||
|
"Zeit": "measured_time",
|
||||||
|
"Systolisch (mmHg)": "systolic",
|
||||||
|
"Diastolisch (mmHg)": "diastolic",
|
||||||
|
"Puls (bpm)": "pulse",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"measured_date": {"type": "date", "format": "dd.mm.yyyy"},
|
||||||
|
"measured_time": {"type": "time", "format": "HH:MM"},
|
||||||
|
"systolic": {"type": "int"},
|
||||||
|
"diastolic": {"type": "int"},
|
||||||
|
"pulse": {"type": "int"},
|
||||||
|
},
|
||||||
|
None,
|
||||||
|
["datum", "diastolisch_mmhg", "puls_bpm", "systolisch_mmhg", "zeit"],
|
||||||
|
)
|
||||||
|
assert r["valid"] is True
|
||||||
|
assert not any(w["code"] == "signature_vs_mappings_mismatch" for w in r["warnings"])
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_duplicate_target_warning():
|
||||||
|
r = validate_csv_template(
|
||||||
|
"weight",
|
||||||
|
{"A": "weight", "B": "weight", "Tag": "date"},
|
||||||
|
{
|
||||||
|
"weight": {"type": "float", "decimal_separator": "."},
|
||||||
|
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
|
||||||
|
},
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
assert r["valid"] is True
|
||||||
|
assert any(w["code"] == "duplicate_target_columns" for w in r["warnings"])
|
||||||
+32
-4
@@ -7,9 +7,9 @@ Semantic Versioning: MAJOR.MINOR.PATCH
|
|||||||
- PATCH: Bugfix, kleine Änderung, Refactor
|
- PATCH: Bugfix, kleine Änderung, Refactor
|
||||||
"""
|
"""
|
||||||
|
|
||||||
APP_VERSION = "0.9p"
|
APP_VERSION = "0.9q"
|
||||||
BUILD_DATE = "2026-04-09"
|
BUILD_DATE = "2026-04-11"
|
||||||
DB_SCHEMA_VERSION = "20260406e" # Migration 041
|
DB_SCHEMA_VERSION = "20260409c" # 048/049 vitals_baseline.source csv + SAVEPOINT Import
|
||||||
|
|
||||||
MODULE_VERSIONS = {
|
MODULE_VERSIONS = {
|
||||||
"auth": "1.2.0",
|
"auth": "1.2.0",
|
||||||
@@ -29,11 +29,39 @@ MODULE_VERSIONS = {
|
|||||||
"exportdata": "1.1.0",
|
"exportdata": "1.1.0",
|
||||||
"importdata": "1.0.0",
|
"importdata": "1.0.0",
|
||||||
"membership": "2.1.0",
|
"membership": "2.1.0",
|
||||||
"workflow": "0.6.0", # Phase 4: End Node Template Engine
|
"workflow": "0.7.0", # Part 3: Inline Prompts (reference + inline mode)
|
||||||
"app_dashboard": "1.11.0", # Entitlements: DB-Override widget→features (AND), sonst Katalog
|
"app_dashboard": "1.11.0", # Entitlements: DB-Override widget→features (AND), sonst Katalog
|
||||||
|
"csv_import": "0.3.2", # Import-Fehler: enrich_row_error / freundlichere 500-Hinweise
|
||||||
|
"admin_csv_templates": "0.3.0", # POST /validate + Speichern nur bei valid (422 + warnings in Response)
|
||||||
}
|
}
|
||||||
|
|
||||||
CHANGELOG = [
|
CHANGELOG = [
|
||||||
|
{
|
||||||
|
"version": "0.9q",
|
||||||
|
"date": "2026-04-11",
|
||||||
|
"changes": [
|
||||||
|
"Workflow Engine Part 3: Inline Prompts",
|
||||||
|
"Frontend: Radio Buttons (Reference/Inline), InlineTemplateEditor Component",
|
||||||
|
"Frontend: Placeholder Picker für Inline-Templates, Cursor-Position Tracking",
|
||||||
|
"Backend: load_prompt_template() unterstützt inline_template",
|
||||||
|
"Backend: WorkflowNode.inline_template Feld hinzugefügt",
|
||||||
|
"Serialization: inline_template speichern/laden in graph_data",
|
||||||
|
"Validation: Prüft dass entweder prompt_slug ODER inline_template gesetzt",
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"version": "0.9p",
|
||||||
|
"date": "2026-04-09",
|
||||||
|
"changes": [
|
||||||
|
"Issue #21 Phase 1: Migration 042/043 (csv_field_mappings, csv_import_log, Seeds)",
|
||||||
|
"csv_parser: core (Decode/Delimiter/Sample), module_registry, type_converter, permissions",
|
||||||
|
"API /api/csv: modules, limits, mappings, analyze, copy",
|
||||||
|
"API /api/admin/csv-templates: CRUD System-Templates, import-limits (system_config)",
|
||||||
|
"Issue #21: POST /api/csv/import + executor (nutrition Aggregat/Tag, weight, Blutdruck); activity 501",
|
||||||
|
"Issue #21: GET-Analyse über alle Vorlagen; Import nur mapping_id; GUI ohne Modul-Dropdown",
|
||||||
|
"v9c_cleanup_features.sql: FK-sichere csv_import→data_import Reihenfolge",
|
||||||
|
],
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"version": "0.9n",
|
"version": "0.9n",
|
||||||
"date": "2026-04-06",
|
"date": "2026-04-06",
|
||||||
|
|||||||
+120
-24
@@ -15,10 +15,10 @@ from datetime import datetime
|
|||||||
import uuid
|
import uuid
|
||||||
import logging
|
import logging
|
||||||
import json
|
import json
|
||||||
from jinja2 import Template, TemplateError
|
from jinja2 import Environment, ChainableUndefined, TemplateError
|
||||||
|
|
||||||
from workflow_models import (
|
from workflow_models import (
|
||||||
WorkflowGraph, NodeExecutionState, ExecutionResult,
|
WorkflowGraph, WorkflowNode, NodeExecutionState, ExecutionResult,
|
||||||
NodeStatus, NormalizedSignal, FallbackStrategy, SignalStatus,
|
NodeStatus, NormalizedSignal, FallbackStrategy, SignalStatus,
|
||||||
EndNodeOutputMode
|
EndNodeOutputMode
|
||||||
)
|
)
|
||||||
@@ -278,9 +278,10 @@ async def execute_node(
|
|||||||
|
|
||||||
# Analysis Nodes
|
# Analysis Nodes
|
||||||
if node.type == "analysis":
|
if node.type == "analysis":
|
||||||
# 1. Lade Prompt
|
# 1. Lade Prompt (Part 3: inline_template support)
|
||||||
prompt_template = await load_prompt_template(node.prompt_slug, context)
|
prompt_template = await load_prompt_template(node, context)
|
||||||
logger.debug(f"Node {node.id}: Loaded prompt '{node.prompt_slug}'")
|
source_type = "inline" if node.inline_template else "reference"
|
||||||
|
logger.debug(f"Node {node.id}: Loaded prompt from {source_type}")
|
||||||
|
|
||||||
# 2. Parse question_augmentations
|
# 2. Parse question_augmentations
|
||||||
questions = []
|
questions = []
|
||||||
@@ -593,10 +594,20 @@ def execute_end_node(
|
|||||||
# AUTO mode: Concatenate all analysis_core values
|
# AUTO mode: Concatenate all analysis_core values
|
||||||
logger.debug(f"End node {node.id}: Using AUTO output mode")
|
logger.debug(f"End node {node.id}: Using AUTO output mode")
|
||||||
|
|
||||||
|
# Get graph from context for node label lookup
|
||||||
|
graph = context.get("graph")
|
||||||
|
|
||||||
combined_analysis = []
|
combined_analysis = []
|
||||||
for node_id, node_state in context.get("node_results", {}).items():
|
for node_id, node_state in context.get("node_results", {}).items():
|
||||||
if node_state.status == NodeStatus.EXECUTED and node_state.analysis_core:
|
if node_state.status == NodeStatus.EXECUTED and node_state.analysis_core:
|
||||||
combined_analysis.append(f"## {node_id}\n{node_state.analysis_core}")
|
# Get node label from graph (fallback to node_id if not found)
|
||||||
|
node_label = node_id # default
|
||||||
|
if graph:
|
||||||
|
node_obj = next((n for n in graph.nodes if n.id == node_id), None)
|
||||||
|
if node_obj and hasattr(node_obj, 'data') and node_obj.data:
|
||||||
|
node_label = node_obj.data.get('label', node_id)
|
||||||
|
|
||||||
|
combined_analysis.append(f"## {node_label}\n{node_state.analysis_core}")
|
||||||
|
|
||||||
final_output = "\n\n".join(combined_analysis) if combined_analysis else "[No analysis generated]"
|
final_output = "\n\n".join(combined_analysis) if combined_analysis else "[No analysis generated]"
|
||||||
|
|
||||||
@@ -618,6 +629,11 @@ def execute_end_node(
|
|||||||
"reasoning_anchors": node_state.reasoning_anchors or "",
|
"reasoning_anchors": node_state.reasoning_anchors or "",
|
||||||
"status": node_state.status.value if node_state.status else "unknown",
|
"status": node_state.status.value if node_state.status else "unknown",
|
||||||
}
|
}
|
||||||
|
# Backward-compatible shortcut:
|
||||||
|
# allow {{node_id.relevanz}} in addition to {{node_id.decision_signals.relevanz}}
|
||||||
|
if node_state.decision_signals:
|
||||||
|
for signal_key, signal_value in node_state.decision_signals.items():
|
||||||
|
node_context[signal_key] = signal_value
|
||||||
|
|
||||||
# Add normalized signals as {{node_id.signal_ID}}
|
# Add normalized signals as {{node_id.signal_ID}}
|
||||||
# NOTE: question_type now IS the ID (not the type!)
|
# NOTE: question_type now IS the ID (not the type!)
|
||||||
@@ -660,7 +676,10 @@ def execute_end_node(
|
|||||||
|
|
||||||
# Render template
|
# Render template
|
||||||
try:
|
try:
|
||||||
jinja_template = Template(node.template)
|
# ChainableUndefined keeps missing nested attributes renderable
|
||||||
|
# so Jinja's default filter can handle them.
|
||||||
|
jinja_env = Environment(undefined=ChainableUndefined)
|
||||||
|
jinja_template = jinja_env.from_string(node.template)
|
||||||
final_output = jinja_template.render(template_context)
|
final_output = jinja_template.render(template_context)
|
||||||
logger.info(f"End node {node.id}: Template rendered successfully ({len(final_output)} chars)")
|
logger.info(f"End node {node.id}: Template rendered successfully ({len(final_output)} chars)")
|
||||||
except TemplateError as te:
|
except TemplateError as te:
|
||||||
@@ -804,39 +823,64 @@ def _has_active_incoming_edge(node, graph: WorkflowGraph, context: Dict[str, Any
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
async def load_prompt_template(prompt_slug: str, context: Dict[str, Any]) -> str:
|
async def load_prompt_template(node: WorkflowNode, context: Dict[str, Any]) -> str:
|
||||||
"""
|
"""
|
||||||
Lädt Prompt-Template aus DB und resolved Platzhalter.
|
Lädt Prompt-Template aus DB (reference mode) oder direkt vom Node (inline mode).
|
||||||
|
|
||||||
|
Part 3: Inline Prompts - Unterstützt zwei Modi:
|
||||||
|
- Reference Mode: prompt_slug → Template aus ai_prompts Tabelle
|
||||||
|
- Inline Mode: inline_template → Template direkt vom Node
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
prompt_slug: Slug des Prompts (z.B. "pipeline_body")
|
node: WorkflowNode mit prompt_slug ODER inline_template
|
||||||
context: {"variables": {"name": "Lars", ...}, "profile_id": "..."}
|
context: {"variables": {"name": "Lars", ...}, "profile_id": "..."}
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Resolved prompt template
|
Resolved prompt template
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
HTTPException: Wenn weder prompt_slug noch inline_template gesetzt
|
||||||
|
|
||||||
Beispiel:
|
Beispiel:
|
||||||
>>> template = await load_prompt_template("pipeline_body", {"profile_id": "123"})
|
>>> node = WorkflowNode(id="n1", prompt_slug="pipeline_body")
|
||||||
|
>>> template = await load_prompt_template(node, {"profile_id": "123"})
|
||||||
>>> "{{name}}" not in template
|
>>> "{{name}}" not in template
|
||||||
True
|
True
|
||||||
"""
|
"""
|
||||||
from placeholder_resolver import get_placeholder_example_values, get_placeholder_catalog
|
from placeholder_resolver import get_placeholder_example_values, get_placeholder_catalog
|
||||||
from prompt_executor import resolve_placeholders
|
from prompt_executor import resolve_placeholders
|
||||||
|
from fastapi import HTTPException
|
||||||
|
|
||||||
with get_db() as conn:
|
# Mode 1: Inline Template (NEU)
|
||||||
cur = get_cursor(conn)
|
if node.inline_template:
|
||||||
cur.execute(
|
logger.debug(f"Node {node.id}: Using inline template ({len(node.inline_template)} chars)")
|
||||||
"SELECT template FROM ai_prompts WHERE slug = %s AND active = true",
|
template = node.inline_template
|
||||||
(prompt_slug,)
|
|
||||||
|
# Mode 2: Reference (bestehend)
|
||||||
|
elif node.prompt_slug:
|
||||||
|
logger.debug(f"Node {node.id}: Loading prompt '{node.prompt_slug}' from DB")
|
||||||
|
with get_db() as conn:
|
||||||
|
cur = get_cursor(conn)
|
||||||
|
cur.execute(
|
||||||
|
"SELECT template FROM ai_prompts WHERE slug = %s AND active = true",
|
||||||
|
(node.prompt_slug,)
|
||||||
|
)
|
||||||
|
row = cur.fetchone()
|
||||||
|
if not row:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Prompt not found: {node.prompt_slug}")
|
||||||
|
template = row['template']
|
||||||
|
|
||||||
|
# Mode 3: Error - weder inline noch reference
|
||||||
|
else:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=400,
|
||||||
|
detail=f"Node {node.id}: Either prompt_slug or inline_template required"
|
||||||
)
|
)
|
||||||
row = cur.fetchone()
|
|
||||||
if not row:
|
|
||||||
raise ValueError(f"Prompt not found: {prompt_slug}")
|
|
||||||
|
|
||||||
template = row['template']
|
|
||||||
|
|
||||||
# Resolve Placeholders using modern prompt_executor method
|
# Resolve Placeholders using modern prompt_executor method
|
||||||
profile_id = context.get("profile_id")
|
profile_id = context.get("profile_id")
|
||||||
|
if not profile_id:
|
||||||
|
raise HTTPException(status_code=400, detail="profile_id required in context")
|
||||||
|
|
||||||
# Build variables dict with ALL registered placeholders
|
# Build variables dict with ALL registered placeholders
|
||||||
variables = {}
|
variables = {}
|
||||||
@@ -844,14 +888,51 @@ async def load_prompt_template(prompt_slug: str, context: Dict[str, Any]) -> str
|
|||||||
try:
|
try:
|
||||||
# Get all placeholder values from registry
|
# Get all placeholder values from registry
|
||||||
processed_placeholders = get_placeholder_example_values(profile_id)
|
processed_placeholders = get_placeholder_example_values(profile_id)
|
||||||
|
logger.info(f"🔍 DEBUG: Loaded {len(processed_placeholders)} placeholders from registry")
|
||||||
|
logger.info(f"🔍 DEBUG: Sample keys (first 3): {list(processed_placeholders.keys())[:3]}")
|
||||||
|
|
||||||
# Remove {{ }} from keys (placeholder_resolver returns them with wrappers)
|
# Remove {{ }} from keys (placeholder_resolver returns them with wrappers)
|
||||||
cleaned_placeholders = {
|
cleaned_placeholders = {
|
||||||
key.replace('{{', '').replace('}}', ''): value
|
key.replace('{{', '').replace('}}', '').strip(): value
|
||||||
for key, value in processed_placeholders.items()
|
for key, value in processed_placeholders.items()
|
||||||
}
|
}
|
||||||
|
logger.info(f"🔍 DEBUG: Cleaned keys (first 3): {list(cleaned_placeholders.keys())[:3]}")
|
||||||
|
logger.info(f"🔍 DEBUG: Sample values: name={cleaned_placeholders.get('name')}, age={cleaned_placeholders.get('age')}, geschlecht={cleaned_placeholders.get('geschlecht')}")
|
||||||
|
|
||||||
variables.update(cleaned_placeholders)
|
variables.update(cleaned_placeholders)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"Failed to load placeholders for workflow: {e}")
|
logger.error(f"❌ CRITICAL: Failed to load placeholders for workflow: {e}", exc_info=True)
|
||||||
|
|
||||||
|
# Add workflow node outputs as placeholders (Part 3: Inline Prompts)
|
||||||
|
# Format: node_id.analysis_core, node_id.signal_xyz, node_id.question_xyz
|
||||||
|
node_results = context.get("node_results", {})
|
||||||
|
if node_results:
|
||||||
|
logger.info(f"🔍 DEBUG: Adding {len(node_results)} node outputs as placeholders")
|
||||||
|
for node_id, node_state in node_results.items():
|
||||||
|
# analysis_core
|
||||||
|
if hasattr(node_state, 'analysis_core') and node_state.analysis_core:
|
||||||
|
key = f"{node_id}.analysis_core"
|
||||||
|
variables[key] = node_state.analysis_core
|
||||||
|
logger.debug(f" Added placeholder: {key} = {node_state.analysis_core[:50]}...")
|
||||||
|
|
||||||
|
# decision_signals (keyed by question ID)
|
||||||
|
if hasattr(node_state, 'decision_signals') and node_state.decision_signals:
|
||||||
|
for signal_id, signal_value in node_state.decision_signals.items():
|
||||||
|
# Signal placeholder: node_id.signal_question_id
|
||||||
|
signal_key = f"{node_id}.signal_{signal_id}"
|
||||||
|
variables[signal_key] = signal_value
|
||||||
|
logger.debug(f" Added placeholder: {signal_key} = {signal_value}")
|
||||||
|
|
||||||
|
# Question texts (from graph metadata if available)
|
||||||
|
# NOTE: Question text placeholders are populated from graph in PlaceholderPicker
|
||||||
|
# Here we only add if available in node_state metadata
|
||||||
|
if hasattr(node_state, 'metadata') and isinstance(node_state.metadata, dict):
|
||||||
|
questions = node_state.metadata.get('questions', [])
|
||||||
|
for q in questions:
|
||||||
|
if isinstance(q, dict) and 'id' in q and 'question' in q:
|
||||||
|
question_key = f"{node_id}.question_{q['id']}"
|
||||||
|
variables[question_key] = q['question']
|
||||||
|
logger.debug(f" Added placeholder: {question_key}")
|
||||||
|
|
||||||
# Load catalog for |d modifier support
|
# Load catalog for |d modifier support
|
||||||
try:
|
try:
|
||||||
@@ -860,13 +941,22 @@ async def load_prompt_template(prompt_slug: str, context: Dict[str, Any]) -> str
|
|||||||
catalog = None
|
catalog = None
|
||||||
logger.warning(f"Failed to load placeholder catalog for workflow: {e}")
|
logger.warning(f"Failed to load placeholder catalog for workflow: {e}")
|
||||||
|
|
||||||
|
logger.info(f"🔍 DEBUG: Template before resolution:\n{template[:200]}...")
|
||||||
|
logger.info(f"🔍 DEBUG: Variables dict has {len(variables)} entries")
|
||||||
|
|
||||||
# Resolve with modern executor
|
# Resolve with modern executor
|
||||||
|
debug_info = {}
|
||||||
resolved = resolve_placeholders(
|
resolved = resolve_placeholders(
|
||||||
template=template,
|
template=template,
|
||||||
variables=variables,
|
variables=variables,
|
||||||
|
debug_info=debug_info,
|
||||||
catalog=catalog
|
catalog=catalog
|
||||||
)
|
)
|
||||||
|
|
||||||
|
logger.info(f"🔍 DEBUG: Resolved placeholders: {debug_info.get('resolved_placeholders', {})}")
|
||||||
|
logger.info(f"🔍 DEBUG: Unresolved placeholders: {debug_info.get('unresolved_placeholders', [])}")
|
||||||
|
logger.info(f"🔍 DEBUG: Template after resolution:\n{resolved[:200]}...")
|
||||||
|
|
||||||
return resolved
|
return resolved
|
||||||
|
|
||||||
|
|
||||||
@@ -919,7 +1009,13 @@ def aggregate_results(node_states: List[NodeExecutionState], graph) -> Dict[str,
|
|||||||
final_output = state.analysis_core
|
final_output = state.analysis_core
|
||||||
else:
|
else:
|
||||||
# Regular node - add to combined analysis
|
# Regular node - add to combined analysis
|
||||||
combined_analysis.append(f"## {state.node_id}\n{state.analysis_core}")
|
# Get node label from graph (fallback to node_id)
|
||||||
|
node_label = state.node_id # default
|
||||||
|
node_obj = next((n for n in graph.nodes if n.id == state.node_id), None)
|
||||||
|
if node_obj and hasattr(node_obj, 'data') and node_obj.data:
|
||||||
|
node_label = node_obj.data.get('label', state.node_id)
|
||||||
|
|
||||||
|
combined_analysis.append(f"## {node_label}\n{state.analysis_core}")
|
||||||
|
|
||||||
if state.normalized_signals:
|
if state.normalized_signals:
|
||||||
all_signals.extend([s.model_dump() for s in state.normalized_signals])
|
all_signals.extend([s.model_dump() for s in state.normalized_signals])
|
||||||
|
|||||||
@@ -190,7 +190,8 @@ class WorkflowNode(BaseModel):
|
|||||||
position: Optional[Position] = Field(None, description="Position im visuellen Editor")
|
position: Optional[Position] = Field(None, description="Position im visuellen Editor")
|
||||||
|
|
||||||
# ANALYSIS-Knoten
|
# ANALYSIS-Knoten
|
||||||
prompt_slug: Optional[str] = Field(None, description="Slug des auszuführenden Prompts")
|
prompt_slug: Optional[str] = Field(None, description="Slug des auszuführenden Prompts (reference mode)")
|
||||||
|
inline_template: Optional[str] = Field(None, description="Inline-Prompt-Template (inline mode, Part 3)")
|
||||||
question_augmentations: Optional[List[QuestionAugmentation]] = Field(None, description="Fragenergänzungen (knotengebunden, überschreiben Prompt-Defaults)")
|
question_augmentations: Optional[List[QuestionAugmentation]] = Field(None, description="Fragenergänzungen (knotengebunden, überschreiben Prompt-Defaults)")
|
||||||
|
|
||||||
# LOGIC-Knoten
|
# LOGIC-Knoten
|
||||||
|
|||||||
@@ -14,6 +14,8 @@
|
|||||||
- ✅ Commits: 7 systematic commits (6 module migrations + 1 chart expansion)
|
- ✅ Commits: 7 systematic commits (6 module migrations + 1 chart expansion)
|
||||||
- ✅ charts.py: 329 → 2246 lines (+1917 lines)
|
- ✅ charts.py: 329 → 2246 lines (+1917 lines)
|
||||||
|
|
||||||
|
**Import vs. Auswertung:** Leitlinie „was beim Ingest erlaubt ist“ und Einordnung bestehender Import-Pfade: `.claude/rules/ARCHITECTURE.md` Abschnitt **8. CSV-Import vs. Data Layer**.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Executive Summary
|
## Executive Summary
|
||||||
|
|||||||
@@ -53,6 +53,9 @@ import RestDaysPage from './pages/RestDaysPage'
|
|||||||
import VitalsPage from './pages/VitalsPage'
|
import VitalsPage from './pages/VitalsPage'
|
||||||
import GoalsPage from './pages/GoalsPage'
|
import GoalsPage from './pages/GoalsPage'
|
||||||
import CustomGoalsPage from './pages/CustomGoalsPage'
|
import CustomGoalsPage from './pages/CustomGoalsPage'
|
||||||
|
import UniversalCsvImportPage from './pages/UniversalCsvImportPage'
|
||||||
|
import AdminCsvTemplatesPage from './pages/AdminCsvTemplatesPage'
|
||||||
|
import AdminCsvTemplateEditorPage from './pages/AdminCsvTemplateEditorPage'
|
||||||
import WorkflowEditorPage from './pages/WorkflowEditorPage'
|
import WorkflowEditorPage from './pages/WorkflowEditorPage'
|
||||||
import DesktopSidebar from './components/DesktopSidebar'
|
import DesktopSidebar from './components/DesktopSidebar'
|
||||||
import { getMainNavItems } from './config/appNav'
|
import { getMainNavItems } from './config/appNav'
|
||||||
@@ -225,6 +228,7 @@ function AppShell() {
|
|||||||
<Route path="/vitals" element={<VitalsPage />} />
|
<Route path="/vitals" element={<VitalsPage />} />
|
||||||
<Route path="/custom-goals" element={<CustomGoalsPage />} />
|
<Route path="/custom-goals" element={<CustomGoalsPage />} />
|
||||||
<Route path="/nutrition" element={<NutritionPage />} />
|
<Route path="/nutrition" element={<NutritionPage />} />
|
||||||
|
<Route path="/csv-import" element={<UniversalCsvImportPage />} />
|
||||||
<Route path="/activity" element={<ActivityPage />} />
|
<Route path="/activity" element={<ActivityPage />} />
|
||||||
<Route path="/guide" element={<GuidePage />} />
|
<Route path="/guide" element={<GuidePage />} />
|
||||||
</Route>
|
</Route>
|
||||||
@@ -256,6 +260,8 @@ function AppShell() {
|
|||||||
<Route path="goal-types" element={<AdminGoalTypesPage/>}/>
|
<Route path="goal-types" element={<AdminGoalTypesPage/>}/>
|
||||||
<Route path="focus-areas" element={<AdminFocusAreasPage/>}/>
|
<Route path="focus-areas" element={<AdminFocusAreasPage/>}/>
|
||||||
<Route path="reference-value-types" element={<AdminReferenceValueTypesPage/>}/>
|
<Route path="reference-value-types" element={<AdminReferenceValueTypesPage/>}/>
|
||||||
|
<Route path="csv-templates" element={<AdminCsvTemplatesPage />} />
|
||||||
|
<Route path="csv-templates/:id" element={<AdminCsvTemplateEditorPage />} />
|
||||||
</Route>
|
</Route>
|
||||||
</Route>
|
</Route>
|
||||||
<Route path="/workflow-editor/:id" element={<WorkflowEditorPage/>}/>
|
<Route path="/workflow-editor/:id" element={<WorkflowEditorPage/>}/>
|
||||||
|
|||||||
+13
-3
@@ -18,6 +18,8 @@
|
|||||||
--header-h: 52px;
|
--header-h: 52px;
|
||||||
--font: system-ui, -apple-system, 'Segoe UI', sans-serif;
|
--font: system-ui, -apple-system, 'Segoe UI', sans-serif;
|
||||||
--capture-content-max: 800px;
|
--capture-content-max: 800px;
|
||||||
|
/* Admin: nutzt volle Hauptspalte bis zu dieser Obergrenze (siehe .app-main:has(.admin-shell)) */
|
||||||
|
--admin-main-max: min(1560px, calc(100vw - 220px));
|
||||||
}
|
}
|
||||||
@media (prefers-color-scheme: dark) {
|
@media (prefers-color-scheme: dark) {
|
||||||
:root {
|
:root {
|
||||||
@@ -619,13 +621,16 @@ a.analysis-split__nav-item {
|
|||||||
|
|
||||||
.admin-page {
|
.admin-page {
|
||||||
width: 100%;
|
width: 100%;
|
||||||
|
min-width: 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Desktop: volle Breite der Admin-Spalte (nicht wie Erfassung 800px); Lesegröße leicht skaliert */
|
||||||
@media (min-width: 1024px) {
|
@media (min-width: 1024px) {
|
||||||
.admin-page {
|
.admin-page {
|
||||||
max-width: var(--capture-content-max);
|
max-width: 100%;
|
||||||
margin-left: auto;
|
margin-left: 0;
|
||||||
margin-right: auto;
|
margin-right: 0;
|
||||||
|
font-size: clamp(15px, 0.88rem + 0.25vw, 18px);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -841,6 +846,11 @@ a.analysis-split__nav-item {
|
|||||||
box-sizing: border-box;
|
box-sizing: border-box;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Admin: mehr horizontaler Raum für Tabellen auf großen Screens (:has ~2022+, sonst bleibt 1200px) */
|
||||||
|
.app-main:has(.admin-shell) {
|
||||||
|
max-width: var(--admin-main-max);
|
||||||
|
}
|
||||||
|
|
||||||
/* Dashboard (P3): Begrüßung + Kennzahlen-Zeile */
|
/* Dashboard (P3): Begrüßung + Kennzahlen-Zeile */
|
||||||
.dashboard-greeting {
|
.dashboard-greeting {
|
||||||
display: flex;
|
display: flex;
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
/**
|
||||||
|
* ConfirmDialog Component
|
||||||
|
*
|
||||||
|
* Modal confirmation dialog
|
||||||
|
*
|
||||||
|
* Props:
|
||||||
|
* - message: string
|
||||||
|
* - onConfirm: callback when confirmed
|
||||||
|
* - onCancel: callback when cancelled
|
||||||
|
* - confirmText: string (default: 'OK')
|
||||||
|
* - cancelText: string (default: 'Abbrechen')
|
||||||
|
* - type: 'warning' | 'danger' | 'info' (default: 'warning')
|
||||||
|
*/
|
||||||
|
export function ConfirmDialog({
|
||||||
|
message,
|
||||||
|
onConfirm,
|
||||||
|
onCancel,
|
||||||
|
confirmText = 'OK',
|
||||||
|
cancelText = 'Abbrechen',
|
||||||
|
type = 'warning'
|
||||||
|
}) {
|
||||||
|
const colors = {
|
||||||
|
warning: '#FFC107',
|
||||||
|
danger: 'var(--danger)',
|
||||||
|
info: 'var(--accent)'
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
position: 'fixed',
|
||||||
|
top: 0,
|
||||||
|
left: 0,
|
||||||
|
right: 0,
|
||||||
|
bottom: 0,
|
||||||
|
background: 'rgba(0,0,0,0.5)',
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
justifyContent: 'center',
|
||||||
|
zIndex: 20000
|
||||||
|
}}
|
||||||
|
onClick={onCancel}
|
||||||
|
>
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
background: 'var(--surface)',
|
||||||
|
borderRadius: '12px',
|
||||||
|
padding: '24px',
|
||||||
|
maxWidth: '500px',
|
||||||
|
width: '90%',
|
||||||
|
boxShadow: '0 8px 24px rgba(0,0,0,0.3)'
|
||||||
|
}}
|
||||||
|
onClick={(e) => e.stopPropagation()}
|
||||||
|
>
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
fontSize: '16px',
|
||||||
|
marginBottom: '24px',
|
||||||
|
color: 'var(--text1)',
|
||||||
|
lineHeight: '1.5'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{message}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
display: 'flex',
|
||||||
|
gap: '12px',
|
||||||
|
justifyContent: 'flex-end'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<button
|
||||||
|
onClick={onCancel}
|
||||||
|
style={{
|
||||||
|
padding: '8px 16px',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
borderRadius: '6px',
|
||||||
|
background: 'var(--surface2)',
|
||||||
|
color: 'var(--text1)',
|
||||||
|
cursor: 'pointer',
|
||||||
|
fontSize: '14px'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{cancelText}
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
onClick={onConfirm}
|
||||||
|
style={{
|
||||||
|
padding: '8px 16px',
|
||||||
|
border: 'none',
|
||||||
|
borderRadius: '6px',
|
||||||
|
background: colors[type],
|
||||||
|
color: type === 'warning' ? '#000' : 'white',
|
||||||
|
cursor: 'pointer',
|
||||||
|
fontSize: '14px',
|
||||||
|
fontWeight: 500
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{confirmText}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
import { useEffect } from 'react'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Toast Notification Component
|
||||||
|
*
|
||||||
|
* Auto-closing notification that appears at the top of the screen
|
||||||
|
*
|
||||||
|
* Props:
|
||||||
|
* - message: string
|
||||||
|
* - type: 'success' | 'error' | 'warning' | 'info'
|
||||||
|
* - duration: number (ms, default 3000)
|
||||||
|
* - onClose: callback when toast closes
|
||||||
|
*/
|
||||||
|
export function Toast({ message, type = 'info', duration = 3000, onClose }) {
|
||||||
|
useEffect(() => {
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
if (onClose) onClose()
|
||||||
|
}, duration)
|
||||||
|
|
||||||
|
return () => clearTimeout(timer)
|
||||||
|
}, [duration, onClose])
|
||||||
|
|
||||||
|
const styles = {
|
||||||
|
success: {
|
||||||
|
background: '#4CAF50',
|
||||||
|
color: 'white',
|
||||||
|
icon: '✅'
|
||||||
|
},
|
||||||
|
error: {
|
||||||
|
background: 'var(--danger)',
|
||||||
|
color: 'white',
|
||||||
|
icon: '❌'
|
||||||
|
},
|
||||||
|
warning: {
|
||||||
|
background: '#FFC107',
|
||||||
|
color: '#856404',
|
||||||
|
icon: '⚠️'
|
||||||
|
},
|
||||||
|
info: {
|
||||||
|
background: 'var(--accent)',
|
||||||
|
color: 'white',
|
||||||
|
icon: 'ℹ️'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const style = styles[type] || styles.info
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
position: 'fixed',
|
||||||
|
top: '80px',
|
||||||
|
left: '50%',
|
||||||
|
transform: 'translateX(-50%)',
|
||||||
|
background: style.background,
|
||||||
|
color: style.color,
|
||||||
|
padding: '12px 24px',
|
||||||
|
borderRadius: '8px',
|
||||||
|
boxShadow: '0 4px 12px rgba(0,0,0,0.3)',
|
||||||
|
zIndex: 10000,
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
gap: '8px',
|
||||||
|
fontSize: '14px',
|
||||||
|
fontWeight: 500,
|
||||||
|
minWidth: '300px',
|
||||||
|
maxWidth: '600px',
|
||||||
|
animation: 'slideDown 0.3s ease-out'
|
||||||
|
}}
|
||||||
|
onClick={onClose}
|
||||||
|
>
|
||||||
|
<span style={{ fontSize: '18px' }}>{style.icon}</span>
|
||||||
|
<span>{message}</span>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add animation CSS if not already in global styles
|
||||||
|
const styleElement = document.createElement('style')
|
||||||
|
styleElement.textContent = `
|
||||||
|
@keyframes slideDown {
|
||||||
|
from {
|
||||||
|
opacity: 0;
|
||||||
|
transform: translateX(-50%) translateY(-20px);
|
||||||
|
}
|
||||||
|
to {
|
||||||
|
opacity: 1;
|
||||||
|
transform: translateX(-50%) translateY(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
`
|
||||||
|
if (!document.querySelector('style[data-toast-styles]')) {
|
||||||
|
styleElement.setAttribute('data-toast-styles', 'true')
|
||||||
|
document.head.appendChild(styleElement)
|
||||||
|
}
|
||||||
@@ -39,6 +39,7 @@ export function WorkflowCanvas({
|
|||||||
className="workflow-canvas"
|
className="workflow-canvas"
|
||||||
minZoom={0.2}
|
minZoom={0.2}
|
||||||
maxZoom={2}
|
maxZoom={2}
|
||||||
|
deleteKeyCode={['Backspace', 'Delete']}
|
||||||
defaultEdgeOptions={{
|
defaultEdgeOptions={{
|
||||||
animated: false,
|
animated: false,
|
||||||
style: { strokeWidth: 2 },
|
style: { strokeWidth: 2 },
|
||||||
|
|||||||
@@ -5,16 +5,22 @@ import { Handle, Position } from 'reactflow'
|
|||||||
*
|
*
|
||||||
* Properties:
|
* Properties:
|
||||||
* - data.label: Node-Label
|
* - data.label: Node-Label
|
||||||
* - data.prompt_slug: Slug des referenzierten Basis-Prompts
|
* - data.prompt_slug: Slug des referenzierten Basis-Prompts (Reference Mode)
|
||||||
|
* - data.inline_template: Inline Prompt-Template (Inline Mode)
|
||||||
* - data.prompt_name: Name des Prompts (optional, für Display)
|
* - data.prompt_name: Name des Prompts (optional, für Display)
|
||||||
* - data.questions: Array von Question Augmentations
|
* - data.questions: Array von Question Augmentations
|
||||||
* - selected: Boolean
|
* - selected: Boolean
|
||||||
*/
|
*/
|
||||||
export function AnalysisNode({ data, selected }) {
|
export function AnalysisNode({ data, selected }) {
|
||||||
const hasQuestions = data.questions?.length > 0
|
const hasQuestions = data.questions?.length > 0
|
||||||
const promptName = data.prompt_name || (data.prompt_slug ? `Prompt: ${data.prompt_slug}` : 'Kein Prompt')
|
|
||||||
const questionCount = data.questions?.length || 0
|
const questionCount = data.questions?.length || 0
|
||||||
|
|
||||||
|
// Part 3: Inline Prompts - Zeige "Inline-Template" oder Prompt-Namen
|
||||||
|
const isInlineMode = data.inline_template !== null && data.inline_template !== undefined
|
||||||
|
const promptName = isInlineMode
|
||||||
|
? '✏️ Inline-Template'
|
||||||
|
: (data.prompt_name || (data.prompt_slug ? `Prompt: ${data.prompt_slug}` : 'Kein Prompt'))
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className={`workflow-node analysis-node ${selected ? 'selected' : ''}`}>
|
<div className={`workflow-node analysis-node ${selected ? 'selected' : ''}`}>
|
||||||
<div className="node-header">
|
<div className="node-header">
|
||||||
|
|||||||
@@ -42,24 +42,54 @@ export function JoinNode({ data, selected }) {
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Mehrere Target Handles für eingehende Pfade */}
|
{/* Mehrere Target Handles für eingehende Pfade (bis zu 8) */}
|
||||||
<Handle
|
<Handle
|
||||||
type="target"
|
type="target"
|
||||||
position={Position.Top}
|
position={Position.Top}
|
||||||
id="path_1"
|
id="path_1"
|
||||||
style={{ left: '25%', background: '#17A2B8' }}
|
style={{ left: '11%', background: '#17A2B8' }}
|
||||||
/>
|
/>
|
||||||
<Handle
|
<Handle
|
||||||
type="target"
|
type="target"
|
||||||
position={Position.Top}
|
position={Position.Top}
|
||||||
id="path_2"
|
id="path_2"
|
||||||
style={{ left: '50%', background: '#17A2B8' }}
|
style={{ left: '22%', background: '#17A2B8' }}
|
||||||
/>
|
/>
|
||||||
<Handle
|
<Handle
|
||||||
type="target"
|
type="target"
|
||||||
position={Position.Top}
|
position={Position.Top}
|
||||||
id="path_3"
|
id="path_3"
|
||||||
style={{ left: '75%', background: '#17A2B8' }}
|
style={{ left: '33%', background: '#17A2B8' }}
|
||||||
|
/>
|
||||||
|
<Handle
|
||||||
|
type="target"
|
||||||
|
position={Position.Top}
|
||||||
|
id="path_4"
|
||||||
|
style={{ left: '44%', background: '#17A2B8' }}
|
||||||
|
/>
|
||||||
|
<Handle
|
||||||
|
type="target"
|
||||||
|
position={Position.Top}
|
||||||
|
id="path_5"
|
||||||
|
style={{ left: '56%', background: '#17A2B8' }}
|
||||||
|
/>
|
||||||
|
<Handle
|
||||||
|
type="target"
|
||||||
|
position={Position.Top}
|
||||||
|
id="path_6"
|
||||||
|
style={{ left: '67%', background: '#17A2B8' }}
|
||||||
|
/>
|
||||||
|
<Handle
|
||||||
|
type="target"
|
||||||
|
position={Position.Top}
|
||||||
|
id="path_7"
|
||||||
|
style={{ left: '78%', background: '#17A2B8' }}
|
||||||
|
/>
|
||||||
|
<Handle
|
||||||
|
type="target"
|
||||||
|
position={Position.Top}
|
||||||
|
id="path_8"
|
||||||
|
style={{ left: '89%', background: '#17A2B8' }}
|
||||||
/>
|
/>
|
||||||
|
|
||||||
{/* Ein Source Handle für konsolidierten Ausgang */}
|
{/* Ein Source Handle für konsolidierten Ausgang */}
|
||||||
|
|||||||
@@ -3,9 +3,8 @@ import { Handle, Position } from 'reactflow'
|
|||||||
/**
|
/**
|
||||||
* StartNode - Workflow Einstiegspunkt
|
* StartNode - Workflow Einstiegspunkt
|
||||||
*
|
*
|
||||||
* Properties:
|
* - data.label: Anzeige auf dem Canvas (Node-Name)
|
||||||
* - data.label: Node-Label (default: "Start")
|
* - data.analysis_title: nur für KI-Analyse-UI, nicht auf dem Canvas
|
||||||
* - selected: Boolean (Node ist ausgewählt)
|
|
||||||
*/
|
*/
|
||||||
export function StartNode({ data, selected }) {
|
export function StartNode({ data, selected }) {
|
||||||
return (
|
return (
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
import { useRef } from 'react'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* InlineTemplateEditor - Template-Editor für Inline-Prompts
|
||||||
|
*
|
||||||
|
* Props:
|
||||||
|
* - value: Template-String
|
||||||
|
* - onChange: (template) => void
|
||||||
|
* - onPlaceholderPick: () => void - Öffnet Placeholder Picker
|
||||||
|
* - textareaRef: Ref für Cursor-Position (von Parent)
|
||||||
|
*/
|
||||||
|
export function InlineTemplateEditor({ value, onChange, onPlaceholderPick, textareaRef }) {
|
||||||
|
return (
|
||||||
|
<div className="inline-template-editor" style={{ marginTop: '12px' }}>
|
||||||
|
<label className="form-label">Template</label>
|
||||||
|
<div style={{ position: 'relative' }}>
|
||||||
|
<textarea
|
||||||
|
ref={textareaRef}
|
||||||
|
value={value || ''}
|
||||||
|
onChange={(e) => onChange(e.target.value)}
|
||||||
|
placeholder="Analysiere folgende Daten: Gewicht: {{ weight_current }} Ziel: {{ goal_weight }} Gib eine Empfehlung..."
|
||||||
|
rows={12}
|
||||||
|
style={{
|
||||||
|
width: '100%',
|
||||||
|
fontFamily: 'monospace',
|
||||||
|
fontSize: '13px',
|
||||||
|
padding: '12px',
|
||||||
|
paddingRight: '120px', // Platz für Button
|
||||||
|
borderRadius: '8px',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
background: 'var(--bg)',
|
||||||
|
color: 'var(--text1)',
|
||||||
|
resize: 'vertical',
|
||||||
|
lineHeight: '1.5'
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<button
|
||||||
|
className="btn-secondary"
|
||||||
|
onClick={onPlaceholderPick}
|
||||||
|
style={{
|
||||||
|
position: 'absolute',
|
||||||
|
top: '8px',
|
||||||
|
right: '8px',
|
||||||
|
fontSize: '11px',
|
||||||
|
padding: '6px 10px',
|
||||||
|
whiteSpace: 'nowrap'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{'{{ }}'} Platzhalter
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<div
|
||||||
|
className="help-text"
|
||||||
|
style={{
|
||||||
|
fontSize: '12px',
|
||||||
|
color: 'var(--text3)',
|
||||||
|
marginTop: '6px'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
💡 Tipp: Verwende <code style={{
|
||||||
|
background: 'var(--surface2)',
|
||||||
|
padding: '2px 6px',
|
||||||
|
borderRadius: '4px',
|
||||||
|
fontFamily: 'monospace'
|
||||||
|
}}>{'{{ placeholder_name }}'}</code> für dynamische Werte
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -6,17 +6,19 @@ import { api } from '../../../utils/api'
|
|||||||
*
|
*
|
||||||
* Props:
|
* Props:
|
||||||
* - nodes: Array of workflow nodes (to extract workflow-specific placeholders)
|
* - nodes: Array of workflow nodes (to extract workflow-specific placeholders)
|
||||||
|
* - currentNodeId: ID des aktuellen Nodes (wird aus Placeholders ausgeschlossen)
|
||||||
* - onSelect: (placeholderString) => void - Callback when placeholder is selected
|
* - onSelect: (placeholderString) => void - Callback when placeholder is selected
|
||||||
* - onClose: () => void
|
* - onClose: () => void
|
||||||
*
|
*
|
||||||
* Features:
|
* Features:
|
||||||
* - Lädt registrierte Platzhalter vom Backend (~120+)
|
* - Lädt registrierte Platzhalter vom Backend (~120+)
|
||||||
* - Extrahiert Workflow-spezifische Node-Outputs
|
* - Extrahiert Workflow-spezifische Node-Outputs
|
||||||
|
* - Filtert Selbst-Referenzierung (Node kann sich nicht selbst referenzieren)
|
||||||
* - Zeigt Node-Namen (nicht nur IDs)
|
* - Zeigt Node-Namen (nicht nur IDs)
|
||||||
* - Kategorisiert: System + Workflow
|
* - Kategorisiert: System + Workflow
|
||||||
* - Suchfunktion über alle Kategorien
|
* - Suchfunktion über alle Kategorien
|
||||||
*/
|
*/
|
||||||
export function PlaceholderPicker({ nodes, onSelect, onClose }) {
|
export function PlaceholderPicker({ nodes, currentNodeId, onSelect, onClose }) {
|
||||||
const [searchQuery, setSearchQuery] = useState('')
|
const [searchQuery, setSearchQuery] = useState('')
|
||||||
const [systemPlaceholders, setSystemPlaceholders] = useState([])
|
const [systemPlaceholders, setSystemPlaceholders] = useState([])
|
||||||
const [loading, setLoading] = useState(true)
|
const [loading, setLoading] = useState(true)
|
||||||
@@ -61,8 +63,8 @@ export function PlaceholderPicker({ nodes, onSelect, onClose }) {
|
|||||||
loadPlaceholders()
|
loadPlaceholders()
|
||||||
}, [])
|
}, [])
|
||||||
|
|
||||||
// Extrahiere Workflow-spezifische Platzhalter
|
// Extrahiere Workflow-spezifische Platzhalter (ohne aktuellen Node)
|
||||||
const workflowPlaceholders = extractWorkflowPlaceholders(nodes)
|
const workflowPlaceholders = extractWorkflowPlaceholders(nodes, currentNodeId)
|
||||||
|
|
||||||
// Kombiniere beide Listen
|
// Kombiniere beide Listen
|
||||||
const allPlaceholders = [
|
const allPlaceholders = [
|
||||||
@@ -341,14 +343,19 @@ export function PlaceholderPicker({ nodes, onSelect, onClose }) {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Extrahiert Workflow-spezifische Platzhalter aus Nodes
|
* Extrahiert Workflow-spezifische Platzhalter aus Nodes
|
||||||
|
*
|
||||||
|
* @param {Array} nodes - Alle Workflow-Nodes
|
||||||
|
* @param {string} currentNodeId - ID des aktuellen Nodes (wird ausgeschlossen)
|
||||||
*/
|
*/
|
||||||
function extractWorkflowPlaceholders(nodes) {
|
function extractWorkflowPlaceholders(nodes, currentNodeId) {
|
||||||
const placeholders = []
|
const placeholders = []
|
||||||
|
|
||||||
console.log('🔍 Extracting workflow placeholders from nodes:', nodes)
|
console.log('🔍 Extracting workflow placeholders from nodes:', nodes)
|
||||||
|
console.log('🚫 Excluding current node:', currentNodeId)
|
||||||
|
|
||||||
nodes.forEach(node => {
|
nodes.forEach(node => {
|
||||||
if (node.type === 'end') return // End Node hat keine Outputs
|
if (node.type === 'end') return // End Node hat keine Outputs
|
||||||
|
if (node.id === currentNodeId) return // Selbst-Referenzierung verhindern
|
||||||
|
|
||||||
const nodeId = node.id
|
const nodeId = node.id
|
||||||
const nodeLabel = node.data?.label || nodeId
|
const nodeLabel = node.data?.label || nodeId
|
||||||
|
|||||||
@@ -0,0 +1,246 @@
|
|||||||
|
import { useState } from 'react'
|
||||||
|
import { AlertCircle, AlertTriangle, ChevronDown, ChevronRight, X } from 'lucide-react'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ValidationPanel - Zeigt Fehler und Warnungen mit Details
|
||||||
|
*
|
||||||
|
* Features:
|
||||||
|
* - Aufklappbar (collapsible)
|
||||||
|
* - Click-to-Jump zu betroffener Node
|
||||||
|
* - Gruppierung nach Severity
|
||||||
|
* - Klare visuelle Trennung
|
||||||
|
*/
|
||||||
|
export function ValidationPanel({ errors, warnings, onNodeClick, onClose }) {
|
||||||
|
const [isExpanded, setIsExpanded] = useState(true)
|
||||||
|
const [showErrors, setShowErrors] = useState(true)
|
||||||
|
const [showWarnings, setShowWarnings] = useState(true)
|
||||||
|
|
||||||
|
const totalCount = errors.length + warnings.length
|
||||||
|
|
||||||
|
if (totalCount === 0) return null
|
||||||
|
|
||||||
|
const handleItemClick = (item) => {
|
||||||
|
if (item.nodeId && onNodeClick) {
|
||||||
|
onNodeClick(item.nodeId)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div style={{
|
||||||
|
position: 'fixed',
|
||||||
|
bottom: 90,
|
||||||
|
right: 20,
|
||||||
|
width: 400,
|
||||||
|
maxHeight: '60vh',
|
||||||
|
background: 'var(--surface)',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
borderRadius: 8,
|
||||||
|
boxShadow: '0 4px 12px rgba(0,0,0,0.15)',
|
||||||
|
display: 'flex',
|
||||||
|
flexDirection: 'column',
|
||||||
|
zIndex: 100
|
||||||
|
}}>
|
||||||
|
{/* Header */}
|
||||||
|
<div
|
||||||
|
onClick={() => setIsExpanded(!isExpanded)}
|
||||||
|
style={{
|
||||||
|
padding: '12px 16px',
|
||||||
|
borderBottom: isExpanded ? '1px solid var(--border)' : 'none',
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
justifyContent: 'space-between',
|
||||||
|
cursor: 'pointer',
|
||||||
|
userSelect: 'none',
|
||||||
|
background: errors.length > 0 ? 'rgba(216, 90, 48, 0.1)' : 'rgba(255, 193, 7, 0.1)'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<div style={{ display: 'flex', alignItems: 'center', gap: 8 }}>
|
||||||
|
{isExpanded ? <ChevronDown size={18} /> : <ChevronRight size={18} />}
|
||||||
|
<AlertCircle size={18} color={errors.length > 0 ? 'var(--danger)' : '#f59e0b'} />
|
||||||
|
<span style={{ fontWeight: 600, fontSize: 14 }}>
|
||||||
|
Validierung
|
||||||
|
</span>
|
||||||
|
<span style={{ fontSize: 12, color: 'var(--text3)' }}>
|
||||||
|
{errors.length > 0 && `${errors.length} Fehler`}
|
||||||
|
{errors.length > 0 && warnings.length > 0 && ', '}
|
||||||
|
{warnings.length > 0 && `${warnings.length} Warnung${warnings.length > 1 ? 'en' : ''}`}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
onClick={(e) => {
|
||||||
|
e.stopPropagation()
|
||||||
|
onClose && onClose()
|
||||||
|
}}
|
||||||
|
style={{
|
||||||
|
background: 'none',
|
||||||
|
border: 'none',
|
||||||
|
cursor: 'pointer',
|
||||||
|
padding: 4,
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<X size={16} color="var(--text3)" />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* Content */}
|
||||||
|
{isExpanded && (
|
||||||
|
<div style={{
|
||||||
|
overflowY: 'auto',
|
||||||
|
maxHeight: 'calc(60vh - 60px)'
|
||||||
|
}}>
|
||||||
|
{/* Errors */}
|
||||||
|
{errors.length > 0 && (
|
||||||
|
<div>
|
||||||
|
<div
|
||||||
|
onClick={() => setShowErrors(!showErrors)}
|
||||||
|
style={{
|
||||||
|
padding: '8px 16px',
|
||||||
|
background: 'rgba(216, 90, 48, 0.05)',
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
gap: 8,
|
||||||
|
cursor: 'pointer',
|
||||||
|
userSelect: 'none',
|
||||||
|
borderBottom: '1px solid var(--border)'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{showErrors ? <ChevronDown size={16} /> : <ChevronRight size={16} />}
|
||||||
|
<AlertCircle size={16} color="var(--danger)" />
|
||||||
|
<span style={{ fontWeight: 600, fontSize: 13, color: 'var(--danger)' }}>
|
||||||
|
Fehler ({errors.length})
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
{showErrors && (
|
||||||
|
<div>
|
||||||
|
{errors.map((error, idx) => (
|
||||||
|
<div
|
||||||
|
key={idx}
|
||||||
|
onClick={() => handleItemClick(error)}
|
||||||
|
style={{
|
||||||
|
padding: '10px 16px',
|
||||||
|
borderBottom: idx < errors.length - 1 ? '1px solid var(--border)' : 'none',
|
||||||
|
cursor: error.nodeId ? 'pointer' : 'default',
|
||||||
|
background: error.nodeId ? 'transparent' : 'transparent',
|
||||||
|
transition: 'background 0.15s',
|
||||||
|
}}
|
||||||
|
onMouseEnter={(e) => {
|
||||||
|
if (error.nodeId) e.currentTarget.style.background = 'rgba(216, 90, 48, 0.05)'
|
||||||
|
}}
|
||||||
|
onMouseLeave={(e) => {
|
||||||
|
e.currentTarget.style.background = 'transparent'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<div style={{
|
||||||
|
fontSize: 13,
|
||||||
|
color: 'var(--text1)',
|
||||||
|
marginBottom: 4
|
||||||
|
}}>
|
||||||
|
{error.message}
|
||||||
|
</div>
|
||||||
|
{error.nodeId && (
|
||||||
|
<div style={{
|
||||||
|
fontSize: 11,
|
||||||
|
color: 'var(--text3)',
|
||||||
|
fontFamily: 'monospace'
|
||||||
|
}}>
|
||||||
|
→ Klicken um zu Node zu springen
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{error.type && (
|
||||||
|
<div style={{
|
||||||
|
fontSize: 10,
|
||||||
|
color: 'var(--text3)',
|
||||||
|
marginTop: 4,
|
||||||
|
textTransform: 'uppercase'
|
||||||
|
}}>
|
||||||
|
{error.type}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Warnings */}
|
||||||
|
{warnings.length > 0 && (
|
||||||
|
<div>
|
||||||
|
<div
|
||||||
|
onClick={() => setShowWarnings(!showWarnings)}
|
||||||
|
style={{
|
||||||
|
padding: '8px 16px',
|
||||||
|
background: 'rgba(255, 193, 7, 0.05)',
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
gap: 8,
|
||||||
|
cursor: 'pointer',
|
||||||
|
userSelect: 'none',
|
||||||
|
borderBottom: '1px solid var(--border)'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{showWarnings ? <ChevronDown size={16} /> : <ChevronRight size={16} />}
|
||||||
|
<AlertTriangle size={16} color="#f59e0b" />
|
||||||
|
<span style={{ fontWeight: 600, fontSize: 13, color: '#f59e0b' }}>
|
||||||
|
Warnungen ({warnings.length})
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
{showWarnings && (
|
||||||
|
<div>
|
||||||
|
{warnings.map((warning, idx) => (
|
||||||
|
<div
|
||||||
|
key={idx}
|
||||||
|
onClick={() => handleItemClick(warning)}
|
||||||
|
style={{
|
||||||
|
padding: '10px 16px',
|
||||||
|
borderBottom: idx < warnings.length - 1 ? '1px solid var(--border)' : 'none',
|
||||||
|
cursor: warning.nodeId ? 'pointer' : 'default',
|
||||||
|
background: 'transparent',
|
||||||
|
transition: 'background 0.15s',
|
||||||
|
}}
|
||||||
|
onMouseEnter={(e) => {
|
||||||
|
if (warning.nodeId) e.currentTarget.style.background = 'rgba(255, 193, 7, 0.05)'
|
||||||
|
}}
|
||||||
|
onMouseLeave={(e) => {
|
||||||
|
e.currentTarget.style.background = 'transparent'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<div style={{
|
||||||
|
fontSize: 13,
|
||||||
|
color: 'var(--text1)',
|
||||||
|
marginBottom: 4
|
||||||
|
}}>
|
||||||
|
{warning.message}
|
||||||
|
</div>
|
||||||
|
{warning.nodeId && (
|
||||||
|
<div style={{
|
||||||
|
fontSize: 11,
|
||||||
|
color: 'var(--text3)',
|
||||||
|
fontFamily: 'monospace'
|
||||||
|
}}>
|
||||||
|
→ Klicken um zu Node zu springen
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{warning.type && (
|
||||||
|
<div style={{
|
||||||
|
fontSize: 10,
|
||||||
|
color: 'var(--text3)',
|
||||||
|
marginTop: 4,
|
||||||
|
textTransform: 'uppercase'
|
||||||
|
}}>
|
||||||
|
{warning.type}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -125,6 +125,11 @@ export const ADMIN_GROUPS = [
|
|||||||
label: 'Produkt-Dashboard (Standard)',
|
label: 'Produkt-Dashboard (Standard)',
|
||||||
description: 'Globales Standard-Layout der Startseite (DB oder Code-Fallback).',
|
description: 'Globales Standard-Layout der Startseite (DB oder Code-Fallback).',
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
to: '/admin/csv-templates',
|
||||||
|
label: 'CSV-Import-Vorlagen',
|
||||||
|
description: 'System-Vorlagen per CSV-Analyse anlernen, Spalten zuordnen, pflegen.',
|
||||||
|
},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
/** DB `ai_prompts.category` – Reihenfolge der Gruppen in der KI-Analyse-Navigation */
|
||||||
|
|
||||||
|
export const ANALYSIS_CATEGORY_ORDER = [
|
||||||
|
'körper',
|
||||||
|
'ernährung',
|
||||||
|
'training',
|
||||||
|
'schlaf',
|
||||||
|
'vitalwerte',
|
||||||
|
'ziele',
|
||||||
|
'ganzheitlich',
|
||||||
|
]
|
||||||
|
|
||||||
|
export const ANALYSIS_CATEGORY_LABELS = {
|
||||||
|
körper: 'Körper',
|
||||||
|
ernährung: 'Ernährung',
|
||||||
|
training: 'Training',
|
||||||
|
schlaf: 'Schlaf',
|
||||||
|
vitalwerte: 'Vitalwerte',
|
||||||
|
ziele: 'Ziele',
|
||||||
|
ganzheitlich: 'Ganzheitlich',
|
||||||
|
}
|
||||||
@@ -40,6 +40,13 @@ export const CAPTURE_HUB_TILES = [
|
|||||||
to: '/nutrition',
|
to: '/nutrition',
|
||||||
color: '#EF9F27',
|
color: '#EF9F27',
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
icon: '📥',
|
||||||
|
label: 'CSV-Import',
|
||||||
|
sub: 'Vorlagen für Ernährung, Gewicht, Blutdruck',
|
||||||
|
to: '/csv-import',
|
||||||
|
color: '#2E7D32',
|
||||||
|
},
|
||||||
{
|
{
|
||||||
icon: '🏋️',
|
icon: '🏋️',
|
||||||
label: 'Aktivität',
|
label: 'Aktivität',
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,125 @@
|
|||||||
|
import { useEffect, useState } from 'react'
|
||||||
|
import { Link } from 'react-router-dom'
|
||||||
|
import { ArrowLeft, FileSpreadsheet, Plus, Pencil } from 'lucide-react'
|
||||||
|
import { api } from '../utils/api'
|
||||||
|
|
||||||
|
const MODULE_LABEL = {
|
||||||
|
nutrition: 'Ernährung',
|
||||||
|
weight: 'Gewicht',
|
||||||
|
blood_pressure: 'Blutdruck',
|
||||||
|
activity: 'Aktivität',
|
||||||
|
sleep: 'Schlaf',
|
||||||
|
vitals_baseline: 'Vitalwerte (Baseline)',
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function AdminCsvTemplatesPage() {
|
||||||
|
const [templates, setTemplates] = useState([])
|
||||||
|
const [loading, setLoading] = useState(true)
|
||||||
|
const [error, setError] = useState(null)
|
||||||
|
const [filterModule, setFilterModule] = useState('')
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
let ok = true
|
||||||
|
setLoading(true)
|
||||||
|
setError(null)
|
||||||
|
api
|
||||||
|
.adminListCsvTemplates(filterModule || null)
|
||||||
|
.then((d) => {
|
||||||
|
if (ok) setTemplates(d.templates || [])
|
||||||
|
})
|
||||||
|
.catch((e) => {
|
||||||
|
if (ok) setError(e.message)
|
||||||
|
})
|
||||||
|
.finally(() => {
|
||||||
|
if (ok) setLoading(false)
|
||||||
|
})
|
||||||
|
return () => {
|
||||||
|
ok = false
|
||||||
|
}
|
||||||
|
}, [filterModule])
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div style={{ padding: '16px 16px 96px', maxWidth: 900, margin: '0 auto' }}>
|
||||||
|
<Link
|
||||||
|
to="/admin"
|
||||||
|
className="btn btn-secondary"
|
||||||
|
style={{ display: 'inline-flex', alignItems: 'center', gap: 8, marginBottom: 16, textDecoration: 'none' }}
|
||||||
|
>
|
||||||
|
<ArrowLeft size={18} /> Admin
|
||||||
|
</Link>
|
||||||
|
|
||||||
|
<h1 className="page-title" style={{ display: 'flex', alignItems: 'center', gap: 10 }}>
|
||||||
|
<FileSpreadsheet size={26} strokeWidth={2} />
|
||||||
|
CSV-Import-Vorlagen (System)
|
||||||
|
</h1>
|
||||||
|
<p style={{ fontSize: 14, color: 'var(--text2)', lineHeight: 1.55, marginBottom: 20 }}>
|
||||||
|
Systemweite Vorlagen für den Universal-CSV-Import. Neue Vorlagen entstehen aus einer Beispiel-CSV inkl.
|
||||||
|
Spaltenzuordnung — analog zur Import-Seite.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 12, alignItems: 'center', marginBottom: 20 }}>
|
||||||
|
<Link to="/admin/csv-templates/new" className="btn btn-primary" style={{ display: 'inline-flex', alignItems: 'center', gap: 8 }}>
|
||||||
|
<Plus size={18} /> Neue Vorlage
|
||||||
|
</Link>
|
||||||
|
<label style={{ fontSize: 14, color: 'var(--text2)', display: 'flex', alignItems: 'center', gap: 8 }}>
|
||||||
|
Modul:
|
||||||
|
<select
|
||||||
|
className="form-input"
|
||||||
|
value={filterModule}
|
||||||
|
onChange={(e) => setFilterModule(e.target.value)}
|
||||||
|
style={{ minWidth: 160 }}
|
||||||
|
>
|
||||||
|
<option value="">Alle</option>
|
||||||
|
<option value="nutrition">Ernährung</option>
|
||||||
|
<option value="weight">Gewicht</option>
|
||||||
|
<option value="blood_pressure">Blutdruck</option>
|
||||||
|
<option value="activity">Aktivität</option>
|
||||||
|
<option value="sleep">Schlaf</option>
|
||||||
|
<option value="vitals_baseline">Vitalwerte (Baseline)</option>
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{error && (
|
||||||
|
<div className="card" style={{ padding: 12, borderColor: 'var(--danger)', color: 'var(--danger)', marginBottom: 16 }}>
|
||||||
|
{error}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{loading ? (
|
||||||
|
<div className="spinner" style={{ width: 32, height: 32, margin: 24 }} />
|
||||||
|
) : (
|
||||||
|
<div style={{ display: 'flex', flexDirection: 'column', gap: 12 }}>
|
||||||
|
{templates.length === 0 ? (
|
||||||
|
<div className="card" style={{ padding: 16, color: 'var(--text2)' }}>
|
||||||
|
Keine Vorlagen. „Neue Vorlage“ legt eine System-Vorlage aus einer CSV an.
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
templates.map((t) => (
|
||||||
|
<div key={t.id} className="card" style={{ padding: 16 }}>
|
||||||
|
<div style={{ display: 'flex', justifyContent: 'space-between', alignItems: 'flex-start', gap: 12 }}>
|
||||||
|
<div>
|
||||||
|
<div style={{ fontWeight: 600 }}>
|
||||||
|
{MODULE_LABEL[t.module] || t.module}: {t.mapping_name}
|
||||||
|
</div>
|
||||||
|
<div style={{ fontSize: 13, color: 'var(--text2)', marginTop: 6 }}>
|
||||||
|
{t.description || '—'} · Trenner: <code>{t.delimiter}</code> · Spalten-Signatur:{' '}
|
||||||
|
{(t.column_signature || []).length} Felder
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<Link
|
||||||
|
to={`/admin/csv-templates/${t.id}`}
|
||||||
|
className="btn btn-secondary"
|
||||||
|
style={{ display: 'inline-flex', alignItems: 'center', gap: 6, flexShrink: 0 }}
|
||||||
|
>
|
||||||
|
<Pencil size={16} /> Bearbeiten
|
||||||
|
</Link>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
))
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -21,6 +21,7 @@ export default function AdminPromptsPage() {
|
|||||||
const [showNewPrompt, setShowNewPrompt] = useState(false)
|
const [showNewPrompt, setShowNewPrompt] = useState(false)
|
||||||
const [importing, setImporting] = useState(false)
|
const [importing, setImporting] = useState(false)
|
||||||
const [importResult, setImportResult] = useState(null)
|
const [importResult, setImportResult] = useState(null)
|
||||||
|
const [importDialogData, setImportDialogData] = useState(null) // {count, fileData, event}
|
||||||
|
|
||||||
const categories = [
|
const categories = [
|
||||||
{ id: 'all', label: 'Alle Kategorien' },
|
{ id: 'all', label: 'Alle Kategorien' },
|
||||||
@@ -194,7 +195,6 @@ export default function AdminPromptsPage() {
|
|||||||
const file = event.target.files[0]
|
const file = event.target.files[0]
|
||||||
if (!file) return
|
if (!file) return
|
||||||
|
|
||||||
setImporting(true)
|
|
||||||
setError(null)
|
setError(null)
|
||||||
setImportResult(null)
|
setImportResult(null)
|
||||||
|
|
||||||
@@ -202,14 +202,34 @@ export default function AdminPromptsPage() {
|
|||||||
const text = await file.text()
|
const text = await file.text()
|
||||||
const data = JSON.parse(text)
|
const data = JSON.parse(text)
|
||||||
|
|
||||||
// Ask user about overwrite
|
// Show custom 3-button dialog
|
||||||
const overwrite = confirm(
|
setImportDialogData({
|
||||||
'Bestehende Prompts überschreiben?\n\n' +
|
count: data.count || 0,
|
||||||
'JA = Existierende Prompts aktualisieren\n' +
|
fileData: data,
|
||||||
'NEIN = Nur neue Prompts erstellen, Duplikate überspringen'
|
event: event
|
||||||
)
|
})
|
||||||
|
} catch (e) {
|
||||||
|
setError('Import-Fehler: ' + e.message)
|
||||||
|
event.target.value = '' // Reset file input
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const result = await api.importPrompts(data, overwrite)
|
const handleImportChoice = async (choice) => {
|
||||||
|
if (!importDialogData) return
|
||||||
|
|
||||||
|
const { fileData, event } = importDialogData
|
||||||
|
setImportDialogData(null) // Close dialog
|
||||||
|
|
||||||
|
if (choice === 'cancel') {
|
||||||
|
event.target.value = '' // Reset file input
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
setImporting(true)
|
||||||
|
|
||||||
|
try {
|
||||||
|
const overwrite = choice === 'yes' // 'yes' = overwrite, 'no' = skip existing
|
||||||
|
const result = await api.importPrompts(fileData, overwrite)
|
||||||
setImportResult(result)
|
setImportResult(result)
|
||||||
await loadPrompts()
|
await loadPrompts()
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
@@ -607,6 +627,70 @@ export default function AdminPromptsPage() {
|
|||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{/* Import Dialog - 3 Button Choice */}
|
||||||
|
{importDialogData && (
|
||||||
|
<div style={{
|
||||||
|
position: 'fixed',
|
||||||
|
top: 0,
|
||||||
|
left: 0,
|
||||||
|
right: 0,
|
||||||
|
bottom: 0,
|
||||||
|
background: 'rgba(0,0,0,0.5)',
|
||||||
|
display: 'flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
justifyContent: 'center',
|
||||||
|
zIndex: 9999
|
||||||
|
}}>
|
||||||
|
<div style={{
|
||||||
|
background: 'var(--bg)',
|
||||||
|
borderRadius: 12,
|
||||||
|
padding: 24,
|
||||||
|
maxWidth: 500,
|
||||||
|
width: '90%',
|
||||||
|
boxShadow: '0 4px 20px rgba(0,0,0,0.2)'
|
||||||
|
}}>
|
||||||
|
<h3 style={{ marginTop: 0, marginBottom: 16, fontSize: 18 }}>
|
||||||
|
{importDialogData.count} Prompts importieren?
|
||||||
|
</h3>
|
||||||
|
<p style={{ marginBottom: 24, color: 'var(--text2)', lineHeight: 1.5 }}>
|
||||||
|
Wie sollen existierende Prompts behandelt werden?
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<div style={{ display: 'flex', flexDirection: 'column', gap: 12 }}>
|
||||||
|
<button
|
||||||
|
className="btn btn-primary"
|
||||||
|
onClick={() => handleImportChoice('yes')}
|
||||||
|
style={{ width: '100%' }}
|
||||||
|
>
|
||||||
|
Ja, überschreiben
|
||||||
|
<div style={{ fontSize: 12, opacity: 0.8, marginTop: 4 }}>
|
||||||
|
Bestehende Prompts aktualisieren
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
|
||||||
|
<button
|
||||||
|
className="btn"
|
||||||
|
onClick={() => handleImportChoice('no')}
|
||||||
|
style={{ width: '100%' }}
|
||||||
|
>
|
||||||
|
Nein, nur neue
|
||||||
|
<div style={{ fontSize: 12, opacity: 0.8, marginTop: 4 }}>
|
||||||
|
Nur neue Prompts erstellen, bestehende überspringen
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
|
||||||
|
<button
|
||||||
|
className="btn btn-secondary"
|
||||||
|
onClick={() => handleImportChoice('cancel')}
|
||||||
|
style={{ width: '100%' }}
|
||||||
|
>
|
||||||
|
Abbrechen
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -2,6 +2,11 @@ import React, { useState, useEffect, useMemo } from 'react'
|
|||||||
import { Brain, Trash2, ChevronDown, ChevronUp } from 'lucide-react'
|
import { Brain, Trash2, ChevronDown, ChevronUp } from 'lucide-react'
|
||||||
import { Link } from 'react-router-dom'
|
import { Link } from 'react-router-dom'
|
||||||
import { api } from '../utils/api'
|
import { api } from '../utils/api'
|
||||||
|
import { getWorkflowDisplayContent } from '../utils/workflowDisplay'
|
||||||
|
import {
|
||||||
|
ANALYSIS_CATEGORY_ORDER,
|
||||||
|
ANALYSIS_CATEGORY_LABELS,
|
||||||
|
} from '../config/analysisCategories'
|
||||||
import { useAuth } from '../context/AuthContext'
|
import { useAuth } from '../context/AuthContext'
|
||||||
import Markdown from '../utils/Markdown'
|
import Markdown from '../utils/Markdown'
|
||||||
import UsageBadge from '../components/UsageBadge'
|
import UsageBadge from '../components/UsageBadge'
|
||||||
@@ -14,19 +19,6 @@ const SLUG_LABELS = {
|
|||||||
pipeline: '🔬 Mehrstufige Gesamtanalyse'
|
pipeline: '🔬 Mehrstufige Gesamtanalyse'
|
||||||
}
|
}
|
||||||
|
|
||||||
/** DB `ai_prompts.category` – Reihenfolge der Gruppen in der Analyse-Navigation */
|
|
||||||
const ANALYSIS_CATEGORY_ORDER = ['körper', 'ernährung', 'training', 'schlaf', 'vitalwerte', 'ziele', 'ganzheitlich']
|
|
||||||
|
|
||||||
const ANALYSIS_CATEGORY_LABELS = {
|
|
||||||
körper: 'Körper',
|
|
||||||
ernährung: 'Ernährung',
|
|
||||||
training: 'Training',
|
|
||||||
schlaf: 'Schlaf',
|
|
||||||
vitalwerte: 'Vitalwerte',
|
|
||||||
ziele: 'Ziele',
|
|
||||||
ganzheitlich: 'Ganzheitlich',
|
|
||||||
}
|
|
||||||
|
|
||||||
function sortAnalysisCategoryKeys(keys) {
|
function sortAnalysisCategoryKeys(keys) {
|
||||||
return [...keys].sort((a, b) => {
|
return [...keys].sort((a, b) => {
|
||||||
const na = String(a).toLowerCase()
|
const na = String(a).toLowerCase()
|
||||||
@@ -45,6 +37,11 @@ function analysisCategoryLabel(key) {
|
|||||||
return ANALYSIS_CATEGORY_LABELS[k] || String(key)
|
return ANALYSIS_CATEGORY_LABELS[k] || String(key)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Analyse-Angebote: klassische Pipelines + graphbasierte KI-Workflows (`type === 'workflow'`) */
|
||||||
|
function isAnalysisOfferPrompt(p) {
|
||||||
|
return p.active && (p.type === 'pipeline' || p.type === 'workflow')
|
||||||
|
}
|
||||||
|
|
||||||
/** Pipeline-Prompts nach `category` gruppieren (Backend-Feld), innerhalb Gruppe nach sort_order */
|
/** Pipeline-Prompts nach `category` gruppieren (Backend-Feld), innerhalb Gruppe nach sort_order */
|
||||||
function buildPipelineGroups(pipelinePrompts) {
|
function buildPipelineGroups(pipelinePrompts) {
|
||||||
const m = new Map()
|
const m = new Map()
|
||||||
@@ -81,7 +78,8 @@ function InsightCard({ ins, onDelete, defaultOpen=false, prompts=[] }) {
|
|||||||
|
|
||||||
// Find matching prompt to get display_name
|
// Find matching prompt to get display_name
|
||||||
const prompt = prompts.find(p => p.slug === ins.scope)
|
const prompt = prompts.find(p => p.slug === ins.scope)
|
||||||
const displayName = prompt?.display_name || SLUG_LABELS[ins.scope] || ins.scope
|
const displayName =
|
||||||
|
prompt?.display_name || prompt?.name || SLUG_LABELS[ins.scope] || ins.scope
|
||||||
|
|
||||||
// Use already-parsed metadata
|
// Use already-parsed metadata
|
||||||
const metadata = metadataRaw
|
const metadata = metadataRaw
|
||||||
@@ -360,7 +358,7 @@ export default function Analysis() {
|
|||||||
},[])
|
},[])
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const list = prompts.filter(p => p.active && p.type === 'pipeline')
|
const list = prompts.filter(isAnalysisOfferPrompt)
|
||||||
setActiveCategoryKey(prev => {
|
setActiveCategoryKey(prev => {
|
||||||
if (!list.length) return null
|
if (!list.length) return null
|
||||||
const groups = buildPipelineGroups(list)
|
const groups = buildPipelineGroups(list)
|
||||||
@@ -372,7 +370,7 @@ export default function Analysis() {
|
|||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!newResult?.scope) return
|
if (!newResult?.scope) return
|
||||||
const list = prompts.filter(p => p.active && p.type === 'pipeline')
|
const list = prompts.filter(isAnalysisOfferPrompt)
|
||||||
const groups = buildPipelineGroups(list)
|
const groups = buildPipelineGroups(list)
|
||||||
const g = groups.find(gg => gg.prompts.some(p => p.slug === newResult.scope))
|
const g = groups.find(gg => gg.prompts.some(p => p.slug === newResult.scope))
|
||||||
if (g) setActiveCategoryKey(g.categoryKey)
|
if (g) setActiveCategoryKey(g.categoryKey)
|
||||||
@@ -394,6 +392,8 @@ export default function Analysis() {
|
|||||||
} else {
|
} else {
|
||||||
content = JSON.stringify(finalOutput, null, 2)
|
content = JSON.stringify(finalOutput, null, 2)
|
||||||
}
|
}
|
||||||
|
} else if (result.type === 'workflow') {
|
||||||
|
content = getWorkflowDisplayContent(result.aggregated_result)
|
||||||
} else {
|
} else {
|
||||||
// For base prompts, use output directly
|
// For base prompts, use output directly
|
||||||
content = typeof result.output === 'string' ? result.output : JSON.stringify(result.output, null, 2)
|
content = typeof result.output === 'string' ? result.output : JSON.stringify(result.output, null, 2)
|
||||||
@@ -405,7 +405,7 @@ export default function Analysis() {
|
|||||||
const placeholders = {}
|
const placeholders = {}
|
||||||
const resolved = result.debug.resolved_placeholders
|
const resolved = result.debug.resolved_placeholders
|
||||||
|
|
||||||
// For pipeline, collect from all stages
|
// For pipeline, collect from all stages (Workflow: kein gleiches debug-Schema)
|
||||||
if (result.type === 'pipeline' && result.debug.stages) {
|
if (result.type === 'pipeline' && result.debug.stages) {
|
||||||
for (const stage of result.debug.stages) {
|
for (const stage of result.debug.stages) {
|
||||||
for (const promptDebug of (stage.prompts || [])) {
|
for (const promptDebug of (stage.prompts || [])) {
|
||||||
@@ -456,9 +456,9 @@ export default function Analysis() {
|
|||||||
grouped[key].push(ins)
|
grouped[key].push(ins)
|
||||||
})
|
})
|
||||||
|
|
||||||
// Show only active pipeline-type prompts (und nach DB-Kategorie gruppiert)
|
// Aktive Pipeline- + Workflow-Prompts (nach DB-Kategorie gruppiert)
|
||||||
const { pipelinePrompts, pipelineGroups } = useMemo(() => {
|
const { pipelinePrompts, pipelineGroups } = useMemo(() => {
|
||||||
const list = prompts.filter(p => p.active && p.type === 'pipeline')
|
const list = prompts.filter(isAnalysisOfferPrompt)
|
||||||
return { pipelinePrompts: list, pipelineGroups: buildPipelineGroups(list) }
|
return { pipelinePrompts: list, pipelineGroups: buildPipelineGroups(list) }
|
||||||
}, [prompts])
|
}, [prompts])
|
||||||
|
|
||||||
@@ -538,9 +538,10 @@ export default function Analysis() {
|
|||||||
{canUseAI && pipelinePrompts.length > 0 && (
|
{canUseAI && pipelinePrompts.length > 0 && (
|
||||||
<>
|
<>
|
||||||
<p style={{fontSize:13,color:'var(--text2)',marginBottom:14,lineHeight:1.6}}>
|
<p style={{fontSize:13,color:'var(--text2)',marginBottom:14,lineHeight:1.6}}>
|
||||||
Zuerst die <strong>Kategorie</strong> wählen (Chip-Leiste bzw. Seitenleiste). Alle Pipeline-Analysen
|
Zuerst die <strong>Kategorie</strong> wählen (Chip-Leiste bzw. Seitenleiste). Alle{' '}
|
||||||
dieser Kategorie erscheinen im Detailbereich (rechts auf Desktop, darunter auf Mobil).
|
<strong>Pipeline- und Workflow-Auswertungen</strong> dieser Kategorie erscheinen im Detailbereich
|
||||||
Kategorien kommen aus dem Feld „Kategorie“ beim jeweiligen Prompt im Admin.
|
(rechts auf Desktop, darunter auf Mobil). Bei Workflows legst du Kategorie, Titel und Kurztext in der{' '}
|
||||||
|
<strong>Start-Node</strong> des Workflow-Editors fest; bei Pipelines im Admin unter KI-Prompts.
|
||||||
</p>
|
</p>
|
||||||
<div className="analysis-split">
|
<div className="analysis-split">
|
||||||
<div className="analysis-split__nav-wrap">
|
<div className="analysis-split__nav-wrap">
|
||||||
@@ -641,9 +642,9 @@ export default function Analysis() {
|
|||||||
|
|
||||||
{canUseAI && pipelinePrompts.length === 0 && (
|
{canUseAI && pipelinePrompts.length === 0 && (
|
||||||
<div className="empty-state">
|
<div className="empty-state">
|
||||||
<p>Keine aktiven Pipeline-Prompts verfügbar.</p>
|
<p>Keine aktiven Pipeline- oder Workflow-Auswertungen verfügbar.</p>
|
||||||
<p style={{fontSize:12,color:'var(--text3)',marginTop:8}}>
|
<p style={{fontSize:12,color:'var(--text3)',marginTop:8}}>
|
||||||
Erstelle Pipeline-Prompts im Admin-Bereich unter Admin → KI-Prompts.
|
Pipelines und Workflows werden im Admin unter KI-Prompts bzw. Workflow-Editor angelegt.
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
@@ -667,7 +668,10 @@ export default function Analysis() {
|
|||||||
onClick={() => setHistoryScopePick(scope)}
|
onClick={() => setHistoryScopePick(scope)}
|
||||||
aria-current={activeHistoryScope === scope ? 'page' : undefined}
|
aria-current={activeHistoryScope === scope ? 'page' : undefined}
|
||||||
>
|
>
|
||||||
{prompts.find(pr => pr.slug === scope)?.display_name || SLUG_LABELS[scope] || scope}
|
{(() => {
|
||||||
|
const pr = prompts.find((p) => p.slug === scope)
|
||||||
|
return pr?.display_name || pr?.name || SLUG_LABELS[scope] || scope
|
||||||
|
})()}
|
||||||
<span className="muted" style={{ fontSize: 12 }}> ({grouped[scope].length})</span>
|
<span className="muted" style={{ fontSize: 12 }}> ({grouped[scope].length})</span>
|
||||||
</button>
|
</button>
|
||||||
))}
|
))}
|
||||||
|
|||||||
@@ -0,0 +1,584 @@
|
|||||||
|
import { useState, useMemo, useRef } from 'react'
|
||||||
|
import { useNavigate } from 'react-router-dom'
|
||||||
|
import { ArrowLeft, FileSpreadsheet, Loader2, Upload } from 'lucide-react'
|
||||||
|
import { api } from '../utils/api'
|
||||||
|
import { csvPreviewTdStyle } from '../utils/csvPreviewCells'
|
||||||
|
|
||||||
|
/** Ziele, die der Universal-Executor bereits schreiben kann (ohne manuelle Modul-Wahl). */
|
||||||
|
const EXECUTOR_READY = new Set([
|
||||||
|
'nutrition',
|
||||||
|
'weight',
|
||||||
|
'blood_pressure',
|
||||||
|
'activity',
|
||||||
|
'sleep',
|
||||||
|
'vitals_baseline',
|
||||||
|
])
|
||||||
|
|
||||||
|
const MODULE_LABEL = {
|
||||||
|
nutrition: 'Ernährung',
|
||||||
|
weight: 'Gewicht',
|
||||||
|
blood_pressure: 'Blutdruck',
|
||||||
|
activity: 'Aktivität',
|
||||||
|
sleep: 'Schlaf',
|
||||||
|
vitals_baseline: 'Vitalwerte (Baseline)',
|
||||||
|
}
|
||||||
|
|
||||||
|
function mergeMappingChoices(detected, mapData) {
|
||||||
|
const sys = mapData.system_templates || []
|
||||||
|
const usr = mapData.user_mappings || []
|
||||||
|
const flat = [
|
||||||
|
...sys.map((m) => ({ ...m, _isSys: true })),
|
||||||
|
...usr.map((m) => ({ ...m, _isSys: false })),
|
||||||
|
]
|
||||||
|
const byId = new Map(flat.map((m) => [m.id, m]))
|
||||||
|
const out = []
|
||||||
|
const seen = new Set()
|
||||||
|
for (const d of detected || []) {
|
||||||
|
const row = byId.get(d.mapping_id)
|
||||||
|
if (row) {
|
||||||
|
out.push({
|
||||||
|
id: row.id,
|
||||||
|
module: row.module,
|
||||||
|
name: row.name,
|
||||||
|
is_system: row.is_system,
|
||||||
|
confidence: d.confidence ?? 0,
|
||||||
|
jaccard: d.jaccard,
|
||||||
|
template_recall: d.template_recall,
|
||||||
|
columns_matched: d.columns_matched,
|
||||||
|
columns_in_template: d.columns_in_template,
|
||||||
|
columns_in_csv: d.columns_in_csv,
|
||||||
|
})
|
||||||
|
seen.add(row.id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const row of flat) {
|
||||||
|
if (!seen.has(row.id)) {
|
||||||
|
out.push({
|
||||||
|
id: row.id,
|
||||||
|
module: row.module,
|
||||||
|
name: row.name,
|
||||||
|
is_system: row.is_system,
|
||||||
|
confidence: 0,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
function SampleTable({ sampleRows, columns }) {
|
||||||
|
if (!sampleRows?.length || !columns?.length) return null
|
||||||
|
const showCols = columns.slice(0, 8)
|
||||||
|
return (
|
||||||
|
<div style={{ overflowX: 'auto', marginTop: 12 }}>
|
||||||
|
<table style={{ width: '100%', fontSize: 12, borderCollapse: 'collapse' }}>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
{showCols.map((c) => (
|
||||||
|
<th
|
||||||
|
key={c}
|
||||||
|
style={{
|
||||||
|
textAlign: 'left',
|
||||||
|
padding: '8px 6px',
|
||||||
|
borderBottom: '1px solid var(--border)',
|
||||||
|
color: 'var(--text2)',
|
||||||
|
whiteSpace: 'nowrap',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{c}
|
||||||
|
</th>
|
||||||
|
))}
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{sampleRows.slice(0, 5).map((row, i) => (
|
||||||
|
<tr key={i}>
|
||||||
|
{showCols.map((c) => (
|
||||||
|
<td key={c} style={csvPreviewTdStyle(row[c] ?? '—')}>
|
||||||
|
{row[c] ?? '—'}
|
||||||
|
</td>
|
||||||
|
))}
|
||||||
|
</tr>
|
||||||
|
))}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function UniversalCsvImportPage() {
|
||||||
|
const navigate = useNavigate()
|
||||||
|
const fileInputRef = useRef(null)
|
||||||
|
const analyzeGenRef = useRef(0)
|
||||||
|
const [file, setFile] = useState(null)
|
||||||
|
const [dragActive, setDragActive] = useState(false)
|
||||||
|
const [analyzeResult, setAnalyzeResult] = useState(null)
|
||||||
|
const [mappingChoices, setMappingChoices] = useState([])
|
||||||
|
const [mappingId, setMappingId] = useState('')
|
||||||
|
const [loadingAnalyze, setLoadingAnalyze] = useState(false)
|
||||||
|
const [loadingImport, setLoadingImport] = useState(false)
|
||||||
|
const [loadingDiagnose, setLoadingDiagnose] = useState(false)
|
||||||
|
const [error, setError] = useState(null)
|
||||||
|
const [success, setSuccess] = useState(null)
|
||||||
|
const [lastImport, setLastImport] = useState(null)
|
||||||
|
const [diagnoseResult, setDiagnoseResult] = useState(null)
|
||||||
|
|
||||||
|
const selectedChoice = useMemo(
|
||||||
|
() => mappingChoices.find((c) => String(c.id) === String(mappingId)),
|
||||||
|
[mappingChoices, mappingId],
|
||||||
|
)
|
||||||
|
const importAllowed = selectedChoice && EXECUTOR_READY.has(selectedChoice.module)
|
||||||
|
|
||||||
|
const runAnalyze = async (fileToAnalyze) => {
|
||||||
|
if (!fileToAnalyze) return
|
||||||
|
const gen = ++analyzeGenRef.current
|
||||||
|
setLoadingAnalyze(true)
|
||||||
|
setError(null)
|
||||||
|
setSuccess(null)
|
||||||
|
setMappingId('')
|
||||||
|
try {
|
||||||
|
const res = await api.analyzeCsv(fileToAnalyze)
|
||||||
|
if (gen !== analyzeGenRef.current) return
|
||||||
|
setAnalyzeResult(res)
|
||||||
|
const mapData = await api.getCsvMappings()
|
||||||
|
if (gen !== analyzeGenRef.current) return
|
||||||
|
const merged = mergeMappingChoices(res.detected_mappings || [], mapData)
|
||||||
|
setMappingChoices(merged)
|
||||||
|
|
||||||
|
const best = res.recommended || (res.detected_mappings || [])[0]
|
||||||
|
let pick = ''
|
||||||
|
if (best && merged.some((m) => m.id === best.mapping_id)) {
|
||||||
|
pick = String(best.mapping_id)
|
||||||
|
} else if (merged.length) {
|
||||||
|
const firstReady = merged.find((m) => EXECUTOR_READY.has(m.module))
|
||||||
|
pick = String((firstReady || merged[0]).id)
|
||||||
|
}
|
||||||
|
setMappingId(pick)
|
||||||
|
} catch (e) {
|
||||||
|
if (gen !== analyzeGenRef.current) return
|
||||||
|
setError(e.message || 'Analyse fehlgeschlagen')
|
||||||
|
setAnalyzeResult(null)
|
||||||
|
setMappingChoices([])
|
||||||
|
} finally {
|
||||||
|
if (gen === analyzeGenRef.current) {
|
||||||
|
setLoadingAnalyze(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const assignCsvFile = (f) => {
|
||||||
|
if (!f) return
|
||||||
|
const name = (f.name || '').toLowerCase()
|
||||||
|
if (!name.endsWith('.csv')) {
|
||||||
|
setError('Bitte eine .csv-Datei wählen.')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
setFile(f)
|
||||||
|
setAnalyzeResult(null)
|
||||||
|
setMappingChoices([])
|
||||||
|
setMappingId('')
|
||||||
|
setSuccess(null)
|
||||||
|
setError(null)
|
||||||
|
setLastImport(null)
|
||||||
|
setDiagnoseResult(null)
|
||||||
|
void runAnalyze(f)
|
||||||
|
}
|
||||||
|
|
||||||
|
const handleImport = async () => {
|
||||||
|
if (!file || !mappingId) {
|
||||||
|
setError('Bitte Datei und Vorlage wählen')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (!importAllowed) {
|
||||||
|
setError('Diese Vorlage wird vom Universal-Importer noch nicht unterstützt.')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
setLoadingImport(true)
|
||||||
|
setError(null)
|
||||||
|
setSuccess(null)
|
||||||
|
try {
|
||||||
|
const res = await api.importUniversalCsv(file, Number(mappingId))
|
||||||
|
setLastImport(res)
|
||||||
|
const st = res.stats || {}
|
||||||
|
const modLabel = MODULE_LABEL[res.module] || res.module || ''
|
||||||
|
setSuccess(
|
||||||
|
(modLabel ? `${modLabel}: ` : '') +
|
||||||
|
`Import fertig — ${st.imported ?? 0} neu, ${st.updated ?? 0} aktualisiert, ` +
|
||||||
|
`${st.skipped ?? 0} übersprungen, ${st.errors ?? 0} Zeilenfehler.`,
|
||||||
|
)
|
||||||
|
} catch (e) {
|
||||||
|
setError(e.message || 'Import fehlgeschlagen')
|
||||||
|
} finally {
|
||||||
|
setLoadingImport(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const runDiagnose = async () => {
|
||||||
|
if (!file || !mappingId) {
|
||||||
|
setError('Bitte Datei und Vorlage wählen')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
setLoadingDiagnose(true)
|
||||||
|
setError(null)
|
||||||
|
setDiagnoseResult(null)
|
||||||
|
try {
|
||||||
|
const res = await api.diagnoseUniversalCsv(file, Number(mappingId))
|
||||||
|
setDiagnoseResult(res)
|
||||||
|
} catch (e) {
|
||||||
|
setError(e.message || 'Diagnose fehlgeschlagen')
|
||||||
|
} finally {
|
||||||
|
setLoadingDiagnose(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="capture-page" style={{ paddingBottom: 88 }}>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => navigate(-1)}
|
||||||
|
className="btn btn-secondary"
|
||||||
|
style={{
|
||||||
|
marginBottom: 16,
|
||||||
|
display: 'inline-flex',
|
||||||
|
alignItems: 'center',
|
||||||
|
gap: 8,
|
||||||
|
padding: '8px 12px',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<ArrowLeft size={18} /> Zurück
|
||||||
|
</button>
|
||||||
|
|
||||||
|
<h1 className="page-title" style={{ display: 'flex', alignItems: 'center', gap: 10 }}>
|
||||||
|
<FileSpreadsheet size={26} strokeWidth={2} />
|
||||||
|
CSV-Import
|
||||||
|
</h1>
|
||||||
|
<p style={{ fontSize: 14, color: 'var(--text2)', marginBottom: 20, lineHeight: 1.55 }}>
|
||||||
|
CSV hier ablegen oder die Fläche antippen — die Analyse startet sofort. Die App vergleicht die Spalten
|
||||||
|
mit gespeicherten Vorlagen (Ernährung, Gewicht, Blutdruck, Aktivität, Schlaf, Vitalwerte) und schlägt
|
||||||
|
passende Ziele vor. Du bestätigst die Vorlage — ohne zuerst ein Modell raten zu müssen. Schlaf-Import
|
||||||
|
erwartet das Apple-Health-Schlaf-CSV (Segment- oder Zusammenfassungs-Export).{' '}
|
||||||
|
<strong>Ausblick:</strong> Eine Datei → mehrere Zieltabellen.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
{error && (
|
||||||
|
<div
|
||||||
|
className="card"
|
||||||
|
style={{
|
||||||
|
marginBottom: 16,
|
||||||
|
padding: 12,
|
||||||
|
borderColor: 'var(--danger)',
|
||||||
|
color: 'var(--danger)',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{error}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{success && (
|
||||||
|
<div
|
||||||
|
className="card"
|
||||||
|
style={{
|
||||||
|
marginBottom: 16,
|
||||||
|
padding: 12,
|
||||||
|
borderColor: 'var(--accent)',
|
||||||
|
color: 'var(--accent-dark)',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{success}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{lastImport?.error_details?.length > 0 && (
|
||||||
|
<details
|
||||||
|
className="card"
|
||||||
|
style={{ marginBottom: 16, padding: 16, cursor: 'pointer' }}
|
||||||
|
open
|
||||||
|
>
|
||||||
|
<summary style={{ fontWeight: 600, color: 'var(--text1)' }}>
|
||||||
|
Zeilenfehler vom letzten Import ({lastImport.error_details.length}) — zum Kopieren aufklappen
|
||||||
|
</summary>
|
||||||
|
<pre
|
||||||
|
style={{
|
||||||
|
marginTop: 12,
|
||||||
|
fontSize: 12,
|
||||||
|
overflow: 'auto',
|
||||||
|
maxHeight: 320,
|
||||||
|
background: 'var(--surface2)',
|
||||||
|
padding: 12,
|
||||||
|
borderRadius: 8,
|
||||||
|
color: 'var(--text1)',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{JSON.stringify(lastImport.error_details, null, 2)}
|
||||||
|
</pre>
|
||||||
|
</details>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div className="card" style={{ marginBottom: 16, padding: 16 }}>
|
||||||
|
<div className="form-label">1. CSV-Datei</div>
|
||||||
|
<input
|
||||||
|
ref={fileInputRef}
|
||||||
|
type="file"
|
||||||
|
accept=".csv,text/csv"
|
||||||
|
className="form-input"
|
||||||
|
style={{ position: 'absolute', width: 0, height: 0, opacity: 0, pointerEvents: 'none' }}
|
||||||
|
onChange={(e) => assignCsvFile(e.target.files?.[0] || null)}
|
||||||
|
/>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => fileInputRef.current?.click()}
|
||||||
|
onDragEnter={(e) => {
|
||||||
|
e.preventDefault()
|
||||||
|
setDragActive(true)
|
||||||
|
}}
|
||||||
|
onDragOver={(e) => {
|
||||||
|
e.preventDefault()
|
||||||
|
setDragActive(true)
|
||||||
|
}}
|
||||||
|
onDragLeave={(e) => {
|
||||||
|
e.preventDefault()
|
||||||
|
if (!e.currentTarget.contains(e.relatedTarget)) setDragActive(false)
|
||||||
|
}}
|
||||||
|
onDrop={(e) => {
|
||||||
|
e.preventDefault()
|
||||||
|
setDragActive(false)
|
||||||
|
assignCsvFile(e.dataTransfer?.files?.[0] || null)
|
||||||
|
}}
|
||||||
|
className="btn btn-secondary"
|
||||||
|
style={{
|
||||||
|
marginTop: 8,
|
||||||
|
width: '100%',
|
||||||
|
minHeight: 120,
|
||||||
|
display: 'flex',
|
||||||
|
flexDirection: 'column',
|
||||||
|
alignItems: 'center',
|
||||||
|
justifyContent: 'center',
|
||||||
|
gap: 10,
|
||||||
|
border: `2px dashed ${dragActive ? 'var(--accent)' : 'var(--border)'}`,
|
||||||
|
borderRadius: 12,
|
||||||
|
background: dragActive ? 'var(--surface2)' : 'var(--surface)',
|
||||||
|
cursor: 'pointer',
|
||||||
|
padding: 16,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Upload size={28} strokeWidth={1.75} color="var(--accent)" />
|
||||||
|
<span style={{ fontSize: 15, color: 'var(--text1)', fontWeight: 600 }}>
|
||||||
|
Datei ablegen oder tippen zum Auswählen
|
||||||
|
</span>
|
||||||
|
<span style={{ fontSize: 13, color: 'var(--text3)' }}>
|
||||||
|
{loadingAnalyze ? (
|
||||||
|
<span style={{ display: 'inline-flex', alignItems: 'center', gap: 8 }}>
|
||||||
|
<Loader2 size={16} style={{ animation: 'spin 0.7s linear infinite' }} />
|
||||||
|
Datei wird analysiert …
|
||||||
|
</span>
|
||||||
|
) : file ? (
|
||||||
|
file.name
|
||||||
|
) : (
|
||||||
|
'Noch keine Datei gewählt'
|
||||||
|
)}
|
||||||
|
</span>
|
||||||
|
</button>
|
||||||
|
{file && !loadingAnalyze && (
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
className="btn btn-secondary"
|
||||||
|
style={{ marginTop: 12, width: '100%' }}
|
||||||
|
onClick={() => void runAnalyze(file)}
|
||||||
|
>
|
||||||
|
Dieselbe Datei erneut analysieren
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{analyzeResult && (
|
||||||
|
<div className="card" style={{ marginBottom: 16, padding: 16 }}>
|
||||||
|
<div className="form-label">2. Erkennung & Vorschau</div>
|
||||||
|
{(analyzeResult.warnings || []).length > 0 && (
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
marginTop: 12,
|
||||||
|
padding: 12,
|
||||||
|
borderRadius: 12,
|
||||||
|
background: 'rgba(216, 90, 48, 0.12)',
|
||||||
|
border: '1px solid var(--danger)',
|
||||||
|
color: 'var(--danger-dark, var(--danger))',
|
||||||
|
fontSize: 14,
|
||||||
|
lineHeight: 1.5,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{(analyzeResult.warnings || []).map((w, i) => (
|
||||||
|
<div key={i}>{w}</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
<div style={{ fontSize: 13, color: 'var(--text2)', marginTop: 8 }}>
|
||||||
|
Trennzeichen: <strong>{analyzeResult.delimiter}</strong> · Spalten:{' '}
|
||||||
|
{analyzeResult.columns?.length ?? 0}
|
||||||
|
{analyzeResult.format_detection?.apple_sleep ? (
|
||||||
|
<>
|
||||||
|
{' '}
|
||||||
|
· <strong>Format:</strong> Apple-Schlaf (Schlafanalyse oder Segment-Export)
|
||||||
|
</>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{analyzeResult.recommended && (
|
||||||
|
<div
|
||||||
|
style={{
|
||||||
|
marginTop: 12,
|
||||||
|
padding: 12,
|
||||||
|
borderRadius: 12,
|
||||||
|
background: 'var(--surface2)',
|
||||||
|
fontSize: 14,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<strong>Vorschlag:</strong>{' '}
|
||||||
|
{MODULE_LABEL[analyzeResult.recommended.module] || analyzeResult.recommended.module} —{' '}
|
||||||
|
{analyzeResult.recommended.mapping_name}.
|
||||||
|
<br />
|
||||||
|
<span style={{ fontSize: 13, color: 'var(--text2)', fontWeight: 500 }}>
|
||||||
|
Vorlage abgedeckt:{' '}
|
||||||
|
<strong>{Math.round((analyzeResult.recommended.confidence || 0) * 100)} %</strong>
|
||||||
|
{analyzeResult.recommended.columns_matched != null &&
|
||||||
|
analyzeResult.recommended.columns_in_template != null
|
||||||
|
? ` (${analyzeResult.recommended.columns_matched}/${analyzeResult.recommended.columns_in_template} erwartete Spalten in der Datei)`
|
||||||
|
: ''}
|
||||||
|
.{' '}
|
||||||
|
{analyzeResult.recommended.jaccard != null && (
|
||||||
|
<>
|
||||||
|
Jaccard{' '}
|
||||||
|
<strong>{Math.round(analyzeResult.recommended.jaccard * 100)} %</strong> (gesamte
|
||||||
|
Spalten-Überlappung — niedriger, wenn die CSV viele Zusatzspalten hat).
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{(analyzeResult.detected_mappings || []).length > 1 && (
|
||||||
|
<details style={{ marginTop: 12, fontSize: 13 }}>
|
||||||
|
<summary style={{ cursor: 'pointer', color: 'var(--text2)' }}>Weitere Treffer</summary>
|
||||||
|
<ul style={{ margin: '8px 0 0 18px', padding: 0 }}>
|
||||||
|
{analyzeResult.detected_mappings.slice(1, 8).map((d) => (
|
||||||
|
<li key={d.mapping_id}>
|
||||||
|
{MODULE_LABEL[d.module] || d.module}: {d.mapping_name} · Vorlage{' '}
|
||||||
|
{Math.round((d.confidence || 0) * 100)} %
|
||||||
|
{d.jaccard != null ? ` · Jaccard ${Math.round(d.jaccard * 100)} %` : ''}
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
</details>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<SampleTable sampleRows={analyzeResult.sample_rows} columns={analyzeResult.columns} />
|
||||||
|
|
||||||
|
<div className="form-label" style={{ marginTop: 16 }}>
|
||||||
|
Import-Vorlage
|
||||||
|
</div>
|
||||||
|
<select
|
||||||
|
className="form-input"
|
||||||
|
value={mappingId}
|
||||||
|
onChange={(e) => setMappingId(e.target.value)}
|
||||||
|
style={{
|
||||||
|
width: '100%',
|
||||||
|
marginTop: 8,
|
||||||
|
minHeight: 48,
|
||||||
|
textAlign: 'left',
|
||||||
|
padding: '12px 14px',
|
||||||
|
fontSize: 15,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{mappingChoices.length === 0 ? (
|
||||||
|
<option value="">Keine Vorlage geladen</option>
|
||||||
|
) : (
|
||||||
|
mappingChoices.map((o) => (
|
||||||
|
<option key={o.id} value={o.id}>
|
||||||
|
{MODULE_LABEL[o.module] || o.module} — {o.name}
|
||||||
|
{o.is_system ? ' (System)' : ''}
|
||||||
|
{o.confidence > 0
|
||||||
|
? ` · Vorlage ${Math.round(o.confidence * 100)} %${
|
||||||
|
o.jaccard != null ? ` · Jaccard ${Math.round(o.jaccard * 100)} %` : ''
|
||||||
|
}`
|
||||||
|
: ''}
|
||||||
|
{!EXECUTOR_READY.has(o.module) ? ' · Import: noch nicht hier' : ''}
|
||||||
|
</option>
|
||||||
|
))
|
||||||
|
)}
|
||||||
|
</select>
|
||||||
|
|
||||||
|
{selectedChoice && !importAllowed && (
|
||||||
|
<p style={{ fontSize: 13, color: 'var(--text3)', marginTop: 10 }}>
|
||||||
|
Diese Vorlage passt strukturell; dieser Import-Weg unterstützt das Zielmodul noch nicht.
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div style={{ display: 'flex', flexDirection: 'column', gap: 12, marginTop: 16 }}>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
className="btn btn-secondary"
|
||||||
|
style={{ width: '100%' }}
|
||||||
|
disabled={!file || !mappingId || !importAllowed || loadingDiagnose}
|
||||||
|
onClick={() => void runDiagnose()}
|
||||||
|
>
|
||||||
|
{loadingDiagnose ? (
|
||||||
|
<>
|
||||||
|
<Loader2
|
||||||
|
size={18}
|
||||||
|
style={{ marginRight: 8, animation: 'spin 0.7s linear infinite' }}
|
||||||
|
/>{' '}
|
||||||
|
Diagnose …
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
'Mapping prüfen (ohne Import)'
|
||||||
|
)}
|
||||||
|
</button>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
className="btn btn-primary"
|
||||||
|
style={{ width: '100%' }}
|
||||||
|
disabled={!file || !mappingId || !importAllowed || loadingImport}
|
||||||
|
onClick={handleImport}
|
||||||
|
>
|
||||||
|
{loadingImport ? (
|
||||||
|
<>
|
||||||
|
<Loader2 size={18} style={{ marginRight: 8, animation: 'spin 0.7s linear infinite' }} /> Import
|
||||||
|
läuft …
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
'Import starten'
|
||||||
|
)}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{diagnoseResult && (
|
||||||
|
<details style={{ marginTop: 20 }} open>
|
||||||
|
<summary style={{ cursor: 'pointer', fontWeight: 600, color: 'var(--text2)' }}>
|
||||||
|
Diagnose-Ergebnis ({diagnoseResult.rows_diagnosed ?? 0} Zeilen)
|
||||||
|
</summary>
|
||||||
|
<p style={{ fontSize: 13, color: 'var(--text3)', marginTop: 8, lineHeight: 1.5 }}>
|
||||||
|
Vorlage #{diagnoseResult.mapping_id} · {diagnoseResult.mapping_name} · Modul{' '}
|
||||||
|
{MODULE_LABEL[diagnoseResult.module] || diagnoseResult.module}. Hinweise: Vitalwerte{' '}
|
||||||
|
<code>vitals.*</code>, Blutdruck <code>blood_pressure.*</code>, Workouts{' '}
|
||||||
|
<code>activity.*</code> (z. B. <code>would_pass_row_gate</code> /{' '}
|
||||||
|
<code>prefilter_fail_reason</code>).
|
||||||
|
</p>
|
||||||
|
<pre
|
||||||
|
style={{
|
||||||
|
marginTop: 8,
|
||||||
|
fontSize: 11,
|
||||||
|
overflow: 'auto',
|
||||||
|
maxHeight: 480,
|
||||||
|
background: 'var(--surface2)',
|
||||||
|
padding: 12,
|
||||||
|
borderRadius: 8,
|
||||||
|
color: 'var(--text1)',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{JSON.stringify(diagnoseResult, null, 2)}
|
||||||
|
</pre>
|
||||||
|
</details>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -18,7 +18,33 @@ import { EndNodeConfig } from '../components/workflow/panels/EndNodeConfig'
|
|||||||
import { PlaceholderPicker } from '../components/workflow/panels/PlaceholderPicker'
|
import { PlaceholderPicker } from '../components/workflow/panels/PlaceholderPicker'
|
||||||
import { WorkflowExecutePanel } from '../components/workflow/panels/WorkflowExecutePanel'
|
import { WorkflowExecutePanel } from '../components/workflow/panels/WorkflowExecutePanel'
|
||||||
import { WorkflowResultViewer } from '../components/workflow/panels/WorkflowResultViewer'
|
import { WorkflowResultViewer } from '../components/workflow/panels/WorkflowResultViewer'
|
||||||
|
import { InlineTemplateEditor } from '../components/workflow/panels/InlineTemplateEditor'
|
||||||
|
import { ValidationPanel } from '../components/workflow/panels/ValidationPanel'
|
||||||
|
import { Toast } from '../components/Toast'
|
||||||
|
import { ConfirmDialog } from '../components/ConfirmDialog'
|
||||||
import '../styles/workflowEditor.css'
|
import '../styles/workflowEditor.css'
|
||||||
|
import {
|
||||||
|
ANALYSIS_CATEGORY_ORDER,
|
||||||
|
ANALYSIS_CATEGORY_LABELS,
|
||||||
|
} from '../config/analysisCategories'
|
||||||
|
|
||||||
|
/** Aus Start-Node → Felder für Analyse-UI / ai_prompts (display_name, description, category) */
|
||||||
|
function getWorkflowAnalysisPublishFields(nodes, fallbackName) {
|
||||||
|
const start = nodes.find((n) => n.type === 'start')
|
||||||
|
const d = start?.data || {}
|
||||||
|
const title =
|
||||||
|
String(d.analysis_title ?? '').trim() ||
|
||||||
|
String(fallbackName ?? '').trim() ||
|
||||||
|
'Workflow'
|
||||||
|
const description = String(d.analysis_description ?? '').trim()
|
||||||
|
const category =
|
||||||
|
String(d.analysis_category ?? 'ganzheitlich').toLowerCase().trim() || 'ganzheitlich'
|
||||||
|
return {
|
||||||
|
display_name: title,
|
||||||
|
description: description || null,
|
||||||
|
category,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Node-Type Mapping
|
// Node-Type Mapping
|
||||||
const nodeTypes = {
|
const nodeTypes = {
|
||||||
@@ -42,7 +68,6 @@ export default function WorkflowEditorPage() {
|
|||||||
const selectedNode = selectedNodeId ? nodes.find(n => n.id === selectedNodeId) : null
|
const selectedNode = selectedNodeId ? nodes.find(n => n.id === selectedNodeId) : null
|
||||||
const [currentPrompt, setCurrentPrompt] = useState(null)
|
const [currentPrompt, setCurrentPrompt] = useState(null)
|
||||||
const [workflowName, setWorkflowName] = useState('Neuer Workflow')
|
const [workflowName, setWorkflowName] = useState('Neuer Workflow')
|
||||||
const [workflowDescription, setWorkflowDescription] = useState('')
|
|
||||||
const [validationErrors, setValidationErrors] = useState([])
|
const [validationErrors, setValidationErrors] = useState([])
|
||||||
const [validationWarnings, setValidationWarnings] = useState([])
|
const [validationWarnings, setValidationWarnings] = useState([])
|
||||||
const [loading, setLoading] = useState(false)
|
const [loading, setLoading] = useState(false)
|
||||||
@@ -50,7 +75,26 @@ export default function WorkflowEditorPage() {
|
|||||||
const [availablePrompts, setAvailablePrompts] = useState([])
|
const [availablePrompts, setAvailablePrompts] = useState([])
|
||||||
const [executionResult, setExecutionResult] = useState(null)
|
const [executionResult, setExecutionResult] = useState(null)
|
||||||
const [showPlaceholderPicker, setShowPlaceholderPicker] = useState(false)
|
const [showPlaceholderPicker, setShowPlaceholderPicker] = useState(false)
|
||||||
|
const [placeholderPickerTarget, setPlaceholderPickerTarget] = useState('end') // 'end' | 'inline'
|
||||||
const endNodeTextareaRef = useRef(null)
|
const endNodeTextareaRef = useRef(null)
|
||||||
|
const inlineTemplateTextareaRef = useRef(null)
|
||||||
|
const [hasUnsavedChanges, setHasUnsavedChanges] = useState(false)
|
||||||
|
const [showValidationPanel, setShowValidationPanel] = useState(true)
|
||||||
|
|
||||||
|
// Toast & Confirm Dialog
|
||||||
|
const [toast, setToast] = useState(null)
|
||||||
|
const [confirmDialog, setConfirmDialog] = useState(null)
|
||||||
|
|
||||||
|
// Wrapped handlers to track unsaved changes
|
||||||
|
const handleNodesChange = useCallback((changes) => {
|
||||||
|
onNodesChange(changes)
|
||||||
|
setHasUnsavedChanges(true)
|
||||||
|
}, [onNodesChange])
|
||||||
|
|
||||||
|
const handleEdgesChange = useCallback((changes) => {
|
||||||
|
onEdgesChange(changes)
|
||||||
|
setHasUnsavedChanges(true)
|
||||||
|
}, [onEdgesChange])
|
||||||
|
|
||||||
// Load available basis prompts for Analysis nodes
|
// Load available basis prompts for Analysis nodes
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
@@ -80,12 +124,33 @@ export default function WorkflowEditorPage() {
|
|||||||
const { errors, warnings } = validateWorkflowGraph(nodes, edges)
|
const { errors, warnings } = validateWorkflowGraph(nodes, edges)
|
||||||
setValidationErrors(errors)
|
setValidationErrors(errors)
|
||||||
setValidationWarnings(warnings)
|
setValidationWarnings(warnings)
|
||||||
|
|
||||||
|
// Re-show validation panel if there are new errors/warnings
|
||||||
|
if (errors.length > 0 || warnings.length > 0) {
|
||||||
|
setShowValidationPanel(true)
|
||||||
|
}
|
||||||
}, [nodes, edges])
|
}, [nodes, edges])
|
||||||
|
|
||||||
|
// Warn on browser back/refresh if unsaved changes
|
||||||
|
useEffect(() => {
|
||||||
|
const handleBeforeUnload = (e) => {
|
||||||
|
if (hasUnsavedChanges) {
|
||||||
|
e.preventDefault()
|
||||||
|
e.returnValue = '' // Chrome requires returnValue to be set
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
window.addEventListener('beforeunload', handleBeforeUnload)
|
||||||
|
return () => window.removeEventListener('beforeunload', handleBeforeUnload)
|
||||||
|
}, [hasUnsavedChanges])
|
||||||
|
|
||||||
// ── Handlers ──────────────────────────────────────────────────────────────
|
// ── Handlers ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
const onConnect = useCallback(
|
const onConnect = useCallback(
|
||||||
(params) => setEdges((eds) => addEdge(params, eds)),
|
(params) => {
|
||||||
|
setEdges((eds) => addEdge(params, eds))
|
||||||
|
setHasUnsavedChanges(true)
|
||||||
|
},
|
||||||
[setEdges]
|
[setEdges]
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -94,16 +159,27 @@ export default function WorkflowEditorPage() {
|
|||||||
}, [])
|
}, [])
|
||||||
|
|
||||||
const handleAddNode = (nodeType) => {
|
const handleAddNode = (nodeType) => {
|
||||||
const newNode = {
|
const base = {
|
||||||
id: `node_${nodeIdCounter++}`,
|
id: `node_${nodeIdCounter++}`,
|
||||||
type: nodeType,
|
type: nodeType,
|
||||||
position: { x: 250, y: 100 + nodes.length * 100 },
|
position: { x: 250, y: 100 + nodes.length * 100 },
|
||||||
data: {
|
data: {
|
||||||
label: `${nodeType.charAt(0).toUpperCase() + nodeType.slice(1)} ${nodeIdCounter - 1}`
|
label: `${nodeType.charAt(0).toUpperCase() + nodeType.slice(1)} ${nodeIdCounter - 1}`,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if (nodeType === 'start') {
|
||||||
|
base.data = {
|
||||||
|
label: 'Start',
|
||||||
|
analysis_title:
|
||||||
|
workflowName && workflowName !== 'Neuer Workflow'
|
||||||
|
? workflowName
|
||||||
|
: '',
|
||||||
|
analysis_description: '',
|
||||||
|
analysis_category: 'ganzheitlich',
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
setNodes((nds) => [...nds, base])
|
||||||
setNodes((nds) => [...nds, newNode])
|
setHasUnsavedChanges(true)
|
||||||
}
|
}
|
||||||
|
|
||||||
const handleNodeUpdate = (nodeId, updates) => {
|
const handleNodeUpdate = (nodeId, updates) => {
|
||||||
@@ -113,6 +189,7 @@ export default function WorkflowEditorPage() {
|
|||||||
console.log('📝 Nodes after update:', updated.find(n => n.id === nodeId))
|
console.log('📝 Nodes after update:', updated.find(n => n.id === nodeId))
|
||||||
return updated
|
return updated
|
||||||
})
|
})
|
||||||
|
setHasUnsavedChanges(true)
|
||||||
}
|
}
|
||||||
|
|
||||||
const handleDeleteNode = () => {
|
const handleDeleteNode = () => {
|
||||||
@@ -121,6 +198,7 @@ export default function WorkflowEditorPage() {
|
|||||||
setNodes((nds) => nds.filter((n) => n.id !== selectedNode.id))
|
setNodes((nds) => nds.filter((n) => n.id !== selectedNode.id))
|
||||||
setEdges((eds) => eds.filter((e) => e.source !== selectedNode.id && e.target !== selectedNode.id))
|
setEdges((eds) => eds.filter((e) => e.source !== selectedNode.id && e.target !== selectedNode.id))
|
||||||
setSelectedNodeId(null)
|
setSelectedNodeId(null)
|
||||||
|
setHasUnsavedChanges(true)
|
||||||
}
|
}
|
||||||
|
|
||||||
const handleSave = async () => {
|
const handleSave = async () => {
|
||||||
@@ -143,31 +221,43 @@ export default function WorkflowEditorPage() {
|
|||||||
})
|
})
|
||||||
console.log('📊 Serialized graph_data:', graph_data)
|
console.log('📊 Serialized graph_data:', graph_data)
|
||||||
|
|
||||||
|
const { display_name, description, category } = getWorkflowAnalysisPublishFields(
|
||||||
|
nodes,
|
||||||
|
workflowName
|
||||||
|
)
|
||||||
|
|
||||||
if (currentPrompt) {
|
if (currentPrompt) {
|
||||||
// Update existing
|
// Update existing
|
||||||
console.log('📝 Updating existing workflow:', currentPrompt.id)
|
console.log('📝 Updating existing workflow:', currentPrompt.id)
|
||||||
await api.updateUnifiedPrompt(currentPrompt.id, {
|
await api.updateUnifiedPrompt(currentPrompt.id, {
|
||||||
type: 'workflow',
|
type: 'workflow',
|
||||||
name: workflowName,
|
name: workflowName,
|
||||||
description: workflowDescription,
|
display_name,
|
||||||
|
description: description ?? undefined,
|
||||||
|
category,
|
||||||
graph_data
|
graph_data
|
||||||
})
|
})
|
||||||
alert('Workflow gespeichert!')
|
setToast({ message: '✅ Workflow gespeichert!', type: 'success' })
|
||||||
} else {
|
} else {
|
||||||
// Create new
|
// Create new
|
||||||
console.log('✨ Creating new workflow')
|
console.log('✨ Creating new workflow')
|
||||||
const result = await api.createUnifiedPrompt({
|
const result = await api.createUnifiedPrompt({
|
||||||
type: 'workflow',
|
type: 'workflow',
|
||||||
name: workflowName,
|
name: workflowName,
|
||||||
description: workflowDescription,
|
display_name,
|
||||||
|
description: description ?? undefined,
|
||||||
|
category,
|
||||||
graph_data
|
graph_data
|
||||||
})
|
})
|
||||||
console.log('✅ Workflow created:', result)
|
console.log('✅ Workflow created:', result)
|
||||||
setCurrentPrompt({ id: result.id, slug: result.slug, name: workflowName })
|
setCurrentPrompt({ id: result.id, slug: result.slug, name: workflowName })
|
||||||
alert('Workflow erstellt!')
|
setToast({ message: '✅ Workflow erstellt!', type: 'success' })
|
||||||
console.log('🚀 Navigating to:', `/workflow-editor/${result.id}`)
|
console.log('🚀 Navigating to:', `/workflow-editor/${result.id}`)
|
||||||
navigate(`/workflow-editor/${result.id}`)
|
navigate(`/workflow-editor/${result.id}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Clear unsaved changes flag after successful save
|
||||||
|
setHasUnsavedChanges(false)
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
console.error('❌ handleSave error:', e)
|
console.error('❌ handleSave error:', e)
|
||||||
setError(e.message)
|
setError(e.message)
|
||||||
@@ -194,11 +284,30 @@ export default function WorkflowEditorPage() {
|
|||||||
const { nodes: loadedNodes, edges: loadedEdges } = deserializeFromWorkflowGraph(prompt.graph_data)
|
const { nodes: loadedNodes, edges: loadedEdges } = deserializeFromWorkflowGraph(prompt.graph_data)
|
||||||
console.log('🎯 Deserialized:', { nodes: loadedNodes, edges: loadedEdges })
|
console.log('🎯 Deserialized:', { nodes: loadedNodes, edges: loadedEdges })
|
||||||
|
|
||||||
setNodes(loadedNodes)
|
const mergedNodes = loadedNodes.map((n) => {
|
||||||
|
if (n.type !== 'start') return n
|
||||||
|
const hasStoredTitle = Object.prototype.hasOwnProperty.call(n.data || {}, 'analysis_title')
|
||||||
|
return {
|
||||||
|
...n,
|
||||||
|
data: {
|
||||||
|
label: n.data?.label || 'Start',
|
||||||
|
...n.data,
|
||||||
|
analysis_title: hasStoredTitle
|
||||||
|
? n.data.analysis_title
|
||||||
|
: (prompt.display_name || prompt.name || ''),
|
||||||
|
analysis_description: Object.prototype.hasOwnProperty.call(n.data || {}, 'analysis_description')
|
||||||
|
? n.data.analysis_description
|
||||||
|
: (prompt.description || ''),
|
||||||
|
analysis_category:
|
||||||
|
(n.data?.analysis_category || prompt.category || 'ganzheitlich').toLowerCase(),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
setNodes(mergedNodes)
|
||||||
setEdges(loadedEdges)
|
setEdges(loadedEdges)
|
||||||
setCurrentPrompt(prompt)
|
setCurrentPrompt(prompt)
|
||||||
setWorkflowName(prompt.name)
|
setWorkflowName(prompt.name)
|
||||||
setWorkflowDescription(prompt.description || '')
|
|
||||||
|
|
||||||
// nodeIdCounter aktualisieren
|
// nodeIdCounter aktualisieren
|
||||||
const maxId = Math.max(
|
const maxId = Math.max(
|
||||||
@@ -207,6 +316,9 @@ export default function WorkflowEditorPage() {
|
|||||||
)
|
)
|
||||||
nodeIdCounter = maxId + 1
|
nodeIdCounter = maxId + 1
|
||||||
console.log('✅ Workflow loaded successfully, nodes:', loadedNodes.length, 'edges:', loadedEdges.length)
|
console.log('✅ Workflow loaded successfully, nodes:', loadedNodes.length, 'edges:', loadedEdges.length)
|
||||||
|
|
||||||
|
// Clear unsaved changes flag after successful load
|
||||||
|
setHasUnsavedChanges(false)
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
console.error('❌ loadWorkflow error:', e)
|
console.error('❌ loadWorkflow error:', e)
|
||||||
setError(e.message)
|
setError(e.message)
|
||||||
@@ -228,15 +340,18 @@ export default function WorkflowEditorPage() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const handleNew = () => {
|
const handleNew = () => {
|
||||||
if (confirm('Neuen Workflow erstellen? Ungespeicherte Änderungen gehen verloren.')) {
|
if (hasUnsavedChanges) {
|
||||||
setNodes([])
|
if (!confirm('Neuen Workflow erstellen? Ungespeicherte Änderungen gehen verloren.')) {
|
||||||
setEdges([])
|
return
|
||||||
setCurrentPrompt(null)
|
}
|
||||||
setWorkflowName('Neuer Workflow')
|
|
||||||
setWorkflowDescription('')
|
|
||||||
setSelectedNodeId(null)
|
|
||||||
navigate('/workflow-editor/new')
|
|
||||||
}
|
}
|
||||||
|
setNodes([])
|
||||||
|
setEdges([])
|
||||||
|
setCurrentPrompt(null)
|
||||||
|
setWorkflowName('Neuer Workflow')
|
||||||
|
setSelectedNodeId(null)
|
||||||
|
setHasUnsavedChanges(false)
|
||||||
|
navigate('/workflow-editor/new')
|
||||||
}
|
}
|
||||||
|
|
||||||
const handleDelete = async () => {
|
const handleDelete = async () => {
|
||||||
@@ -261,31 +376,81 @@ export default function WorkflowEditorPage() {
|
|||||||
setExecutionResult(result)
|
setExecutionResult(result)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const handleBack = () => {
|
||||||
|
if (hasUnsavedChanges) {
|
||||||
|
const confirm = window.confirm(
|
||||||
|
'Du hast ungespeicherte Änderungen.\n\n' +
|
||||||
|
'Möchtest du wirklich zurück gehen? Alle Änderungen gehen verloren.'
|
||||||
|
)
|
||||||
|
if (!confirm) return
|
||||||
|
}
|
||||||
|
navigate('/admin/prompts')
|
||||||
|
}
|
||||||
|
|
||||||
|
const handleValidationNodeClick = (nodeId) => {
|
||||||
|
// Select the node to show its config panel
|
||||||
|
setSelectedNodeId(nodeId)
|
||||||
|
|
||||||
|
// TODO: Optional - scroll to node in canvas
|
||||||
|
// ReactFlow doesn't expose a direct scrollTo API, but we could use fitView
|
||||||
|
// or manual DOM manipulation if needed
|
||||||
|
}
|
||||||
|
|
||||||
const handlePlaceholderSelect = (placeholderString) => {
|
const handlePlaceholderSelect = (placeholderString) => {
|
||||||
if (!selectedNode || selectedNode.type !== 'end') return
|
if (!selectedNode) return
|
||||||
|
|
||||||
const textarea = endNodeTextareaRef.current
|
// Target bestimmen: End Node oder Inline Template
|
||||||
const currentTemplate = selectedNode.data.template || ''
|
if (placeholderPickerTarget === 'end' && selectedNode.type === 'end') {
|
||||||
|
const textarea = endNodeTextareaRef.current
|
||||||
|
const currentTemplate = selectedNode.data.template || ''
|
||||||
|
|
||||||
// Wenn Textarea Ref verfügbar, an Cursor-Position einfügen
|
// Wenn Textarea Ref verfügbar, an Cursor-Position einfügen
|
||||||
if (textarea) {
|
if (textarea) {
|
||||||
const cursorPos = textarea.selectionStart || currentTemplate.length
|
const cursorPos = textarea.selectionStart || currentTemplate.length
|
||||||
const before = currentTemplate.substring(0, cursorPos)
|
const before = currentTemplate.substring(0, cursorPos)
|
||||||
const after = currentTemplate.substring(cursorPos)
|
const after = currentTemplate.substring(cursorPos)
|
||||||
const newTemplate = before + placeholderString + after
|
const newTemplate = before + placeholderString + after
|
||||||
|
|
||||||
handleNodeUpdate(selectedNode.id, { template: newTemplate })
|
handleNodeUpdate(selectedNode.id, { template: newTemplate })
|
||||||
|
|
||||||
// Cursor nach eingefügtem Platzhalter positionieren
|
// Cursor nach eingefügtem Platzhalter positionieren
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
const newPos = cursorPos + placeholderString.length
|
const newPos = cursorPos + placeholderString.length
|
||||||
textarea.setSelectionRange(newPos, newPos)
|
textarea.setSelectionRange(newPos, newPos)
|
||||||
textarea.focus()
|
textarea.focus()
|
||||||
}, 0)
|
}, 0)
|
||||||
} else {
|
} else {
|
||||||
// Fallback: Am Ende einfügen
|
// Fallback: Am Ende einfügen
|
||||||
const newTemplate = currentTemplate + placeholderString
|
const newTemplate = currentTemplate + placeholderString
|
||||||
handleNodeUpdate(selectedNode.id, { template: newTemplate })
|
handleNodeUpdate(selectedNode.id, { template: newTemplate })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inline Template (Analysis Node)
|
||||||
|
else if (placeholderPickerTarget === 'inline' && selectedNode.type === 'analysis') {
|
||||||
|
const textarea = inlineTemplateTextareaRef.current
|
||||||
|
const currentTemplate = selectedNode.data.inline_template || ''
|
||||||
|
|
||||||
|
// Wenn Textarea Ref verfügbar, an Cursor-Position einfügen
|
||||||
|
if (textarea) {
|
||||||
|
const cursorPos = textarea.selectionStart || currentTemplate.length
|
||||||
|
const before = currentTemplate.substring(0, cursorPos)
|
||||||
|
const after = currentTemplate.substring(cursorPos)
|
||||||
|
const newTemplate = before + placeholderString + after
|
||||||
|
|
||||||
|
handleNodeUpdate(selectedNode.id, { inline_template: newTemplate })
|
||||||
|
|
||||||
|
// Cursor nach eingefügtem Platzhalter positionieren
|
||||||
|
setTimeout(() => {
|
||||||
|
const newPos = cursorPos + placeholderString.length
|
||||||
|
textarea.setSelectionRange(newPos, newPos)
|
||||||
|
textarea.focus()
|
||||||
|
}, 0)
|
||||||
|
} else {
|
||||||
|
// Fallback: Am Ende einfügen
|
||||||
|
const newTemplate = currentTemplate + placeholderString
|
||||||
|
handleNodeUpdate(selectedNode.id, { inline_template: newTemplate })
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -295,15 +460,19 @@ export default function WorkflowEditorPage() {
|
|||||||
<div className="workflow-editor">
|
<div className="workflow-editor">
|
||||||
{/* Toolbar */}
|
{/* Toolbar */}
|
||||||
<div className="workflow-toolbar">
|
<div className="workflow-toolbar">
|
||||||
<button className="btn-secondary" onClick={() => navigate('/admin/prompts')}>
|
<button className="btn-secondary" onClick={handleBack}>
|
||||||
← Zurück
|
← Zurück
|
||||||
</button>
|
</button>
|
||||||
|
|
||||||
<input
|
<input
|
||||||
type="text"
|
type="text"
|
||||||
value={workflowName}
|
value={workflowName}
|
||||||
onChange={(e) => setWorkflowName(e.target.value)}
|
onChange={(e) => {
|
||||||
placeholder="Workflow-Name"
|
setWorkflowName(e.target.value)
|
||||||
|
setHasUnsavedChanges(true)
|
||||||
|
}}
|
||||||
|
placeholder="Interner Workflow-Name (Slug-Basis)"
|
||||||
|
title="Technischer Name in der Datenbank. Den sichtbaren Titel für die KI-Analyse setzt du in der Start-Node."
|
||||||
style={{ flex: 1, padding: '8px', borderRadius: '4px', border: '1px solid var(--border)' }}
|
style={{ flex: 1, padding: '8px', borderRadius: '4px', border: '1px solid var(--border)' }}
|
||||||
/>
|
/>
|
||||||
|
|
||||||
@@ -400,8 +569,8 @@ export default function WorkflowEditorPage() {
|
|||||||
nodes={nodes}
|
nodes={nodes}
|
||||||
edges={edges}
|
edges={edges}
|
||||||
nodeTypes={nodeTypes}
|
nodeTypes={nodeTypes}
|
||||||
onNodesChange={onNodesChange}
|
onNodesChange={handleNodesChange}
|
||||||
onEdgesChange={onEdgesChange}
|
onEdgesChange={handleEdgesChange}
|
||||||
onConnect={onConnect}
|
onConnect={onConnect}
|
||||||
onNodeClick={onNodeClick}
|
onNodeClick={onNodeClick}
|
||||||
/>
|
/>
|
||||||
@@ -429,9 +598,115 @@ export default function WorkflowEditorPage() {
|
|||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{/* Inline Validation Display */}
|
||||||
|
{(validationErrors.length > 0 || validationWarnings.length > 0) && (
|
||||||
|
<div style={{
|
||||||
|
background: validationErrors.length > 0 ? '#ffebee' : '#fff3cd',
|
||||||
|
border: `1px solid ${validationErrors.length > 0 ? 'var(--danger)' : '#FFC107'}`,
|
||||||
|
borderRadius: '8px',
|
||||||
|
padding: '12px',
|
||||||
|
marginBottom: '16px'
|
||||||
|
}}>
|
||||||
|
<div style={{ fontWeight: 600, marginBottom: '8px', color: validationErrors.length > 0 ? 'var(--danger)' : '#856404' }}>
|
||||||
|
{validationErrors.length > 0 ? `❌ ${validationErrors.length} Fehler` : `⚠️ ${validationWarnings.length} Warnungen`}
|
||||||
|
</div>
|
||||||
|
{validationErrors.map((err, i) => (
|
||||||
|
<div key={`err-${i}`} style={{ fontSize: '13px', marginBottom: '4px', color: 'var(--danger)', cursor: err.nodeId ? 'pointer' : 'default' }}
|
||||||
|
onClick={() => err.nodeId && setSelectedNodeId(err.nodeId)}>
|
||||||
|
• {err.message}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
{validationWarnings.map((warn, i) => (
|
||||||
|
<div key={`warn-${i}`} style={{ fontSize: '13px', marginBottom: '4px', color: '#856404', cursor: warn.nodeId ? 'pointer' : 'default' }}
|
||||||
|
onClick={() => warn.nodeId && setSelectedNodeId(warn.nodeId)}>
|
||||||
|
• {warn.message}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Start-Node: Metadaten für KI-Analyse (wie Pipeline: display_name, description, category) */}
|
||||||
|
{selectedNode.type === 'start' && (
|
||||||
|
<div className="config-section" style={{ marginBottom: 20 }}>
|
||||||
|
<h3 style={{ margin: '0 0 10px', fontSize: 15 }}>Anzeige in KI-Analyse</h3>
|
||||||
|
<p style={{ fontSize: 12, color: 'var(--text3)', lineHeight: 1.5, marginBottom: 12 }}>
|
||||||
|
Titel, Kurzbeschreibung und Kategorie erscheinen auf der Analyse-Seite, im Verlauf und in der
|
||||||
|
Kategorie-Navigation – analog zu Pipeline-Prompts im Admin.
|
||||||
|
</p>
|
||||||
|
<label style={{ display: 'block', marginBottom: 6, fontWeight: 600, fontSize: 13 }}>
|
||||||
|
Titel (sichtbar für Nutzer)
|
||||||
|
</label>
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
value={selectedNode.data.analysis_title ?? ''}
|
||||||
|
onChange={(e) =>
|
||||||
|
handleNodeUpdate(selectedNode.id, { analysis_title: e.target.value })
|
||||||
|
}
|
||||||
|
placeholder="z. B. Ganzheitliche Auswertung"
|
||||||
|
style={{
|
||||||
|
width: '100%',
|
||||||
|
padding: '8px',
|
||||||
|
borderRadius: '4px',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
background: 'var(--surface)',
|
||||||
|
color: 'var(--text1)',
|
||||||
|
fontSize: '14px',
|
||||||
|
marginBottom: 12,
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<label style={{ display: 'block', marginBottom: 6, fontWeight: 600, fontSize: 13 }}>
|
||||||
|
Kurzbeschreibung (optional)
|
||||||
|
</label>
|
||||||
|
<textarea
|
||||||
|
value={selectedNode.data.analysis_description ?? ''}
|
||||||
|
onChange={(e) =>
|
||||||
|
handleNodeUpdate(selectedNode.id, { analysis_description: e.target.value })
|
||||||
|
}
|
||||||
|
placeholder="Ein Satz, worum es bei dieser Auswertung geht …"
|
||||||
|
rows={4}
|
||||||
|
style={{
|
||||||
|
width: '100%',
|
||||||
|
padding: '8px',
|
||||||
|
borderRadius: '4px',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
background: 'var(--surface)',
|
||||||
|
color: 'var(--text1)',
|
||||||
|
fontSize: '13px',
|
||||||
|
lineHeight: 1.5,
|
||||||
|
marginBottom: 12,
|
||||||
|
resize: 'vertical',
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<label style={{ display: 'block', marginBottom: 6, fontWeight: 600, fontSize: 13 }}>
|
||||||
|
Kategorie
|
||||||
|
</label>
|
||||||
|
<select
|
||||||
|
value={selectedNode.data.analysis_category || 'ganzheitlich'}
|
||||||
|
onChange={(e) =>
|
||||||
|
handleNodeUpdate(selectedNode.id, { analysis_category: e.target.value })
|
||||||
|
}
|
||||||
|
style={{
|
||||||
|
width: '100%',
|
||||||
|
padding: '8px',
|
||||||
|
borderRadius: '4px',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
background: 'var(--surface)',
|
||||||
|
color: 'var(--text1)',
|
||||||
|
fontSize: '14px',
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{ANALYSIS_CATEGORY_ORDER.map((key) => (
|
||||||
|
<option key={key} value={key}>
|
||||||
|
{ANALYSIS_CATEGORY_LABELS[key] || key}
|
||||||
|
</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
{/* Basis-Konfiguration */}
|
{/* Basis-Konfiguration */}
|
||||||
<div className="config-section">
|
<div className="config-section">
|
||||||
<label>Node-Name</label>
|
<label>Node-Name {selectedNode.type === 'start' && '(auf dem Canvas)'}</label>
|
||||||
<input
|
<input
|
||||||
type="text"
|
type="text"
|
||||||
value={selectedNode.data.label || ''}
|
value={selectedNode.data.label || ''}
|
||||||
@@ -453,49 +728,112 @@ export default function WorkflowEditorPage() {
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Type-spezifische Konfiguration */}
|
{/* Type-spezifische Konfiguration */}
|
||||||
{selectedNode.type === 'analysis' && (
|
{selectedNode.type === 'analysis' && (() => {
|
||||||
<>
|
// Helper: Bestimme aktuellen Mode basierend auf node.data
|
||||||
<div className="config-section">
|
const isInlineMode = selectedNode.data.inline_template !== null && selectedNode.data.inline_template !== undefined
|
||||||
<label>KI-Prompt auswählen</label>
|
const isReferenceMode = !isInlineMode
|
||||||
<select
|
|
||||||
value={selectedNode.data.prompt_slug ? String(selectedNode.data.prompt_slug) : ''}
|
return (
|
||||||
onChange={(e) => {
|
<>
|
||||||
const promptSlug = e.target.value
|
{/* Prompt Source Selector */}
|
||||||
console.log('🎯 Prompt selected:', promptSlug, 'Type:', typeof promptSlug)
|
<div className="config-section">
|
||||||
const selectedPrompt = availablePrompts.find(p => p.slug === promptSlug)
|
<label className="form-label">Prompt-Quelle</label>
|
||||||
console.log('📋 Selected prompt object:', selectedPrompt)
|
<div style={{ display: 'flex', flexDirection: 'column', gap: '8px' }}>
|
||||||
handleNodeUpdate(selectedNode.id, {
|
<label style={{ display: 'flex', alignItems: 'center', cursor: 'pointer' }}>
|
||||||
prompt_slug: promptSlug || null,
|
<input
|
||||||
prompt_name: selectedPrompt?.name || null
|
type="radio"
|
||||||
})
|
name={`promptSource-${selectedNode.id}`}
|
||||||
}}
|
checked={isReferenceMode}
|
||||||
style={{
|
onChange={() => {
|
||||||
width: '100%',
|
// Wechsel zu Reference Mode
|
||||||
padding: '8px',
|
handleNodeUpdate(selectedNode.id, {
|
||||||
borderRadius: '4px',
|
inline_template: null, // Inline löschen
|
||||||
border: '1px solid var(--border)',
|
prompt_slug: selectedNode.data.prompt_slug || '' // Behalte existierenden slug
|
||||||
background: 'var(--surface)',
|
})
|
||||||
color: 'var(--text1)'
|
}}
|
||||||
}}
|
style={{ marginRight: '8px' }}
|
||||||
>
|
/>
|
||||||
<option value="">-- Basis-Prompt wählen --</option>
|
<span style={{ fontSize: '14px' }}>📚 Basis-Prompt referenzieren</span>
|
||||||
{availablePrompts.map(prompt => (
|
</label>
|
||||||
<option key={prompt.id} value={prompt.slug}>
|
<label style={{ display: 'flex', alignItems: 'center', cursor: 'pointer' }}>
|
||||||
{prompt.name}
|
<input
|
||||||
</option>
|
type="radio"
|
||||||
))}
|
name={`promptSource-${selectedNode.id}`}
|
||||||
</select>
|
checked={isInlineMode}
|
||||||
{selectedNode.data.prompt_slug && (
|
onChange={() => {
|
||||||
<div style={{ marginTop: 8, fontSize: 12, color: 'var(--text3)' }}>
|
// Wechsel zu Inline Mode
|
||||||
Prompt: {selectedNode.data.prompt_slug} ({selectedNode.data.prompt_name || 'unbekannt'})
|
// WICHTIG: prompt_slug NICHT löschen, damit er beim Zurückwechseln noch da ist
|
||||||
|
handleNodeUpdate(selectedNode.id, {
|
||||||
|
inline_template: selectedNode.data.inline_template || '' // Aktiviere Inline Mode
|
||||||
|
})
|
||||||
|
}}
|
||||||
|
style={{ marginRight: '8px' }}
|
||||||
|
/>
|
||||||
|
<span style={{ fontSize: '14px' }}>✏️ Inline-Template erstellen</span>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* Conditional Rendering: Reference Mode */}
|
||||||
|
{isReferenceMode && (
|
||||||
|
<div className="config-section">
|
||||||
|
<label>Basis-Prompt auswählen</label>
|
||||||
|
<select
|
||||||
|
value={selectedNode.data.prompt_slug || ''}
|
||||||
|
onChange={(e) => {
|
||||||
|
const promptSlug = e.target.value
|
||||||
|
const selectedPrompt = availablePrompts.find(p => p.slug === promptSlug)
|
||||||
|
handleNodeUpdate(selectedNode.id, {
|
||||||
|
prompt_slug: promptSlug,
|
||||||
|
prompt_name: selectedPrompt?.name || null,
|
||||||
|
inline_template: null
|
||||||
|
})
|
||||||
|
}}
|
||||||
|
style={{
|
||||||
|
width: '100%',
|
||||||
|
padding: '8px',
|
||||||
|
borderRadius: '4px',
|
||||||
|
border: '1px solid var(--border)',
|
||||||
|
background: 'var(--surface)',
|
||||||
|
color: 'var(--text1)'
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<option value="">-- Basis-Prompt wählen --</option>
|
||||||
|
{availablePrompts.map(prompt => (
|
||||||
|
<option key={prompt.id} value={prompt.slug}>
|
||||||
|
{prompt.name}
|
||||||
|
</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
{selectedNode.data.prompt_slug && (
|
||||||
|
<div style={{ marginTop: 8, fontSize: 12, color: 'var(--text3)' }}>
|
||||||
|
Prompt: {selectedNode.data.prompt_slug} ({selectedNode.data.prompt_name || 'unbekannt'})
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
</div>
|
|
||||||
|
|
||||||
<QuestionAugmentationPanel node={selectedNode} onChange={handleNodeUpdate} />
|
{/* Conditional Rendering: Inline Mode */}
|
||||||
<FallbackConfig node={selectedNode} edges={edges} nodes={nodes} onChange={handleNodeUpdate} />
|
{isInlineMode && (
|
||||||
</>
|
<InlineTemplateEditor
|
||||||
)}
|
value={selectedNode.data.inline_template || ''}
|
||||||
|
onChange={(template) => handleNodeUpdate(selectedNode.id, {
|
||||||
|
inline_template: template,
|
||||||
|
prompt_slug: null
|
||||||
|
})}
|
||||||
|
onPlaceholderPick={() => {
|
||||||
|
setPlaceholderPickerTarget('inline')
|
||||||
|
setShowPlaceholderPicker(true)
|
||||||
|
}}
|
||||||
|
textareaRef={inlineTemplateTextareaRef}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<QuestionAugmentationPanel node={selectedNode} onChange={handleNodeUpdate} />
|
||||||
|
<FallbackConfig node={selectedNode} edges={edges} nodes={nodes} onChange={handleNodeUpdate} />
|
||||||
|
</>
|
||||||
|
)
|
||||||
|
})()}
|
||||||
|
|
||||||
{selectedNode.type === 'logic' && (
|
{selectedNode.type === 'logic' && (
|
||||||
<>
|
<>
|
||||||
@@ -517,7 +855,10 @@ export default function WorkflowEditorPage() {
|
|||||||
<EndNodeConfig
|
<EndNodeConfig
|
||||||
node={selectedNode}
|
node={selectedNode}
|
||||||
onChange={handleNodeUpdate}
|
onChange={handleNodeUpdate}
|
||||||
onOpenPlaceholderPicker={() => setShowPlaceholderPicker(true)}
|
onOpenPlaceholderPicker={() => {
|
||||||
|
setPlaceholderPickerTarget('end')
|
||||||
|
setShowPlaceholderPicker(true)
|
||||||
|
}}
|
||||||
textareaRef={endNodeTextareaRef}
|
textareaRef={endNodeTextareaRef}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
@@ -525,36 +866,7 @@ export default function WorkflowEditorPage() {
|
|||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Validation Panel */}
|
{/* Validation Panel - REMOVED (moved to config panel header) */}
|
||||||
{(validationErrors.length > 0 || validationWarnings.length > 0) && (
|
|
||||||
<div className="validation-panel">
|
|
||||||
{validationErrors.map((err, i) => (
|
|
||||||
<div key={i} className="validation-error" onClick={() => {
|
|
||||||
if (err.nodeId) {
|
|
||||||
setSelectedNodeId(err.nodeId)
|
|
||||||
}
|
|
||||||
}}>
|
|
||||||
❌ {err.message}
|
|
||||||
</div>
|
|
||||||
))}
|
|
||||||
|
|
||||||
{validationWarnings.map((warn, i) => (
|
|
||||||
<div key={i} className="validation-warning" onClick={() => {
|
|
||||||
if (warn.nodeId) {
|
|
||||||
setSelectedNodeId(warn.nodeId)
|
|
||||||
}
|
|
||||||
}}>
|
|
||||||
⚠️ {warn.message}
|
|
||||||
</div>
|
|
||||||
))}
|
|
||||||
|
|
||||||
{validationErrors.length === 0 && validationWarnings.length > 0 && (
|
|
||||||
<div className="validation-success">
|
|
||||||
✅ Workflow ist valide ({validationWarnings.length} Warnungen)
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* Execution Result Viewer */}
|
{/* Execution Result Viewer */}
|
||||||
{executionResult && (
|
{executionResult && (
|
||||||
@@ -568,10 +880,43 @@ export default function WorkflowEditorPage() {
|
|||||||
{showPlaceholderPicker && (
|
{showPlaceholderPicker && (
|
||||||
<PlaceholderPicker
|
<PlaceholderPicker
|
||||||
nodes={nodes}
|
nodes={nodes}
|
||||||
|
currentNodeId={selectedNode?.id}
|
||||||
onSelect={handlePlaceholderSelect}
|
onSelect={handlePlaceholderSelect}
|
||||||
onClose={() => setShowPlaceholderPicker(false)}
|
onClose={() => setShowPlaceholderPicker(false)}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{/* Validation Panel */}
|
||||||
|
{showValidationPanel && (validationErrors.length > 0 || validationWarnings.length > 0) && (
|
||||||
|
<ValidationPanel
|
||||||
|
errors={validationErrors}
|
||||||
|
warnings={validationWarnings}
|
||||||
|
onNodeClick={handleValidationNodeClick}
|
||||||
|
onClose={() => setShowValidationPanel(false)}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Toast Notification */}
|
||||||
|
{toast && (
|
||||||
|
<Toast
|
||||||
|
message={toast.message}
|
||||||
|
type={toast.type}
|
||||||
|
duration={toast.duration || 3000}
|
||||||
|
onClose={() => setToast(null)}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Confirm Dialog */}
|
||||||
|
{confirmDialog && (
|
||||||
|
<ConfirmDialog
|
||||||
|
message={confirmDialog.message}
|
||||||
|
onConfirm={confirmDialog.onConfirm}
|
||||||
|
onCancel={() => setConfirmDialog(null)}
|
||||||
|
confirmText={confirmDialog.confirmText}
|
||||||
|
cancelText={confirmDialog.cancelText}
|
||||||
|
type={confirmDialog.type}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -27,7 +27,7 @@
|
|||||||
/* ── Sidebar (Node Palette) ─────────────────────────────────────────────── */
|
/* ── Sidebar (Node Palette) ─────────────────────────────────────────────── */
|
||||||
|
|
||||||
.workflow-sidebar {
|
.workflow-sidebar {
|
||||||
width: 250px;
|
width: 220px;
|
||||||
background: var(--surface);
|
background: var(--surface);
|
||||||
border-right: 1px solid var(--border);
|
border-right: 1px solid var(--border);
|
||||||
padding: 16px;
|
padding: 16px;
|
||||||
@@ -268,7 +268,7 @@
|
|||||||
/* ── Config Panel ────────────────────────────────────────────────────────── */
|
/* ── Config Panel ────────────────────────────────────────────────────────── */
|
||||||
|
|
||||||
.workflow-config-panel {
|
.workflow-config-panel {
|
||||||
width: 400px;
|
width: 520px;
|
||||||
background: var(--surface);
|
background: var(--surface);
|
||||||
border-left: 1px solid var(--border);
|
border-left: 1px solid var(--border);
|
||||||
padding: 16px;
|
padding: 16px;
|
||||||
@@ -453,7 +453,7 @@
|
|||||||
}
|
}
|
||||||
|
|
||||||
.workflow-config-panel {
|
.workflow-config-panel {
|
||||||
width: 350px;
|
width: 450px;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -29,6 +29,15 @@ export function formatFastApiDetail(detail, fallback = '') {
|
|||||||
return parts.length ? parts.join(' · ') : fallback || 'Validierungsfehler'
|
return parts.length ? parts.join(' · ') : fallback || 'Validierungsfehler'
|
||||||
}
|
}
|
||||||
if (typeof detail === 'object') {
|
if (typeof detail === 'object') {
|
||||||
|
if (Array.isArray(detail.errors) && detail.errors.length > 0) {
|
||||||
|
const parts = detail.errors
|
||||||
|
.map((e) => {
|
||||||
|
if (e && typeof e === 'object') return e.message || e.msg || ''
|
||||||
|
return String(e)
|
||||||
|
})
|
||||||
|
.filter(Boolean)
|
||||||
|
if (parts.length) return parts.join(' · ')
|
||||||
|
}
|
||||||
if (typeof detail.msg === 'string') return detail.msg
|
if (typeof detail.msg === 'string') return detail.msg
|
||||||
if (typeof detail.message === 'string') return detail.message
|
if (typeof detail.message === 'string') return detail.message
|
||||||
}
|
}
|
||||||
@@ -481,4 +490,84 @@ export const api = {
|
|||||||
|
|
||||||
// Placeholder Metadata Export (v1.0)
|
// Placeholder Metadata Export (v1.0)
|
||||||
exportPlaceholdersExtendedJson: () => req('/prompts/placeholders/export-values-extended'),
|
exportPlaceholdersExtendedJson: () => req('/prompts/placeholders/export-values-extended'),
|
||||||
|
|
||||||
|
// Universal CSV Import (Issue #21)
|
||||||
|
getCsvModules: () => req('/csv/modules'),
|
||||||
|
getCsvLimits: () => req('/csv/limits'),
|
||||||
|
getCsvMappings: (module = null) =>
|
||||||
|
req(module ? `/csv/mappings?module=${encodeURIComponent(module)}` : '/csv/mappings'),
|
||||||
|
copyCsvMapping: (mappingId, body = null) =>
|
||||||
|
req(`/csv/mappings/${mappingId}/copy`, body ? json(body) : { method: 'POST' }),
|
||||||
|
/** Universal-CSV (Issue #21): Zielmodul steckt in der Vorlage; nur file + mapping_id */
|
||||||
|
/** Import-Diagnose: keine Datenbank-Schreibung, erste Zeilen + Mapping-Auflösung */
|
||||||
|
diagnoseUniversalCsv: async (file, mappingId, module = null) => {
|
||||||
|
const fd = new FormData()
|
||||||
|
fd.append('file', file)
|
||||||
|
fd.append('mapping_id', String(mappingId))
|
||||||
|
if (module) fd.append('module', module)
|
||||||
|
const res = await fetch(BASE + '/csv/import-diagnose', { method: 'POST', headers: hdrs(), body: fd })
|
||||||
|
if (!res.ok) {
|
||||||
|
const errText = await res.text()
|
||||||
|
let parsed = null
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(errText)
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
throw new Error(formatFastApiDetail(parsed?.detail, errText.trim() || `HTTP ${res.status}`))
|
||||||
|
}
|
||||||
|
return res.json()
|
||||||
|
},
|
||||||
|
importUniversalCsv: async (file, mappingId) => {
|
||||||
|
const fd = new FormData()
|
||||||
|
fd.append('file', file)
|
||||||
|
fd.append('mapping_id', String(mappingId))
|
||||||
|
const res = await fetch(BASE + '/csv/import', { method: 'POST', headers: hdrs(), body: fd })
|
||||||
|
if (!res.ok) {
|
||||||
|
const errText = await res.text()
|
||||||
|
let parsed = null
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(errText)
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
throw new Error(formatFastApiDetail(parsed?.detail, errText.trim() || `HTTP ${res.status}`))
|
||||||
|
}
|
||||||
|
return res.json()
|
||||||
|
},
|
||||||
|
/** optional module = nur diese Vorlagen-Gruppe (Abwärtskompatibilität); ohne = globale Erkennung */
|
||||||
|
analyzeCsv: async (file, module = null, delimiter = null) => {
|
||||||
|
const fd = new FormData()
|
||||||
|
fd.append('file', file)
|
||||||
|
if (module) fd.append('module', module)
|
||||||
|
if (delimiter) fd.append('delimiter', delimiter)
|
||||||
|
const res = await fetch(BASE + '/csv/analyze', { method: 'POST', headers: hdrs(), body: fd })
|
||||||
|
if (!res.ok) {
|
||||||
|
const errText = await res.text()
|
||||||
|
let parsed = null
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(errText)
|
||||||
|
} catch { /* ignore */ }
|
||||||
|
throw new Error(formatFastApiDetail(parsed?.detail, errText.trim() || `HTTP ${res.status}`))
|
||||||
|
}
|
||||||
|
return res.json()
|
||||||
|
},
|
||||||
|
adminListCsvTemplates: (module = null) =>
|
||||||
|
req(module ? `/admin/csv-templates?module=${encodeURIComponent(module)}` : '/admin/csv-templates'),
|
||||||
|
/** CSV für Admin-Editor analysieren (Vorschlags-Mappings, wie Nutzer-Import) */
|
||||||
|
adminAnalyzeCsvTemplate: async (file, module, delimiter = null, seedTemplateId = null) => {
|
||||||
|
const fd = new FormData()
|
||||||
|
fd.append('file', file)
|
||||||
|
fd.append('module', module)
|
||||||
|
if (delimiter) fd.append('delimiter', delimiter)
|
||||||
|
if (seedTemplateId != null && seedTemplateId !== '') fd.append('seed_template_id', String(seedTemplateId))
|
||||||
|
const res = await fetch(BASE + '/admin/csv-templates/analyze-upload', { method: 'POST', headers: hdrs(), body: fd })
|
||||||
|
const j = await res.json().catch(() => ({}))
|
||||||
|
if (!res.ok) throw new Error(j.detail || res.statusText || 'Analyse fehlgeschlagen')
|
||||||
|
return j
|
||||||
|
},
|
||||||
|
adminGetCsvTemplate: (id) => req(`/admin/csv-templates/${id}`),
|
||||||
|
/** Formatprüfung (ohne Speichern); body wie Create, import_row_processing optional */
|
||||||
|
adminValidateCsvTemplate: (d) => req('/admin/csv-templates/validate', json(d)),
|
||||||
|
adminCreateCsvTemplate: (d) => req('/admin/csv-templates', json(d)),
|
||||||
|
adminUpdateCsvTemplate: (id, d) => req(`/admin/csv-templates/${id}`, jput(d)),
|
||||||
|
adminDeleteCsvTemplate: (id) => req(`/admin/csv-templates/${id}`, { method: 'DELETE' }),
|
||||||
|
adminGetCsvImportLimits: () => req('/admin/csv-templates/import-limits'),
|
||||||
|
adminPutCsvImportLimits: (d) => req('/admin/csv-templates/import-limits', jput(d)),
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
/**
|
||||||
|
* Vorschau-Zellen: Zahlen rechts (tabular-nums), Text links — typisch für CSV-Tabellen.
|
||||||
|
*/
|
||||||
|
|
||||||
|
function stripForNumericTest(s) {
|
||||||
|
return String(s)
|
||||||
|
.trim()
|
||||||
|
.replace(/\u00a0/g, '')
|
||||||
|
.replace(/\s/g, '')
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Einfache Heuristik: reine Zahl / Dezimal (ein Punkt oder ein Komma als Dezimaltrenner).
|
||||||
|
* Keine Datums-/Zeitstrings (mehrere Punkte ohne klares Muster werden nicht als Zahl gewertet).
|
||||||
|
*/
|
||||||
|
export function isCsvPreviewNumericCell(val) {
|
||||||
|
if (val == null || val === '—') return false
|
||||||
|
let s = stripForNumericTest(val)
|
||||||
|
if (!s) return false
|
||||||
|
if (/^\d{1,2}[./]\d{1,2}[./]\d{2,4}/.test(s)) return false
|
||||||
|
if (/^\d{4}-\d{2}-\d{2}/.test(s)) return false
|
||||||
|
if (/:/.test(s) && /\d/.test(s)) return false
|
||||||
|
|
||||||
|
s = s.replace(/^[-+]/, '')
|
||||||
|
const comma = (s.match(/,/g) || []).length
|
||||||
|
const dot = (s.match(/\./g) || []).length
|
||||||
|
if (comma > 1 || dot > 1 || (comma >= 1 && dot >= 1)) return false
|
||||||
|
|
||||||
|
s = s.replace(',', '.')
|
||||||
|
if (!/^\d*\.?\d+$/.test(s) && !/^\d+\.\d+$/.test(s)) return false
|
||||||
|
return Number.isFinite(Number(s))
|
||||||
|
}
|
||||||
|
|
||||||
|
/** td-Style für CSV-Vorschau (Import + Admin-Editor) */
|
||||||
|
export function csvPreviewTdStyle(val, extra = {}) {
|
||||||
|
const numeric = isCsvPreviewNumericCell(val)
|
||||||
|
return {
|
||||||
|
padding: '6px 8px',
|
||||||
|
borderBottom: '1px solid var(--border)',
|
||||||
|
maxWidth: 220,
|
||||||
|
overflow: 'hidden',
|
||||||
|
textOverflow: 'ellipsis',
|
||||||
|
textAlign: numeric ? 'right' : 'left',
|
||||||
|
fontVariantNumeric: numeric ? 'tabular-nums' : undefined,
|
||||||
|
...extra,
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
/**
|
||||||
|
* Nutzer-sichtbare Textausgabe eines Workflow-Laufs – gleiche Logik wie
|
||||||
|
* WorkflowResultViewer („Final Output“): primär aggregated_result.analysis_core.
|
||||||
|
*/
|
||||||
|
const AGG_META_KEYS = new Set([
|
||||||
|
'combined_analysis',
|
||||||
|
'all_signals',
|
||||||
|
'total_nodes',
|
||||||
|
'executed_nodes',
|
||||||
|
'failed_nodes',
|
||||||
|
'skipped_nodes',
|
||||||
|
])
|
||||||
|
|
||||||
|
export function getWorkflowDisplayContent(aggregatedResult) {
|
||||||
|
if (aggregatedResult == null) return ''
|
||||||
|
if (typeof aggregatedResult !== 'object' || Array.isArray(aggregatedResult)) {
|
||||||
|
return typeof aggregatedResult === 'string'
|
||||||
|
? aggregatedResult
|
||||||
|
: JSON.stringify(aggregatedResult, null, 2)
|
||||||
|
}
|
||||||
|
|
||||||
|
const core = aggregatedResult.analysis_core
|
||||||
|
if (typeof core === 'string' && core.trim() !== '') return core
|
||||||
|
|
||||||
|
const combined = aggregatedResult.combined_analysis
|
||||||
|
if (typeof combined === 'string' && combined.trim() !== '') return combined
|
||||||
|
|
||||||
|
const keys = Object.keys(aggregatedResult).filter((k) => !AGG_META_KEYS.has(k))
|
||||||
|
if (keys.length === 1) {
|
||||||
|
const v = aggregatedResult[keys[0]]
|
||||||
|
if (typeof v === 'string') return v
|
||||||
|
if (v != null && typeof v === 'object') return JSON.stringify(v, null, 2)
|
||||||
|
return String(v)
|
||||||
|
}
|
||||||
|
|
||||||
|
return JSON.stringify(aggregatedResult, null, 2)
|
||||||
|
}
|
||||||
@@ -22,6 +22,7 @@ export function serializeToWorkflowGraph(nodes, edges, metadata = {}) {
|
|||||||
// Type-spezifische Felder
|
// Type-spezifische Felder
|
||||||
...(node.type === 'analysis' && {
|
...(node.type === 'analysis' && {
|
||||||
prompt_slug: node.data.prompt_slug || null,
|
prompt_slug: node.data.prompt_slug || null,
|
||||||
|
inline_template: node.data.inline_template || null, // Part 3: Inline Prompts
|
||||||
prompt_name: node.data.prompt_name || null,
|
prompt_name: node.data.prompt_name || null,
|
||||||
question_augmentations: node.data.questions || [], // Backend erwartet question_augmentations
|
question_augmentations: node.data.questions || [], // Backend erwartet question_augmentations
|
||||||
fallback_strategy: node.data.fallback_strategy || 'conservative_skip'
|
fallback_strategy: node.data.fallback_strategy || 'conservative_skip'
|
||||||
@@ -41,6 +42,12 @@ export function serializeToWorkflowGraph(nodes, edges, metadata = {}) {
|
|||||||
...(node.type === 'end' && {
|
...(node.type === 'end' && {
|
||||||
output_mode: node.data.output_mode || 'auto',
|
output_mode: node.data.output_mode || 'auto',
|
||||||
template: node.data.template || null
|
template: node.data.template || null
|
||||||
|
}),
|
||||||
|
|
||||||
|
...(node.type === 'start' && {
|
||||||
|
analysis_title: node.data.analysis_title || '',
|
||||||
|
analysis_description: node.data.analysis_description || '',
|
||||||
|
analysis_category: node.data.analysis_category || 'ganzheitlich',
|
||||||
})
|
})
|
||||||
}))
|
}))
|
||||||
|
|
||||||
@@ -84,6 +91,7 @@ export function deserializeFromWorkflowGraph(jsonbData) {
|
|||||||
|
|
||||||
...(node.type === 'analysis' && {
|
...(node.type === 'analysis' && {
|
||||||
prompt_slug: node.prompt_slug || node.prompt_id || null, // Fallback für alte Workflows mit prompt_id
|
prompt_slug: node.prompt_slug || node.prompt_id || null, // Fallback für alte Workflows mit prompt_id
|
||||||
|
inline_template: node.inline_template || null, // Part 3: Inline Prompts
|
||||||
prompt_name: node.prompt_name || null, // Falls vom Backend mitgeliefert
|
prompt_name: node.prompt_name || null, // Falls vom Backend mitgeliefert
|
||||||
questions: node.question_augmentations || node.questions || [], // Backend sendet question_augmentations
|
questions: node.question_augmentations || node.questions || [], // Backend sendet question_augmentations
|
||||||
fallback_strategy: node.fallback_strategy || 'conservative_skip'
|
fallback_strategy: node.fallback_strategy || 'conservative_skip'
|
||||||
@@ -103,6 +111,12 @@ export function deserializeFromWorkflowGraph(jsonbData) {
|
|||||||
...(node.type === 'end' && {
|
...(node.type === 'end' && {
|
||||||
output_mode: node.output_mode || 'auto',
|
output_mode: node.output_mode || 'auto',
|
||||||
template: node.template || null
|
template: node.template || null
|
||||||
|
}),
|
||||||
|
|
||||||
|
...(node.type === 'start' && {
|
||||||
|
analysis_title: node.analysis_title || '',
|
||||||
|
analysis_description: node.analysis_description || '',
|
||||||
|
analysis_category: node.analysis_category || 'ganzheitlich',
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}))
|
}))
|
||||||
|
|||||||
@@ -49,6 +49,14 @@ function validateStructure(nodes, edges, errors, warnings) {
|
|||||||
message: 'Kein END-Node vorhanden',
|
message: 'Kein END-Node vorhanden',
|
||||||
severity: 'error'
|
severity: 'error'
|
||||||
})
|
})
|
||||||
|
} else if (endNodes.length > 1) {
|
||||||
|
// Backend unterstützt aktuell nur 1 End-Node (aggregate_results nimmt letzten)
|
||||||
|
// Future: Multi-Exit Support würde Backend-Anpassung erfordern
|
||||||
|
errors.push({
|
||||||
|
type: 'structure',
|
||||||
|
message: `${endNodes.length} END-Nodes gefunden (max. 1 erlaubt)`,
|
||||||
|
severity: 'error'
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// Zyklen-Erkennung
|
// Zyklen-Erkennung
|
||||||
@@ -86,17 +94,29 @@ function validateLogic(nodes, edges, errors, warnings) {
|
|||||||
// Analysis Nodes
|
// Analysis Nodes
|
||||||
if (node.type === 'analysis') {
|
if (node.type === 'analysis') {
|
||||||
const questions = node.data.questions || []
|
const questions = node.data.questions || []
|
||||||
|
const hasPromptSlug = node.data.prompt_slug != null && node.data.prompt_slug !== ''
|
||||||
|
const hasInlineTemplate = node.data.inline_template != null && node.data.inline_template.trim() !== ''
|
||||||
|
|
||||||
// Prompt ausgewählt?
|
// Part 3: Validation - Entweder prompt_slug ODER inline_template
|
||||||
if (!node.data.prompt_slug) {
|
if (!hasPromptSlug && !hasInlineTemplate) {
|
||||||
errors.push({
|
errors.push({
|
||||||
type: 'config',
|
type: 'config',
|
||||||
message: `Analysis-Node "${node.data.label}" hat keinen Prompt`,
|
message: `Analysis-Node "${node.data.label}" benötigt entweder Basis-Prompt oder Inline-Template`,
|
||||||
nodeId: node.id,
|
nodeId: node.id,
|
||||||
severity: 'error'
|
severity: 'error'
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Warning wenn beide gesetzt (sollte nicht passieren, aber zur Sicherheit)
|
||||||
|
if (hasPromptSlug && hasInlineTemplate) {
|
||||||
|
warnings.push({
|
||||||
|
type: 'config',
|
||||||
|
message: `Analysis-Node "${node.data.label}" hat sowohl Basis-Prompt als auch Inline-Template - Inline hat Vorrang`,
|
||||||
|
nodeId: node.id,
|
||||||
|
severity: 'warning'
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// Fragen validieren
|
// Fragen validieren
|
||||||
questions.forEach((q, idx) => {
|
questions.forEach((q, idx) => {
|
||||||
if (!q.question?.trim()) {
|
if (!q.question?.trim()) {
|
||||||
|
|||||||
@@ -0,0 +1,85 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Test placeholder resolution in inline templates
|
||||||
|
|
||||||
|
This script simulates what happens in workflow_executor.load_prompt_template()
|
||||||
|
"""
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, 'backend')
|
||||||
|
|
||||||
|
from placeholder_resolver import get_placeholder_example_values, get_placeholder_catalog
|
||||||
|
from prompt_executor import resolve_placeholders
|
||||||
|
|
||||||
|
# Test profile_id (use first profile in dev DB)
|
||||||
|
PROFILE_ID = "019601b5-d65a-738f-a1e7-b3f69bb97f69" # Lars profile from dev
|
||||||
|
|
||||||
|
def test_placeholder_resolution():
|
||||||
|
"""Test the exact same logic as in workflow_executor.load_prompt_template()"""
|
||||||
|
|
||||||
|
# Test template with spaces in placeholders (as user showed)
|
||||||
|
template = "Hallo {{ name }}, du bist {{ age }} Jahre alt und {{ geschlecht }}."
|
||||||
|
|
||||||
|
print("=" * 80)
|
||||||
|
print("PLACEHOLDER RESOLUTION TEST")
|
||||||
|
print("=" * 80)
|
||||||
|
print(f"\nTemplate:\n{template}\n")
|
||||||
|
|
||||||
|
# Step 1: Load placeholders (same as workflow_executor)
|
||||||
|
print("Step 1: Loading placeholders...")
|
||||||
|
processed_placeholders = get_placeholder_example_values(PROFILE_ID)
|
||||||
|
print(f" Loaded {len(processed_placeholders)} placeholders")
|
||||||
|
print(f" Sample keys (first 5): {list(processed_placeholders.keys())[:5]}")
|
||||||
|
|
||||||
|
# Step 2: Clean keys (same as workflow_executor)
|
||||||
|
print("\nStep 2: Cleaning keys...")
|
||||||
|
cleaned_placeholders = {
|
||||||
|
key.replace('{{', '').replace('}}', '').strip(): value
|
||||||
|
for key, value in processed_placeholders.items()
|
||||||
|
}
|
||||||
|
print(f" Cleaned keys (first 5): {list(cleaned_placeholders.keys())[:5]}")
|
||||||
|
print(f" Sample values:")
|
||||||
|
print(f" name = {cleaned_placeholders.get('name')}")
|
||||||
|
print(f" age = {cleaned_placeholders.get('age')}")
|
||||||
|
print(f" geschlecht = {cleaned_placeholders.get('geschlecht')}")
|
||||||
|
|
||||||
|
variables = cleaned_placeholders
|
||||||
|
|
||||||
|
# Step 3: Load catalog
|
||||||
|
print("\nStep 3: Loading catalog...")
|
||||||
|
try:
|
||||||
|
catalog = get_placeholder_catalog(PROFILE_ID)
|
||||||
|
print(f" Catalog loaded with {len(catalog)} categories")
|
||||||
|
except Exception as e:
|
||||||
|
catalog = None
|
||||||
|
print(f" Catalog failed: {e}")
|
||||||
|
|
||||||
|
# Step 4: Resolve placeholders
|
||||||
|
print("\nStep 4: Resolving placeholders...")
|
||||||
|
debug_info = {}
|
||||||
|
resolved = resolve_placeholders(
|
||||||
|
template=template,
|
||||||
|
variables=variables,
|
||||||
|
debug_info=debug_info,
|
||||||
|
catalog=catalog
|
||||||
|
)
|
||||||
|
|
||||||
|
print(f" Resolved placeholders: {debug_info.get('resolved_placeholders', {})}")
|
||||||
|
print(f" Unresolved placeholders: {debug_info.get('unresolved_placeholders', [])}")
|
||||||
|
|
||||||
|
# Result
|
||||||
|
print("\n" + "=" * 80)
|
||||||
|
print("RESULT")
|
||||||
|
print("=" * 80)
|
||||||
|
print(f"\nResolved template:\n{resolved}\n")
|
||||||
|
|
||||||
|
# Check if placeholders were resolved
|
||||||
|
if '{{' in resolved:
|
||||||
|
print("❌ FAILED: Some placeholders were not resolved!")
|
||||||
|
return False
|
||||||
|
else:
|
||||||
|
print("✅ SUCCESS: All placeholders resolved!")
|
||||||
|
return True
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
success = test_placeholder_resolution()
|
||||||
|
sys.exit(0 if success else 1)
|
||||||
Reference in new issue
Block a user