feat(csv-import): Refactor CSV import logic and enhance data handling
- Updated the CSV import architecture to clarify the distinction between import and data layer responsibilities, as outlined in the new section of ARCHITECTURE.md. - Enhanced the build_row_after_mapping function to include module-specific context for improved data processing. - Introduced source unit options in the admin CSV template editor to facilitate user-defined conversions, improving flexibility in handling various data formats. - Added new tests to validate the handling of source units and ensure accurate conversions during CSV imports. - Updated module definitions to include unit specifications for nutritional and activity data fields, enhancing data integrity.
This commit is contained in:
parent
41cc0ed2a8
commit
d6d7e738a5
|
|
@ -384,26 +384,55 @@ Dev-URL: dev.mitai.jinkendo.de
|
|||
|
||||
---
|
||||
|
||||
## 8. Test-Regeln
|
||||
## 8. CSV-Import vs. Data Layer (Issue #53)
|
||||
|
||||
### 8.1 Tests schreiben ist Pflicht
|
||||
### 8.1 Leitlinie: Wo Interpretation stattfindet
|
||||
|
||||
| Schicht | Erlaubt | Nicht Sinn der Schicht |
|
||||
|--------|---------|-------------------------|
|
||||
| **Import (Ingest)** | Zuordnung CSV→Speicherfeld, **Typ-/Einheits-Konvertierung** (`type_conversions`), Duplikat-/Constraint-Logik | Fachliche **Interpretation**, Aggregation von „Bedeutung“, Metriken für Auswertung |
|
||||
| **Data Layer (Issue #53, Layer 1+)** | Daten lesen, aufbereiten, ableiten, für Charts/KI/Prompts bereitstellen | — |
|
||||
|
||||
Verbindlich: **Semantik und Auswertung** nicht dauerhaft im Import verstecken; neue Features werden an dieser Grenze geprüft.
|
||||
|
||||
**Detail & Zielbild (Multi-Layer, Single Source of Truth):** `docs/issues/issue-53-phase-0c-multi-layer-architecture.md`
|
||||
|
||||
**Umsetzung Schlaf-Import (Refactoring, Offen):** Gitea http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/69
|
||||
|
||||
### 8.2 Ist-Einordnung Import-Pfade (Übergang)
|
||||
|
||||
Bis sukzessive auf das Zielbild umgestellt ist, gilt:
|
||||
|
||||
| Pfad | Einordnung |
|
||||
|------|----------------|
|
||||
| Universal-CSV (`csv_parser`, `routers/csv_import.py`, Executor für u. a. Gewicht/Ernährung/Blutdruck/Aktivität/Vitals) | **Zielrichtung:** Mapping + Typkonvertierung |
|
||||
| Apple-Schlaf-Aggregat (`csv_parser/sleep_apple_import.py`, `import_mode: apple_sleep_aggregate`) | **Legacy-Adapter** (quellenspezifische Aufbereitung) – Austausch gegen mapping-nah + Layer 1 geplant |
|
||||
| Dedizierte Import-Endpoints (z. B. `/api/activity/import-csv`, Vitals Apple) | **Legacy/Parallel** – neue Quellen bevorzugt über Universal-Pfad + Vorlagen |
|
||||
|
||||
Änderungen an Import-Pfaden: Legacy nur erweitern mit **expliziter** Issue-/Review-Begründung; kein neues „wir rechnen Auswertung beim Insert“ ohne Data-Layer-Bezug.
|
||||
|
||||
---
|
||||
|
||||
## 9. Test-Regeln
|
||||
|
||||
### 9.1 Tests schreiben ist Pflicht
|
||||
Jedes neue Feature bekommt mindestens einen Playwright-Test in
|
||||
`tests/dev-smoke-test.spec.js`.
|
||||
|
||||
### 8.2 Reihenfolge: Test vor Commit
|
||||
### 9.2 Reihenfolge: Test vor Commit
|
||||
```
|
||||
Implementieren → Tests schreiben → Tests grün → Committen
|
||||
NIEMALS: Implementieren → Committen → Tests später
|
||||
```
|
||||
|
||||
### 8.3 Claude Code schreibt Tests selbst
|
||||
### 9.3 Claude Code schreibt Tests selbst
|
||||
Nach jeder Implementierung:
|
||||
1. Passende Tests in dev-smoke-test.spec.js ergänzen
|
||||
2. `npx playwright test` ausführen
|
||||
3. Fehler korrigieren bis alle Tests grün
|
||||
4. Erst dann committen
|
||||
|
||||
### 8.4 Test-Kategorien
|
||||
### 9.4 Test-Kategorien
|
||||
```javascript
|
||||
// UI-Test (Playwright)
|
||||
test('FEATURE: Beschreibung', async ({ page }) => { ... })
|
||||
|
|
@ -412,18 +441,18 @@ test('FEATURE: Beschreibung', async ({ page }) => { ... })
|
|||
test('API: Endpoint', async ({ request }) => { ... })
|
||||
```
|
||||
|
||||
### 8.5 Screenshots bei Fehlern
|
||||
### 9.5 Screenshots bei Fehlern
|
||||
Fehlgeschlagene Tests erzeugen automatisch Screenshots in:
|
||||
`test-results/TESTNAME/test-failed-1.png`
|
||||
→ Immer ansehen bevor Code geändert wird
|
||||
|
||||
### 8.6 Prod nie testen
|
||||
### 9.6 Prod nie testen
|
||||
Tests laufen IMMER gegen dev.mitai.jinkendo.de
|
||||
NIEMALS gegen mitai.jinkendo.de
|
||||
|
||||
---
|
||||
|
||||
## 9. Dashboard-Lab-Widgets und Feature-System
|
||||
## 10. Dashboard-Lab-Widgets und Feature-System
|
||||
|
||||
**Kontext:** Dashboard-Widgets (`backend/widget_catalog.py`, Lab unter `/api/app/...`) und das **Subscription-/Feature-Modell** (`features`, `tier_limits`, `check_feature_access` in `backend/auth.py`) sind **getrennte Schichten**, müssen aber bei tariffrelevanten Widgets **verknüpft** werden.
|
||||
|
||||
|
|
|
|||
|
|
@ -200,7 +200,7 @@ def _import_nutrition(
|
|||
rows_total = 0
|
||||
for csv_row in iter_csv_dict_rows(text, delim, has_header=has_header):
|
||||
rows_total += 1
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc)
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
|
||||
d = coerce_date(mapped.get("date"))
|
||||
if d is None:
|
||||
error_details.append({"row": rows_total, "error": "Datum fehlt oder ungültig"})
|
||||
|
|
@ -283,7 +283,7 @@ def _import_weight(
|
|||
new_entries = 0
|
||||
for csv_row in iter_csv_dict_rows(text, delim, has_header=has_header):
|
||||
rows_total += 1
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc)
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc, module="weight")
|
||||
d = coerce_date(mapped.get("date"))
|
||||
w = mapped.get("weight")
|
||||
note = mapped.get("note")
|
||||
|
|
@ -356,7 +356,7 @@ def _import_blood_pressure(
|
|||
skipped = 0
|
||||
for csv_row in iter_csv_dict_rows(text, delim, has_header=has_header):
|
||||
rows_total += 1
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc)
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc, module="blood_pressure")
|
||||
md = coerce_date(mapped.get("measured_date"))
|
||||
mt = mapped.get("measured_time")
|
||||
if md is None:
|
||||
|
|
@ -483,7 +483,7 @@ def _import_vitals_baseline(
|
|||
skipped = 0
|
||||
for csv_row in iter_csv_dict_rows(text, delim, has_header=has_header):
|
||||
rows_total += 1
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc)
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc, module="vitals_baseline")
|
||||
d = coerce_date(mapped.get("date"))
|
||||
if d is None:
|
||||
error_details.append({"row": rows_total, "error": "Datum fehlt"})
|
||||
|
|
@ -581,7 +581,7 @@ def _import_activity(
|
|||
|
||||
for csv_row in iter_csv_dict_rows(text, delim, has_header=has_header):
|
||||
rows_total += 1
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc)
|
||||
mapped = build_row_after_mapping(csv_row, fm, tc, module="activity")
|
||||
activity_type = mapped.get("activity_type")
|
||||
if not activity_type or not str(activity_type).strip():
|
||||
error_details.append({"row": rows_total, "error": "Trainingsart fehlt"})
|
||||
|
|
|
|||
101
backend/csv_parser/field_units.py
Normal file
101
backend/csv_parser/field_units.py
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
"""
|
||||
Kanonische Speichereinheiten pro CSV-Zielfeld (module_registry: field.unit) und
|
||||
abwählbare Quelleinheiten → Faktor für type_conversions.source_unit.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from csv_parser.module_registry import get_module_definition
|
||||
|
||||
# 1 kcal = 4.184 kJ (IEC/ISO 80000)
|
||||
_KJ_TO_KCAL = 1.0 / 4.184
|
||||
|
||||
# — Energie (Ziel kcal) —
|
||||
_ENERGY: list[dict[str, Any]] = [
|
||||
{"id": "kcal", "label": "Kilokalorien (kcal), wie in DB", "factor": 1.0},
|
||||
{"id": "kj", "label": "Kilojoule (kJ) → kcal", "factor": _KJ_TO_KCAL},
|
||||
{"id": "j", "label": "Joule (J) → kcal", "factor": _KJ_TO_KCAL / 1000.0},
|
||||
]
|
||||
|
||||
# — Masse klein (Ziel g): Makronährstoffe —
|
||||
_GRAM: list[dict[str, Any]] = [
|
||||
{"id": "g", "label": "Gramm (g), wie in DB", "factor": 1.0},
|
||||
{"id": "kg", "label": "Kilogramm (kg) → g", "factor": 1000.0},
|
||||
{"id": "mg", "label": "Milligramm (mg) → g", "factor": 0.001},
|
||||
]
|
||||
|
||||
# — Körpergewicht (Ziel kg) —
|
||||
_KG: list[dict[str, Any]] = [
|
||||
{"id": "kg", "label": "Kilogramm (kg), wie in DB", "factor": 1.0},
|
||||
{"id": "g", "label": "Gramm (g) → kg", "factor": 0.001},
|
||||
{"id": "lb", "label": "Pfund / lb → kg", "factor": 0.45359237},
|
||||
{"id": "oz", "label": "Unze / oz → kg", "factor": 0.028349523125},
|
||||
]
|
||||
|
||||
# — Strecke (Ziel km) —
|
||||
_KM: list[dict[str, Any]] = [
|
||||
{"id": "km", "label": "Kilometer (km), wie in DB", "factor": 1.0},
|
||||
{"id": "m", "label": "Meter (m) → km", "factor": 0.001},
|
||||
{"id": "mi", "label": "Meilen (mi) → km", "factor": 1.609344},
|
||||
]
|
||||
|
||||
_UNIT_FAMILY: dict[str, list[dict[str, Any]]] = {
|
||||
"kcal": _ENERGY,
|
||||
"g": _GRAM,
|
||||
"kg": _KG,
|
||||
"km": _KM,
|
||||
}
|
||||
|
||||
|
||||
def get_canonical_unit(module: str, db_field: str) -> str | None:
|
||||
mod = get_module_definition(module)
|
||||
if not mod:
|
||||
return None
|
||||
finfo: dict[str, Any] | None = mod.get("fields", {}).get(db_field)
|
||||
if not finfo:
|
||||
return None
|
||||
u = finfo.get("unit")
|
||||
return str(u) if u else None
|
||||
|
||||
|
||||
def source_unit_choices_for_field(module: str, db_field: str) -> list[dict[str, Any]]:
|
||||
"""Optionen für GUI: id, label, canonical_unit, is_canonical (Umrechnung serverseitig)."""
|
||||
cu = get_canonical_unit(module, db_field)
|
||||
if not cu:
|
||||
return []
|
||||
choices = _UNIT_FAMILY.get(cu)
|
||||
if not choices:
|
||||
return []
|
||||
return [
|
||||
{
|
||||
"id": c["id"],
|
||||
"label": c["label"],
|
||||
"canonical_unit": cu,
|
||||
"is_canonical": c["id"] == cu,
|
||||
}
|
||||
for c in choices
|
||||
]
|
||||
|
||||
|
||||
def factor_source_to_canonical(module: str, db_field: str, source_unit: str | None) -> float:
|
||||
"""
|
||||
Multiplikator: CSV-Zahl * Faktor → Wert in kanonischer DB-Einheit.
|
||||
Unbekannte/None/leer/Passthrough → 1.0
|
||||
"""
|
||||
if source_unit is None:
|
||||
return 1.0
|
||||
su = str(source_unit).strip().lower()
|
||||
if not su:
|
||||
return 1.0
|
||||
cu = get_canonical_unit(module, db_field)
|
||||
if not cu:
|
||||
return 1.0
|
||||
choices = _UNIT_FAMILY.get(cu)
|
||||
if not choices:
|
||||
return 1.0
|
||||
for c in choices:
|
||||
if str(c["id"]).lower() == su:
|
||||
return float(c["factor"])
|
||||
return 1.0
|
||||
|
|
@ -207,4 +207,33 @@ def build_type_conversions_for_mapping(
|
|||
if t not in out and t in defaults:
|
||||
out[t] = deepcopy(defaults[t])
|
||||
|
||||
_apply_energy_kj_hint_from_headers(module, field_mappings, out)
|
||||
return out
|
||||
|
||||
|
||||
_ENERGY_FIELDS = frozenset({"kcal", "kcal_active", "kcal_resting"})
|
||||
|
||||
|
||||
def _apply_energy_kj_hint_from_headers(
|
||||
module: str,
|
||||
field_mappings: Mapping[str, str],
|
||||
out: dict[str, Any],
|
||||
) -> None:
|
||||
"""Wenn Überschrift kJ/Kilojoule nahelegt (nicht kcal), source_unit kj setzen (FDDB & Co.)."""
|
||||
if module not in ("nutrition", "activity"):
|
||||
return
|
||||
for csv_col, db_field in field_mappings.items():
|
||||
if db_field not in _ENERGY_FIELDS:
|
||||
continue
|
||||
spec = out.get(db_field)
|
||||
if not isinstance(spec, dict):
|
||||
continue
|
||||
if spec.get("source_unit"):
|
||||
continue
|
||||
norm = normalize_header_for_signature(str(csv_col)).lower()
|
||||
if "kcal" in norm:
|
||||
continue
|
||||
if "kj" in norm or "kilojoule" in norm:
|
||||
spec2 = deepcopy(spec)
|
||||
spec2["source_unit"] = "kj"
|
||||
out[db_field] = spec2
|
||||
|
|
|
|||
|
|
@ -16,10 +16,10 @@ MODULE_DEFINITIONS: Dict[str, Dict[str, Any]] = {
|
|||
"table": "nutrition_log",
|
||||
"fields": {
|
||||
"date": {"type": "date", "required": True},
|
||||
"kcal": {"type": "float", "required": False},
|
||||
"protein_g": {"type": "float", "required": False, "min": 0},
|
||||
"fat_g": {"type": "float", "required": False, "min": 0},
|
||||
"carbs_g": {"type": "float", "required": False, "min": 0},
|
||||
"kcal": {"type": "float", "required": False, "unit": "kcal"},
|
||||
"protein_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
|
||||
"fat_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
|
||||
"carbs_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
|
||||
},
|
||||
"duplicate_key": ["profile_id", "date"],
|
||||
"duplicate_strategy": "update",
|
||||
|
|
@ -32,9 +32,9 @@ MODULE_DEFINITIONS: Dict[str, Dict[str, Any]] = {
|
|||
"end_time": {"type": "datetime", "required": False},
|
||||
"activity_type": {"type": "string", "required": True},
|
||||
"duration_min": {"type": "float", "required": False, "min": 0},
|
||||
"kcal_active": {"type": "float", "required": False},
|
||||
"kcal_resting": {"type": "float", "required": False},
|
||||
"distance_km": {"type": "float", "required": False},
|
||||
"kcal_active": {"type": "float", "required": False, "unit": "kcal"},
|
||||
"kcal_resting": {"type": "float", "required": False, "unit": "kcal"},
|
||||
"distance_km": {"type": "float", "required": False, "unit": "km"},
|
||||
"hr_avg": {"type": "float", "required": False, "min": 30, "max": 220},
|
||||
"hr_max": {"type": "float", "required": False, "min": 30, "max": 220},
|
||||
},
|
||||
|
|
@ -77,7 +77,7 @@ MODULE_DEFINITIONS: Dict[str, Dict[str, Any]] = {
|
|||
"table": "weight_log",
|
||||
"fields": {
|
||||
"date": {"type": "date", "required": True},
|
||||
"weight": {"type": "float", "required": True, "min": 20, "max": 400},
|
||||
"weight": {"type": "float", "required": True, "min": 20, "max": 400, "unit": "kg"},
|
||||
"note": {"type": "string", "required": False, "max_length": 2000},
|
||||
},
|
||||
"duplicate_key": ["profile_id", "date"],
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ from typing import Any, Mapping, Sequence
|
|||
from dateutil import parser as dateutil_parser
|
||||
|
||||
from csv_parser.core import normalize_header_for_signature
|
||||
from csv_parser.field_units import factor_source_to_canonical
|
||||
|
||||
# Alias → strptime (JSON in Kleinbuchstaben)
|
||||
DATE_FORMAT_STRPTIME: dict[str, str] = {
|
||||
|
|
@ -300,6 +301,7 @@ def convert_value(
|
|||
raw: str,
|
||||
db_field: str,
|
||||
spec: Mapping[str, Any] | None,
|
||||
module: str | None = None,
|
||||
) -> Any:
|
||||
"""
|
||||
Konvertiert eine Roh-Zelle in einen Python-Wert.
|
||||
|
|
@ -310,6 +312,8 @@ def convert_value(
|
|||
- decimal_separator: ".", ",", "auto" — bei auto Heuristik EU/US-Mischformen.
|
||||
- formats: [ "yyyy-mm-dd", "%d.%m.%y", ... ] — weitere strptime-/Alias-Ketten.
|
||||
- dayfirst: true|false — nur für dateutil-Fallback; Standard: true dann false.
|
||||
- source_unit: z. B. "kj", "kg" — Quelleinheit; Ziel aus Modul-Feld (unit in module_registry);
|
||||
Faktor wird serverseitig ermittelt. Optional zusätzlich conversion_factor (legacy/zusätzlich).
|
||||
"""
|
||||
if spec is None:
|
||||
return raw.strip() if raw else None
|
||||
|
|
@ -325,6 +329,10 @@ def convert_value(
|
|||
|
||||
if t in ("float", "number"):
|
||||
v = _float_from_spec(raw, spec)
|
||||
if module:
|
||||
su = spec.get("source_unit")
|
||||
if su is not None:
|
||||
v = float(v) * factor_source_to_canonical(module, db_field, str(su))
|
||||
factor = spec.get("conversion_factor")
|
||||
if factor is not None:
|
||||
v = float(v) * float(factor)
|
||||
|
|
@ -395,6 +403,7 @@ def build_row_after_mapping(
|
|||
csv_row: Mapping[str, str],
|
||||
field_mappings: Mapping[str, str],
|
||||
type_conversions: Mapping[str, Any] | None,
|
||||
module: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Wendet Zuordnung csv_spalte → db_feld und Typkonvertierung an.
|
||||
|
|
@ -408,7 +417,9 @@ def build_row_after_mapping(
|
|||
continue
|
||||
spec = tc.get(db_field)
|
||||
try:
|
||||
out[db_field] = convert_value(raw, db_field, spec if isinstance(spec, dict) else None)
|
||||
out[db_field] = convert_value(
|
||||
raw, db_field, spec if isinstance(spec, dict) else None, module=module
|
||||
)
|
||||
except Exception:
|
||||
out[db_field] = None
|
||||
return out
|
||||
|
|
|
|||
|
|
@ -24,6 +24,7 @@ from csv_parser.core import (
|
|||
normalize_header_for_signature,
|
||||
parse_csv_sample,
|
||||
)
|
||||
from csv_parser.field_units import source_unit_choices_for_field
|
||||
from csv_parser.module_registry import get_module_definition, list_modules, validate_field_mappings
|
||||
from csv_parser.sleep_apple_import import detect_apple_sleep_csv_format
|
||||
|
||||
|
|
@ -60,11 +61,18 @@ def csv_modules(session: dict = Depends(require_auth)):
|
|||
for mid in list_modules():
|
||||
d = get_module_definition(mid)
|
||||
if d:
|
||||
fields_out = {}
|
||||
for fname, finfo in (d.get("fields") or {}).items():
|
||||
fd = dict(finfo)
|
||||
opts = source_unit_choices_for_field(mid, fname)
|
||||
if opts:
|
||||
fd["source_unit_options"] = opts
|
||||
fields_out[fname] = fd
|
||||
out.append(
|
||||
{
|
||||
"id": mid,
|
||||
"table": d["table"],
|
||||
"fields": d["fields"],
|
||||
"fields": fields_out,
|
||||
"import_mode": d.get("import_mode"),
|
||||
}
|
||||
)
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from csv_parser.core import (
|
|||
get_csv_import_limits,
|
||||
iter_csv_dict_rows,
|
||||
)
|
||||
from csv_parser.mapping_suggest import build_type_conversions_for_mapping
|
||||
from csv_parser.type_converter import convert_value, build_row_after_mapping
|
||||
|
||||
|
||||
|
|
@ -73,6 +74,25 @@ def test_convert_date_and_kcal_factor():
|
|||
assert abs(k - 8000 * 0.239) < 0.01
|
||||
|
||||
|
||||
def test_convert_kcal_via_source_unit_kj():
|
||||
spec = {"type": "float", "source_unit": "kj", "decimal_separator": "."}
|
||||
k = convert_value("4184", "kcal", spec, module="nutrition")
|
||||
assert abs(k - 1000.0) < 0.05
|
||||
|
||||
|
||||
def test_convert_protein_kg_to_g():
|
||||
spec = {"type": "float", "source_unit": "kg", "decimal_separator": "."}
|
||||
g = convert_value("0.1", "protein_g", spec, module="nutrition")
|
||||
assert abs(g - 100.0) < 0.001
|
||||
|
||||
|
||||
def test_build_row_source_unit_without_module_no_factor():
|
||||
"""Ohne module bleibt source_unit wirkungslos (Abwärtskompatibilität)."""
|
||||
spec = {"type": "float", "source_unit": "kj", "decimal_separator": "."}
|
||||
k = convert_value("4184", "kcal", spec, module=None)
|
||||
assert abs(k - 4184.0) < 0.01
|
||||
|
||||
|
||||
def test_iter_csv_dict_rows_full_file():
|
||||
text = "a;b\n1;2\n3;4\n"
|
||||
rows = list(iter_csv_dict_rows(text, ";", has_header=True))
|
||||
|
|
@ -86,9 +106,16 @@ def test_build_row_after_mapping():
|
|||
"date": {"type": "date", "format": "dd.mm.yyyy"},
|
||||
"kcal": {"type": "float", "conversion_factor": 0.239, "decimal_separator": "."},
|
||||
}
|
||||
out = build_row_after_mapping(csv_row, fm, tc)
|
||||
out = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
|
||||
assert out["date"].month == 1
|
||||
assert out["kcal"] is not None
|
||||
assert abs(float(out["kcal"]) - 4200 * 0.239) < 0.02
|
||||
|
||||
|
||||
def test_build_type_conversions_kj_header_sets_source_unit():
|
||||
fm = {"kJ": "kcal", "Datum": "date"}
|
||||
tc = build_type_conversions_for_mapping("nutrition", fm, None)
|
||||
assert tc["kcal"].get("source_unit") == "kj"
|
||||
|
||||
|
||||
def test_build_row_fddb_raw_header_keys_match_normalized_template():
|
||||
|
|
@ -111,15 +138,16 @@ def test_build_row_fddb_raw_header_keys_match_normalized_template():
|
|||
"date": {"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only"},
|
||||
"kcal": {
|
||||
"type": "float",
|
||||
"conversion_factor": 0.239,
|
||||
"source_unit": "kj",
|
||||
"decimal_separator": ",",
|
||||
},
|
||||
"fat_g": {"type": "float", "decimal_separator": ","},
|
||||
"carbs_g": {"type": "float", "decimal_separator": ","},
|
||||
"protein_g": {"type": "float", "decimal_separator": ","},
|
||||
}
|
||||
out = build_row_after_mapping(csv_row, fm, tc)
|
||||
out = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
|
||||
assert out["date"].year == 2024 and out["date"].month == 1 and out["date"].day == 1
|
||||
assert out["kcal"] is not None and abs(float(out["kcal"]) - (42000 / 4.184)) < 0.1
|
||||
|
||||
|
||||
def test_convert_date_ddmm_with_seconds():
|
||||
|
|
|
|||
|
|
@ -14,6 +14,8 @@
|
|||
- ✅ Commits: 7 systematic commits (6 module migrations + 1 chart expansion)
|
||||
- ✅ charts.py: 329 → 2246 lines (+1917 lines)
|
||||
|
||||
**Import vs. Auswertung:** Leitlinie „was beim Ingest erlaubt ist“ und Einordnung bestehender Import-Pfade: `.claude/rules/ARCHITECTURE.md` Abschnitt **8. CSV-Import vs. Data Layer**.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
|
|
|||
|
|
@ -159,6 +159,57 @@ export default function AdminCsvTemplateEditorPage() {
|
|||
return requiredTargets.filter((r) => !assignedTargets.has(r))
|
||||
}, [requiredTargets, assignedTargets])
|
||||
|
||||
const unitTargets = useMemo(() => {
|
||||
if (!modMeta?.fields || aggregateSleepImport) return []
|
||||
const list = []
|
||||
const seen = new Set()
|
||||
for (const t of assignedTargets) {
|
||||
const opts = modMeta.fields[t]?.source_unit_options
|
||||
if (!opts?.length || seen.has(t)) continue
|
||||
seen.add(t)
|
||||
list.push({ field: t, options: opts })
|
||||
}
|
||||
return list.sort((a, b) => a.field.localeCompare(b.field))
|
||||
}, [modMeta, assignedTargets, aggregateSleepImport])
|
||||
|
||||
const getSourceUnitSelectValue = (fieldKey) => {
|
||||
let tc
|
||||
try {
|
||||
tc = JSON.parse(typeConversionsText || '{}')
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
const opts = modMeta?.fields?.[fieldKey]?.source_unit_options || []
|
||||
const canonical = opts.find((o) => o.is_canonical)?.id || opts[0]?.id
|
||||
const su = tc[fieldKey]?.source_unit
|
||||
if (su && opts.some((o) => o.id === su)) return su
|
||||
return canonical || ''
|
||||
}
|
||||
|
||||
const updateSourceUnit = (fieldKey, sourceUnitId) => {
|
||||
let tc
|
||||
try {
|
||||
tc = JSON.parse(typeConversionsText || '{}')
|
||||
} catch {
|
||||
setError('type_conversions: ungültiges JSON (Quelleinheit kann nicht gesetzt werden).')
|
||||
return
|
||||
}
|
||||
const opts = modMeta?.fields?.[fieldKey]?.source_unit_options || []
|
||||
const canonical = opts.find((o) => o.is_canonical)?.id
|
||||
const base =
|
||||
tc[fieldKey] && typeof tc[fieldKey] === 'object'
|
||||
? { ...tc[fieldKey] }
|
||||
: { type: 'float', decimal_separator: 'auto', flexible: true }
|
||||
if (sourceUnitId === canonical || sourceUnitId === '' || sourceUnitId == null) {
|
||||
delete base.source_unit
|
||||
} else {
|
||||
base.source_unit = sourceUnitId
|
||||
}
|
||||
tc[fieldKey] = base
|
||||
setTypeConversionsText(JSON.stringify(tc, null, 2))
|
||||
setError(null)
|
||||
}
|
||||
|
||||
const handleAnalyze = async () => {
|
||||
if (!file) {
|
||||
setError('Bitte eine CSV-Datei wählen.')
|
||||
|
|
@ -530,10 +581,48 @@ export default function AdminCsvTemplateEditorPage() {
|
|||
)}
|
||||
</div>
|
||||
|
||||
{unitTargets.length > 0 && (
|
||||
<div className="card" style={{ padding: 16, marginBottom: 16 }}>
|
||||
<div className="form-label">3b. Quelleinheit (optional)</div>
|
||||
<p style={{ fontSize: 13, color: 'var(--text2)', marginTop: 8, lineHeight: 1.55 }}>
|
||||
Ziel-Einheit kommt aus dem Datenmodell (z. B. kcal, g, kg, km). Hier nur angeben, falls die CSV
|
||||
abweicht (z. B. FDDB kJ → kcal, Makros in kg → g, Gewicht in lb → kg).
|
||||
</p>
|
||||
<div style={{ marginTop: 14, display: 'flex', flexDirection: 'column', gap: 12 }}>
|
||||
{unitTargets.map(({ field: fkey, options }) => (
|
||||
<label key={fkey} style={{ display: 'block' }}>
|
||||
<span className="form-label" style={{ display: 'block', marginBottom: 6 }}>
|
||||
{fkey}
|
||||
</span>
|
||||
<select
|
||||
className="form-input"
|
||||
value={getSourceUnitSelectValue(fkey) || ''}
|
||||
onChange={(e) => updateSourceUnit(fkey, e.target.value)}
|
||||
style={{
|
||||
width: '100%',
|
||||
minHeight: 46,
|
||||
textAlign: 'left',
|
||||
padding: '11px 14px',
|
||||
fontSize: 15,
|
||||
}}
|
||||
>
|
||||
{options.map((o) => (
|
||||
<option key={o.id} value={o.id}>
|
||||
{o.label}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="card" style={{ padding: 16, marginBottom: 16 }}>
|
||||
<div className="form-label">4. type_conversions (JSON)</div>
|
||||
<p style={{ fontSize: 13, color: 'var(--text2)', marginTop: 8 }}>
|
||||
Vom Vorschlag übernommen; bei Bedarf manuell anpassen (z. B. kJ-Faktor, Datumsformat).
|
||||
Vom Vorschlag übernommen; Quelleinheiten setzen zusätzlich <code>source_unit</code> (siehe 3b).
|
||||
Manuell z. B. Datumsformat oder legacy <code>conversion_factor</code>.
|
||||
</p>
|
||||
<textarea
|
||||
className="form-input"
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user