140 Commits
Author SHA1 Message Date
Lars 19a88e4f97 Merge pull request 'feat: Enhance goal progress tracking and display' (#84) from develop into main
Deploy Production / deploy (push) Successful in 53s
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 16s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #84
2026-04-14 11:39:58 +02:00
Lars 1b01f5e6d0 feat: Enhance goal progress tracking and display
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Added a function to calculate goal progress percentage based on start, target, and current values.
- Updated GoalsPage to display progress in a user-friendly format, including visual progress bars.
- Implemented error handling for goal progress updates in the backend to ensure robustness.
2026-04-14 10:43:49 +02:00
Lars 424270f0e8 Merge pull request 'bug fix und Debug Workflow' (#83) from develop into main
Deploy Production / deploy (push) Successful in 1m2s
Build Test / build-frontend (push) Successful in 16s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #83
2026-04-13 18:15:06 +02:00
Lars df8e732709 fix: Use correct field 'label' instead of 'name' for node display
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Frontend saves node name as 'label' (workflowSerializer.js:19)
- Changed WorkflowNode.name to WorkflowNode.label
- Changed node.name to node.label in workflow_executor.py
- Priority: node.label > prompt_slug > node_type-id
- Verified against frontend serialization code
2026-04-13 18:09:12 +02:00
Lars d5325acee6 fix: Use node.name in node_label calculation (minimal change)
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Add name field to WorkflowNode model
- Extend node_label priority: node.name > prompt_slug > node_type-id
- No new fields in NodeExecutionState (uses existing debug_prompt_slug)
- Simpler approach than previous attempt to avoid 504 timeout
2026-04-13 18:03:02 +02:00
Lars b7062d32bf Revert "feat: Show node.name from workflow editor in debug panel"
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
This reverts commit 5fa2ea2e6b.
2026-04-13 15:54:22 +02:00
Lars 5fa2ea2e6b feat: Show node.name from workflow editor in debug panel
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
- Add name field to WorkflowNode model
- Add node_name field to NodeExecutionState
- Set node_name in execute_workflow from node.name
- Display priority: node_name > debug_prompt_slug > node_label > node_id

User sees 'Qualitätseinschätzung' instead of 'node_abc123'
2026-04-13 15:43:42 +02:00
Lars f97d15288d fix: Show debug_prompt_slug instead of node_id
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 47s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Priority: debug_prompt_slug > node_label > node_type-shortid > node_id
2026-04-13 15:36:50 +02:00
Lars 736dc58d81 feat: Show debug info in WorkflowResultViewer
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 59s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 1s
Display per node:
- debug_prompt (prompt sent to AI)
- debug_raw_response (raw AI response)
- analysis_core (parsed results)
- normalized_signals (decision signals with status)
- Failed nodes: red border + red background

NO other changes - executeWorkflow still used
2026-04-13 15:31:37 +02:00
Lars 0a27533262 feat: Highlight failed nodes in WorkflowDebugPanel
Build Test / build-frontend (push) Waiting to run
Build Test / pytest-backend (push) Has been cancelled
Build Test / lint-backend (push) Waiting to run
Deploy Development / deploy (push) Successful in 54s
- Failed nodes now have:
  - Red border (2px instead of 1px)
  - Light red background (#D85A3010)
  - Red shadow/glow effect

Makes it immediately obvious which nodes had errors.
2026-04-13 12:58:25 +02:00
Lars 7388776b29 fix: Add human-readable labels to workflow nodes in debug output
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 53s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- workflow_executor.py: Store prompt_slug or generated label in debug_prompt_slug for all nodes
  - This makes it easy to identify nodes in the debug panel
  - Example: 'wf_nutrition_basis' instead of 'node_5'

- Helps identify which node is which when debugging workflows
2026-04-13 12:52:29 +02:00
Lars a515a5d563 feat: Add WorkflowDebugPanel component to display per-node debug information
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Created WorkflowDebugPanel.jsx: Collapsible panel showing debug info for each workflow node
  - Shows prompt sent to AI
  - Shows raw AI response
  - Shows parsed results
  - Shows normalized signals
  - Color-coded status (executed/failed/skipped)
  - Expandable/collapsible per node

- Updated Analysis.jsx:
  - Added WorkflowDebugPanel import
  - Store node_states in newResult for debugging
  - Display WorkflowDebugPanel below InsightCard (both locations)

This makes it easy to debug workflow issues by seeing exactly what happened at each node.
2026-04-13 12:41:12 +02:00
Lars 12d4d7c63b feat: Add comprehensive debug information for workflow nodes
Build Test / pytest-backend (push) Successful in 8s
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / lint-backend (push) Successful in 0s
Backend changes:
- workflow_models.py: Add debug_prompt, debug_raw_response, debug_node_type, debug_prompt_slug, metadata fields to NodeExecutionState
- workflow_executor.py: Capture and store debug info for analysis, logic, and join nodes when enable_debug=True
  - Analysis nodes: store full prompt + raw AI response
  - Logic nodes: store expression + evaluation result
  - Join nodes: store strategy + path statistics

Frontend changes:
- Analysis.jsx: Enable debug mode by default (debug=true) for all workflow executions

This allows developers to see exactly what prompt was sent to the AI, what response was received, and how each node was processed - essential for debugging workflow issues.
2026-04-13 12:38:55 +02:00
Lars f34e46b04f Merge pull request 'Async Workflow' (#82) from develop into main
Deploy Production / deploy (push) Successful in 1m3s
Build Test / build-frontend (push) Successful in 16s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #82
2026-04-13 11:58:01 +02:00
Lars 3664f53c51 fix: Use NodeStatus.EXECUTED instead of COMPLETED
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 53s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
NodeStatus enum has EXECUTED, not COMPLETED. Fixed in workflow_executor.py progress callback.
2026-04-13 11:49:31 +02:00
Lars fb2e0803c0 fix: SSE streaming - WorkflowNode label attribute and ai_insights column name
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 52s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- workflow_executor.py: Generate node_label from prompt_slug or node.type (WorkflowNode has no label attribute)
- prompts.py: Fix INSERT statement - use 'created' column instead of 'created_at'

SSE endpoint now works correctly for workflow execution streaming.
2026-04-13 11:47:31 +02:00
Lars bb01283727 fix: Correct except/finally indentation in SSE endpoint
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 46s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
2026-04-13 11:41:56 +02:00
Lars bc60b9f5c9 fix: Correct indentation in SSE execute_workflow_async function
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Failing after 1s
Build Test / lint-backend (push) Successful in 0s
2026-04-13 11:27:44 +02:00
Lars fbeabcde97 fix: IndentationError in prompts.py SSE endpoint
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 52s
Build Test / pytest-backend (push) Failing after 1s
Build Test / lint-backend (push) Successful in 1s
2026-04-13 11:25:34 +02:00
LarsandClaude Sonnet 4.5 ba474b0a57 feat: Implement Server-Sent Events (SSE) for long-running workflows
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Failing after 1s
Build Test / lint-backend (push) Successful in 0s
Backend:
- workflow_executor.py: Add progress_callback parameter, emit events for execution_started, node_complete, execution_complete, execution_failed
- prompt_executor.py: Thread progress_callback through execute chain
- routers/prompts.py: New /execute-stream endpoint with asyncio Queue for SSE

Frontend:
- utils/api.js: New executeUnifiedPromptStream() function with EventSource
- pages/Analysis.jsx: Use SSE with live progress display (X/Y Nodes)

Fixes:
- No more gateway timeouts for complex workflows (10+ nodes)
- Live progress feedback for users
- Unlimited workflow complexity

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-04-13 11:23:16 +02:00
LarsandClaude Opus 4.6 790e6df8ef fix: Make debug parameter work as Query parameter in /api/prompts/execute
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
Bug: debug=true in URL was ignored because FastAPI expected it in
request body (POST without Query() expects body params by default).

Result: node_states were never returned, even with ?debug=true

Fix: Changed debug and save to Query parameters:
- debug: bool = Query(False, ...)
- save: bool = Query(False, ...)

Now ?debug=true in URL correctly enables debug output with node_states.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 09:14:30 +02:00
LarsandClaude Opus 4.6 057df0afc8 fix: Support UI-format edge routing with sourceHandle
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 24s
Deploy Development / deploy (push) Successful in 52s
Build Test / lint-backend (push) Successful in 0s
Logic-Nodes evaluated correctly but activated_edges was empty because
_get_edges_by_label() only checked e.label, which is null in UI format.

UI format uses:
- sourceHandle: "true" / "false" (instead of label: "then" / "else")
- targetHandle: "in" / "path_1" / etc.

Changes:
1. Added source_handle/target_handle fields to WorkflowEdge model
   - With aliases sourceHandle/targetHandle for camelCase JSON
2. Updated _get_edges_by_label() to check both formats:
   - Legacy: e.label == "then" / "else"
   - UI: e.source_handle == "true" / "false"

Now Logic-Nodes correctly activate outgoing edges → Join-Node receives
completed paths → End-Node executes → Workflow completes!

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 09:07:50 +02:00
LarsandClaude Opus 4.6 ba04e0c0b6 fix: Add extra='forbid' to Condition for proper Union resolution
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Critical fix: Without extra='forbid', Pydantic accepted UI format
{operator: "and", operands: [...]} as valid Condition by ignoring
unknown fields, resulting in Condition(expression=None).

With extra='forbid':
- Condition rejects unknown fields → fails
- Union tries next type → LogicExpression → success

Test Results (9/9 passed):
- Simple comparisons (eq, neq, gt, lt, in) ✅
- AND/OR combinations ✅
- Deep nesting (3+ levels) ✅
- NOT operator ✅
- All operators (eq, neq, in, not_in, gt, lt, gte, lte, and, or, not) ✅
- Legacy format (Condition wrapper) ✅
- Complex real-world scenarios ✅

Added comprehensive test suite in:
- test_condition_parsing.py (9 test cases)
- test_condition_union.py (Union resolution verification)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 09:01:53 +02:00
LarsandClaude Opus 4.6 f5ce1ec941 refactor: Proper type-safe condition handling with Union types
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 52s
Build Test / pytest-backend (push) Successful in 5s
Build Test / lint-backend (push) Successful in 0s
Previous fix used Any type, breaking type safety and only handling
simple cases. This is the correct implementation:

Changes:
1. LogicExpression.operands: List[Any] → List['LogicExpression']
   - Enables recursive/nested expressions
   - Proper type checking for all operator combinations

2. WorkflowNode.condition: Any → Union[LogicExpression, Condition]
   - Type-safe deserialization
   - Supports both UI format (direct LogicExpression) and legacy (Condition wrapper)
   - Pydantic automatically tries LogicExpression first, then Condition

3. Executor: Simplified with isinstance() checks
   - Clean type detection without dict manipulation
   - Fallback for edge cases

This now correctly handles:
- Simple conditions: {operator: "eq", ref: "...", value: "..."}
- Combined: {operator: "and", operands: [...]}
- Nested: {operator: "or", operands: [{operator: "and", ...}, ...]}
- All operators: eq, neq, in, not_in, gt, lt, gte, lte, contains, and, or, not
- Legacy format: {expression: {...}, then_path: "...", else_path: "..."}

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 08:45:55 +02:00
LarsandClaude Opus 4.6 2deb6510f8 fix: Support UI-format LogicExpression in Logic-Node condition field
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Root cause: UI saves LogicExpression directly as condition:
  {operands: [...], operator: "and"}

But Pydantic model expected Condition with wrapped expression:
  {expression: {operands: [...], operator: "and"}}

Result: Pydantic deserialized it as Condition with expression=None
→ Logic-Nodes failed with "'NoneType' object has no attribute 'operator'"

Fix:
1. Changed WorkflowNode.condition type from Condition to Any
2. Executor now handles both dict and Pydantic model formats
3. Detects UI format (operator+operands) vs legacy format (expression wrapper)
4. Converts dict to LogicExpression before evaluation

Fixes: Logic-Node execution failures in Training-Tiefenanalyse workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 08:40:43 +02:00
LarsandClaude Opus 4.6 0eac40abf6 fix: Add None-check for Logic-Node condition/expression
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 51s
Build Test / lint-backend (push) Successful in 0s
Previous fix handled hasattr() but didn't check for None values.
Now explicitly checks that operator/expression is not None before using it.

Error was: "'NoneType' object has no attribute 'operator'"

Clearer error message: "condition is None or missing"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 08:32:54 +02:00
LarsandClaude Opus 4.6 e915d3fb13 fix: Support both Logic-Node condition serialization formats
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 5s
Build Test / lint-backend (push) Successful in 0s
Logic-Nodes were timing out because UI saves condition as:
  {operands: [...], operator: "and"}

But Backend expected:
  {expression: {operands: [...], operator: "and"}}

This caused node.condition.expression to be None, triggering:
- Logic-Node failures
- Join-Node wait_all timeout
- 504 Gateway Timeout

Fix: Accept both formats by checking for operator/operands attributes
directly on condition, falling back to condition.expression.

Fixes: 504 Gateway Timeout in Training-Tiefenanalyse workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-13 08:26:43 +02:00
Lars 60f6cf3c6d fix: Add null check for logic node expression to prevent AttributeError
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 58s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Problem: Logic nodes without logic_expression defined caused AttributeError
"'NoneType' object has no attribute 'operator'" when evaluating condition.

Solution: Check both node.condition AND node.condition.expression before
calling evaluate_logic_expression(). Return clear FAILED state with error
message instead of crashing.

Impact: Workflows with incomplete logic node definitions now fail gracefully
with clear error message instead of cryptic AttributeError.
2026-04-13 08:16:06 +02:00
Lars 46b703e5a2 Merge pull request 'Bug Fixes. Workflow Engine' (#81) from develop into main
Deploy Production / deploy (push) Successful in 1m0s
Build Test / build-frontend (push) Successful in 16s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #81
2026-04-12 14:08:43 +02:00
Lars e09cbc112e fix: Preserve case in question IDs during parsing
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Problem: Parser converted question IDs to lowercase ('qAnalyst' → 'qanalyst'),
causing normalization to fail because id_catalog lookup is case-sensitive.

Impact: All workflow question signals were lost - normalized_signals stayed empty,
so template placeholders like {{node_2.signal_qAnalyst}} remained unresolved.

Solution: Removed .lower() call in parse_decision_questions() to preserve
original case from AI response.

Root cause: Line 162 in result_container_parser.py
Fixes: Question augmentation signals not appearing in workflow end nodes
2026-04-12 14:04:14 +02:00
Lars f6b3182a80 fix: Add wrapper in prompts.py execute endpoint for workflow signature mismatch
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Problem: Workflows executed via /api/prompts/execute (not /api/workflows/execute)
were passing call_openrouter directly to execute_prompt_with_data, which then
passes it to workflow_executor. workflow_executor expects (prompt, model) signature
but call_openrouter has (prompt, max_tokens=4096) signature.

Previous fix in workflows.py was correct but unused - workflows use prompts.py endpoint.

Solution: Added workflow_llm_call() wrapper in execute_unified_prompt() endpoint
that matches expected (prompt, model) -> str signature.

Related: cb3aa48 (workflows.py fix for different endpoint)
2026-04-12 13:44:08 +02:00
Lars cb3aa48999 fix: Add wrapper function for workflow LLM calls to prevent max_tokens signature mismatch
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Problem: workflow_executor calls openrouter_call_func(prompt, model) but
call_openrouter expects (prompt, max_tokens=4096). This caused the model string
'anthropic/claude-sonnet-4' to be passed as max_tokens, resulting in OpenRouter
requesting 64000 tokens and failing with 402 credit errors.

Solution: Added workflow_llm_call() wrapper in workflows.py that matches the
expected (prompt, model) -> str signature and calls call_openrouter correctly.

Fixes: All workflows failing with 402 'insufficient credits' errors
2026-04-12 13:37:31 +02:00
LarsandClaude Opus 4.6 77f1ed14c5 fix: Cursor-Problem beim Frage-ID Editieren
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Problem: Cursor springt nach jedem Tastendruck aus dem ID-Feld

Ursache: key={q.id} in QuestionEditor map
- Wenn ID geändert wird, ändert sich der React Key
- React unmountet alte Component und mountet neue
- Focus geht verloren

Lösung: key={idx} verwenden
- Stabiler Key während Editing
- Komponente bleibt gemountet
- Cursor bleibt im Feld

UX: Jetzt kann man IDs flüssig editieren ohne Unterbrechung

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-12 12:29:52 +02:00
Lars 25c4ecfd48 Merge pull request 'Verbesserung für Tests von Prompts' (#80) from develop into main
Deploy Production / deploy (push) Successful in 1m2s
Build Test / build-frontend (push) Successful in 15s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
Reviewed-on: #80
2026-04-12 11:58:07 +02:00
Lars 08c9cccdcc feat: Add expandable collapsible component for improved content display
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 53s
Build Test / lint-backend (push) Successful in 0s
- Introduced `ExpandableCollapsible` component to manage the visibility of lengthy content, allowing users to toggle between expanded and collapsed views.
- Updated `renderTestOutput` to utilize the new component for displaying test results, JSON outputs, and object representations, enhancing user experience by reducing clutter.
- Enhanced `Markdown` component to support fenced code blocks, improving the rendering of code snippets with language labels and better styling.

These changes improve the readability and organization of content within the application, providing users with a more interactive and manageable interface.
2026-04-12 11:10:39 +02:00
Lars 4b6e1bed11 feat: Enhance OpenRouter API interaction and error handling
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Increased the maximum token limit in the `call_openrouter` function from 1500 to 4096 to allow for more extensive responses.
- Implemented robust error handling for API requests, including timeout and request errors, with detailed HTTP exceptions for better debugging.
- Improved JSON response handling to ensure valid data is returned, with specific error messages for missing content in the response.
- Enhanced the overall reliability of the OpenRouter API integration, providing clearer feedback for users in case of issues.

These changes improve the user experience by ensuring more comprehensive responses and clearer error reporting during API interactions.
2026-04-12 11:03:07 +02:00
Lars 72eb94d186 Merge pull request 'feat: Improve float parsing logic for enhanced accuracy in numeric conversions' (#79) from develop into main
Deploy Production / deploy (push) Successful in 55s
Build Test / build-frontend (push) Successful in 17s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #79
2026-04-12 07:30:44 +02:00
Lars 90a27846ca feat: Improve float parsing logic for enhanced accuracy in numeric conversions
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Updated the `_parse_float_auto` function in `type_converter.py` to better handle various decimal and thousand separators, particularly for cases with long decimal parts from sources like Apple Health.
- Enhanced the logic for splitting and processing numeric strings to ensure correct interpretation of values, including edge cases with multiple separators.
- Added handling for cases where numeric strings may contain both commas and periods, improving overall robustness in float parsing.

These changes enhance the accuracy of numeric conversions, ensuring more reliable data processing across the application.
2026-04-12 07:28:24 +02:00
Lars 4d81ea2cf3 Merge pull request 'feat: Update Gitea issues index and enhance data layer metrics' (#78) from develop into main
Deploy Production / deploy (push) Successful in 1m4s
Build Test / build-frontend (push) Successful in 15s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #78
2026-04-11 22:17:18 +02:00
Lars d7cefdd9e9 feat: Update Gitea issues index and enhance data layer metrics
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 1m3s
Build Test / pytest-backend (push) Successful in 9s
Build Test / lint-backend (push) Successful in 0s
- Updated the Gitea issues index to reflect the latest state as of 2026-04-11, adding issue #76 to the list.
- Refined data handling in `activity_metrics.py`, `body_metrics.py`, `nutrition_metrics.py`, and `scores.py` to ensure consistent float conversions for calculations, improving accuracy in metric evaluations.
- Enhanced the calculation logic for various metrics to handle potential None values more robustly, ensuring smoother data processing and improved reliability across the application.

These changes improve the clarity of the Gitea issues documentation and enhance the overall accuracy and reliability of health and fitness metrics.
2026-04-11 22:14:45 +02:00
Lars caeed3fbaa Merge pull request 'Platzhalter finalisiert - Option |d und Option |x implementiert' (#77) from develop into main
Deploy Production / deploy (push) Successful in 1m3s
Build Test / build-frontend (push) Successful in 16s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #77
2026-04-11 22:10:10 +02:00
Lars 4868e44882 feat: Refine placeholder resolution with enhanced modifiers support
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Updated `resolve_placeholders` in `prompt_executor.py` to support combined modifiers for placeholders, allowing for more flexible output formats.
- Enhanced `build_ai_placeholder_caption` in `placeholder_registry.py` to clarify the generation of AI context captions, focusing on descriptions and explanations.
- Introduced new helper functions in `placeholder_resolver.py` to streamline the retrieval of descriptions and explanations for placeholders.
- Modified tests to cover new functionality, ensuring accurate behavior for combined modifiers and improved placeholder resolution.

These changes enhance the usability and clarity of placeholder outputs, providing users with richer contextual information.
2026-04-11 21:58:29 +02:00
Lars a9a414b956 feat: Enhance placeholder caption generation and formatting
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
- Updated `build_ai_placeholder_caption` in `placeholder_registry.py` to improve the generation of AI context captions by prioritizing descriptions and avoiding redundancy.
- Introduced `format_value_with_d_modifier` in `placeholder_resolver.py` to format values with contextual information, enhancing the clarity of exported placeholder values.
- Modified `export_placeholder_values` in `prompts.py` to utilize the new formatting function, ensuring that exported data includes both raw values and contextual descriptions.
- Added tests for the new formatting function and updated existing tests to ensure accurate caption generation.

These changes improve the contextual relevance of placeholder data and enhance the user experience when interacting with exported values.
2026-04-11 21:47:08 +02:00
Lars baeddd7c13 feat: Enhance placeholder system with AI context support
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Introduced `build_ai_placeholder_caption` function in `placeholder_registry.py` to generate AI context captions based on placeholder metadata.
- Updated `resolve_placeholders` in `placeholder_resolver.py` to support modifiers for AI context, allowing for enhanced descriptions when placeholders are resolved.
- Modified `get_placeholder_catalog` to include AI captions in the output, improving the metadata available for placeholders.
- Adjusted `export_placeholder_values` to include AI captions in the exported data, enhancing the information provided to users.

These changes improve the flexibility and functionality of the placeholder system, enabling richer context generation for dynamic content.
2026-04-11 21:36:29 +02:00
Lars 41bf593d4c feat: Refactor sleep metrics calculations and improve error handling
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 3s
Build Test / lint-backend (push) Successful in 0s
- Updated `get_sleep_avg_duration` and `get_sleep_avg_quality` functions in `placeholder_resolver.py` to provide clearer error messages when data is unavailable.
- Enhanced sleep quality calculations in `recovery_metrics.py` to handle cases with insufficient data more robustly.
- Improved data handling in various metrics files (`activity_metrics.py`, `body_metrics.py`, `nutrition_metrics.py`, `recovery_metrics.py`, and `scores.py`) to ensure consistent float conversions for calculations.
- Added utility functions in `recovery_metrics.py` for parsing and normalizing sleep segment data, enhancing the accuracy of sleep quality assessments.

These changes improve the reliability and clarity of sleep-related metrics and enhance overall data handling across the application.
2026-04-11 21:27:49 +02:00
Lars 04e23d8115 feat: Enhance placeholder resolution and error handling
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / lint-backend (push) Successful in 0s
- Updated `extract_value_raw` to improve JSON parsing and handle unavailable data more effectively.
- Introduced new functions in `placeholder_resolver.py` for standardized responses when data is unavailable, enhancing clarity for users and AI.
- Modified various data retrieval functions to utilize the new response format, providing detailed reasons for unavailability.
- Improved availability checks in `export_placeholder_values_extended` to account for new response formats.

These changes enhance the robustness of the placeholder system and improve user experience by providing clearer error messages and data handling.
2026-04-11 21:22:27 +02:00
Lars 052ba195cc feat: Update placeholder metadata and nutrition metrics
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 54s
Build Test / lint-backend (push) Successful in 1s
- Adjusted the total number of placeholders from 116 to 114 across various documentation and code files to reflect the current state of the system.
- Enhanced TDEE calculation logic in `nutrition_metrics.py` to prioritize Mifflin–St Jeor BMR with PAL when demographic data is available, with a fallback to a weight-based estimate.
- Updated placeholder registrations to ensure consistency with the new metadata structure and improved data handling.
- Revised documentation to clarify the authoritative source of placeholder metadata and the implications of the changes on existing functionalities.

These updates improve the accuracy and consistency of the placeholder system and enhance the nutritional assessment capabilities within the application.
2026-04-11 21:11:05 +02:00
Lars 2ea5f905c4 feat: Add new profile and time period placeholders in placeholder_resolver.py
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 50s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
- Introduced functions to retrieve profile name, age, height, and gender for better placeholder resolution.
- Added functions for displaying current date and time period labels (last 7, 30, and 90 days).
- Updated PLACEHOLDER_MAP to utilize new functions for improved readability and maintainability.
- Enhanced placeholder registrations in __init__.py to include new modules for sleep, vital metrics, and profile time periods.

These changes enhance the flexibility and functionality of the placeholder system, allowing for more dynamic content generation.
2026-04-11 21:08:34 +02:00
Lars e9e094c6a4 feat: Enhance nutrition and activity metrics with new data layers
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Added new functions for BMI and goal weight/body fat percentage retrieval in `body_metrics.py`.
- Introduced training frequency and inter-session gap calculations in `activity_metrics.py`.
- Updated placeholder registrations to include new metrics for nutrition and activity.
- Improved data handling in `placeholder_resolver.py` for better integration of new metrics.
- Enhanced documentation across modules to reflect the new functionalities.

These updates improve the accuracy and comprehensiveness of health and fitness assessments within the application.
2026-04-11 20:46:17 +02:00
Lars 61a5bb39ae feat: Update nutrition metrics and energy balance calculations
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 59s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Introduced a single TDEE calculation based on current weight, replacing the fixed 2500 kcal value.
- Updated `get_energy_balance_data` to use daily totals for intake calculations and improved energy balance logic.
- Enhanced `get_nutrition_average_data` to calculate averages over calendar days instead of raw log entries.
- Adjusted placeholder resolution to ensure consistent metadata usage across requests.
- Fixed issues in the charts router to reflect the new energy balance logic and TDEE calculations.

These changes improve the accuracy of nutritional assessments and streamline data handling in the application.
2026-04-11 19:04:27 +02:00
Lars b5f745fc3e Merge pull request 'Workflow 1.1 bug fixes' (#74) from develop into main
Deploy Production / deploy (push) Successful in 1m5s
Build Test / build-frontend (push) Successful in 15s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #74
2026-04-11 15:39:23 +02:00
LarsandClaude Opus 4.6 549c31431e feat: Validierungs-Panel mit Details und Click-to-Jump
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Issue #1 + #3: Validierung zeigt keine Details / Fehler-Lokalisierung

NEU: ValidationPanel Component
- Zeigt alle Fehler und Warnungen mit vollständigen Details
- Gruppiert nach Severity (Fehler/Warnungen)
- Aufklappbar pro Gruppe (collapsible)
- Click-to-Jump: Fehler mit nodeId sind klickbar
- Klick selektiert betroffene Node → Config-Panel öffnet sich
- Schließbar, öffnet sich automatisch bei neuen Fehler/Warnungen

Features:
- Fixed position (bottom-right, über Canvas)
- Farb-Kodierung: Rot (Errors) / Gelb (Warnings)
- Hover-Effekt auf klickbare Items
- Type-Labels (structure, isolation, etc.)
- Scrollbar bei vielen Fehler/Warnungen
- X-Button zum Schließen

UX-Verbesserungen:
- Kein Raten mehr: Zeigt WAS der Fehler ist
- Kein Suchen mehr: Klick springt direkt zur Node
- Übersichtlich auch bei vielen Fehler/Warnungen
- Funktioniert in großen Workflows

Technisch:
- handleValidationNodeClick: setSelectedNodeId
- Auto-show bei errors.length > 0 || warnings.length > 0
- State: showValidationPanel (closable)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 15:27:14 +02:00
LarsandClaude Opus 4.6 3fa01dd686 feat: Warnung bei ungespeicherten Workflow-Änderungen
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 52s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
Issue #0: Ungespeicherte Änderungen gehen verloren beim "Zurück"-Klick

Implementiert:
- hasUnsavedChanges State tracking
- Warnung beim "Zurück"-Button (navigate zu /admin/prompts)
- Warnung beim "Neu"-Button (nur wenn unsaved changes)
- Browser beforeunload Event (warnt bei Browser-Back/Refresh)

Tracking für alle Änderungen:
- onNodesChange/onEdgesChange (Node-Bewegung, Löschen via Delete-Taste)
- onConnect (neue Edges)
- handleAddNode (Node hinzufügen)
- handleNodeUpdate (Node-Daten ändern)
- handleDeleteNode (Node löschen via Button)
- workflowName onChange (Titel ändern)

Flag wird cleared:
- Nach erfolgreichem Save (Update/Create)
- Nach erfolgreichem Load
- Bei "Neu" (nach User-Bestätigung)

UX:
- Klare Warnung: "Du hast ungespeicherte Änderungen"
- Kein Datenverlust mehr durch versehentliches Zurück
- Browser warnt auch bei Refresh/Close

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 15:21:31 +02:00
LarsandClaude Opus 4.6 c9357d4c0e feat: Import Dialog mit 3 Buttons (Ja/Nein/Abbrechen)
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Ersetzt zwei aufeinanderfolgende confirm()-Dialoge durch einen
Custom Dialog mit drei klaren Optionen:

- "Ja, überschreiben" → bestehende Prompts aktualisieren
- "Nein, nur neue" → existierende überspringen
- "Abbrechen" → Import komplett abbrechen

UX-Verbesserung:
- Alle Optionen auf einen Blick sichtbar
- Kein Raten mehr was "OK" oder "Abbrechen" bedeutet
- Klare Beschreibungstexte unter jedem Button
- Vollbildschirm-Modal mit Overlay

Technisch:
- importDialogData State für Dialog-Daten
- handleImportChoice verarbeitet yes/no/cancel
- Custom Modal-JSX statt Browser confirm()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 14:57:48 +02:00
LarsandClaude Opus 4.6 f3a61091c7 fix: Import confirmation UX - two-step process
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Admin-Prompts Import hatte unklare Abbrechen-Logik:
- Nutzer erwartete: Abbrechen = Import komplett abbrechen
- Vorher: Abbrechen = overwrite=false, Import lief weiter

Lösung: Zwei-Schritt-Bestätigung
1. "X Prompts importieren?" → Abbrechen = kompletter Abbruch
2. "Existierende überschreiben?" → Abbrechen = nur neue importieren

UX: Klare Trennung zwischen "Import abbrechen" und "Modus wählen"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 14:54:53 +02:00
LarsandClaude Opus 4.6 10d24bbef7 fix(workflow): Duplicate - JSON-encode JSONB fields
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 52s
Build Test / lint-backend (push) Successful in 0s
**Error:**
```
psycopg2.ProgrammingError: can't adapt type 'dict'
```

**Root Cause:**
- duplicate_prompt passed Python dicts directly to SQL INSERT
- JSONB fields from r2d() are already deserialized by psycopg2
- PostgreSQL expects JSON strings for JSONB columns

**Fix:**
- Added json.dumps() for all JSONB fields before INSERT:
  - stages, output_schema, question_augmentations, graph_data
- Same pattern as import function

Files changed:
- backend/routers/prompts.py: JSON-encode JSONB in duplicate_prompt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 14:46:13 +02:00
LarsandClaude Opus 4.6 ff8104a533 fix(workflow): Route precedence - move export/import before path param
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
**Root Cause:**
- FastAPI route matching: /{prompt_id} caught ALL requests including /export-all
- Specific routes MUST be defined BEFORE path parameter routes

**Error:**
```
psycopg2.errors.InvalidTextRepresentation: invalid input syntax for type uuid: "export-all"
LINE 1: SELECT * FROM ai_prompts WHERE id='export-all'
```

**Fix:**
- Moved /export-all and /import endpoints to line 106 (BEFORE /{prompt_id} at ~260)
- Added warning comments to both functions
- Fixed typo: for r in → for row in

**Affected:**
- /export-all: Internal Server Error → now works ✅
- /import: Would have had same issue → preemptively fixed ✅

Files changed:
- backend/routers/prompts.py: Reordered route definitions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 14:42:55 +02:00
LarsandClaude Opus 4.6 3b7f89a214 fix(workflow): UnboundLocalError in execute_end_node - graph not defined
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Critical bug fix from pytest failures:

**Problem:**
- execute_end_node() tried to use 'graph' variable without defining it
- UnboundLocalError at line 602: "if graph:"
- Caused 2 test failures in test_end_node_template.py

**Root Cause:**
- In Issue #5 fix, added graph lookup for node labels in AUTO mode
- But forgot to get graph from context first
- TEMPLATE mode already had: graph = context.get("graph")

**Fix:**
- Added: graph = context.get("graph") at start of AUTO mode block
- Same pattern as TEMPLATE mode
- graph is optional (None if not in context), so if-check is safe

**Tests:**
- test_auto_mode_concatenates_all_analyses - should pass now
- test_auto_mode_skips_skipped_nodes - should pass now

Files changed:
- backend/workflow_executor.py: Added graph = context.get("graph") line 596

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 14:28:19 +02:00
LarsandClaude Opus 4.6 ba773e677b fix(workflow): Test-Suite Fixes - Issues #5, #8, #9, #11, #12
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 1m1s
Build Test / pytest-backend (push) Failing after 4s
Build Test / lint-backend (push) Successful in 0s
Addressed test results from Test_status_Wkf.md:

**Issue #5: End-Node Überschriften**
- Fixed aggregate_results to show node labels instead of "Node 10"
- Added graph lookup to get node.data.label from node objects
- Modified backend/workflow_executor.py (2 locations)

**Issue #8: Löschen-Taste funktioniert nicht**
- Added Delete key support to WorkflowCanvas
- Set deleteKeyCode={['Backspace', 'Delete']}
- Frontend: WorkflowCanvas.jsx

**Issue #9: Mehrere End-Nodes verhindern**
- Added validation error when multiple End-Nodes exist
- Backend supports only 1 End-Node (aggregate_results takes last)
- Frontend: workflowValidation.js

**Issue #11: Export Fehler "Internal Server Error"**
- Added missing fields to export-all endpoint:
  - graph_data (workflow node graph)
  - question_augmentations (analysis prompts)
- Added missing fields to import endpoint
- Proper JSON serialization for all JSONB fields
- Backend: routers/prompts.py

**Issue #12: Workflow duplizieren funktioniert nicht**
- Fixed duplicate endpoint to include all prompt fields:
  - type, stages, output_format, output_schema
  - question_augmentations, graph_data (critical for workflows!)
- Backend: routers/prompts.py

Files changed:
- backend/workflow_executor.py: Node label lookup in aggregate_results
- backend/routers/prompts.py: Export/import/duplicate fixes
- frontend/src/components/workflow/WorkflowCanvas.jsx: Delete key
- frontend/src/utils/workflowValidation.js: Max 1 End-Node validation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 14:15:57 +02:00
Lars 2443b5ac3a Merge pull request 'Workflow Analysen V1.1' (#73) from develop into main
Deploy Production / deploy (push) Successful in 51s
Build Test / build-frontend (push) Successful in 15s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
Reviewed-on: #73
2026-04-11 12:38:08 +02:00
Lars 4c9e0e3c98 refactor: Simplify StartNode component to improve clarity
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 57s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Removed the trimming of the analysis title, now displaying only the node label or defaulting to 'Start'.
- Updated documentation to clarify the purpose of properties, specifically distinguishing between display on the canvas and analysis metadata.

These changes enhance the readability and maintainability of the StartNode component in the workflow editor.
2026-04-11 12:30:10 +02:00
Lars 0ce98e8973 feat: Enhance StartNode and Workflow Editor with analysis metadata
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / lint-backend (push) Successful in 0s
- Updated StartNode to display a trimmed analysis title if available, falling back to the label or 'Start'.
- Refactored WorkflowEditorPage to include analysis metadata (title, description, category) in the start node configuration.
- Improved serialization and deserialization of workflow graphs to handle new analysis fields.
- Enhanced user interface to allow setting and displaying analysis metadata for better clarity in the workflow editor.

These changes improve the user experience by providing clearer metadata handling in workflows and ensuring consistent display in analysis components.
2026-04-11 12:19:06 +02:00
Lars d803f39de3 feat: Refactor workflow result handling in prompts and analysis components
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 53s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
- Introduced a new utility function to streamline the extraction of user-facing content from aggregated workflow results.
- Updated backend prompt handling to utilize the new function for improved clarity and maintainability.
- Adjusted frontend analysis component to leverage the utility for consistent content display across different workflow result formats.

These changes enhance the overall user experience by ensuring more reliable and readable output from workflow executions.
2026-04-11 12:04:35 +02:00
Lars 300d96a9d8 feat: Enhance prompt execution for workflows and analysis offers
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Added support for handling aggregated results in workflow prompts, allowing for various data formats (string, object).
- Introduced a utility function to filter active prompts for both pipeline and workflow types in the analysis page.
- Updated content handling in the analysis component to accommodate new workflow data structures.

This improves the flexibility and usability of the prompt execution process in both backend and frontend components.
2026-04-11 11:42:54 +02:00
Lars 28b6fb28d5 neuer Viewport für Admin-Seiten
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
2026-04-11 11:32:46 +02:00
Lars 8b731313df Merge pull request 'Workflow V1' (#72) from develop into main
Deploy Production / deploy (push) Successful in 1m1s
Build Test / build-frontend (push) Successful in 15s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #72
2026-04-11 10:59:39 +02:00
Lars 3541c416f9 feat: Workflow Editor UX improvements - validation and notifications
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 47s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
**Problem 1: Validation footer covers mobile menu**
- Fixed bottom validation panel (z-index 1000) overlapped mobile nav
- Solution: Removed bottom panel, added inline validation in config panel header

**Problem 2: Alert dialogs for save success**
- alert() blocks UI and requires OK click for every save
- Solution: Toast notifications (auto-close after 3s, non-blocking)

**Problem 3: Validation shows only counts, not details**
- Footer showed "1 Error, 2 Warnings" without details
- Solution: Inline display shows all error/warning messages with click-to-navigate

**New Components:**
- Toast.jsx: Auto-closing notifications (success/error/warning/info)
- ConfirmDialog.jsx: Modal confirmation dialogs (for future save-on-close)

**Changes:**
- WorkflowEditorPage: Inline validation in config panel, toast state
- Removed fixed bottom .validation-panel (no mobile overlap)
- Toast for save success instead of alert()

**Still TODO (separate commit):**
- Save confirmation when closing/switching nodes with unsaved changes
- Dirty state tracking

Part 3: Inline Prompts - UX polish (validation + notifications)
2026-04-11 10:48:28 +02:00
Lars 8d89b23db1 fix: UX improvements for inline template mode switching
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 46s
Build Test / pytest-backend (push) Successful in 8s
Build Test / lint-backend (push) Successful in 0s
**Issue 1: Prompt selector loses value**
- When switching Reference→Inline→Reference, the dropdown was empty
- Root cause: Switching to Inline deleted prompt_slug
- Fix: Keep prompt_slug when activating inline_template
- Now: Both modes preserve their values independently

**Issue 2: Workflow overview shows 'kein Prompt'**
- Nodes with inline templates showed misleading 'kein Prompt' text
- Root cause: AnalysisNode only checked prompt_name/prompt_slug
- Fix: Check inline_template and display '✏️ Inline-Template'
- Now: Clear visual distinction between modes

Files:
- WorkflowEditorPage.jsx: Preserve prompt_slug on mode switch
- AnalysisNode.jsx: Add isInlineMode detection and display

Part 3: Inline Prompts - UX polish
2026-04-11 10:31:05 +02:00
Lars c0525cf2d2 feat: Extend JoinNode to support 8 input paths
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 51s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Increased from 3 to 8 input handles (path_1 through path_8)
- Evenly distributed across top edge (11%, 22%, 33%, ..., 89%)
- Backend already supports unlimited inputs dynamically
- Unused handles are ignored (no edges = no check)

Join strategies for missing inputs:
- wait_all: Fails if any connected path fails
- wait_any: Succeeds if at least 1 path succeeds
- best_effort: Always succeeds

User can still chain multiple Joins if >8 inputs needed.
Part 3: Inline Prompts - workflow flexibility
2026-04-11 10:25:28 +02:00
Lars 88f0b5a0a4 fix: Add workflow node outputs as placeholders in inline templates
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 46s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
ISSUE: Inline templates referencing node outputs ({{ node_id.analysis_core }},
{{ node_id.signal_xyz }}) were not resolved - AI received empty data from
previous workflow stages.

ROOT CAUSE: load_prompt_template() only loaded system placeholders
(name, age, etc.) but not node execution results from context['node_results'].

FIX:
- Extract node outputs from context['node_results']
- Add as placeholders: node_id.analysis_core, node_id.signal_xyz, node_id.question_xyz
- Format matches PlaceholderPicker extraction logic
- Debug logging shows which node placeholders are added

TESTING:
- System placeholder test: ✅ SUCCESS (name, age, geschlecht resolved)
- Node output placeholders: Fixed (previously missing)
- User workflow: Join → Analysis → End now receives upstream data

Part 3: Inline Prompts - placeholder resolution completion
2026-04-11 10:13:03 +02:00
Lars aeb0ee6ad9 debug: Add comprehensive placeholder resolution logging to workflow executor
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 50s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
- Log placeholder loading (count, sample keys)
- Log key cleaning process (before/after)
- Log sample values (name, age, geschlecht)
- Log template before/after resolution
- Log resolved and unresolved placeholders
- Add .strip() to key cleaning to handle spaces

This will help diagnose why {{ name }}, {{ age }}, {{ geschlecht }} are not resolving in inline templates.
Issue: Part 3 Inline Prompts - placeholder resolution debugging
2026-04-11 09:38:18 +02:00
LarsandClaude Opus 4.6 a4c8b4bd9a fix: Syntax error in WorkflowEditorPage IIFE
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
QuestionAugmentationPanel und FallbackConfig waren außerhalb der IIFE.
Müssen innerhalb sein, vor dem schließenden Fragment.

Build lokal getestet: ✓ erfolgreich

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 09:03:10 +02:00
LarsandClaude Opus 4.6 8f6d60681e fix: Inline Prompts - UX-Verbesserungen
Build Test / build-frontend (push) Failing after 11s
Deploy Development / deploy (push) Failing after 39s
Build Test / pytest-backend (push) Successful in 3s
Build Test / lint-backend (push) Successful in 0s
Problem 1: Selbst-Referenzierung verhindern
- PlaceholderPicker erhält currentNodeId prop
- Node kann sich nicht mehr selbst in Placeholders sehen
- extractWorkflowPlaceholders() filtert aktuellen Node aus

Problem 2: Radio-Button State-Management
- IIFE mit Helper-Funktion für Mode-Bestimmung
- isInlineMode/isReferenceMode basierend auf data.inline_template
- Korrekte Conditional Rendering Logic
- Beim Wechsel Reference→Inline bleibt prompt_slug erhalten
- Beim Wechsel Inline→Reference bleibt inline_template erhalten

Problem 3: Layout-Breite optimiert
- Sidebar: 250px → 220px (schmaler)
- Config Panel: 400px → 520px (breiter für bessere Lesbarkeit)
- Responsive: Config Panel bei <1200px: 450px statt 350px

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 08:58:46 +02:00
LarsandClaude Opus 4.6 65500c899b fix: Add missing WorkflowNode import in workflow_executor
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
Import war vergessen nach Umstellung von load_prompt_template() auf WorkflowNode Parameter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 08:47:45 +02:00
LarsandClaude Opus 4.6 a1723db387 feat: Workflow Engine Part 3 - Inline Prompts (v0.9q)
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Failing after 4s
Build Test / lint-backend (push) Successful in 0s
Ermöglicht Analysis Nodes zwischen zwei Prompt-Modi zu wählen:
- Reference Mode: Basis-Prompt aus DB referenzieren (bestehend)
- Inline Mode: Template direkt im Node editieren (NEU)

Frontend:
- InlineTemplateEditor Component (~80 Zeilen)
- Radio Buttons in WorkflowEditorPage für Mode-Auswahl
- Placeholder Picker für beide Modi (End Node + Inline Template)
- Cursor-Position Tracking mit textareaRef
- Conditional Rendering basierend auf promptSource
- Validation: Entweder prompt_slug ODER inline_template

Backend:
- load_prompt_template() akzeptiert ganzen WorkflowNode (statt nur slug)
- Unterstützt inline_template (Mode 1) und prompt_slug (Mode 2)
- WorkflowNode.inline_template Feld hinzugefügt
- Validation: HTTPException wenn weder slug noch template

Serialization:
- inline_template in graph_data speichern/laden
- Backward-compatible mit bestehenden Workflows

Version: 0.9q
Module: workflow 0.7.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-11 08:45:00 +02:00
Lars b453ce63c6 feat(universal-csv-import): Introduce Universal CSV Import module and related documentation
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 53s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Added the Universal CSV Import module, including the `UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` for guidelines on new import modules, executors, and templates.
- Updated relevant rules in `ARCHITECTURE.md` and `CODING_RULES.md` to include references to the new import module and its requirements.
- Revised the Gitea Issues Index to reflect the latest updates and added context for ongoing issues related to the CSV import functionality.
- Enhanced the README files to provide clearer navigation and documentation for the Universal CSV Import features.
2026-04-11 08:14:20 +02:00
Lars 12893cba78 Merge pull request 'Universal CSV Importer' (#70) from develop into main
Deploy Production / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 16s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #70
2026-04-11 07:06:47 +02:00
Lars ebca44829e fix(csv_parser): Normalize header comparison in CSV template validation
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Updated the `validate_csv_template` function to normalize both the column signature and field mappings for accurate comparison, preventing false warnings about mismatches.
- Enhanced warning messages to provide clearer guidance on the relationship between normalized signatures and raw field mappings.
- Added a new test to ensure that normalized signatures do not trigger false warnings when compared to raw mappings.
2026-04-11 06:52:17 +02:00
Lars 0629f88b37 feat(csv-templates): Add CSV template validation endpoint and enhance error handling
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / lint-backend (push) Successful in 0s
- Introduced a new endpoint for validating CSV templates without saving, allowing users to check field mappings and type conversions.
- Updated the `create_system_template` and `update_system_template` functions to include validation reports in responses.
- Enhanced error handling in CSV import processes by integrating `enrich_row_error` for more informative error messages.
- Improved the AdminCsvTemplateEditorPage to support format checking and display validation results, enhancing user experience.
- Incremented version numbers for `csv_import` and `admin_csv_templates` to reflect these updates.
2026-04-11 06:47:27 +02:00
Lars 6945b748cb feat(schema, csv_parser): Update activity log schema and parsing logic
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 3s
Build Test / lint-backend (push) Successful in 0s
- Increased precision for `kcal_active`, `kcal_resting`, `hr_avg`, and `hr_max` fields in the activity log schema.
- Added a new function `_activity_hr_bpm` to validate heart rate values during CSV import, ensuring they fall within plausible ranges.
- Updated the CSV parser to utilize the new heart rate validation function for improved data integrity.
- Enhanced the type converter to accommodate additional aliases for energy fields in CSV imports.
- Added a test to verify conversion of active energy from kJ to kcal, ensuring accurate data handling.
2026-04-11 06:41:23 +02:00
Lars 08a2485f43 refactor(csv_parser): Implement SAVEPOINT handling for activity import
Build Test / pytest-backend (push) Successful in 9s
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 47s
Build Test / lint-backend (push) Successful in 0s
- Added SAVEPOINT management to the `_import_activity` function to improve error handling during CSV imports.
- Moved the training type resolution logic to occur within a transaction block, ensuring that errors can be rolled back without affecting the entire import process.
- Enhanced error logging to capture evaluation failures, providing better insights during CSV import operations.
2026-04-11 06:31:42 +02:00
Lars 894ee1dd02 refactor(csv_parser): Update training type resolution to use existing database cursor
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Modified `_resolve_training_type_for_activity` to accept a database cursor, improving efficiency and avoiding potential deadlocks during CSV imports.
- Introduced `get_training_type_for_activity_with_cursor` to handle training type resolution with an existing cursor, streamlining database interactions.
- Updated related calls in the activity import logic to utilize the new function, ensuring consistent behavior across the application.
2026-04-11 06:27:11 +02:00
Lars a9bd3faabb Bug Fix für type_converter.py und executor.py
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
2026-04-10 16:52:11 +02:00
Lars 5b96bd4f75 feat(csv-import): Add blood pressure and activity row diagnosis functionality
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Introduced `diagnose_blood_pressure_row` and `diagnose_activity_row` functions to validate and analyze blood pressure and activity data from CSV imports.
- Updated the CSV import logic to handle combined datetime columns for blood pressure and activity, improving data integrity during import.
- Enhanced type conversion specifications to include `start_time` for blood pressure and activity, ensuring accurate data mapping.
- Added tests to validate the new diagnosis functions and their integration with existing import processes, ensuring robustness and reliability.
- Updated frontend messages to provide clearer guidance on blood pressure and activity data handling during CSV imports.
2026-04-10 16:43:00 +02:00
Lars c5b0540b11 feat(csv-import): Add CSV import diagnosis endpoint and related functionality
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 50s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Implemented a new endpoint for diagnosing CSV imports without writing to the database, allowing users to validate mappings and type conversions.
- Introduced the `diagnose_vitals_row` function to analyze vital metrics and provide detailed feedback on data validity.
- Enhanced the CSV import logic to include alias handling for vital fields, improving compatibility with different CSV formats.
- Updated the frontend to support the new diagnosis feature, including UI elements for displaying diagnosis results and error details.
- Added tests to ensure the correctness of the new diagnosis functionality and its integration with existing import processes.
2026-04-10 16:35:31 +02:00
Lars 1855f6e57a refactor(migrations): Improve idempotency and constraint handling for vitals_baseline source
Build Test / pytest-backend (push) Successful in 4s
Build Test / build-frontend (push) Successful in 18s
Deploy Development / deploy (push) Successful in 57s
Build Test / lint-backend (push) Successful in 0s
- Updated migration scripts to ensure idempotent behavior for the source CHECK constraint, allowing for consistent application even if previous migrations were partially successful.
- Enhanced SQL logic to drop existing constraints safely and re-add them, ensuring compatibility with the universal CSV import.
- Clarified comments for better understanding of migration context and functionality.
2026-04-10 16:17:35 +02:00
Lars 5a0c71dd90 feat(csv-import): Implement SAVEPOINT handling for vitals baseline import
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Failing after 1s
Build Test / lint-backend (push) Successful in 1s
- Updated the CSV import logic to include SAVEPOINT management, allowing for better error handling during the vitals baseline import process.
- Enhanced the SQL migration script to drop existing CHECK constraints related to the 'source' field, ensuring compatibility with the new universal CSV import.
- Incremented DB_SCHEMA_VERSION to "20260409c" to reflect these changes and improve the import process reliability.
2026-04-10 16:11:08 +02:00
Lars e60976e1cc chore(version): Update database schema version for CSV import enhancements
Build Test / build-frontend (push) Successful in 18s
Deploy Development / deploy (push) Successful in 51s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
- Incremented DB_SCHEMA_VERSION to "20260409b" to reflect changes related to the vitals_baseline.source CSV migration.
- Updated comments to clarify the migration context for better maintainability.
2026-04-10 16:05:51 +02:00
Lars b7cd710c32 feat(csv-import): Enhance row aggregation and validation features
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 47s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 0s
- Updated the aggregate_mapped_rows function to support multiple row policies, allowing for flexible handling of duplicate keys during CSV imports.
- Introduced deduplication of identical rows before aggregation, improving data integrity.
- Enhanced validation for multi_row_policy and dedupe_identical_rows in import_row_processing specifications.
- Updated the AdminCsvTemplateEditorPage to include options for multi-row policies and deduplication settings, improving user experience in template management.
- Added comprehensive tests to validate new aggregation behaviors and ensure correct error handling for multiple rows.
2026-04-10 15:36:12 +02:00
Lars ad7aa2d255 feat(csv-import): Add custom row aggregation options in AdminCsvTemplateEditorPage
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 55s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
- Introduced a new section for row aggregation settings, allowing users to customize aggregation functions for imported CSV data.
- Implemented functionality for users to save custom aggregation configurations and select key fields for aggregation.
- Enhanced user interface with detailed instructions and options for managing row aggregation, improving overall usability in template management.
2026-04-10 15:26:59 +02:00
Lars a51ee1d304 feat(csv-import): Update versioning and enhance row processing features
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 3s
Build Test / lint-backend (push) Successful in 0s
- Bumped version numbers for csv_import to 0.3.1 and admin_csv_templates to 0.2.0, reflecting recent enhancements.
- Added support for import_row_processing_default in the CSV modules endpoint, improving data handling capabilities.
- Introduced new row aggregation operations in the AdminCsvTemplateEditorPage, allowing for more flexible data processing options.
- Implemented parsing and validation for custom row processing configurations, enhancing user experience in template management.
2026-04-10 15:22:31 +02:00
Lars e35d167055 feat(csv-import): Enhance CSV import processing and validation
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 56s
Build Test / pytest-backend (push) Successful in 4s
Build Test / lint-backend (push) Successful in 1s
- Updated the CSV import logic to support new row processing specifications for weight and vitals baseline, allowing for better data aggregation and validation.
- Implemented handling for multiple rows on the same day, enabling aggregation of values such as averages for vitals and last values for weight.
- Enhanced test coverage for the new import functionalities, ensuring correct behavior during data processing and validation.
- Refactored the module registry to include default import row processing options for better flexibility in handling CSV data.
2026-04-10 15:09:34 +02:00
Lars c0fcdea1fe refactor(csv-import): Enhance nutrition data processing and template rendering
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 48s
Build Test / pytest-backend (push) Successful in 3s
Build Test / lint-backend (push) Successful in 0s
- Updated the nutrition import logic to utilize a new row processing specification, improving data aggregation and validation.
- Refactored the template rendering process in the workflow executor to use Jinja2's Environment with ChainableUndefined for better handling of missing attributes.
- Added backward-compatible shortcuts for accessing decision signals in node contexts, enhancing flexibility in template usage.
- Introduced import row processing options in CSV templates, allowing for more customizable data handling during imports.
2026-04-10 11:56:43 +02:00
Lars 8b67f7ab55 refactor(csv-import): Simplify test execution and enhance custom equivalence handling
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 57s
Build Test / pytest-backend (push) Failing after 4s
Build Test / lint-backend (push) Successful in 0s
- Updated the test execution command in the CI workflow to run all tests excluding slow ones, improving efficiency.
- Enhanced the AdminCsvTemplateEditorPage to support custom equivalence for unit conversions, allowing for more flexible data handling.
- Added markers in pytest configuration for categorizing tests, facilitating better test management.
2026-04-10 11:38:54 +02:00
Lars 8ee9fb84ba fix(metadata): Update extraction logic and enhance circumference detection
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / pytest-backend (push) Successful in 3s
Build Test / lint-backend (push) Successful in 0s
- Adjusted the extract_value_raw function to return failure for unavailable values in strict mode.
- Expanded the circumference detection logic in infer_unit_strict to include additional terms for better accuracy in unit inference.
2026-04-10 11:25:38 +02:00
Lars fe7a69fb07 feat(csv-import): Enhance source unit handling and custom conversion options
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 52s
Build Test / pytest-backend (push) Failing after 2s
Build Test / lint-backend (push) Successful in 0s
- Updated the source_unit_choices_for_field function to include a custom option for user-defined conversion factors, improving flexibility in unit conversions.
- Modified the AdminCsvTemplateEditorPage to support custom conversion factors, allowing users to input specific scaling factors for their data.
- Added tests to ensure the custom option is correctly included in the source unit choices and functions as expected in the template editor.
2026-04-10 11:19:44 +02:00
Lars bb6eefc837 fix(csv-import): Normalize source unit representation and update CI workflows
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 49s
Build Test / pytest-backend (push) Failing after 3s
Build Test / lint-backend (push) Successful in 0s
- Changed source unit representation from "kJ" to "kj" for consistency across CSV templates and migrations.
- Updated CI workflow to enhance testing conditions, ensuring tests run in the correct environment based on deployment context.
- Improved job steps for backend testing and syntax checking by utilizing deployed application directories, streamlining the CI process.
2026-04-10 10:42:59 +02:00
Lars 0d0ab62674 feat(workflows): Update CI configuration and enhance testing conditions
Build Test / pytest-backend-csv (push) Failing after 24s
Build Test / build-frontend (push) Successful in 1m25s
Deploy Development / deploy (push) Successful in 48s
Build Test / lint-backend (push) Failing after 3s
- Added workflow_run triggers for "Deploy Development" and "Deploy Production" to ensure tests run only after successful deployments.
- Updated Python version in CI from 3.12 to 3.11 for better compatibility with the Debian 12 ARM64 runner.
- Enhanced job conditions to skip tests on failed workflow runs.
- Improved frontend build process by updating Node.js setup and ensuring correct directory navigation.
- Refined CSV parsing logic to handle custom and unknown source units, enhancing conversion flexibility.
- Added new tests for custom source unit handling in CSV conversions, ensuring accurate processing.
2026-04-10 10:27:59 +02:00
Lars d6d7e738a5 feat(csv-import): Refactor CSV import logic and enhance data handling
Build Test / pytest-backend-csv (push) Failing after 3s
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 48s
Build Test / lint-backend (push) Successful in 0s
- Updated the CSV import architecture to clarify the distinction between import and data layer responsibilities, as outlined in the new section of ARCHITECTURE.md.
- Enhanced the build_row_after_mapping function to include module-specific context for improved data processing.
- Introduced source unit options in the admin CSV template editor to facilitate user-defined conversions, improving flexibility in handling various data formats.
- Added new tests to validate the handling of source units and ensure accurate conversions during CSV imports.
- Updated module definitions to include unit specifications for nutritional and activity data fields, enhancing data integrity.
2026-04-10 09:54:32 +02:00
Lars 41cc0ed2a8 feat(csv-import): Enhance Apple sleep CSV import functionality
Build Test / pytest-backend-csv (push) Failing after 4s
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 54s
Build Test / lint-backend (push) Successful in 0s
- Integrated date parsing improvements using dateutil for better handling of various date formats in sleep data.
- Added total sleep hours to the nights dictionary for comprehensive sleep analysis.
- Updated the import logic to handle cases where sleep duration is zero, providing appropriate warnings.
- Enhanced the CSV import interface to detect Apple sleep CSV format and provide user feedback on template selection.
- Improved the admin CSV template editor to accommodate new sleep import requirements and clarify usage instructions.
2026-04-10 07:52:04 +02:00
Lars 26ab11eb7b feat(csv-import): Enhance CSV import functionality with new modules and tests
Build Test / pytest-backend-csv (push) Failing after 1m4s
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 55s
Build Test / lint-backend (push) Successful in 0s
- Added support for new CSV import modules: sleep and vitals_baseline, expanding the import capabilities.
- Implemented backend logic for handling CSV imports related to sleep and vitals baseline, including error handling and data processing.
- Updated frontend components to include new modules in the CSV import interface, improving user experience.
- Introduced unit tests for the new import functionalities to ensure reliability and correctness.
- Enhanced existing CSV analysis features to accommodate the new modules, ensuring consistent behavior across the application.
2026-04-10 07:30:48 +02:00
Lars b4cc3cb934 feat(csv-parser): Introduce header signature ranking metrics for enhanced CSV analysis
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 52s
Build Test / lint-backend (push) Successful in 0s
- Added new functions for calculating header signature recall and ranking metrics, improving the analysis of CSV templates.
- Updated existing CSV analysis endpoints to utilize the new ranking metrics, enhancing the accuracy of template matching.
- Refactored related code to replace Jaccard score calculations with the new metrics, providing a more comprehensive evaluation of CSV structure.
- Improved documentation for new functions to clarify their purpose and usage in the context of CSV template analysis.
2026-04-10 07:08:21 +02:00
Lars c10da55ec6 feat(csv-templates): Introduce CSV template analysis and validation features
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 56s
Build Test / lint-backend (push) Successful in 0s
- Added a new endpoint for analyzing uploaded CSV files, providing suggestions for field mappings and type conversions.
- Implemented validation for required field targets to ensure all mandatory fields are mapped correctly.
- Enhanced the admin CSV templates interface with new routes and navigation options in the frontend.
- Updated API utility functions to support the new CSV analysis functionality.
- Improved error handling for CSV uploads, including file size and row count checks.
2026-04-10 06:39:41 +02:00
Lars 338163ac0b feat(csv-parser): Enhance CSV parsing with header normalization and flexible date handling
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 53s
Build Test / lint-backend (push) Successful in 0s
- Added a new function to strip header keys of unwanted characters, improving CSV import consistency.
- Updated CSV row iteration to utilize the new header normalization function, ensuring cleaner data processing.
- Enhanced date parsing capabilities to support flexible formats, accommodating various date representations in CSV files.
- Introduced additional tests to validate the new header normalization and date parsing functionalities.
2026-04-10 06:23:46 +02:00
Lars 5e5f3b4e5a feat(csv-import): Update CSV import functionality and enhance analysis features
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 50s
Build Test / lint-backend (push) Successful in 0s
- Bumped version of csv_import to 0.3.0, reflecting new analysis capabilities.
- Modified analyze_csv endpoint to allow optional module filtering, improving flexibility in template selection.
- Enhanced the import process to support both system and user-defined templates, ensuring backward compatibility.
- Updated frontend to streamline mapping choices and improve user experience during CSV analysis and import.
- Added detailed error handling and user feedback for import operations.
2026-04-10 06:15:21 +02:00
Lars 7e9da46fe5 feat(csv-import): Add Universal CSV Import page and navigation tile
Build Test / build-frontend (push) Successful in 16s
Deploy Development / deploy (push) Successful in 57s
Build Test / lint-backend (push) Successful in 1s
- Introduced a new route for the Universal CSV Import page in App.jsx.
- Added a corresponding navigation tile in captureNav.js for easy access to the CSV import functionality.
2026-04-10 06:10:06 +02:00
Lars 66979f3f51 feat(api): Rename importCsv to importUniversalCsv for clarity and documentation
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 55s
Build Test / lint-backend (push) Successful in 0s
- Updated the function name from importCsv to importUniversalCsv to better reflect its purpose.
- Added documentation comment to clarify the parameters used in the universal CSV import process.
2026-04-10 06:05:19 +02:00
Lars 851018b3b9 feat(csv_import): Enhance CSV import functionality with new endpoint and parsing improvements
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 52s
Build Test / lint-backend (push) Successful in 0s
- Updated version for csv_import to 0.2.0, reflecting new features.
- Implemented a new POST endpoint for universal CSV import, supporting nutrition, weight, and blood pressure modules.
- Added CSV parsing function to yield rows as dictionaries for easier data handling.
- Enhanced error handling and logging for import operations.
- Introduced tests for the new CSV parsing functionality to ensure reliability.
2026-04-10 06:03:21 +02:00
Lars 36417bfdf3 refactor: Rename csv_import to data_import and update foreign key references
Build Test / build-frontend (push) Successful in 17s
Deploy Development / deploy (push) Successful in 49s
Build Test / lint-backend (push) Successful in 0s
- Changed feature ID from 'csv_import' to 'data_import' in the features table.
- Updated foreign key references in tier_limits, user_feature_restrictions, user_feature_usage, and widget_feature_requirements.
- Removed the old 'csv_import' feature entry after ensuring all references are updated.
- Simplified the migration process by using an INSERT with ON CONFLICT for the new feature entry.
2026-04-09 21:42:11 +02:00
Lars 4a771f6a83 feat(csv-parser): Implement CSV import functionality with mapping and type conversion
Build Test / build-frontend (push) Successful in 15s
Deploy Development / deploy (push) Successful in 50s
Build Test / lint-backend (push) Successful in 0s
- Added permissions for editing and deleting CSV field mappings.
- Created type converter for CSV cells to handle various data types.
- Implemented database migrations for CSV field mappings and import logs.
- Seeded initial system templates for nutrition and activity data imports.
- Developed admin endpoints for managing system CSV templates.
- Introduced user endpoints for CSV import analysis and mapping retrieval.
- Added tests for core CSV parser functionalities, including delimiter detection and value conversion.
2026-04-09 21:37:19 +02:00
Lars 09439ad1a5 Merge pull request 'Indvidual Dashboard V0.9' (#67) from develop into main
Deploy Production / deploy (push) Successful in 1m0s
Build Test / build-frontend (push) Successful in 15s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #67
2026-04-08 10:56:02 +02:00
Lars 75736dadec Merge pull request 'feat: Integrate caliper data enrichment and weight loading in API responses' (#64) from develop into main
Deploy Production / deploy (push) Successful in 57s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #64
2026-04-06 06:10:39 +02:00
Lars 8d56c352fc Merge pull request 'Bug Fixes' (#63) from develop into main
Deploy Production / deploy (push) Successful in 50s
Build Test / build-frontend (push) Successful in 15s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #63
2026-04-05 17:54:00 +02:00
Lars ebea9946a7 Merge pull request 'feat: Add admin access denial alert to Dashboard with timeout handling' (#62) from develop into main
Build Test / build-frontend (push) Successful in 13s
Deploy Production / deploy (push) Successful in 55s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #62
2026-04-05 11:29:03 +02:00
Lars 9fb04af7df Merge pull request 'Responsive Gui - partially Workflow' (#61) from develop into main
Deploy Production / deploy (push) Failing after 1s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #61
2026-04-05 11:27:43 +02:00
Lars 3f5b9b8b58 Merge pull request 'Platzhaltersystem für Ernährung, Körper und Aktivity' (#60) from develop into main
Deploy Production / deploy (push) Successful in 51s
Build Test / build-frontend (push) Successful in 15s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #60
2026-04-03 09:28:13 +02:00
Lars 5bbea036c2 Merge pull request 'develop' (#59) from develop into main
Deploy Production / deploy (push) Successful in 53s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #59
2026-04-03 08:23:48 +02:00
Lars c762495b6f Merge pull request 'Goals -System refactored - Platzhaltersystem enhanced (als draft)' (#53) from develop into main
Deploy Production / deploy (push) Successful in 51s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #53
2026-03-31 11:46:47 +02:00
Lars 979e734bd9 Merge pull request 'Bug Fix' (#52) from develop into main
Deploy Production / deploy (push) Successful in 50s
Build Test / build-frontend (push) Successful in 15s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #52
2026-03-27 22:16:18 +01:00
Lars 79cb3e0100 Merge pull request 'V 0.9h dynamic focus area system' (#51) from develop into main
Deploy Production / deploy (push) Successful in 52s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #51
2026-03-27 21:14:40 +01:00
Lars 9d22e7e8af Merge pull request 'Goalsystem V1' (#50) from develop into main
Deploy Production / deploy (push) Successful in 49s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #50
2026-03-27 17:40:50 +01:00
Lars f46c367c27 Merge pull request 'Flexibles KI Prompt System' (#48) from develop into main
Deploy Production / deploy (push) Successful in 50s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #48
2026-03-26 14:49:47 +01:00
Lars ac4c6760d7 Merge pull request 'globaler Filter für Qualitätsgates von Trainings' (#41) from develop into main
Deploy Production / deploy (push) Successful in 52s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 1s
Reviewed-on: #41
2026-03-24 08:44:22 +01:00
Lars 5a6a140dfd Merge pull request 'Bugfixes: Vitals Import (German columns + decimal values)' (#23) from develop into main
Deploy Production / deploy (push) Successful in 56s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
2026-03-23 16:52:27 +01:00
Lars 931012c16b Merge pull request 'v9d Phase 2d: Vitals Module Refactoring (Baseline + Blood Pressure)' (#22) from develop into main
Deploy Production / deploy (push) Successful in 54s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
2026-03-23 16:27:03 +01:00
Lars 03f4b871a9 Merge pull request 'Production Release: RestDays Widget + Trainingstyp Fix' (#16) from develop into main
Deploy Production / deploy (push) Successful in 52s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Merge pull request #16: Production Release
2026-03-23 09:24:17 +01:00
Lars 601fc80178 Merge pull request 'WP 9c Phase 1' (#12) from develop into main
Deploy Production / deploy (push) Successful in 49s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #12
2026-03-22 14:14:34 +01:00
Lars 3ff2a1bf45 Merge pull request 'Abschluss 9c' (#11) from develop into main
Deploy Production / deploy (push) Successful in 54s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #11
2026-03-21 21:20:10 +01:00
Lars 51aa57f304 Merge pull request 'Final Feature 9c' (#10) from develop into main
Deploy Production / deploy (push) Successful in 49s
Build Test / build-frontend (push) Successful in 14s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #10
2026-03-21 12:41:41 +01:00
Lars ca4411f30f Merge pull request 'fix: update version string to v9c' (#9) from develop into main
Deploy Production / deploy (push) Successful in 49s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #9
2026-03-21 09:00:55 +01:00
Lars b551365fb5 Merge pull request 'Membership-System und Bug Fixing (inkl. Nutrition)' (#8) from develop into main
Deploy Production / deploy (push) Failing after 1s
Build Test / build-frontend (push) Successful in 12s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #8
2026-03-21 08:48:56 +01:00
Lars 272c123952 Merge pull request '9c Phase 2' (#6) from develop into main
Deploy Production / deploy (push) Successful in 50s
Build Test / build-frontend (push) Successful in 12s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #6
2026-03-19 14:59:25 +01:00
Lars 9387670a7b Merge pull request '9c datatables' (#5) from develop into main
Deploy Production / deploy (push) Successful in 55s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #5
2026-03-19 13:00:31 +01:00
Lars d4a8401a6a Merge pull request 'Refactored Main.py' (#4) from develop into main
Deploy Production / deploy (push) Successful in 48s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #4
2026-03-19 11:44:32 +01:00
Lars aaf88a6f12 Merge pull request 'fix: Migration-Fehler - meas_id Spalte in ai_insights' (#3) from develop into main
Deploy Production / deploy (push) Successful in 49s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #3
2026-03-19 08:41:13 +01:00
Lars b789c1bd44 Merge pull request 'bug Fix Login' (#2) from develop into main
Deploy Production / deploy (push) Successful in 55s
Build Test / build-frontend (push) Successful in 13s
Build Test / lint-backend (push) Successful in 1s
Reviewed-on: #2
2026-03-19 08:29:48 +01:00
Lars 5062aa8068 docker-compose.yml aktualisiert
Deploy Production / deploy (push) Successful in 52s
Build Test / build-frontend (push) Successful in 12s
Build Test / lint-backend (push) Successful in 0s
2026-03-19 08:21:12 +01:00
Lars 8a042589e7 docker-compose.yml aktualisiert
Deploy Production / deploy (push) Failing after 1s
Build Test / build-frontend (push) Successful in 12s
Build Test / lint-backend (push) Successful in 1s
2026-03-19 08:19:19 +01:00
Lars 3898b5ad45 docker-compose.yml aktualisiert
Deploy Production / deploy (push) Failing after 38s
Build Test / build-frontend (push) Successful in 12s
Build Test / lint-backend (push) Successful in 0s
2026-03-19 08:06:02 +01:00
Lars 9d15336144 Merge pull request 'Version 9b' (#1) from develop into main
Deploy Production / deploy (push) Failing after 42s
Build Test / build-frontend (push) Successful in 12s
Build Test / lint-backend (push) Successful in 0s
Reviewed-on: #1
2026-03-19 08:04:01 +01:00
132 changed files with 17863 additions and 1517 deletions

No files matched your search

+1
View File
@@ -12,6 +12,7 @@ Dieser Ordner ist der **primäre Orientierungspunkt** für Claude Code / Cursor-
| 2 | **`rules/DOCUMENTATION.md`** – Ablage- und Dokumentationsregeln |
| 3 | `rules/ARCHITECTURE.md`, `rules/CODING_RULES.md`, `rules/LESSONS_LEARNED.md` |
| 4 | Issue-Landkarte: **`.claude/docs/GITEA_ISSUES_INDEX.md`** |
| 5 | **Universal CSV Import** (Modul/Executor/Vorlagen): **`docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`** (unter `.claude/`) |
Themen mit UI/Nav/PWA: siehe `../docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md` (im **Projekt**-`docs/`, nicht hier).
+2 -2
View File
@@ -1,6 +1,6 @@
# Gitea Issues – Landkarte (Auswertung)
**Quelle:** Gitea `Lars/mitai-jinkendo`, Stand **2026-04-08** (Abfrage `state=all`).
**Quelle:** Gitea `Lars/mitai-jinkendo`, Stand **2026-04-11** (Abfrage `state=all`, ergänzt: #71, #76).
**URL:** http://192.168.2.144:3000/Lars/mitai-jinkendo/issues
Dieses Dokument ist ein **Orientierungs-Index** für Agenten und Entwickler. Verbindliches Tracking bleibt **in Gitea**; hier: Kategorien, Dubletten-Hinweise, grobe Prioritätseinschätzung.
@@ -88,7 +88,7 @@ Dieses Dokument ist ein **Orientierungs-Index** für Agenten und Entwickler. Ver
| # | Titel |
|---|--------|
| 15 | [FEAT-002] Quality-Filter für KI-Auswertungen & Charts integrieren |
| 21 | [FEATURE] Universeller CSV-Parser mit lernbarem Feldmapping |
| 76 | Trainings-Qualität: zielbezogene Logik + Listen-Filter statt globalem „Hochwertig“-Hide |
| 36 | BUG-009: Trainingstyp-Erstellung führt zu Internal Server Error |
---
+3 -1
View File
@@ -54,6 +54,7 @@ _Dieser Ordner `.claude/docs/` ist per `.gitignore`-Ausnahme **versioniert** (Sp
| Platzhalter / Registry | `technical/PLACEHOLDER_REGISTRY_FRAMEWORK.md`, `technical/PLACEHOLDER_DEVELOPMENT_GUIDE.md` | `backend/placeholder_registrations/`, `backend/placeholder_resolver.py` |
| Dashboard-Lab-Widgets | `technical/DASHBOARD_WIDGETS_AGENT_GUIDE.md` | Widget-Katalog + Registrierung (siehe Guide) |
| Training Profiler / Resolver | `technical/TRAINING_PROFILE_RESOLVER_LAYER1.md`, `functional/TRAINING_TYPE_PROFILES.md` | Resolver-Module wie im Guide genannt |
| Universal CSV Import | `technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` | `backend/csv_parser/`, `routers/csv_import.py`, `routers/admin_csv_templates.py` |
| Mitgliedschaft / Features | `technical/MEMBERSHIP_SYSTEM.md`, `architecture/FEATURE_ENFORCEMENT.md` | `backend/auth.py`, Feature-Logging, Router mit Enforcement |
---
@@ -111,6 +112,7 @@ _Dieser Ordner `.claude/docs/` ist per `.gitignore`-Ausnahme **versioniert** (Sp
| `PROFILE_REFERENCE_VALUES.md` | Profil-Referenzwerte |
| `TRAINING_PROFILE_RESOLVER_LAYER1.md` | Training-Resolver Schicht 1 |
| `TRAINING_TYPE_PROFILES_TECHNICAL.md` | Trainingsprofile technisch |
| `UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` | Universal CSV: Registry, Executor, Vorlagen, Agent-Checkliste |
| `V9D_PHASE2_VITALS_SLEEP.md` | v9d Vitalwerte/Schlaf (Release-Bezug) |
---
@@ -174,4 +176,4 @@ Siehe [`audit/README.md`](./audit/README.md).
---
**Letzte Aktualisierung:** 8. April 2026 (Struktur-Index, Duplikatbereinigung, Abgleich-Hinweise)
**Letzte Aktualisierung:** 9. April 2026 (Universal CSV Agent-Guide, Abgleich-Tabelle)
@@ -92,16 +92,10 @@ registry = get_registry()
**Package:** `backend/placeholder_registrations/`
**Struktur:**
```
placeholder_registrations/
├── __init__.py # Auto-Import aller Registrations
├── nutrition_part_a.py # Nutrition Basis-Metriken (4 Placeholder)
├── nutrition_part_b.py # Protein-Ziele (5 Placeholder) - TODO
├── body_metrics.py # Körper-Metriken - TODO
├── activity_metrics.py # Aktivitäts-Metriken - TODO
└── ... # Weitere Cluster
```
**Struktur:** Vollständige Cluster-Module (u. a. Ernährung, Körper, Aktivität, Schlaf,
Vitalwerte, Profil/Zeitraum, Phase-0b-Ziele, Korrelationen); siehe `__init__.py` für die
Import-Liste. **Anzahl:** 114 Platzhalter, identisch zu `PLACEHOLDER_MAP` in
`placeholder_resolver.py`.
**Auto-Registration:**
- Import des Package triggert automatische Registrierung aller Placeholder
@@ -0,0 +1,63 @@
# Universal CSV Import – Agent-Leitfaden
**Stand:** 2026-04-09 · **Kontext:** Issue #21 (Universeller CSV-Parser), Prod-Migrationen u. a. 051–053.
Dieses Dokument ist **normativ für Agenten**, die ein neues Import-Zielmodul anlegen oder bestehende Import-Pfade (Executor, Vorlagen, DB) ändern.
---
## 1. Architektur (Kurz)
| Komponente | Pfad / Rolle |
|------------|----------------|
| Modul-Definitionen | `backend/csv_parser/module_registry.py` (`MODULE_DEFINITIONS`) |
| Typ-/Einheiten-Konvertierung | `backend/csv_parser/type_converter.py`, `field_units.py` |
| Zeilen-Aggregation (z. B. Ernährung pro Tag) | `backend/csv_parser/import_row_processing.py` |
| Import-Ausführung | `backend/csv_parser/executor.py` |
| Fehlertexte / Transaktions-Hinweise | `backend/csv_parser/import_errors.py` (`enrich_row_error`) |
| Admin-Systemvorlagen | `backend/routers/admin_csv_templates.py` |
| Nutzer-Import (Profil-Mappings) | `backend/routers/csv_import.py` |
| Vorlagen-Validierung (strukturell + Sample) | `backend/csv_parser/template_validator.py` (`validate_csv_template`) |
**Single Source of Truth** für erlaubte Zielfelder, Typen und Duplikat-Keys ist **`module_registry.py`**. Keine parallele Feldliste in Routern duplizieren.
---
## 2. Checkliste: Neues Zielmodul
1. **`MODULE_DEFINITIONS`** um Eintrag erweitern: `table`, `fields` (Typen `date` / `datetime` / `float` / `int` / `string`), `duplicate_key`, `duplicate_strategy`, ggf. `derive_date_from_datetime_field`, `import_mode` (Spezialpfade wie Schlaf).
2. **DB:** Migration nur nach Projektregel (`backend/migrations/NNN_*.sql`). Spaltenbreiten/Typen so wählen, dass importierte Werte (z. B. kJ→kcal, große Energiebeträge) **keinen NUMERIC-Overflow** verursachen.
3. **`source` / CHECK-Constraints:** Wenn die Zieltabelle `source` hat, muss der Wert **`csv`** (oder der vereinbarte Import-Tag) in der DB erlaubt sein (Migration anpassen, nicht nur App-Code).
4. **Executor:** Einfügen/Aktualisieren in `executor.py` nur über bestehende Muster (ein Cursor, **kein** verschachteltes `get_db()` im gleichen Request). Bei mehreren Zeilen pro Transaktion: bei **Zeilenfehlern** SAVEPOINT pro Zeile nutzen (siehe Activity-Pattern), damit die Transaktion nicht dauerhaft abgebrochen ist.
5. **Trainingstyp / FK-Auflösung:** DB-Zugriffe für abhängige Entitäten (z. B. `get_training_type_for_activity_with_cursor`) **mit dem gleichen Cursor** wie der Import – keine zweite Connection aus dem Importpfad.
6. **Vorlagen:** System-Templates in Migration/Seed pflegen (`csv_field_mappings`, `is_system=true`). `type_conversions` und `source_unit` dort setzen, wo Einheiten aus Exporten abweichen (z. B. Apple kJ).
7. **Validierung:** Neue/angepasste Admin-Vorlagen müssen **`validate_csv_template`** passieren (Create/Update liefert bei Fehlern **422** mit `validation`). Tests für Randfälle ergänzen (`tests/test_template_validator.py` o. ä.).
8. **API / Frontend:** Neue Admin-Endpunkte in `main.py` registrieren; Frontend **nur** über `api.js`. Bei strukturierten FastAPI-Fehlern (`detail` als Objekt/Liste) bestehende Hilfen (`formatFastApiDetail`) nutzen.
---
## 3. Checkliste: Bestehendes Modul ändern
- Schema-Änderung: Migration + ggf. **`module_registry`**-Felder anpassen.
- Neue Spalte im Import: Executor-Mapping, optional `type_conversions` / Validator.
- Änderung an Duplikatlogik: `duplicate_key` / `ON CONFLICT`-Pfad im Executor prüfen.
- Datums-/Zeit-Parsing: **`type_converter`** – ISO-Daten `YYYY-MM-DD` konsistent (**`dayfirst=False`**), Zeiten `HH:MM` ohne Sekunden unterstützen wo nötig.
---
## 4. Bekannte Einschränkungen (Follow-up in Gitea)
- Admin **„Format prüfen“** kann `import_row_processing` derzeit weglassen; volle Parität mit dem gespeicherten Template erst beim Speichern / echten Import.
- Nutzer-Mappings (Copy aus Systemvorlage) laufen nicht automatisch durch **`validate_csv_template`** – Tracking: **Gitea #71** (http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/71).
---
## 5. Verwandte Regeln
- `.claude/rules/ARCHITECTURE.md` – Router, DB, `source`-Tracking
- `.claude/rules/CODING_RULES.md` – Kurzverweis Universal CSV
- `.claude/rules/DOCUMENTATION.md` – Ablage technischer Specs
---
**Version:** 1.0
+13
View File
@@ -0,0 +1,13 @@
Folgende Ergebnisse des Tests:
- Valididierung gibt immer noch keine Aufschlüsse was für Fehler und Warning es sind, Es zeigt immer nur noch die Anzahl der entsprechenden Fehler/Warnungen
- Speichern als kurzes PopUp -- gut
- In der Node selbst wird nun eine Fehlermeldung ausgegeben. Das ist gut. In größen Workflows aber schwierig den Fehler zu lokalisieren.
- In der automatischen Zusammenfassung in der Endnode kommt als Überschrift, z.B. Node 10, anstatt den Node-Name auszugeben.
- Alle Änderungen an Nodes scheinen automatisch in den Gesamtflow übernommen zu werden. Diese werden dann nach dem Speichern aktiv. Da muss man sehr vorsichtig sein, bei kurzen Änderungen und dem Ausprobieren.
- Der Testlauf "Execute" sollte auf dem aktuellen Workflowstand ausgeführt werden, auch wenn dieser vom gespeicherten Abweicht. Ich würde natürlich vor dem Speichern den Workflow testen können. Prüfe und bewerte diesen Punkt, setze ihn aber noch nicht um.
- Das löschen von Knoten und Kanten funktioniert aktuell nur über Backspace nicht über entfernen
- Wir sollten auch dafür sorgen, dass jeweils nur eine Start-Node, End-Node in einem Workflow existiert, Prüfe ob mehrere End-Nodes sinnvoll sind, da wir ja auch Logik-Pfade abbilden und ggf. auch eine route beschreiten, die ein anderes Ende hat. (Prüfe, ob das heute schon möglich wäre!)
- Als zukünftige Ausbaustufe sollten wir überlegen, ob wir auch Trigger implementieren, z.B. um Kurzstatements zu generieren, wenn neue Daten hereinkommen und wir diese Bewertungen aktualisieren wollen
- Exportieren aller KI-Prompts/Templates/Workflows im Admin --> KI-Prompts führt zu einem "internal Server Error", Importieren konnte daraufhin nicht getestet werden
- Das duplizieren von Workflows funktioniert nicht
-
@@ -0,0 +1,458 @@
-- Migration XXX: CSV Parser - System Templates Seed Data
-- Legt Standard-Import-Konfigurationen für bekannte CSV-Formate an
-- Diese Templates sind für alle User verfügbar (is_system = true, profile_id = NULL)
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- NUTRITION (Ernährung)
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- 1. FDDB Export (Deutsch)
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'nutrition',
'FDDB Export (Standard)',
'Standard-Format für FDDB.de CSV-Exporte (Deutsch). Delimiter Semikolon, kJ → kcal Konvertierung.',
ARRAY['datum_tag_monat_jahr_stunde_minute', 'fett_g', 'kh_g', 'kj', 'protein_g']::TEXT[],
';',
'utf-8',
true,
'{
"datum_tag_monat_jahr_stunde_minute": "date",
"kj": "kcal",
"fett_g": "fat_g",
"kh_g": "carbs_g",
"protein_g": "protein_g"
}'::JSONB,
'{
"date": {
"type": "date",
"format": "dd.mm.yyyy HH:MM",
"extract": "date_only"
},
"kcal": {
"type": "float",
"source_unit": "kj",
"decimal_separator": ","
},
"fat_g": {
"type": "float",
"decimal_separator": ","
},
"carbs_g": {
"type": "float",
"decimal_separator": ","
},
"protein_g": {
"type": "float",
"decimal_separator": ","
}
}'::JSONB
);
-- 2. MyFitnessPal Export (English)
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'nutrition',
'MyFitnessPal Export',
'Standard CSV export from MyFitnessPal (English)',
ARRAY['Carbohydrates (g)', 'Calories', 'Date', 'Fat (g)', 'Protein (g)']::TEXT[],
',',
'utf-8',
true,
'{
"Date": "date",
"Calories": "kcal",
"Fat (g)": "fat_g",
"Carbohydrates (g)": "carbs_g",
"Protein (g)": "protein_g"
}'::JSONB,
'{
"date": {
"type": "date",
"format": "yyyy-mm-dd"
},
"kcal": {
"type": "float",
"decimal_separator": "."
},
"fat_g": {
"type": "float",
"decimal_separator": "."
},
"carbs_g": {
"type": "float",
"decimal_separator": "."
},
"protein_g": {
"type": "float",
"decimal_separator": "."
}
}'::JSONB
);
-- 3. Cronometer Export
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'nutrition',
'Cronometer Export',
'Cronometer daily nutrition export (English)',
ARRAY['Day', 'Energy (kcal)', 'Fat (g)', 'Net Carbs (g)', 'Protein (g)']::TEXT[],
',',
'utf-8',
true,
'{
"Day": "date",
"Energy (kcal)": "kcal",
"Fat (g)": "fat_g",
"Net Carbs (g)": "carbs_g",
"Protein (g)": "protein_g"
}'::JSONB,
'{
"date": {
"type": "date",
"format": "yyyy-mm-dd"
},
"kcal": {"type": "float", "decimal_separator": "."},
"fat_g": {"type": "float", "decimal_separator": "."},
"carbs_g": {"type": "float", "decimal_separator": "."},
"protein_g": {"type": "float", "decimal_separator": "."}
}'::JSONB
);
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- ACTIVITY (Aktivität)
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- 1. Apple Health Workout Export (English)
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'activity',
'Apple Health Workout Export (English)',
'Apple Health CSV-Export für Workouts (English). Automatisches Training-Type-Mapping.',
ARRAY['Active Energy (kcal)', 'Distance (km)', 'Duration', 'End', 'Heart Rate Average (bpm)', 'Start', 'Workout Type']::TEXT[],
',',
'utf-8',
true,
'{
"Workout Type": "activity_type",
"Start": "start_time",
"End": "end_time",
"Duration": "duration_min",
"Distance (km)": "distance_km",
"Active Energy (kcal)": "kcal_active",
"Heart Rate Average (bpm)": "hr_avg"
}'::JSONB,
'{
"start_time": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_and_time"
},
"end_time": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS"
},
"duration_min": {
"type": "duration",
"format": "HH:MM:SS",
"target_unit": "minutes"
},
"distance_km": {
"type": "float",
"decimal_separator": "."
},
"kcal_active": {
"type": "float",
"decimal_separator": "."
},
"hr_avg": {
"type": "int"
}
}'::JSONB
);
-- 2. Apple Health Workout Export (Deutsch)
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'activity',
'Apple Health Workout Export (Deutsch)',
'Apple Health CSV-Export für Workouts (Deutsch). Automatisches Training-Type-Mapping.',
ARRAY['Aktive Energie (kcal)', 'Dauer', 'Durchschnittliche Herzfrequenz (bpm)', 'Ende', 'Start', 'Strecke (km)', 'Trainingsart']::TEXT[],
',',
'utf-8',
true,
'{
"Trainingsart": "activity_type",
"Start": "start_time",
"Ende": "end_time",
"Dauer": "duration_min",
"Strecke (km)": "distance_km",
"Aktive Energie (kcal)": "kcal_active",
"Durchschnittliche Herzfrequenz (bpm)": "hr_avg"
}'::JSONB,
'{
"start_time": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_and_time"
},
"end_time": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS"
},
"duration_min": {
"type": "duration",
"format": "HH:MM:SS",
"target_unit": "minutes"
},
"distance_km": {
"type": "float",
"decimal_separator": ","
},
"kcal_active": {
"type": "float",
"decimal_separator": ","
},
"hr_avg": {
"type": "int"
}
}'::JSONB
);
-- 3. Garmin Connect Export
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'activity',
'Garmin Connect Export',
'Garmin Connect activity CSV export (English)',
ARRAY['Activity Type', 'Avg HR', 'Calories', 'Date', 'Distance', 'Duration', 'Time']::TEXT[],
',',
'utf-8',
true,
'{
"Activity Type": "activity_type",
"Date": "date",
"Time": "start_time",
"Duration": "duration_min",
"Distance": "distance_km",
"Calories": "kcal_active",
"Avg HR": "hr_avg"
}'::JSONB,
'{
"date": {
"type": "date",
"format": "yyyy-mm-dd"
},
"start_time": {
"type": "time",
"format": "HH:MM:SS"
},
"duration_min": {
"type": "duration",
"format": "HH:MM:SS",
"target_unit": "minutes"
},
"distance_km": {
"type": "float",
"decimal_separator": "."
},
"kcal_active": {
"type": "float",
"decimal_separator": "."
},
"hr_avg": {
"type": "int"
}
}'::JSONB
);
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- BLOOD PRESSURE (Blutdruck)
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- 1. Omron Export (Deutsch)
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'blood_pressure',
'Omron Export (Deutsch)',
'Omron Blutdruckmessgerät CSV-Export (Deutsch)',
ARRAY['Datum', 'Diastolisch (mmHg)', 'Puls (bpm)', 'Systolisch (mmHg)', 'Zeit']::TEXT[],
',',
'utf-8',
true,
'{
"Datum": "measured_date",
"Zeit": "measured_time",
"Systolisch (mmHg)": "systolic",
"Diastolisch (mmHg)": "diastolic",
"Puls (bpm)": "pulse"
}'::JSONB,
'{
"measured_date": {
"type": "date",
"format": "dd.mm.yyyy"
},
"measured_time": {
"type": "time",
"format": "HH:MM"
},
"systolic": {"type": "int"},
"diastolic": {"type": "int"},
"pulse": {"type": "int"}
}'::JSONB
);
-- 2. Omron Export (English)
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'blood_pressure',
'Omron Export (English)',
'Omron blood pressure monitor CSV export (English)',
ARRAY['Date', 'Diastolic (mmHg)', 'Pulse (bpm)', 'Systolic (mmHg)', 'Time']::TEXT[],
',',
'utf-8',
true,
'{
"Date": "measured_date",
"Time": "measured_time",
"Systolic (mmHg)": "systolic",
"Diastolic (mmHg)": "diastolic",
"Pulse (bpm)": "pulse"
}'::JSONB,
'{
"measured_date": {
"type": "date",
"format": "mm/dd/yyyy"
},
"measured_time": {
"type": "time",
"format": "HH:MM"
},
"systolic": {"type": "int"},
"diastolic": {"type": "int"},
"pulse": {"type": "int"}
}'::JSONB
);
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- WEIGHT (Gewicht)
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- 1. Apple Health Weight Export
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'weight',
'Apple Health Weight Export',
'Apple Health body mass CSV export',
ARRAY['Body Mass (kg)', 'Start']::TEXT[],
',',
'utf-8',
true,
'{
"Start": "date",
"Body Mass (kg)": "weight"
}'::JSONB,
'{
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only"
},
"weight": {
"type": "float",
"decimal_separator": "."
}
}'::JSONB
);
-- 2. Withings Export
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
) VALUES (
NULL,
true,
'weight',
'Withings Export',
'Withings smart scale CSV export (weight, body fat, muscle mass)',
ARRAY['Body Fat (%)', 'Date', 'Muscle Mass (kg)', 'Weight (kg)']::TEXT[],
',',
'utf-8',
true,
'{
"Date": "date",
"Weight (kg)": "weight"
}'::JSONB,
'{
"date": {
"type": "date",
"format": "yyyy-mm-dd"
},
"weight": {
"type": "float",
"decimal_separator": "."
}
}'::JSONB
);
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-- SUMMARY
-- ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
DO $$
DECLARE
template_count INTEGER;
BEGIN
SELECT COUNT(*) INTO template_count FROM csv_field_mappings WHERE is_system = true;
RAISE NOTICE '✓ CSV Parser: % System-Templates created', template_count;
RAISE NOTICE ' - Nutrition: 3 (FDDB, MyFitnessPal, Cronometer)';
RAISE NOTICE ' - Activity: 3 (Apple Health DE/EN, Garmin)';
RAISE NOTICE ' - Blood Pressure: 2 (Omron DE/EN)';
RAISE NOTICE ' - Weight: 2 (Apple Health, Withings)';
END $$;
File diff suppressed because it is too large. Load diff
@@ -7,7 +7,7 @@
## Gesamt-Übersicht
**Aktuelle Platzhalter:** 116
**Aktuelle Platzhalter:** 114 (PLACEHOLDER_MAP / Registry)
**Nach Phase 0c Migration:**
- ✅ **Bleiben einfach (kein Data Layer):** 8 Platzhalter
- 🔄 **Gehen zu Data Layer:** 108 Platzhalter
+41 -9
View File
@@ -216,8 +216,11 @@ updated_at TIMESTAMP DEFAULT NOW()
Tabellen die Daten aus externen Quellen empfangen brauchen:
```sql
source VARCHAR(50) DEFAULT 'manual'
-- Werte: 'manual' | 'apple_health' | 'garmin' | 'withings'
-- Werte u. a.: 'manual' | 'apple_health' | 'garmin' | 'withings' | 'csv'
```
Importe über den **Universal CSV**-Pfad setzen `source = 'csv'`, sofern die Tabelle ein `source`-Feld hat; CHECK-Constraints und Migrationen müssen diesen Wert erlauben.
**Agent-Pflicht bei neuen Import-Zielen oder Executor-Änderungen:** `.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`
Manuelle Einträge (`source = 'manual'`) haben IMMER Vorrang bei Reimport:
```sql
@@ -384,26 +387,55 @@ Dev-URL: dev.mitai.jinkendo.de
---
## 8. Test-Regeln
## 8. CSV-Import vs. Data Layer (Issue #53)
### 8.1 Tests schreiben ist Pflicht
### 8.1 Leitlinie: Wo Interpretation stattfindet
| Schicht | Erlaubt | Nicht Sinn der Schicht |
|--------|---------|-------------------------|
| **Import (Ingest)** | Zuordnung CSV→Speicherfeld, **Typ-/Einheits-Konvertierung** (`type_conversions`), Duplikat-/Constraint-Logik | Fachliche **Interpretation**, Aggregation von „Bedeutung“, Metriken für Auswertung |
| **Data Layer (Issue #53, Layer 1+)** | Daten lesen, aufbereiten, ableiten, für Charts/KI/Prompts bereitstellen | — |
Verbindlich: **Semantik und Auswertung** nicht dauerhaft im Import verstecken; neue Features werden an dieser Grenze geprüft.
**Detail & Zielbild (Multi-Layer, Single Source of Truth):** `docs/issues/issue-53-phase-0c-multi-layer-architecture.md`
**Umsetzung Schlaf-Import (Refactoring, Offen):** Gitea http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/69
### 8.2 Ist-Einordnung Import-Pfade (Übergang)
Bis sukzessive auf das Zielbild umgestellt ist, gilt:
| Pfad | Einordnung |
|------|----------------|
| Universal-CSV (`csv_parser`, `routers/csv_import.py`, Executor für u. a. Gewicht/Ernährung/Blutdruck/Aktivität/Vitals) | **Zielrichtung:** Mapping + Typkonvertierung |
| Apple-Schlaf-Aggregat (`csv_parser/sleep_apple_import.py`, `import_mode: apple_sleep_aggregate`) | **Legacy-Adapter** (quellenspezifische Aufbereitung) – Austausch gegen mapping-nah + Layer 1 geplant |
| Dedizierte Import-Endpoints (z. B. `/api/activity/import-csv`, Vitals Apple) | **Legacy/Parallel** – neue Quellen bevorzugt über Universal-Pfad + Vorlagen |
Änderungen an Import-Pfaden: Legacy nur erweitern mit **expliziter** Issue-/Review-Begründung; kein neues „wir rechnen Auswertung beim Insert“ ohne Data-Layer-Bezug.
---
## 9. Test-Regeln
### 9.1 Tests schreiben ist Pflicht
Jedes neue Feature bekommt mindestens einen Playwright-Test in
`tests/dev-smoke-test.spec.js`.
### 8.2 Reihenfolge: Test vor Commit
### 9.2 Reihenfolge: Test vor Commit
```
Implementieren → Tests schreiben → Tests grün → Committen
NIEMALS: Implementieren → Committen → Tests später
```
### 8.3 Claude Code schreibt Tests selbst
### 9.3 Claude Code schreibt Tests selbst
Nach jeder Implementierung:
1. Passende Tests in dev-smoke-test.spec.js ergänzen
2. `npx playwright test` ausführen
3. Fehler korrigieren bis alle Tests grün
4. Erst dann committen
### 8.4 Test-Kategorien
### 9.4 Test-Kategorien
```javascript
// UI-Test (Playwright)
test('FEATURE: Beschreibung', async ({ page }) => { ... })
@@ -412,18 +444,18 @@ test('FEATURE: Beschreibung', async ({ page }) => { ... })
test('API: Endpoint', async ({ request }) => { ... })
```
### 8.5 Screenshots bei Fehlern
### 9.5 Screenshots bei Fehlern
Fehlgeschlagene Tests erzeugen automatisch Screenshots in:
`test-results/TESTNAME/test-failed-1.png`
→ Immer ansehen bevor Code geändert wird
### 8.6 Prod nie testen
### 9.6 Prod nie testen
Tests laufen IMMER gegen dev.mitai.jinkendo.de
NIEMALS gegen mitai.jinkendo.de
---
## 9. Dashboard-Lab-Widgets und Feature-System
## 10. Dashboard-Lab-Widgets und Feature-System
**Kontext:** Dashboard-Widgets (`backend/widget_catalog.py`, Lab unter `/api/app/...`) und das **Subscription-/Feature-Modell** (`features`, `tier_limits`, `check_feature_access` in `backend/auth.py`) sind **getrennte Schichten**, müssen aber bei tariffrelevanten Widgets **verknüpft** werden.
+339
View File
@@ -0,0 +1,339 @@
# Architektur-Regeln – Mitai Jinkendo
> **PFLICHTLEKTÜRE für Claude Code vor jeder Implementierung.**
> Diese Regeln sind verbindlich und dürfen nicht ohne explizite
> Genehmigung des Nutzers abgeändert werden.
---
## 1. Router-Architektur
### 1.1 Ein Modul = Ein Router
Jedes fachliche Modul hat genau eine Router-Datei in `backend/routers/`.
```
backend/routers/
├── auth.py # Authentifizierung
├── profiles.py # Nutzerprofile
├── weight.py # Gewichts-Tracking
├── sleep.py # Schlaf-Modul
├── training_types.py # Trainingstypen + HF
└── ... # je neues Modul = neue Datei
```
**Regeln:**
- Kein Endpoint darf außerhalb seines thematischen Routers definiert werden
- Neue Module immer als neue Router-Datei anlegen, nie in bestehende einfügen
- Router in `main.py` registrieren: `app.include_router(modul.router, prefix="/api")`
- Router-Datei-Name = Modul-Name in `version.py` MODULE_VERSIONS
### 1.2 API-First Prinzip
Jede Funktion ist zuerst als API-Endpoint implementiert – die UI nutzt ausschließlich
diese Endpoints über `api.js`. Keine Business-Logik im Frontend.
```python
# ✅ Richtig: Logik im Backend-Endpoint
@router.get("/sleep/stats")
def get_sleep_stats(session=Depends(require_auth)):
# Berechnung hier
return {"avg_duration": ..., "sleep_debt": ...}
# ❌ Falsch: Berechnung im Frontend
const sleepDebt = entries.reduce((sum, e) => sum + (goal - e.duration), 0)
```
### 1.3 Einheitliche Fehlerbehandlung
```python
# ✅ Immer dieses Format:
raise HTTPException(status_code=404, detail="Eintrag nicht gefunden")
# Response: {"detail": "Eintrag nicht gefunden"}
# ❌ Nie eigene Formate:
return {"error": "not found"}
return {"message": "Fehler", "success": False}
```
---
## 2. Versionskontrollsystem
### 2.1 Versionierungsschema
**Semantic Versioning: `MAJOR.MINOR.PATCH`**
| Typ | Wann | Beispiel |
|-----|------|---------|
| MAJOR | Breaking Change, DB-Migration inkompatibel | 9.0.0 → 10.0.0 |
| MINOR | Neues Feature, neues Modul | 9.2.0 → 9.3.0 |
| PATCH | Bugfix, kleine Änderung, Refactor | 9.3.0 → 9.3.1 |
### 2.2 Versions-Dateien
**Backend: `backend/version.py`**
```python
APP_VERSION = "9.3.0"
BUILD_DATE = "2026-03-22"
MODULE_VERSIONS = {
"auth": "1.2.0",
"profiles": "1.1.0",
"weight": "1.0.3",
"circumference": "1.0.1",
"caliper": "1.0.1",
"activity": "1.1.0",
"nutrition": "1.0.2",
"photos": "1.0.0",
"insights": "1.3.0",
"prompts": "1.1.0",
"admin": "1.2.0",
"stats": "1.0.1",
"exportdata": "1.1.0",
"importdata": "1.0.0",
"membership": "2.1.0",
}
CHANGELOG = [
{
"version": "9.3.0",
"date": "2026-03-22",
"changes": [
"Feature: Sleep Module (sleep_log, JSONB-Segmente)",
"Feature: Vitalwerte-Seite in Navigation",
"Feature: Trainingstypen-Kategorisierung",
]
},
{
"version": "9.2.1",
"date": "2026-03-20",
"changes": [
"Fix: Feature-Enforcement Rollback",
"Fix: Erholungsstatus-Gewichtung korrigiert",
]
},
]
```
**Frontend: `frontend/src/version.js`**
```javascript
export const APP_VERSION = "9.3.0"
export const BUILD_DATE = "2026-03-22"
export const PAGE_VERSIONS = {
Dashboard: "1.3.0",
LoginScreen: "1.1.0",
WeightPage: "1.0.3",
ActivityPage: "1.2.0",
NutritionPage: "1.1.0",
AnalysisPage: "1.3.0",
SettingsPage: "1.4.0",
AdminPanel: "1.2.0",
SubscriptionPage: "1.0.0",
// Neue Seiten hier eintragen
}
```
### 2.3 Versions-Endpoint
**`GET /api/version`** – öffentlich (kein Auth erforderlich)
```json
{
"app_version": "9.3.0",
"build_date": "2026-03-22",
"backend_version": "9.3.0",
"modules": {
"auth": "1.2.0",
"sleep": "1.0.0"
},
"db_schema_version": "20260322",
"environment": "production"
}
```
Dieser Endpoint wird in `backend/routers/version.py` implementiert und liest
direkt aus `version.py`.
### 2.4 Versions-Anzeige in der App
**Settings-Seite – Versions-Panel:**
```
System-Versionen
─────────────────────────────────────
App (gesamt) 9.3.0
Backend 9.3.0 ✓ erreichbar
Frontend 9.3.0 ✓ geladen
DB-Schema 20260322
Umgebung production
─────────────────────────────────────
Module
auth 1.2.0
sleep 1.0.0
membership 2.1.0
[alle Module...]
─────────────────────────────────────
[Changelog] [Cache leeren]
```
Frontend ruft beim Laden der Settings-Seite `/api/version` ab und vergleicht
mit der eigenen `APP_VERSION` aus `version.js`. Bei Abweichung: Warnung anzeigen.
### 2.5 Pflicht-Regel: Versions-Bump bei jedem Commit
**Jede Code-Änderung erfordert:**
1. Versions-Bump in `backend/version.py` (APP_VERSION + betroffenes MODULE_VERSION)
2. Versions-Bump in `frontend/src/version.js` (APP_VERSION + betroffene PAGE_VERSION)
3. Changelog-Eintrag in `backend/version.py` CHANGELOG
**Claude Code prüft das im `/deploy` Command automatisch.**
Kein Commit ohne Versions-Bump – keine Ausnahme.
### 2.6 DB-Schema-Version
Format: `YYYYMMDD` (Datum der letzten Migration)
Gespeichert in `backend/version.py`:
```python
DB_SCHEMA_VERSION = "20260322"
```
Bei jeder Schema-Änderung (ALTER TABLE, neue Tabelle) → DB_SCHEMA_VERSION aktualisieren.
---
## 3. Datenbankregeln
### 3.1 Pflichtfelder für neue Tabellen
```sql
-- Jede neue Tabelle braucht:
id SERIAL PRIMARY KEY,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
```
### 3.2 Source-Tracking bei Import-Daten
Tabellen die Daten aus externen Quellen empfangen brauchen:
```sql
source VARCHAR(50) DEFAULT 'manual'
-- Werte: 'manual' | 'apple_health' | 'garmin' | 'withings'
```
Manuelle Einträge (`source = 'manual'`) haben IMMER Vorrang bei Reimport:
```sql
-- Reimport überschreibt nur nicht-manuelle Einträge:
INSERT INTO sleep_log (...) ON CONFLICT (profile_id, date)
DO UPDATE SET ... WHERE sleep_log.source != 'manual'
```
### 3.3 Profile-ID Isolation
Jede Tabelle mit Nutzerdaten hat `profile_id` als Foreign Key.
Kein Endpoint gibt Daten eines anderen Profils zurück.
Profile-ID kommt IMMER aus der Session, nie aus Request-Parametern.
### 3.4 Boolean-Werte
```sql
-- PostgreSQL Boolean (nicht SQLite 0/1):
WHERE active = true ✓
WHERE active = 1 ✗
```
---
## 4. Frontend-Regeln
### 4.1 Alle API-Calls über api.js
```javascript
// ✅ Richtig:
import { api } from '../utils/api'
const data = await api.listSleep()
// ❌ Falsch:
const r = await fetch('/api/sleep')
```
### 4.2 Neue Seite = Eintrag in PAGE_VERSIONS
Jede neue Seite in `frontend/src/version.js` registrieren.
### 4.3 CSS-Variablen statt Hardcoded-Farben
```javascript
// ✅ Richtig:
style={{color: 'var(--accent)'}}
// ❌ Falsch:
style={{color: '#1D9E75'}}
```
### 4.4 Fehlerbehandlung in allen async Funktionen
```javascript
try {
const data = await api.meinEndpoint()
setData(data)
} catch(e) {
setError(e.message)
} finally {
setLoading(false)
}
```
---
## 5. Git & Deployment-Regeln
### 5.1 Nie direkt auf main pushen
Immer über Pull Request in Gitea: develop → main.
develop Branch niemals löschen.
### 5.2 Commit-Message Format
```
feat: neues Feature oder Modul
fix: Bugfix
refactor: Umbau ohne Funktionsänderung
docs: Dokumentation
version: Versions-Bump
ci: CI/CD Änderungen
chore: Maintenance
```
### 5.3 Versions-Bump im Commit
```
feat: Sleep Module v1.0.0
- sleep_log Tabelle mit JSONB-Segmenten
- Import aus Apple Health CSV
- Korrelationen Schlaf <-> Ruhepuls
version: 9.3.0 (backend + frontend)
module: sleep 1.0.0
```
---
## 6. Dokumentations-Regeln
### 6.1 Neue Module dokumentieren
Bei jedem neuen Modul:
1. Fachliche Spec: `.claude/docs/functional/MODUL_NAME.md`
2. Technische Spec: `.claude/docs/technical/MODUL_NAME.md`
3. Nach Fertigstellung: `.claude/library/` aktualisieren
### 6.2 CLAUDE.md aktuell halten
Nach größeren Änderungen CLAUDE.md Versions-Tabelle aktualisieren.
### 6.3 Lessons Learned dokumentieren
Jeder Rollback oder schwerer Bug → Eintrag in `.claude/rules/LESSONS_LEARNED.md`
---
## Zusammenfassung: Checkliste vor jedem Commit
```
[ ] Versions-Bump in backend/version.py (APP_VERSION + MODULE)
[ ] Versions-Bump in frontend/src/version.js (APP_VERSION + PAGE)
[ ] Changelog-Eintrag in backend/version.py
[ ] DB_SCHEMA_VERSION aktualisiert (wenn Schema geändert)
[ ] Neues Modul in PAGE_VERSIONS / MODULE_VERSIONS eingetragen
[ ] Auth auf alle neuen Endpoints (require_auth)
[ ] Fehlerformat einheitlich (HTTPException mit detail)
[ ] Neue Tabellen haben created_at + updated_at
[ ] Import-Tabellen haben source-Feld
[ ] api.js für alle Frontend API-Calls
```
+7
View File
@@ -39,6 +39,13 @@ from slowapi import Limiter
def sensitive(request: Request, ...):
```
### 6. Universal CSV Import / Admin-Vorlagen
Neues **Import-Zielmodul**, Änderungen an **`csv_parser`**, Executor, DB-`source`/`CHECK`, oder System-CSV-Vorlagen:
- Pflichtlektüre und Checkliste: **`.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`**
- Keine zweite DB-Connection im Importpfad; Zeilenfehler ohne „aborted transaction“ (SAVEPOINT-Muster wo nötig)
- Admin Create/Update von Systemvorlagen: Validierung über `validate_csv_template` nicht umgehen
## Frontend
### 1. api.js für alle API-Calls
+65 -5
View File
@@ -3,22 +3,82 @@ name: Build Test
on:
push:
branches: [main, develop]
workflow_run:
workflows: ["Deploy Development", "Deploy Production"]
types: [completed]
jobs:
lint-backend:
pytest-backend:
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Check backend syntax
- name: Run backend pytest suite in deployed container
run: |
python3 -m py_compile /home/lars/docker/bodytrack/backend/main.py
EVENT_NAME="${{ github.event_name }}"
REF_NAME="${{ github.ref_name }}"
RUN_WORKFLOW="${{ github.event.workflow_run.name }}"
APP_DIR="/home/lars/docker/bodytrack"
COMPOSE_FILE="docker-compose.yml"
if [ "$EVENT_NAME" = "workflow_run" ]; then
if [ "$RUN_WORKFLOW" = "Deploy Development" ]; then
APP_DIR="/home/lars/docker/bodytrack-dev"
COMPOSE_FILE="docker-compose.dev-env.yml"
fi
elif [ "$REF_NAME" = "develop" ]; then
APP_DIR="/home/lars/docker/bodytrack-dev"
COMPOSE_FILE="docker-compose.dev-env.yml"
fi
cd "$APP_DIR"
docker compose -f "$COMPOSE_FILE" exec -T backend sh -lc "
pip install -r /app/requirements-dev.txt &&
cd /app &&
python -m pytest tests -m 'not slow' -ra -vv --tb=short
"
lint-backend:
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Check backend syntax on deployed app
run: |
EVENT_NAME="${{ github.event_name }}"
REF_NAME="${{ github.ref_name }}"
RUN_WORKFLOW="${{ github.event.workflow_run.name }}"
APP_DIR="/home/lars/docker/bodytrack"
if [ "$EVENT_NAME" = "workflow_run" ]; then
if [ "$RUN_WORKFLOW" = "Deploy Development" ]; then
APP_DIR="/home/lars/docker/bodytrack-dev"
fi
elif [ "$REF_NAME" = "develop" ]; then
APP_DIR="/home/lars/docker/bodytrack-dev"
fi
python3 -m py_compile "$APP_DIR/backend/main.py"
echo "✓ Backend syntax OK"
build-frontend:
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Build frontend
- name: Build frontend on deployed app
run: |
cd /home/lars/docker/bodytrack/frontend
EVENT_NAME="${{ github.event_name }}"
REF_NAME="${{ github.ref_name }}"
RUN_WORKFLOW="${{ github.event.workflow_run.name }}"
APP_DIR="/home/lars/docker/bodytrack"
if [ "$EVENT_NAME" = "workflow_run" ]; then
if [ "$RUN_WORKFLOW" = "Deploy Development" ]; then
APP_DIR="/home/lars/docker/bodytrack-dev"
fi
elif [ "$REF_NAME" = "develop" ]; then
APP_DIR="/home/lars/docker/bodytrack-dev"
fi
cd "$APP_DIR/frontend"
npm install
npm run build
echo "✓ Frontend build OK"
+23
View File
@@ -8,6 +8,7 @@
> | Coding-Regeln | `.claude/rules/CODING_RULES.md` |
> | Lessons Learned | `.claude/rules/LESSONS_LEARNED.md` |
> | **Gitea-Landkarte (lokal gepflegt)** | **`.claude/docs/GITEA_ISSUES_INDEX.md`** |
> | **Universal CSV Import** (neues Modul / Executor / Vorlagen) | **`.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md`** |
> | **GUI / IA / Admin / Nav / PWA-Leiste** | **`docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md`** |
> | **Dashboard-Lab-Widgets** (Katalog, Registrierung, `config`) | **`.claude/docs/technical/DASHBOARD_WIDGETS_AGENT_GUIDE.md`** |
> | **Agent-Einstieg** | **`.claude/README.md`** |
@@ -98,6 +99,28 @@ frontend/src/
**Branch:** develop
**Nächster Schritt:** Frontend Chart Integration → Testing → Prod Deploy v0.9i
### Updates (09.04.2026 - Universal CSV Import, Prod-Migration abgeschlossen)
- **Agent-Leitfaden:** `.claude/docs/technical/UNIVERSAL_CSV_IMPORT_AGENT_GUIDE.md` (Checkliste für neue Import-Module, Executor, Vorlagen, `source=csv`, SAVEPOINT-/Cursor-Regeln)
- **Regeln:** Verweise in `.claude/rules/ARCHITECTURE.md` (§3.2 `source`), `.claude/rules/CODING_RULES.md` (§6)
- **Follow-ups:** **Gitea #71** – Dry-Run inkl. `import_row_processing`, Nutzer-Mapping-Validierung, Fehler-Hints in der Import-UI ([Issue](http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/71))
### Updates (11.04.2026 - Placeholder Phase A)
- **`main.py`:** `import placeholder_registrations` beim Start, damit die Registry (**114 Keys**, deckungsgleich `PLACEHOLDER_MAP`) und `get_placeholder_catalog()` ohne vorherigen Export-Request konsistent sind.
- **`placeholder_resolver.py`:** `{{top_goal_progress_pct}}` nutzt `_safe_int` statt `_safe_str` (Verdrahtung zu `scores.get_top_priority_goal` korrigiert).
### Updates (11.04.2026 - Gitea #75, nutrition_score Registry)
- **Gitea #75** (offen): Zucker/Ballaststoffe/Lebensmittelqualität, automatisches Lebensmittelprofil, später Mahlzeiten-Timing/Abgleich mit Training — http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/75
- **`nutrition_score`:** Registry in `backend/placeholder_registrations/nutrition_score.py`, Import in `placeholder_registrations/__init__.py`; Legacy-Duplikat unter „Scores“ im Platzhalter-Katalog entfernt.
### Updates (11.04.2026 - Ernährung: TDEE, Bilanz, Kalorien-Score)
- **`data_layer/nutrition_metrics.py`:** TDEE für Bilanz: primär **Mifflin–St Jeor BMR × PAL 1,55**, wenn Profil (Größe, Geschlecht, DOB) und Gewicht vorhanden; sonst Fallback **kg × 32,5** (`estimate_tdee_kcal_from_latest_weight`). `get_energy_balance_data` / `calculate_energy_balance_7d` nutzen **tägliche kcal-Summen**. **`_score_calorie_adherence`** (Komponente von `calculate_nutrition_score`) wertet die 7-Tage-Bilanz nach **`profiles.goal_mode`** aus (weight_loss vs. strength/recomposition vs. maintenance/health/endurance).
- **`routers/charts.py`:** `/charts/energy-balance` und Protein-Timeline nutzen dieselbe TDEE-/Tageslogik; ohne `weight_log` liefert Energiebilanz-Chart eine klare Fehlermeldung. Adherence-Endpoint: Kcal-CV über **Tages-Summen**.
- **Doku:** Normative Platzhalter-Zahl **114** (`docs/PLACEHOLDER_*.md`); `placeholder_metadata_complete.py` als **Legacy** gekennzeichnet — maßgeblich `placeholder_registrations/` + `PLACEHOLDER_REGISTRY_FRAMEWORK.md`.
### GUI / Informationsarchitektur (Abnahme dieser Iteration, 2026-04-05)
Admin-Bereich (`AdminShell`, Hub-Routen), Hauptnavigation inkl. **Ziele** (`config/appNav.js`), Einstellungen nur aktives Profil + E-Mail, KI-Analyse Ergebnis in rechter Spalte, **PWA** Bottom-Nav inkl. iOS Safe Area. Zentrale Agent-Doku: **`docs/issues/GUI_IA_ADMIN_NAV_2026-04-05.md`**. Responsive-Epic **Gitea #30:** Phasenplan `docs/issues/PHASE_PLAN_RESPONSIVE_UI.md` — **P7 Kern erledigt**, **P8** (Regression/Abnahme) ausstehend; Issue bewusst **nicht** geschlossen.
+18 -11
View File
@@ -509,17 +509,24 @@ def calculate_sleep_quality_7d(profile_id: str) -> Optional[int]:
quality_scores = []
for s in sleep_data:
if s['deep_minutes'] and s['rem_minutes']:
quality_pct = ((s['deep_minutes'] + s['rem_minutes']) / s['duration_minutes']) * 100
# 40-60% deep+REM is good
if quality_pct >= 45:
quality_scores.append(100)
elif quality_pct >= 35:
quality_scores.append(75)
elif quality_pct >= 25:
quality_scores.append(50)
else:
quality_scores.append(30)
dur = s["duration_minutes"]
if not dur or dur <= 0:
continue
d = s["deep_minutes"]
r = s["rem_minutes"]
if d is None and r is None:
continue
di, ri = (d or 0), (r or 0)
quality_pct = ((di + ri) / dur) * 100
# 40-60% deep+REM is good
if quality_pct >= 45:
quality_scores.append(100)
elif quality_pct >= 35:
quality_scores.append(75)
elif quality_pct >= 25:
quality_scores.append(50)
else:
quality_scores.append(30)
if not quality_scores:
return None
+27
View File
@@ -0,0 +1,27 @@
"""Universal CSV import foundation (Issue #21)."""
from csv_parser.core import (
decode_raw_bytes,
sniff_delimiter,
parse_csv_sample,
column_signature,
normalize_header_for_signature,
)
from csv_parser.module_registry import MODULE_DEFINITIONS, get_module_definition, list_modules
from csv_parser.type_converter import convert_value, build_row_after_mapping
from csv_parser.permissions import user_may_delete_mapping, user_may_edit_mapping_row
__all__ = [
"decode_raw_bytes",
"sniff_delimiter",
"parse_csv_sample",
"column_signature",
"normalize_header_for_signature",
"MODULE_DEFINITIONS",
"get_module_definition",
"list_modules",
"convert_value",
"build_row_after_mapping",
"user_may_delete_mapping",
"user_may_edit_mapping_row",
]
+208
View File
@@ -0,0 +1,208 @@
"""
CSV bytes → text, delimiter sniffing, strukturierte Erstzeilen für Analyse (Issue #21).
"""
from __future__ import annotations
import csv
import io
import re
from typing import Any, Dict, Iterator, List, Sequence, Tuple
_DEFAULT_DELIMS = [",", ";", "\t"]
def decode_raw_bytes(raw: bytes) -> str:
"""UTF-8 bevorzugt, Fallback Latin-1; BOM entfernen."""
if not raw:
return ""
for enc in ("utf-8-sig", "utf-8", "latin-1"):
try:
text = raw.decode(enc)
break
except UnicodeDecodeError:
text = ""
continue
else:
text = raw.decode("utf-8", errors="replace")
if text.startswith("\ufeff"):
text = text[1:]
return text
def sniff_delimiter(sample_line: str) -> str:
"""
Heuristik: Zähle Vorkommen der Kandidaten in der ersten Datenzeile.
Kein csv.Sniffer (robuster gegen kurze Zeilen).
"""
if not sample_line or not sample_line.strip():
return ","
best = ","
best_count = -1
for d in _DEFAULT_DELIMS:
c = sample_line.count(d)
if c > best_count:
best_count = c
best = d
return best
def _split_first_lines(text: str, max_lines: int = 5) -> List[str]:
lines: List[str] = []
for line in text.splitlines():
if line.strip():
lines.append(line)
if len(lines) >= max_lines:
break
return lines
def parse_csv_sample(
text: str,
delimiter: str | None = None,
has_header: bool = True,
max_data_rows: int = 5,
) -> Tuple[List[str], List[dict[str, str]], str]:
"""
Gibt (headers, rows_as_dicts, verwendetes_delimiter) zurück.
rows sind Rohstrings pro Zelle.
"""
lines = _split_first_lines(text, max_lines=50)
if not lines:
return [], [], ","
delim = delimiter if delimiter is not None else sniff_delimiter(lines[0])
reader = csv.reader(io.StringIO(text.replace("\r\n", "\n").replace("\r", "\n")), delimiter=delim)
rows_raw: List[List[str]] = []
for i, row in enumerate(reader):
if i >= 1 + max_data_rows + (1 if has_header else 0):
break
if not any(c.strip() for c in row):
continue
rows_raw.append(row)
if not rows_raw:
return [], [], delim
if has_header:
headers = [h.strip() for h in rows_raw[0]]
data = rows_raw[1 : 1 + max_data_rows]
else:
n = len(rows_raw[0])
headers = [f"col_{i}" for i in range(n)]
data = rows_raw[:max_data_rows]
dict_rows: List[dict[str, str]] = []
for r in data:
row_dict: dict[str, str] = {}
for j, h in enumerate(headers):
row_dict[h] = r[j].strip() if j < len(r) else ""
dict_rows.append(row_dict)
return headers, dict_rows, delim
def normalize_header_for_signature(name: str) -> str:
s = name.strip().lower()
s = re.sub(r"\s+", "_", s)
s = re.sub(r"[^a-z0-9_äöüß().%-]+", "_", s)
return s.strip("_")
def column_signature(headers: List[str]) -> List[str]:
"""Sortierte normalisierte Spaltennamen für Signatur-Vergleich."""
return sorted({normalize_header_for_signature(h) for h in headers if h is not None and str(h).strip()})
def headers_signature_match_score(sig_csv: List[str], sig_template: List[str]) -> float:
"""Jaccard-Überlappung 0..1 (|A∩B|/|A∪B|). Fällt stark, wenn die CSV viele Zusatzspalten hat."""
a, b = set(sig_csv), set(sig_template)
if not a and not b:
return 1.0
if not a or not b:
return 0.0
inter = len(a & b)
union = len(a | b)
return inter / union if union else 0.0
def headers_signature_template_recall(sig_csv: Sequence[str], sig_template: Sequence[str]) -> float:
"""
Anteil der Template-Spalten (Signatur), die in der CSV vorkommen: |A∩B|/|B|.
100 %, sobald alle für die Vorlage relevanten Spalten in der Datei sind — unabhängig von
Zusatzspalten (Gewicht + Ernährung in einer Datei erzeugt keinen „Abzug“ für die jeweilige Vorlage).
"""
a = set(sig_csv)
b = {normalize_header_for_signature(str(x)) for x in sig_template}
b.discard("")
if not b:
return 1.0 if not a else 0.0
inter = len(a & b)
return inter / len(b)
def headers_signature_rank_metrics(sig_csv: List[str], sig_template: List[str]) -> dict[str, Any]:
"""
Einheitliche Kennzahlen für Vorlagen-Ranking und UI.
confidence = template_recall (empfohlen für Anzeige / Sortierung primär).
"""
a = set(sig_csv)
b = {normalize_header_for_signature(str(x)) for x in sig_template}
b.discard("")
inter = a & b
n_inter = len(inter)
n_b = len(b)
n_a = len(a)
union = len(a | b)
template_recall = n_inter / n_b if n_b else (1.0 if not n_a else 0.0)
jaccard = n_inter / union if union else 0.0
return {
"confidence": round(template_recall, 4),
"template_recall": round(template_recall, 4),
"jaccard": round(jaccard, 4),
"columns_matched": n_inter,
"columns_in_template": n_b,
"columns_in_csv": n_a,
}
def get_csv_import_limits(conn_row: dict | None) -> dict[str, int]:
"""Liest Limits aus system_config.csv_import; Fallback bei fehlendem Key."""
defaults = {"max_rows_per_file": 50_000, "max_file_bytes": 52_428_800}
if not conn_row or "value" not in conn_row:
return defaults
val = conn_row["value"]
if isinstance(val, dict):
out = {**defaults, **{k: int(v) for k, v in val.items() if k in defaults}}
return out
return defaults
def _strip_header_key(k: str | None) -> str:
if k is None:
return ""
return str(k).strip().removeprefix("\ufeff")
def iter_csv_dict_rows(
text: str,
delimiter: str,
*,
has_header: bool = True,
) -> Iterator[Dict[str, str]]:
"""
Vollständige Datei zeilenweise als Dict (Header = Keys).
Spaltenreihenfolge ist egal; zusätzliche Spalten werden ignoriert, wenn sie nicht
in field_mappings vorkommen. Keine Obergrenze für die Spaltenanzahl (nur Zeilenlimits
kommen aus system_config / Import-Router).
"""
if not has_header:
raise ValueError("CSV ohne Kopfzeile wird für Import noch nicht unterstützt")
normalized = text.replace("\r\n", "\n").replace("\r", "\n")
reader = csv.DictReader(io.StringIO(normalized), delimiter=delimiter)
for row in reader:
if row is None:
continue
if not any(v and str(v).strip() for v in row.values()):
continue
yield {_strip_header_key(k): (v or "").strip() for k, v in row.items() if _strip_header_key(k)}
File diff suppressed because it is too large. Load diff
+115
View File
@@ -0,0 +1,115 @@
"""
Kanonische Speichereinheiten pro CSV-Zielfeld (module_registry: field.unit) und
abwählbare Quelleinheiten → Faktor für type_conversions.source_unit.
"""
from __future__ import annotations
from typing import Any
from csv_parser.module_registry import get_module_definition
# 1 kcal = 4.184 kJ (IEC/ISO 80000)
_KJ_TO_KCAL = 1.0 / 4.184
# — Energie (Ziel kcal) —
_ENERGY: list[dict[str, Any]] = [
{"id": "kcal", "label": "Kilokalorien (kcal), wie in DB", "factor": 1.0},
{"id": "kj", "label": "Kilojoule (kJ) → kcal", "factor": _KJ_TO_KCAL},
{"id": "j", "label": "Joule (J) → kcal", "factor": _KJ_TO_KCAL / 1000.0},
]
# — Masse klein (Ziel g): Makronährstoffe —
_GRAM: list[dict[str, Any]] = [
{"id": "g", "label": "Gramm (g), wie in DB", "factor": 1.0},
{"id": "kg", "label": "Kilogramm (kg) → g", "factor": 1000.0},
{"id": "mg", "label": "Milligramm (mg) → g", "factor": 0.001},
]
# — Körpergewicht (Ziel kg) —
_KG: list[dict[str, Any]] = [
{"id": "kg", "label": "Kilogramm (kg), wie in DB", "factor": 1.0},
{"id": "g", "label": "Gramm (g) → kg", "factor": 0.001},
{"id": "lb", "label": "Pfund / lb → kg", "factor": 0.45359237},
{"id": "oz", "label": "Unze / oz → kg", "factor": 0.028349523125},
]
# — Strecke (Ziel km) —
_KM: list[dict[str, Any]] = [
{"id": "km", "label": "Kilometer (km), wie in DB", "factor": 1.0},
{"id": "m", "label": "Meter (m) → km", "factor": 0.001},
{"id": "mi", "label": "Meilen (mi) → km", "factor": 1.609344},
]
_UNIT_FAMILY: dict[str, list[dict[str, Any]]] = {
"kcal": _ENERGY,
"g": _GRAM,
"kg": _KG,
"km": _KM,
}
def get_canonical_unit(module: str, db_field: str) -> str | None:
mod = get_module_definition(module)
if not mod:
return None
finfo: dict[str, Any] | None = mod.get("fields", {}).get(db_field)
if not finfo:
return None
u = finfo.get("unit")
return str(u) if u else None
def source_unit_choices_for_field(module: str, db_field: str) -> list[dict[str, Any]]:
"""Optionen für GUI: id, label, canonical_unit, is_canonical (Umrechnung serverseitig)."""
cu = get_canonical_unit(module, db_field)
if not cu:
return []
choices = _UNIT_FAMILY.get(cu)
if not choices:
return []
out: list[dict[str, Any]] = [
{
"id": c["id"],
"label": c["label"],
"canonical_unit": cu,
"is_canonical": c["id"] == cu,
}
for c in choices
]
out.append(
{
"id": "custom",
"label": "Benutzerdefiniert (Konvertierungsfaktor, z. B. ml→g je nach Dichte)",
"canonical_unit": cu,
"is_canonical": False,
}
)
return out
def factor_source_to_canonical(module: str, db_field: str, source_unit: str | None) -> float:
"""
Multiplikator: CSV-Zahl * Faktor → Wert in kanonischer DB-Einheit.
Unbekannte/None/leer/Passthrough → 1.0
``source_unit`` ``custom`` / ``none``: kein Registry-Faktor (1.0); freie Skalierung nur über
``conversion_factor`` in type_conversions (JSON).
"""
if source_unit is None:
return 1.0
su = str(source_unit).strip().lower()
if not su:
return 1.0
if su in ("custom", "none"):
return 1.0
cu = get_canonical_unit(module, db_field)
if not cu:
return 1.0
choices = _UNIT_FAMILY.get(cu)
if not choices:
return 1.0
for c in choices:
if str(c["id"]).lower() == su:
return float(c["factor"])
return 1.0
+53
View File
@@ -0,0 +1,53 @@
"""
Menschenlesbare Hinweise zu typischen Import-/DB-Fehlern (Universal-CSV).
"""
from __future__ import annotations
def enrich_row_error(message: str, module: str | None = None) -> dict[str, str | None]:
"""
Ergänzt eine Rohexception-Zeichenkette um ``code`` und ``hint`` für die Fehlerliste im Import.
"""
low = (message or "").lower()
out: dict[str, str | None] = {"error": message, "code": None, "hint": None}
if "numeric field overflow" in low or "numeric value out of range" in low:
out["code"] = "db_numeric_overflow"
out["hint"] = (
"Wert passt nicht in die Datenbank-Spalte (z. B. NUMERIC mit begrenzter Größe). "
"Häufig: Kilojoule aus dem Export landen im Kalorien-Feld – in der Vorlage für kcal_active/kcal_resting "
'"source_unit": "kj" setzen. Oder eine falsche CSV-Spalte ist einem kleinen Zielfeld zugeordnet '
"(z. B. große Zahl in einem HF-Feld)."
)
return out
if "violates check constraint" in low and "source" in low:
out["code"] = "db_check_constraint_source"
out["hint"] = (
"Die Tabelle erlaubt den gesetzten «source»-Wert nicht. "
"System-Vorlage / Migration zur erlaubten Quelle prüfen (z. B. csv für Universal-Import)."
)
return out
if "current transaction is aborted" in low:
out["code"] = "transaction_aborted"
out["hint"] = (
"Eine frühere Zeile hat einen Datenbankfehler ausgelöst. "
"Zuerst die niedrigste Zeilennummer in error_details beheben (Vorlage/Daten prüfen)."
)
return out
if "invalid input syntax" in low and "time" in low:
out["code"] = "db_time_cast"
out["hint"] = (
"start_time/end_time passen nicht zum erwarteten Zeitformat in der Datenbank. "
"Vorlage: Datums- und Zeitanteil konsistent (oft nur Uhrzeit, wenn date separat)."
)
return out
if module == "activity" and "foreign key" in low:
out["code"] = "db_foreign_key"
out["hint"] = "Verknüpfung zur Datenbank verletzt (z. B. training_type). Support kontaktieren."
return out
+213
View File
@@ -0,0 +1,213 @@
"""
Zeilenaggregation nach CSV-Mapping (group_by + aggregates), vor dem DB-Upsert.
Spezifikation in der Vorlage (import_row_processing JSONB). Optional: Modul-Default
(import_row_processing_default in module_registry) nur als **Legacy-Fallback**, wenn
die Vorlage nichts speichert — mittelfristig sollen Vorlagen explizit sein.
"""
from __future__ import annotations
import datetime as dt
import statistics
from typing import Any, Mapping
from csv_parser.module_registry import get_module_definition
ALLOWED_AGGREGATES = frozenset({"sum", "mean", "min", "max", "median", "first", "last"})
# Mehr als eine CSV-Zeile pro group_by-Schlüssel
ALLOWED_MULTI_ROW_POLICIES = frozenset({"aggregate", "reject", "first_row", "last_row"})
def resolve_import_row_processing(module: str, mapping_row: Mapping[str, Any]) -> dict[str, Any] | None:
"""Explizite Vorlage hat Vorrang; sonst Modul-Default; leeres Dict zählt wie „nicht gesetzt“."""
raw = mapping_row.get("import_row_processing")
if isinstance(raw, dict) and raw:
return dict(raw)
mod = get_module_definition(module)
if not mod:
return None
default = mod.get("import_row_processing_default")
if isinstance(default, dict) and default:
return dict(default)
return None
def validate_import_row_processing(
module: str,
spec: Mapping[str, Any],
field_mappings: Mapping[str, Any],
) -> None:
"""Wirft ValueError bei ungültiger Konfiguration."""
mod = get_module_definition(module)
if not mod:
raise ValueError(f"Unbekanntes Modul: {module}")
allowed = set(mod.get("fields") or [])
fm_targets = {str(v) for v in field_mappings.values() if v and v not in ("-", "_skip")}
group_by = spec.get("group_by") or []
if not isinstance(group_by, list) or not all(isinstance(x, str) for x in group_by):
raise ValueError("import_row_processing.group_by muss eine Liste von Feldnamen sein")
aggregates = spec.get("aggregates") or {}
if not isinstance(aggregates, dict):
raise ValueError("import_row_processing.aggregates muss ein Objekt sein")
for g in group_by:
if g not in allowed:
raise ValueError(f"group_by: unbekanntes Feld '{g}' für Modul '{module}'")
if g not in fm_targets:
raise ValueError(
f"group_by: Zielfeld '{g}' ist keiner CSV-Spalte zugeordnet — Aggregation nicht möglich."
)
for field, op in aggregates.items():
if field not in allowed:
raise ValueError(f"aggregates: unbekanntes Feld '{field}' für Modul '{module}'")
if str(op) not in ALLOWED_AGGREGATES:
raise ValueError(
f"aggregates['{field}']: ungültige Operation '{op}'. "
f"Erlaubt: {', '.join(sorted(ALLOWED_AGGREGATES))}"
)
mrp = spec.get("multi_row_policy")
if mrp is not None and str(mrp) not in ALLOWED_MULTI_ROW_POLICIES:
raise ValueError(
f"multi_row_policy: ungültiger Wert '{mrp}'. "
f"Erlaubt: {', '.join(sorted(ALLOWED_MULTI_ROW_POLICIES))}"
)
dedupe = spec.get("dedupe_identical_rows")
if dedupe is not None and not isinstance(dedupe, bool):
raise ValueError("dedupe_identical_rows muss ein Boolean sein")
def _sort_key_for_group(v: Any) -> Any:
if isinstance(v, dt.datetime):
return v.isoformat()
if isinstance(v, dt.date):
return v.isoformat()
if isinstance(v, dt.time):
return v.isoformat()
return v
def _apply_aggregate(op: str, values: list[Any]) -> Any:
nums: list[float] = []
for x in values:
if x is None or x == "":
continue
try:
nums.append(float(x))
except (TypeError, ValueError):
continue
if op == "sum":
return sum(nums) if nums else None
if op == "mean":
return statistics.mean(nums) if nums else None
if op == "median":
return float(statistics.median(nums)) if nums else None
if op == "min":
return min(nums) if nums else None
if op == "max":
return max(nums) if nums else None
if op == "first":
for x in values:
if x is not None and x != "":
return x
return None
if op == "last":
for x in reversed(values):
if x is not None and x != "":
return x
return None
raise ValueError(f"Unbekannte Aggregations-Operation: {op}")
def _row_identity_signature(r: dict[str, Any]) -> tuple[Any, ...]:
return tuple(sorted((k, _sort_key_for_group(r.get(k))) for k in sorted(r.keys())))
def _dedupe_identical_mapped_rows(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Exakt gleiche gemappte Zeilen (alle Keys/Werte) — erste behalten."""
seen: set[tuple[Any, ...]] = set()
out: list[dict[str, Any]] = []
for r in rows:
sig = _row_identity_signature(r)
if sig in seen:
continue
seen.add(sig)
out.append(r)
return out
def aggregate_mapped_rows(
rows: list[dict[str, Any]],
spec: Mapping[str, Any],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""
Gruppiert gemappte Zeilen-Dicts nach group_by und wendet aggregates an.
Felder, die weder in group_by noch in aggregates vorkommen: Wert aus der ersten Zeile der Gruppe.
Rückgabe: (merged_rows, strukturelle Fehler / Hinweise, z. B. abgelehnte Schlüsselgruppen).
"""
errors: list[dict[str, Any]] = []
rows = list(rows)
if spec.get("dedupe_identical_rows"):
rows = _dedupe_identical_mapped_rows(rows)
group_by = spec.get("group_by") or []
aggregates = spec.get("aggregates") or {}
policy = str(spec.get("multi_row_policy") or "aggregate")
if policy not in ALLOWED_MULTI_ROW_POLICIES:
policy = "aggregate"
if not group_by:
return rows, errors
buckets: dict[tuple[Any, ...], list[dict[str, Any]]] = {}
order: list[tuple[Any, ...]] = []
for r in rows:
key = tuple(_sort_key_for_group(r.get(g)) for g in group_by)
if key not in buckets:
buckets[key] = []
order.append(key)
buckets[key].append(r)
gb_label = ", ".join(group_by)
out: list[dict[str, Any]] = []
for key in order:
group_rows = buckets[key]
if len(group_rows) > 1:
if policy == "reject":
errors.append(
{
"error": "mehrere_zeilen_pro_schluessel",
"message": (
f"{len(group_rows)} CSV-Zeilen mit gleichem Schlüssel ({gb_label}); "
"laut Vorlage abgelehnt (multi_row_policy=reject)."
),
"rows_in_group": len(group_rows),
}
)
continue
if policy == "first_row":
group_rows = [group_rows[0]]
elif policy == "last_row":
group_rows = [group_rows[-1]]
first = group_rows[0]
merged: dict[str, Any] = {}
for g in group_by:
merged[g] = first.get(g)
for field, op in aggregates.items():
merged[field] = _apply_aggregate(str(op), [row.get(field) for row in group_rows])
for row in group_rows:
for k, v in row.items():
if k in merged:
continue
if k in group_by or k in aggregates:
continue
merged[k] = v
out.append(merged)
return out, errors
+240
View File
@@ -0,0 +1,240 @@
"""
Heuristische Vorschläge für CSV field_mappings / type_conversions (Admin-Editor, Issue #21).
"""
from __future__ import annotations
from copy import deepcopy
from typing import Any, Mapping
from csv_parser.core import normalize_header_for_signature
from csv_parser.module_registry import get_module_definition
# Normalisierte Header-Fragmente → DB-Feld (Substring- oder exakter Norm-Vergleich)
_MODULE_HEADER_ALIASES: dict[str, dict[str, frozenset[str]]] = {
"nutrition": {
"date": frozenset(
{"datum", "date", "tag", "day", "zeit", "timestamp", "uhrzeit", "monat", "jahr"}
),
"kcal": frozenset({"kcal", "kalorie", "calorie", "energie", "energy", "kj", "joule"}),
"protein_g": frozenset({"protein", "eiwei", "eiweiss"}),
"fat_g": frozenset({"fett", "fat", "lipid"}),
"carbs_g": frozenset({"kh", "carb", "kohlenhydr", "carbs", "sugar", "zucker"}),
},
"weight": {
"date": frozenset({"datum", "date", "tag", "day", "zeit"}),
"weight": frozenset({"gewicht", "weight", "masse", "kg", "kilo"}),
"note": frozenset({"notiz", "note", "comment", "kommentar"}),
},
"blood_pressure": {
"measured_date": frozenset({"datum", "date", "tag", "day", "messdatum"}),
"measured_time": frozenset({"zeit", "time", "uhr", "uhrzeit"}),
"systolic": frozenset({"systol", "sys", "sbp", "oberdruck"}),
"diastolic": frozenset({"diastol", "dia", "dbp", "unterdruck"}),
"pulse": frozenset({"puls", "pulse", "hr", "herz", "bpm"}),
},
"activity": {
"date": frozenset({"datum", "date", "tag", "day"}),
"start_time": frozenset({"start", "beginn", "von"}),
"end_time": frozenset({"end", "ende", "bis", "stop"}),
"activity_type": frozenset({"workout", "training", "typ", "type", "art", "aktiv"}),
"duration_min": frozenset({"dauer", "duration", "min"}),
"distance_km": frozenset({"strecke", "distance", "km", "distanz"}),
"kcal_active": frozenset({"kcal", "kalorie", "energie", "active"}),
"kcal_resting": frozenset({"ruhe", "resting"}),
"hr_avg": frozenset({"puls", "heart", "hr", "bpm", "herzfrequenz", "durchschn"}),
"hr_max": frozenset({"max", "peak"}),
},
"vitals_baseline": {
"date": frozenset({"datum", "date", "tag", "start", "zeit"}),
"resting_hr": frozenset({"ruhepuls", "resting", "rhr"}),
"hrv": frozenset({"hrv", "variabilit", "vfc"}),
"vo2_max": frozenset({"vo2"}),
"spo2": frozenset({"sauerstoff", "spo2", "oxygen"}),
"respiratory_rate": frozenset({"atem", "respiratory"}),
},
}
_DEFAULT_TYPE_CONVERSIONS: dict[str, dict[str, dict[str, Any]]] = {
"nutrition": {
"date": {"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only", "flexible": True},
"kcal": {"type": "float", "decimal_separator": "auto", "flexible": True},
"protein_g": {"type": "float", "decimal_separator": "auto", "flexible": True},
"fat_g": {"type": "float", "decimal_separator": "auto", "flexible": True},
"carbs_g": {"type": "float", "decimal_separator": "auto", "flexible": True},
},
"weight": {
"date": {"type": "date", "format": "dd.mm.yyyy", "flexible": True},
"weight": {"type": "float", "decimal_separator": "auto", "flexible": True},
"note": {"type": "string"},
},
"blood_pressure": {
"measured_date": {"type": "date", "format": "dd.mm.yyyy", "flexible": True},
"measured_time": {"type": "time", "format": "HH:MM", "flexible": True},
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
"systolic": {"type": "int", "flexible": True},
"diastolic": {"type": "int", "flexible": True},
"pulse": {"type": "int", "flexible": True},
},
"activity": {
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
"activity_type": {"type": "string"},
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes", "flexible": True},
"distance_km": {"type": "float", "decimal_separator": "auto", "flexible": True},
"kcal_active": {"type": "float", "decimal_separator": "auto", "flexible": True},
"kcal_resting": {"type": "float", "decimal_separator": "auto", "flexible": True},
"hr_avg": {"type": "int", "flexible": True},
"hr_max": {"type": "int", "flexible": True},
},
"vitals_baseline": {
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": True,
},
"resting_hr": {"type": "int", "flexible": True},
"hrv": {"type": "int", "flexible": True},
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": True},
"spo2": {"type": "int", "flexible": True},
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": True},
},
}
def _norm_key(header: str) -> str:
return normalize_header_for_signature(header)
def _match_seed_to_db_field(header: str, seed_fm: Mapping[str, str]) -> str | None:
"""Findet Ziel-Feld, wenn Seed-Key zu diesem Header passt (exakt oder normalisiert)."""
if header in seed_fm:
v = seed_fm[header]
if v and v not in ("-", "_skip"):
return v
nh = _norm_key(header)
if nh in seed_fm:
v = seed_fm[nh]
if v and v not in ("-", "_skip"):
return v
for sk, sv in seed_fm.items():
if not sv or sv in ("-", "_skip"):
continue
if _norm_key(str(sk)) == nh:
return sv
return None
def _alias_suggest(norm: str, module: str, used: set[str]) -> str | None:
aliases = _MODULE_HEADER_ALIASES.get(module, {})
mod = get_module_definition(module)
if not mod:
return None
field_order = list(mod["fields"].keys())
for db_field in field_order:
if db_field in used:
continue
tokens = aliases.get(db_field, frozenset())
nlow = norm.lower()
if nlow == db_field or nlow.replace("_", "") == db_field.replace("_", ""):
return db_field
for tok in tokens:
if len(tok) >= 2 and tok in nlow:
return db_field
if len(tok) >= 4 and tok in norm:
return db_field
return None
def suggest_field_mappings(
headers: list[str],
module: str,
seed_fm: Mapping[str, str] | None = None,
) -> dict[str, str]:
"""
Mappt jede CSV-Spalte (Roh-Header als Key) auf DB-Feld oder '-'.
Nutzt zuerst eine passende Seed-Vorlage, dann Alias-Heuristik.
"""
if module == "sleep":
return {h: "-" for h in headers}
mod = get_module_definition(module)
if not mod:
return {h: "-" for h in headers}
fm: dict[str, str] = {h: "-" for h in headers}
used: set[str] = set()
if seed_fm:
for h in headers:
db = _match_seed_to_db_field(h, seed_fm)
if db and db not in used:
fm[h] = db
used.add(db)
for h in headers:
if fm[h] != "-":
continue
norm = _norm_key(h)
db = _alias_suggest(norm, module, used)
if db:
fm[h] = db
used.add(db)
return fm
def build_type_conversions_for_mapping(
module: str,
field_mappings: Mapping[str, str],
seed_tc: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
"""type_conversions nur für zugewiesene Zielfelder; Seed überschreibt Defaults."""
if module == "sleep":
return {}
defaults = _DEFAULT_TYPE_CONVERSIONS.get(module, {})
out: dict[str, Any] = {}
targets = {v for v in field_mappings.values() if v and v not in ("-", "_skip")}
if seed_tc:
for k, v in seed_tc.items():
if k in targets and isinstance(v, dict):
out[k] = deepcopy(v)
for t in targets:
if t not in out and t in defaults:
out[t] = deepcopy(defaults[t])
_apply_energy_kj_hint_from_headers(module, field_mappings, out)
return out
_ENERGY_FIELDS = frozenset({"kcal", "kcal_active", "kcal_resting"})
def _apply_energy_kj_hint_from_headers(
module: str,
field_mappings: Mapping[str, str],
out: dict[str, Any],
) -> None:
"""Wenn Überschrift kJ/Kilojoule nahelegt (nicht kcal), source_unit kj setzen (FDDB & Co.)."""
if module not in ("nutrition", "activity"):
return
for csv_col, db_field in field_mappings.items():
if db_field not in _ENERGY_FIELDS:
continue
spec = out.get(db_field)
if not isinstance(spec, dict):
continue
if spec.get("source_unit"):
continue
norm = normalize_header_for_signature(str(csv_col)).lower()
if "kcal" in norm:
continue
if "kj" in norm or "kilojoule" in norm:
spec2 = deepcopy(spec)
spec2["source_unit"] = "kj"
out[db_field] = spec2
+161
View File
@@ -0,0 +1,161 @@
"""
Ziel-Module für CSV-Import: Tabellen-Felder, Pflichtfelder, Duplikat-Strategie (Issue #21).
Hinweis: blood_pressure nutzt in der DB measured_at; Logik-Felder measured_date + measured_time
werden im Executor zu measured_at zusammengefügt (Phase Import-Executor).
Activity: date kann aus start_time (ISO-Datetime) abgeleitet werden, wenn nur start_time gesetzt ist.
"""
from __future__ import annotations
from typing import Any, Dict, cast
MODULE_DEFINITIONS: Dict[str, Dict[str, Any]] = {
"nutrition": {
"table": "nutrition_log",
"fields": {
"date": {"type": "date", "required": True},
"kcal": {"type": "float", "required": False, "unit": "kcal"},
"protein_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
"fat_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
"carbs_g": {"type": "float", "required": False, "min": 0, "unit": "g"},
},
"duplicate_key": ["profile_id", "date"],
"duplicate_strategy": "update",
# Legacy-Fallback wenn die Vorlage kein import_row_processing speichert — Vorlagen mittelfristig explizit.
"import_row_processing_default": {
"group_by": ["date"],
"aggregates": {
"kcal": "sum",
"protein_g": "sum",
"fat_g": "sum",
"carbs_g": "sum",
},
},
},
"activity": {
"table": "activity_log",
"fields": {
"date": {"type": "date", "required": False},
"start_time": {"type": "datetime", "required": False},
"end_time": {"type": "datetime", "required": False},
"activity_type": {"type": "string", "required": True},
"duration_min": {"type": "float", "required": False, "min": 0},
"kcal_active": {"type": "float", "required": False, "unit": "kcal"},
"kcal_resting": {"type": "float", "required": False, "unit": "kcal"},
"distance_km": {"type": "float", "required": False, "unit": "km"},
"hr_avg": {"type": "float", "required": False, "min": 30, "max": 220},
"hr_max": {"type": "float", "required": False, "min": 30, "max": 220},
},
"derive_date_from_datetime_field": "start_time",
"duplicate_key": ["profile_id", "date", "start_time"],
"duplicate_strategy": "update",
},
"sleep": {
"table": "sleep_log",
"fields": {},
"import_mode": "apple_sleep_aggregate",
},
"vitals_baseline": {
"table": "vitals_baseline",
"fields": {
"date": {"type": "date", "required": True},
"resting_hr": {"type": "int", "required": False},
"hrv": {"type": "int", "required": False},
"vo2_max": {"type": "float", "required": False},
"spo2": {"type": "int", "required": False},
"respiratory_rate": {"type": "float", "required": False},
},
"duplicate_key": ["profile_id", "date"],
"duplicate_strategy": "update",
# Legacy-Fallback — Vorlagen mittelfristig explizit setzen.
"import_row_processing_default": {
"group_by": ["date"],
"aggregates": {
"resting_hr": "mean",
"hrv": "mean",
"vo2_max": "mean",
"spo2": "mean",
"respiratory_rate": "mean",
},
},
},
"blood_pressure": {
"table": "blood_pressure_log",
"fields": {
"measured_date": {"type": "date", "required": True},
"measured_time": {"type": "time", "required": True},
# Apple Health: eine Spalte „Start“ / „Datum/Uhrzeit“ (Datetime); Executor splittet.
"start_time": {"type": "datetime", "required": False},
"systolic": {"type": "int", "required": True},
"diastolic": {"type": "int", "required": True},
"pulse": {"type": "int", "required": False},
},
"logical_to_db": "blood_pressure_composite_measured_at",
"duplicate_key": ["profile_id", "measured_at"],
"duplicate_strategy": "update",
},
"weight": {
"table": "weight_log",
"fields": {
"date": {"type": "date", "required": True},
"weight": {"type": "float", "required": True, "min": 20, "max": 400, "unit": "kg"},
"note": {"type": "string", "required": False, "max_length": 2000},
},
"duplicate_key": ["profile_id", "date"],
"duplicate_strategy": "update",
# Legacy-Fallback — Vorlagen mittelfristig explizit setzen.
"import_row_processing_default": {
"group_by": ["date"],
"aggregates": {
"weight": "last",
"note": "last",
},
},
},
}
def get_module_definition(module: str) -> Dict[str, Any] | None:
return MODULE_DEFINITIONS.get(module)
def list_modules() -> list[str]:
return sorted(MODULE_DEFINITIONS.keys())
def validate_field_mappings(module: str, field_mappings: dict) -> None:
"""Wirft ValueError bei unbekanntem Modul oder unbekanntem DB-Feld."""
mod = get_module_definition(module)
if not mod:
raise ValueError(f"Unbekanntes Modul: {module}")
fields = cast(dict, mod["fields"])
allowed = set(fields.keys())
if not allowed:
for _csv_col, db_field in field_mappings.items():
if db_field not in ("", None, "-", "_skip"):
raise ValueError(
f"Modul '{module}' nutzt einen Aggregat-Import ohne Spalten-Mapping; "
f"alle Spalten müssen „ignorieren“ sein."
)
return
for _csv_col, db_field in field_mappings.items():
if db_field in ("", None, "-", "_skip"):
continue
if db_field not in allowed:
raise ValueError(f"Ungültiges Zielfeld '{db_field}' für Modul '{module}'")
def validate_required_field_targets(module: str, field_mappings: dict) -> None:
"""Stellt sicher, dass jedes als required markierte Zielfeld mindestens einer Spalte zugeordnet ist."""
mod = get_module_definition(module)
if not mod:
raise ValueError(f"Unbekanntes Modul: {module}")
field_defs = cast(dict, mod["fields"])
targets = {v for v in field_mappings.values() if v and v not in ("-", "_skip")}
if module == "blood_pressure" and "start_time" in targets:
targets = set(targets) | {"measured_date", "measured_time"}
for fname, finfo in field_defs.items():
if finfo.get("required") and fname not in targets:
raise ValueError(f"Pflicht-Zielfeld nicht zugeordnet: {fname}")
+19
View File
@@ -0,0 +1,19 @@
"""Zugriffsregeln für csv_field_mappings (Issue #21)."""
from __future__ import annotations
from typing import Any, Mapping
def user_may_edit_mapping_row(row: Mapping[str, Any], session: Mapping[str, Any]) -> bool:
if session.get("role") == "admin":
return True
if row.get("is_system"):
return False
return str(row.get("profile_id")) == str(session.get("profile_id"))
def user_may_delete_mapping(row: Mapping[str, Any], session: Mapping[str, Any]) -> bool:
if row.get("is_system"):
return False
return str(row.get("profile_id")) == str(session.get("profile_id"))
+350
View File
@@ -0,0 +1,350 @@
"""
Apple-Health-Schlaf-CSV → sleep_log (für Universal-Import und /api/sleep/import).
Nutzt dieselbe Logik wie der Sleep-Router, ohne HTTPException — arbeitet auf einem übergebenen Cursor.
"""
from __future__ import annotations
import csv
import io
import json
import logging
from datetime import date, datetime
from decimal import Decimal, InvalidOperation
from typing import Any, Literal
from dateutil import parser as dateutil_parser
logger = logging.getLogger(__name__)
def _strip_row_keys(row: dict) -> dict:
return {(k or "").strip(): (v.strip() if isinstance(v, str) else v) for k, v in row.items()}
def _safe_float(value: Any) -> float | None:
if value is None or value == "":
return None
if isinstance(value, (int, float)):
return float(value)
if isinstance(value, Decimal):
return float(value)
try:
s = str(value).strip().replace(",", ".")
return float(s)
except (ValueError, InvalidOperation):
return None
def _parse_apple_sleep_datetime(value: str) -> datetime:
raw = (value or "").strip()
if not raw:
raise ValueError("empty datetime")
fmts = (
"%Y-%m-%d %H:%M:%S %z",
"%d.%m.%y %H:%M:%S",
"%d.%m.%Y %H:%M:%S",
"%Y-%m-%d %H:%M:%S",
)
for fmt in fmts:
try:
return datetime.strptime(raw, fmt)
except ValueError:
continue
try:
return dateutil_parser.parse(raw, dayfirst=False)
except (ValueError, TypeError, OverflowError) as e:
raise ValueError(f"Unbekanntes Datumsformat: {raw!r}") from e
def _hr_to_minutes(hours: float | None) -> int:
if hours is None:
return 0
return int(round(float(hours) * 60))
def detect_apple_sleep_csv_format(fieldnames: list[str] | None) -> Literal["segments", "summary"]:
if not fieldnames:
raise ValueError("CSV enthält keine Spaltenüberschriften.")
fn = {(f or "").strip() for f in fieldnames}
if {"Start", "End", "Duration (hr)", "Value"}.issubset(fn):
return "segments"
if "Start" in fn and "End" in fn and "Total Sleep (hr)" in fn:
return "summary"
raise ValueError(
"Unbekanntes Apple-Health-Schlaf-CSV. Erwartet: Segment-Export oder Schlafanalyse-Zusammenfassung."
)
def _build_nights_from_apple_summary(reader: csv.DictReader) -> dict[date, dict]:
nights_dict: dict[date, dict] = {}
for raw in reader:
row = _strip_row_keys(raw)
start_s = row.get("Start") or ""
end_s = row.get("End") or ""
if not start_s or not end_s:
continue
try:
start_dt = _parse_apple_sleep_datetime(start_s)
end_dt = _parse_apple_sleep_datetime(end_s)
except ValueError:
continue
dt_key = (row.get("Date/Time") or row.get("Datum/Uhrzeit") or "").strip()
if dt_key:
try:
wake_d = datetime.strptime(dt_key[:10], "%Y-%m-%d").date()
except ValueError:
wake_d = end_dt.date()
else:
wake_d = end_dt.date()
core_hr = _safe_float(row.get("Core (hr)"))
if core_hr is None:
core_hr = _safe_float(row.get("Light (hr)")) or 0.0
deep_min = _hr_to_minutes(_safe_float(row.get("Deep (hr)")))
rem_min = _hr_to_minutes(_safe_float(row.get("REM (hr)")))
light_min = _hr_to_minutes(core_hr)
awake_min = _hr_to_minutes(_safe_float(row.get("Awake (hr)")))
total_sleep_hr = _safe_float(row.get("Total Sleep (hr)"))
nights_dict[wake_d] = {
"bedtime": start_dt,
"wake_time": end_dt,
"segments": [],
"deep_minutes": deep_min,
"rem_minutes": rem_min,
"light_minutes": light_min,
"awake_minutes": awake_min,
"total_sleep_hr": total_sleep_hr,
}
return nights_dict
def _build_nights_from_apple_segments(reader: csv.DictReader, phase_map: dict) -> dict[date, dict]:
segments = []
for raw in reader:
row = _strip_row_keys(raw)
phase_key = (row.get("Value") or "").strip()
phase_en = phase_map.get(phase_key)
if phase_en is None:
continue
try:
start_dt = _parse_apple_sleep_datetime(row.get("Start") or "")
end_dt = _parse_apple_sleep_datetime(row.get("End") or "")
duration_hr = _safe_float(row.get("Duration (hr)"))
if duration_hr is None:
continue
except (ValueError, TypeError):
continue
duration_min = int(duration_hr * 60)
segments.append({
"start": start_dt,
"end": end_dt,
"duration_min": duration_min,
"phase": phase_en,
})
segments.sort(key=lambda s: s["start"])
nights = []
current_night = None
for seg in segments:
if current_night is None or (seg["start"] - current_night["wake_time"]).total_seconds() > 7200:
current_night = {
"bedtime": seg["start"],
"wake_time": seg["end"],
"segments": [],
"deep_minutes": 0,
"rem_minutes": 0,
"light_minutes": 0,
"awake_minutes": 0,
}
nights.append(current_night)
current_night["segments"].append(seg)
current_night["wake_time"] = max(current_night["wake_time"], seg["end"])
current_night["bedtime"] = min(current_night["bedtime"], seg["start"])
if seg["phase"] == "deep":
current_night["deep_minutes"] += seg["duration_min"]
elif seg["phase"] == "rem":
current_night["rem_minutes"] += seg["duration_min"]
elif seg["phase"] == "light":
current_night["light_minutes"] += seg["duration_min"]
elif seg["phase"] == "awake":
current_night["awake_minutes"] += seg["duration_min"]
nights_dict: dict[date, dict] = {}
for night in nights:
wake_date = night["wake_time"].date()
nights_dict[wake_date] = night
return nights_dict
def import_apple_sleep_nights(cur, profile_id: str, text: str) -> dict[str, Any]:
"""
Schreibt in sleep_log. Kein conn.commit — Aufrufer rollt Transaktion.
Gibt Statistik im Executor-Format zurück.
"""
csv_text = text.replace("\r\n", "\n").replace("\r", "\n")
if csv_text.startswith("\ufeff"):
csv_text = csv_text[1:]
reader = csv.DictReader(io.StringIO(csv_text))
fmt = detect_apple_sleep_csv_format(reader.fieldnames)
phase_map = {
"Kern": "light",
"Core": "light",
"Light": "light",
"REM": "rem",
"Tief": "deep",
"Deep": "deep",
"Wach": "awake",
"Awake": "awake",
"Schlafend": None,
"Asleep": None,
"In Bed": None,
}
if fmt == "summary":
nights_dict = _build_nights_from_apple_summary(reader)
else:
nights_dict = _build_nights_from_apple_segments(reader, phase_map)
if not nights_dict:
raise ValueError(
"Keine importierbaren Schlafzeilen gefunden (prüfe Start/Ende und Format)."
)
inserted = 0
updated = 0
skipped = 0
error_details: list[dict[str, Any]] = []
affected_ids: list[str] = []
row_hint = 0
for wake_date, night in nights_dict.items():
row_hint += 1
phase_sum = (
night["deep_minutes"] + night["rem_minutes"] + night["light_minutes"]
)
total_hr = night.get("total_sleep_hr")
fallback_min = int(round(float(total_hr) * 60)) if total_hr is not None else 0
duration_minutes = phase_sum if phase_sum > 0 else fallback_min
if duration_minutes <= 0:
logger.warning(
"Sleep import: überspringe %s — Dauer 0 (Phasen-Summe und Total Sleep (hr) leer/0).",
wake_date,
)
error_details.append(
{
"row": row_hint,
"error": f"Schlafdauer für {wake_date} ist 0 — Phasen oder Total Sleep (hr) fehlen.",
}
)
continue
wake_count = sum(1 for seg in night["segments"] if seg["phase"] == "awake")
sleep_segments = [
{
"phase": seg["phase"],
"start": seg["start"].isoformat(),
"end": seg["end"].isoformat(),
"duration_min": seg["duration_min"],
}
for seg in night["segments"]
]
cur.execute(
"""
SELECT id, source FROM sleep_log
WHERE profile_id = %s AND date = %s
""",
(profile_id, wake_date),
)
existing = cur.fetchone()
if existing and existing["source"] == "manual":
skipped += 1
continue
try:
if existing:
cur.execute(
"""
UPDATE sleep_log SET
bedtime = %s,
wake_time = %s,
duration_minutes = %s,
wake_count = %s,
deep_minutes = %s,
rem_minutes = %s,
light_minutes = %s,
awake_minutes = %s,
sleep_segments = %s,
source = 'apple_health',
updated_at = CURRENT_TIMESTAMP
WHERE id = %s AND profile_id = %s
RETURNING id
""",
(
night["bedtime"].time(),
night["wake_time"].time(),
duration_minutes,
wake_count,
night["deep_minutes"],
night["rem_minutes"],
night["light_minutes"],
night["awake_minutes"],
json.dumps(sleep_segments),
existing["id"],
profile_id,
),
)
row = cur.fetchone()
updated += 1
if row and row.get("id"):
affected_ids.append(str(row["id"]))
else:
cur.execute(
"""
INSERT INTO sleep_log (
profile_id, date, bedtime, wake_time, duration_minutes,
wake_count, deep_minutes, rem_minutes, light_minutes, awake_minutes,
sleep_segments, source, created_at, updated_at
) VALUES (
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, 'apple_health', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
)
RETURNING id
""",
(
profile_id,
wake_date,
night["bedtime"].time(),
night["wake_time"].time(),
duration_minutes,
wake_count,
night["deep_minutes"],
night["rem_minutes"],
night["light_minutes"],
night["awake_minutes"],
json.dumps(sleep_segments),
),
)
row = cur.fetchone()
inserted += 1
if row and row.get("id"):
affected_ids.append(str(row["id"]))
except Exception as e:
logger.warning("Sleep import row failed: %s", e)
error_details.append({"row": row_hint, "error": str(e)})
return {
"rows_total": len(nights_dict),
"inserted": inserted,
"updated": updated,
"skipped": skipped,
"new_entries": inserted,
"error_details": error_details,
"affected_ids": affected_ids,
}
+235
View File
@@ -0,0 +1,235 @@
"""
Formatprüfung für CSV-Import-Vorlagen (field_mappings, type_conversions).
Liefert strukturierte Fehler/Warnungen für Admin-UI und Speicher-Guards.
"""
from __future__ import annotations
from typing import Any, Mapping
from csv_parser.core import normalize_header_for_signature
from csv_parser.import_row_processing import validate_import_row_processing as validate_import_row_processing_spec
from csv_parser.module_registry import (
get_module_definition,
validate_field_mappings,
validate_required_field_targets,
)
ALLOWED_SPEC_TYPES = frozenset(
{"string", "float", "number", "int", "date", "time", "datetime", "duration"}
)
def _issue(
severity: str,
code: str,
message: str,
*,
hint: str | None = None,
field: str | None = None,
csv_columns: list[str] | None = None,
) -> dict[str, Any]:
out: dict[str, Any] = {
"severity": severity,
"code": code,
"message": message,
}
if hint:
out["hint"] = hint
if field:
out["field"] = field
if csv_columns:
out["csv_columns"] = csv_columns
return out
def validate_csv_template(
module: str,
field_mappings: Mapping[str, Any] | None,
type_conversions: Mapping[str, Any] | None = None,
import_row_processing: Mapping[str, Any] | None = None,
column_signature: list[str] | None = None,
) -> dict[str, Any]:
"""
Prüft eine Vorlage ohne Datei-Upload.
Returns:
``{"valid": bool, "errors": [...], "warnings": [...]}``
"""
errors: list[dict[str, Any]] = []
warnings: list[dict[str, Any]] = []
fm = dict(field_mappings or {})
tc: dict[str, Any] = dict(type_conversions or {}) if type_conversions else {}
mod = get_module_definition(module)
if not mod:
errors.append(
_issue(
"error",
"unknown_module",
f"Unbekanntes Modul «{module}».",
hint="Nur registrierte Module in module_registry sind erlaubt.",
)
)
return {"valid": False, "errors": errors, "warnings": warnings}
try:
validate_field_mappings(module, fm)
except ValueError as e:
errors.append(
_issue(
"error",
"invalid_field_mapping",
str(e),
hint="Jede Zuordnung muss auf ein bekanntes Zielfeld des Moduls zeigen (oder „–“ / ignorieren).",
)
)
try:
validate_required_field_targets(module, fm)
except ValueError as e:
errors.append(
_issue(
"error",
"missing_required_target",
str(e),
hint="Pflichtfelder des Moduls müssen mindestens einer CSV-Spalte zugeordnet sein.",
)
)
if import_row_processing:
try:
validate_import_row_processing_spec(module, import_row_processing, fm)
except ValueError as e:
errors.append(
_issue(
"error",
"invalid_import_row_processing",
str(e),
hint="import_row_processing: group_by und aggregates prüfen (siehe Doku Issue #21).",
)
)
field_defs = mod.get("fields") or {}
for db_field, spec in tc.items():
if db_field not in field_defs:
errors.append(
_issue(
"error",
"unknown_type_conversion_field",
f"type_conversions enthält unbekanntes Zielfeld «{db_field}».",
hint="Nur Felder aus der Moduldefinition sind erlaubt.",
field=db_field,
)
)
continue
if not isinstance(spec, Mapping):
errors.append(
_issue(
"error",
"type_conversion_not_object",
f"type_conversions[\"{db_field}\"] muss ein JSON-Objekt sein.",
field=db_field,
)
)
continue
stype = spec.get("type", "string")
if stype not in ALLOWED_SPEC_TYPES:
warnings.append(
_issue(
"warning",
"unusual_conversion_type",
f"Ungewöhnlicher Typ «{stype}» für «{db_field}» (erwartet u. a. string, float, date, datetime).",
field=db_field,
)
)
finfo = field_defs.get(db_field) or {}
expected = finfo.get("type")
if expected == "date" and stype not in ("date", "datetime"):
warnings.append(
_issue(
"warning",
"date_field_conversion",
f"Zielfeld «{db_field}» ist ein Datum; der Konvertierungstyp ist «{stype}».",
hint="Meist «date» oder «datetime» mit passendem format.",
field=db_field,
)
)
if expected == "float" and stype == "int" and db_field in ("hr_avg", "hr_max"):
warnings.append(
_issue(
"warning",
"hr_as_int",
"Herzfrequenz als «int» konvertiert; Nachkommastellen aus Apple-Export gehen verloren.",
hint="Optional «float» mit flexible: true verwenden.",
field=db_field,
)
)
# Mehrere CSV-Spalten → dasselbe Zielfeld
by_target: dict[str, list[str]] = {}
for csv_col, dbf in fm.items():
if dbf in (None, "", "-", "_skip"):
continue
by_target.setdefault(str(dbf), []).append(str(csv_col))
for dbf, cols in by_target.items():
if len(cols) > 1:
warnings.append(
_issue(
"warning",
"duplicate_target_columns",
f"Mehrere Spalten mappen auf «{dbf}»: {', '.join(cols)}.",
hint="Beim Import gewinnt die letzte Spalte in der CSV-Kopfzeilen-Reihenfolge.",
field=dbf,
csv_columns=cols,
)
)
# Kilojoule in kcal-Feldern (häufiger Apple-DE-Fehler)
for csv_col, dbf in fm.items():
if dbf not in ("kcal_active", "kcal_resting"):
continue
col_l = str(csv_col).lower()
if "kj" in col_l or "kilojoule" in col_l:
sub = tc.get(dbf)
su = (sub or {}).get("source_unit") if isinstance(sub, Mapping) else None
if str(su or "").strip().lower() != "kj":
warnings.append(
_issue(
"warning",
"energy_kj_without_source_unit",
f"Spalte «{csv_col}» deutet auf Kilojoule, Zielfeld «{dbf}» speichert kcal.",
hint='In type_conversions für dieses Feld "source_unit": "kj" setzen (Faktor 1/4.184).',
field=str(dbf),
csv_columns=[str(csv_col)],
)
)
# Signatur vs. gemappte Spalten: beide Seiten wie beim Import normalisieren
# (column_signature kann sortierte Normalform aus Analyse sein, field_mappings rohe Header).
if column_signature:
sig_forms = {
normalize_header_for_signature(str(c))
for c in column_signature
if str(c).strip()
}
sig_forms.discard("")
mapped_forms = {
normalize_header_for_signature(str(k))
for k in fm.keys()
if str(k).strip()
}
mapped_forms.discard("")
if sig_forms and mapped_forms and not sig_forms.intersection(mapped_forms):
warnings.append(
_issue(
"warning",
"signature_vs_mappings_mismatch",
"column_signature und field_mappings (Schlüssel) haben nach Normalisierung keine gemeinsame Spalte.",
hint="Prüfen Sie, ob die gespeicherte Signatur zur CSV passt; Zuordnungen nutzen rohe Kopfzeilen, die Signatur oft die gleiche Normalform wie in der Analyse.",
)
)
return {"valid": len(errors) == 0, "errors": errors, "warnings": warnings}
+710
View File
@@ -0,0 +1,710 @@
"""
Typkonvertierung für CSV-Zellen gemäß type_conversions-JSON (Issue #21).
Locale-robust: dieselbe Vorlage kann Exporte mit wechselndem Datumsformat oder
Dezimaltrenner verarbeiten, wenn flexible oder auto-Optionen gesetzt sind.
"""
from __future__ import annotations
import datetime as dt
import re
from decimal import Decimal, InvalidOperation
from typing import Any, Mapping, Sequence
from dateutil import parser as dateutil_parser
from csv_parser.core import normalize_header_for_signature
from csv_parser.field_units import factor_source_to_canonical
# Alias → strptime (JSON in Kleinbuchstaben)
DATE_FORMAT_STRPTIME: dict[str, str] = {
"yyyy-mm-dd": "%Y-%m-%d",
"mm/dd/yyyy": "%m/%d/%Y",
"dd/mm/yyyy": "%d/%m/%Y",
"dd.mm.yyyy": "%d.%m.%Y",
"dd.mm.yyyy hh:mm": "%d.%m.%Y %H:%M",
"dd.mm.yyyy HH:MM": "%d.%m.%Y %H:%M",
"yyyy-mm-dd hh:mm:ss": "%Y-%m-%d %H:%M:%S",
"yyyy-mm-dd HH:MM:SS": "%Y-%m-%d %H:%M:%S",
}
TIME_FORMAT_STRPTIME: dict[str, str] = {
"HH:MM": "%H:%M",
"HH:MM:SS": "%H:%M:%S",
}
# Wenn flexible: zusätzliche strptime-Versuche (ungefähr häufig → seltener)
_STRPTIME_FALLBACK_DATES: list[str] = [
"%Y-%m-%d",
"%d.%m.%Y",
"%d.%m.%y",
"%d/%m/%Y",
"%m/%d/%Y",
"%Y/%m/%d",
"%Y%m%d",
]
_STRPTIME_FALLBACK_DATETIME: list[str] = [
"%Y-%m-%d",
"%Y-%m-%d %H:%M:%S",
"%Y-%m-%d %H:%M",
"%d.%m.%Y %H:%M:%S",
"%d.%m.%Y %H:%M",
"%d.%m.%y %H:%M:%S",
"%d.%m.%y %H:%M",
"%Y-%m-%dT%H:%M:%S",
"%Y-%m-%dT%H:%M:%SZ",
"%Y-%m-%dT%H:%M:%S%z",
]
def _normalize_num_token(raw: str) -> str:
return re.sub(r"[\s\u00a0\u202f]", "", raw.strip())
def _parse_float_auto(s: str) -> float:
"""
Heuristik ohne festes Locale: Punkt/Komma als Tausender vs. Dezimal,
basierend auf der letzten erkannten Trennstelle und Gruppierung.
Apple Health u. a. liefern berechnete Mittelwerte mit vielen Nachkommastellen
(z. B. «96.874937…») und Energie als «596.668904…» — dabei ist der Punkt
immer Dezimaltrenner. Früher wurden lange Nachkommateile fälschlich so
behandelt, dass der Punkt entfernt wurde (Tausender-Heuristik).
"""
raw = s
s = _normalize_num_token(s)
if not s or s in ("-", "—", "–"):
raise ValueError("leer")
neg = False
if s.startswith("(") and s.endswith(")"):
neg = True
s = s[1:-1].strip()
if s.startswith("-"):
neg = not neg
s = s[1:]
elif s.startswith("+"):
s = s[1:]
last_comma = s.rfind(",")
last_dot = s.rfind(".")
if last_comma >= 0 and last_dot >= 0:
if last_comma > last_dot:
s = s.replace(".", "").replace(",", ".")
else:
s = s.replace(",", "")
elif last_comma >= 0:
parts = s.split(",")
if len(parts) == 2:
left, right = parts[0], parts[1]
if not right:
raise ValueError("leer")
left_digits = left.replace(".", "")
# Langer Nachkommateil → Dezimalkomma; «1.234,56»-Fälle oben mit Punkt+Komma
if len(right) > 3 or len(right) <= 2:
s = left_digits + "." + right.replace(".", "")
elif len(right) == 3 and len(left_digits) <= 3:
s = left_digits + right
else:
s = left_digits + "." + right.replace(".", "")
else:
s = s.replace(",", "")
elif last_dot >= 0:
parts = s.split(".")
if len(parts) == 2:
left, right = parts[0], parts[1]
if not right:
raise ValueError("leer")
left_digits = left.replace(",", "")
# Genau ein Punkt: viele Nachkommastellen → Apple/US-Dezimalpunkt (nicht „.“ streichen)
if len(right) > 3 or len(right) <= 2:
s = left_digits + "." + right
elif len(right) == 3:
if len(left_digits) == 1 and left_digits != "0" and left_digits.isdigit():
s = left_digits + right
else:
s = left_digits + "." + right
elif len(parts) > 2:
if len(parts[-1]) <= 2:
s = "".join(parts[:-1]) + "." + parts[-1]
else:
s = "".join(parts)
else:
s = s.replace(".", "")
try:
v = float(Decimal(s))
except (InvalidOperation, ValueError) as e:
raise ValueError(f"Zahl nicht parsbar: {raw!r}") from e
return -v if neg else v
def _parse_float(raw: str, decimal_sep: str) -> float:
s = _normalize_num_token(raw)
if not s:
raise ValueError("leer")
if "." in s and "," in s:
return _parse_float_auto(s)
if decimal_sep == ",":
if "," not in s and "." in s:
return _parse_float_auto(s)
s = s.replace(".", "").replace(",", ".")
else:
if "," in s and "." not in s:
return _parse_float_auto(s)
s = s.replace(",", "")
return float(Decimal(s))
def _float_from_spec(raw: str, spec: Mapping[str, Any]) -> float:
dec = spec.get("decimal_separator", ".")
flexible = bool(spec.get("flexible"))
if dec in (None, "auto"):
return _parse_float_auto(raw)
try:
return _parse_float(raw, str(dec))
except (InvalidOperation, ValueError):
if flexible:
return _parse_float_auto(raw)
raise
def _resolve_strptime_pattern(fmt_key: str) -> str | None:
k = fmt_key.strip()
if k.startswith("%"):
return k
return DATE_FORMAT_STRPTIME.get(k.lower())
def _collect_strptime_date_formats(spec: Mapping[str, Any], *, for_datetime: bool) -> list[str]:
seen: set[str] = set()
out: list[str] = []
def add(fmt_key: str) -> None:
p = _resolve_strptime_pattern(fmt_key)
if p and p not in seen:
seen.add(p)
out.append(p)
if not for_datetime and p.endswith(" %H:%M") and "%H:%M:%S" not in p:
p2 = p + ":%S"
if p2 not in seen:
seen.add(p2)
out.append(p2)
elif for_datetime and p.endswith(":%S"):
# z. B. Apple Health „2026-04-09 16:48“ ohne Sekunden
p_short = p[:-3]
if p_short not in seen:
seen.add(p_short)
out.append(p_short)
primary = spec.get("format")
if primary:
add(str(primary))
extra = spec.get("formats")
if isinstance(extra, Sequence) and not isinstance(extra, (str, bytes)):
for item in extra:
if item:
add(str(item))
if bool(spec.get("flexible")):
for p in _STRPTIME_FALLBACK_DATETIME if for_datetime else _STRPTIME_FALLBACK_DATES:
if p not in seen:
seen.add(p)
out.append(p)
return out
def _try_strptime(s: str, patterns: Sequence[str]) -> dt.datetime | None:
for pat in patterns:
try:
return dt.datetime.strptime(s, pat)
except ValueError:
continue
return None
def _try_strptime_trim_time(s: str, patterns: Sequence[str]) -> dt.datetime | None:
head = s.split(maxsplit=1)[0].strip() if s else ""
if head and head != s:
hit = _try_strptime(head, patterns)
if hit:
return hit
return _try_strptime(s, patterns)
def _normalize_locale_date_months(s: str) -> str:
"""
Omron Connect / Berichte: «10 Apr. 2026», «31 März 2026» — ohne DE→EN scheitert dateutil.
"""
if not s:
return s
out = s
for pat, rep in (
(r"März", "March"),
(r"Maerz", "March"),
(r"Januar", "January"),
(r"Februar", "February"),
(r"Oktober", "October"),
(r"Dezember", "December"),
(r"Juni", "June"),
(r"Juli", "July"),
(r"\bMai\b", "May"),
):
out = re.sub(pat, rep, out, flags=re.IGNORECASE)
return out
def _dateutil_parse(s: str, spec: Mapping[str, Any]) -> dt.datetime | None:
s_trim = s.strip()
dayfirst_opt = spec.get("dayfirst")
# ISO YYYY-MM-DD: dayfirst=True vertauscht Monat/Tag (09.04. → 04.09.)
iso_ymd_prefix = bool(re.match(r"^\d{4}-\d{2}-\d{2}(\D|$)", s_trim))
tries: list[bool | None]
if dayfirst_opt is True:
tries = [True]
elif dayfirst_opt is False:
tries = [False]
else:
tries = [False, True] if iso_ymd_prefix else [True, False]
for df in tries:
try:
return dateutil_parser.parse(s_trim, dayfirst=df)
except (ValueError, TypeError, OverflowError):
continue
return None
def _parse_date_typed(s: str, spec: Mapping[str, Any]) -> dt.date | dt.datetime:
extract = spec.get("extract", "date_only")
s0 = _normalize_locale_date_months(s.strip())
patterns = _collect_strptime_date_formats(spec, for_datetime=False)
part = _try_strptime_trim_time(s0, patterns) if patterns else None
if part is None:
part = _try_strptime(s0, _collect_strptime_date_formats(spec, for_datetime=True))
if part is None and (bool(spec.get("flexible")) or spec.get("formats")):
part = _dateutil_parse(s0, spec)
if part is None:
merged: dict[str, Any] = {**dict(spec), "flexible": True}
if "dayfirst" not in merged:
merged["dayfirst"] = True
part = _dateutil_parse(s0, merged)
if part is None:
fmt_key = str(spec.get("format", ""))
raise ValueError(f"Datum nicht parsbar: {fmt_key} / {s!r}")
if extract == "date_only":
return part.date()
return part
def _parse_datetime_typed(s: str, spec: Mapping[str, Any]) -> dt.datetime:
s0 = _normalize_locale_date_months(s.strip())
patterns = _collect_strptime_date_formats(spec, for_datetime=True)
part = _try_strptime(s0, patterns)
if part is None and (bool(spec.get("flexible")) or spec.get("formats")):
du = _dateutil_parse(s0, spec)
if du:
part = du
if part is None:
merged: dict[str, Any] = {**dict(spec), "flexible": True}
if "dayfirst" not in merged:
merged["dayfirst"] = True
du = _dateutil_parse(s0, merged)
if du:
part = du
if part is None:
fmt_key = str(spec.get("format", ""))
raise ValueError(f"Datetime nicht parsbar: {fmt_key} / {s!r}")
return part
def _parse_time_typed(s: str, spec: Mapping[str, Any]) -> dt.time:
patterns: list[str] = []
seen: set[str] = set()
primary = spec.get("format")
if primary:
fk = str(primary)
p = TIME_FORMAT_STRPTIME.get(fk, _resolve_strptime_pattern(fk) or fk)
if p not in seen:
seen.add(p)
patterns.append(p)
extra = spec.get("formats")
if isinstance(extra, Sequence) and not isinstance(extra, (str, bytes)):
for item in extra:
if not item:
continue
p = TIME_FORMAT_STRPTIME.get(str(item), str(item))
if p not in seen:
seen.add(p)
patterns.append(p)
if bool(spec.get("flexible")):
for p in ("%H:%M:%S", "%H:%M"):
if p not in seen:
seen.add(p)
patterns.append(p)
part = _try_strptime(s.strip(), patterns)
if part is None:
raise ValueError(f"Zeit nicht parsbar: {s!r}")
return part.time()
def _parse_int(raw: str, spec: Mapping[str, Any]) -> int:
s = raw.strip()
if bool(spec.get("flexible")) or spec.get("thousands_separator") == "auto":
s2 = _normalize_num_token(s)
if not s2 or s2 in ("-", "—", "–"):
raise ValueError("leer")
# EU-Dezimal (z. B. Apple DE «37,26» für HRV) — nicht alle Ziffern konkatenieren (würde 3726 → CHECK).
if "," in s2 or "." in s2:
try:
fv = _parse_float_auto(s2)
return int(round(fv))
except (ValueError, InvalidOperation):
pass
neg = s2.startswith("-")
body = s2[1:] if neg else s2
digits = re.sub(r"\D", "", body)
if not digits:
raise ValueError("leer")
v = int(digits)
return -v if neg else v
# Ohne flexible: «108.0» / «96,8» trotzdem als Zahl mit Nachkommastellen
s2 = _normalize_num_token(s)
if "," in s2 or "." in s2:
dec = spec.get("decimal_separator", ".")
try:
if dec in (None, "auto"):
fv = _parse_float_auto(s2)
else:
fv = _parse_float(raw, str(dec))
return int(round(fv))
except (ValueError, InvalidOperation):
pass
s = re.sub(r"[^\d-]", "", s)
if not s:
raise ValueError("leer")
return int(s)
def convert_value(
raw: str,
db_field: str,
spec: Mapping[str, Any] | None,
module: str | None = None,
) -> Any:
"""
Konvertiert eine Roh-Zelle in einen Python-Wert.
spec kommt aus type_conversions[db_field].
Optionen (JSON):
- flexible: true — nach Primärformat Fallbacks (Datum/Zahl/Zeit/Duration).
- decimal_separator: ".", ",", "auto" — bei auto Heuristik EU/US-Mischformen.
- formats: [ "yyyy-mm-dd", "%d.%m.%y", ... ] — weitere strptime-/Alias-Ketten.
- dayfirst: true|false — nur für dateutil-Fallback; Standard: true dann false.
- source_unit: Registry-IDs (z. B. "kj", "kg") oder "custom"/"none" — letztere ohne
vordefinierten Faktor; beliebige Skalierung dann nur über conversion_factor.
- conversion_factor: Zusätzlicher Multiplikator nach dem Parsen (und nach source_unit);
für nicht vordefinierte Umrechnungen source_unit weglassen oder "custom" setzen und
hier den Faktor angeben.
"""
if spec is None:
return raw.strip() if raw else None
if raw is None:
return None
s = raw.strip()
if s == "":
return None
t = spec.get("type", "string")
if t == "string":
return s
if t in ("float", "number"):
v = _float_from_spec(raw, spec)
if module:
su = spec.get("source_unit")
if su is not None and str(su).strip() != "":
v = float(v) * factor_source_to_canonical(module, db_field, str(su))
factor = spec.get("conversion_factor")
if factor is not None:
v = float(v) * float(factor)
return v
if t == "int":
return _parse_int(raw, spec)
if t == "date":
return _parse_date_typed(s, spec)
if t == "time":
return _parse_time_typed(s, spec)
if t == "datetime":
return _parse_datetime_typed(s, spec)
if t == "duration":
target = spec.get("target_unit", "minutes")
parts = [p.strip() for p in s.split(":")]
flexible = bool(spec.get("flexible"))
if len(parts) == 3:
try:
h, m, sec = int(parts[0]), int(parts[1]), int(parts[2])
if target == "minutes":
return round(h * 60 + m + sec / 60.0, 4)
except ValueError:
if not flexible:
raise ValueError(f"Duration nicht parsbar: {s!r}") from None
if flexible:
try:
h, m, sec = int(parts[0]), int(parts[1]), float(parts[2])
if target == "minutes":
return round(h * 60 + m + sec / 60.0, 4)
except ValueError:
pass
if len(parts) == 2:
try:
h, m = int(parts[0]), int(parts[1])
return h * 60 + m
except ValueError:
pass
raise ValueError(f"Duration nicht parsbar: {s!r}")
return s
def _lookup_db_field(csv_col: str, field_mappings: Mapping[str, str]) -> str | None:
"""
CSV-Spaltennamen können Roh-Header sein; Vorlagen-Schlüssel oft normalisiert
(wie column_signature). Exakter Treffer, dann Schlüssel nach Normalisierung,
dann Abgleich aller Vorlagen-Keys über deren Normalform.
"""
v = field_mappings.get(csv_col)
if v:
return v if v not in ("-", "_skip") else None
norm = normalize_header_for_signature(csv_col)
v = field_mappings.get(norm)
if v:
return v if v not in ("-", "_skip") else None
for k, fv in field_mappings.items():
if normalize_header_for_signature(str(k)) == norm:
return fv if fv not in ("-", "_skip") else None
return None
def _vitals_baseline_alias_db_field(csv_col: str) -> str | None:
"""
Apple Health: deutsch „Vitalwerte.csv“ (Breitexport) vs. schmale Vorlage
(Start / Resting Heart Rate …). Ohne Alias wählt die Analyse oft die
englische Vorlage → jede Zeile „Datum fehlt“.
Abgleich über normalisierten Header (normalize_header_for_signature).
"""
n = normalize_header_for_signature(str(csv_col))
if n in ("datum_uhrzeit", "start", "date_time", "datetime"):
return "date"
if "ruhepuls" in n or n.startswith("resting_heart_rate"):
return "resting_hr"
if "herzfrequenzvariabilit" in n or "heart_rate_variability" in n:
return "hrv"
if "vo2" in n and "max" in n:
return "vo2_max"
if "blutsauerstoff" in n or "oxygen_saturation" in n:
return "spo2"
if "atemfrequenz" in n or "respiratory_rate" in n:
return "respiratory_rate"
return None
def _blood_pressure_alias_db_field(csv_col: str) -> str | None:
"""
Omron (schmal) vs. Apple-Gesundheit (Breitexport): unterschiedliche Spaltennamen;
kombinierte Messzeit oft als „Start“ oder „Datum/Uhrzeit“.
"""
n = normalize_header_for_signature(str(csv_col))
low = str(csv_col).lower()
if n in ("datum_uhrzeit", "datetime", "date_time", "messzeitpunkt"):
return "start_time"
if n in ("start", "beginn"):
return "start_time"
if n in ("datum", "date", "messdatum"):
return "measured_date"
if n in ("zeit", "time", "uhrzeit"):
return "measured_time"
if "systolisch" in n or ("blutdruck" in n and "systol" in low) or n.startswith("systolic"):
return "systolic"
if "diastolisch" in n or ("blutdruck" in n and "diastol" in low) or n.startswith("diastolic"):
return "diastolic"
if n.startswith("puls") or n.startswith("pulse") or "puls_" in n:
return "pulse"
return None
def _activity_alias_db_field(csv_col: str) -> str | None:
"""
Apple-Workout schmal vs. Breitexport (viele Spalten): Trainingsart/Dauer/Strecke
trotzdem zuverlässig erkennen.
"""
n = normalize_header_for_signature(str(csv_col))
low = str(csv_col).lower()
if n in ("trainingsart", "workout_type", "activity_type", "workouttype"):
return "activity_type"
if ("trainings" in n and "art" in n) or ("workout" in low and "type" in low):
return "activity_type"
if n in ("datum_uhrzeit", "start", "beginn", "startzeit", "von"):
return "start_time"
if n in ("ende", "end", "endzeit", "bis"):
return "end_time"
if n in ("date", "datum"):
return "date"
if "dauer" in n or n == "duration" or n.startswith("duration_"):
return "duration_min"
if ("strecke" in n or "distance" in low) and ("km" in low or "(km" in low or " km" in low):
return "distance_km"
if "aktive_energie" in n or "active_energy" in n:
return "kcal_active"
if "ruheeintr" in n or "ruheenergie" in n or "resting_energy" in n:
return "kcal_resting"
if ("herzfrequenz" in n or "heart_rate" in n) and ("max" in low or "max" in n):
return "hr_max"
if (
"durchschnittliche_herzfrequenz" in n
or "heart_rate_average" in n
or ("herzfrequenz" in n and ("durchschn" in n or "avg" in low or "average" in low))
or ("heart_rate" in n and ("avg" in low or "average" in low))
):
return "hr_avg"
return None
def _effective_conversion_spec(
db_field: str,
spec: Mapping[str, Any] | None,
module: str | None,
) -> Mapping[str, Any] | None:
if spec is not None:
return spec
if module == "blood_pressure" and db_field == "start_time":
return {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True}
return None
def build_row_after_mapping(
csv_row: Mapping[str, str],
field_mappings: Mapping[str, str],
type_conversions: Mapping[str, Any] | None,
module: str | None = None,
) -> dict[str, Any]:
"""
Wendet Zuordnung csv_spalte → db_feld und Typkonvertierung an.
Unzugeordnete oder „—“ werden übersprungen.
Die Reihenfolge der Spalten in der CSV spielt keine Rolle (Dict-Zugriff nach Name).
Falls mehrere Spalten auf dasselbe db_field abbilden, gewinnt die zuletzt verarbeitete
(iterreihenfolge = Kopfzeilen-Reihenfolge in der Datei) — in der Praxis selten.
"""
out: dict[str, Any] = {}
tc = type_conversions or {}
for csv_col, raw in csv_row.items():
db_field = _lookup_db_field(str(csv_col), field_mappings)
if not db_field and module == "vitals_baseline":
db_field = _vitals_baseline_alias_db_field(csv_col)
elif not db_field and module == "blood_pressure":
db_field = _blood_pressure_alias_db_field(csv_col)
elif not db_field and module == "activity":
db_field = _activity_alias_db_field(csv_col)
if not db_field:
continue
raw_spec = tc.get(db_field) if isinstance(tc, dict) else None
if not isinstance(raw_spec, dict):
raw_spec = None
spec = _effective_conversion_spec(db_field, raw_spec, module)
try:
out[db_field] = convert_value(
raw, db_field, spec if isinstance(spec, dict) else None, module=module
)
except Exception:
out[db_field] = None
return out
def diagnose_row_mapping(
csv_row: Mapping[str, str],
field_mappings: Mapping[str, str],
type_conversions: Mapping[str, Any] | None,
module: str | None = None,
*,
mapped_typed: Mapping[str, Any] | None = None,
max_columns: int = 512,
) -> dict[str, Any]:
"""
Nur für Diagnose-Endpunkt: Quelle (Vorlage vs. Alias), Konvertierung pro Spalte,
Ergebnis wie build_row_after_mapping (json-freundliche Vorschau).
max_columns begrenzt nur die Länge der Liste „per_column“ in der Antwort — der echte
Import verarbeitet alle Spalten (siehe iter_csv_dict_rows / build_row_after_mapping).
"""
tc = type_conversions or {}
per_column: list[dict[str, Any]] = []
n = 0
for csv_col, raw in csv_row.items():
if n >= max_columns:
break
n += 1
sc = str(csv_col)
via_t = _lookup_db_field(sc, field_mappings)
via_a = None
if not via_t and module == "vitals_baseline":
via_a = _vitals_baseline_alias_db_field(sc)
elif not via_t and module == "blood_pressure":
via_a = _blood_pressure_alias_db_field(sc)
elif not via_t and module == "activity":
via_a = _activity_alias_db_field(sc)
target = via_t or via_a
src = "template" if via_t else ("alias" if via_a else "none")
raw_spec = tc.get(target) if isinstance(tc, dict) and target else None
if not isinstance(raw_spec, dict):
raw_spec = None
spec = _effective_conversion_spec(target, raw_spec, module) if target else None
conv_err: str | None = None
conv_preview: Any = None
if target:
try:
conv_val = convert_value(
(raw or "").strip(),
target,
spec if isinstance(spec, dict) else None,
module=module,
)
conv_preview = conv_val.isoformat() if hasattr(conv_val, "isoformat") else conv_val
except Exception as e:
conv_err = str(e)
per_column.append(
{
"csv_column": sc,
"raw_preview": ((raw or "")[:120]),
"db_field": target,
"source": src,
"convert_error": conv_err,
"converted_preview": conv_preview,
}
)
src_map = (
build_row_after_mapping(csv_row, field_mappings, type_conversions, module=module)
if mapped_typed is None
else mapped_typed
)
mapped_preview: dict[str, Any] = {}
for k, v in src_map.items():
mapped_preview[k] = v.isoformat() if hasattr(v, "isoformat") else v
tmpl_keys = [
str(k)
for k, v in field_mappings.items()
if v not in (None, "-", "_skip")
]
return {
"per_column": per_column,
"columns_truncated": len(csv_row) > max_columns,
"template_mapped_keys": tmpl_keys[:40],
"template_mapped_keys_truncated": len(tmpl_keys) > 40,
"mapped": mapped_preview,
}
+6
View File
@@ -51,6 +51,9 @@ __all__ = [
# Body Metrics (Basic)
'get_latest_weight_data',
'get_bmi_data',
'get_profile_goal_weight_data',
'get_profile_goal_bf_pct_data',
'get_weight_trend_data',
'get_body_composition_data',
'get_circumference_summary_data',
@@ -99,6 +102,9 @@ __all__ = [
'get_activity_summary_data',
'get_activity_detail_data',
'get_training_type_distribution_data',
'get_training_frequency_by_type_data',
'get_training_inter_session_gap_data',
'get_training_sessions_recent_weeks_data',
# Activity Metrics (Calculated)
'calculate_training_minutes_week',
+290 -26
View File
@@ -7,6 +7,9 @@ Functions:
- get_activity_summary_data(): Count, total duration, calories, averages
- get_activity_detail_data(): Detailed activity log entries
- get_training_type_distribution_data(): Training category percentages
- get_training_frequency_by_type_data(): Häufigkeit & Intensität pro activity_type
- get_training_inter_session_gap_data(): Pausen zwischen Einheiten (Stunden)
- get_training_sessions_recent_weeks_data(): Wochen-JSON für KI-Kontext
All functions return structured data (dict) without formatting.
Use placeholder_resolver.py for formatted strings for AI.
@@ -15,11 +18,11 @@ Phase 0c: Multi-Layer Architecture
Version: 1.0
"""
from typing import Dict, List, Optional
from datetime import datetime, timedelta, date
from typing import Dict, List, Optional, Any
from datetime import datetime, timedelta, date, time
import statistics
from db import get_db, get_cursor, r2d
from data_layer.utils import calculate_confidence, safe_float, safe_int
from data_layer.utils import calculate_confidence, safe_float, safe_int, serialize_dates
def get_activity_summary_data(
@@ -601,26 +604,23 @@ def calculate_activity_score(profile_id: str, focus_weights: Optional[Dict] = No
from data_layer.scores import get_user_focus_weights
focus_weights = get_user_focus_weights(profile_id)
# Activity-related focus areas (English keys from DB)
# Strength training
strength = focus_weights.get('strength', 0)
strength_endurance = focus_weights.get('strength_endurance', 0)
power = focus_weights.get('power', 0)
# Activity-related focus areas (English keys from DB); Gewichte float (kein Decimal×float)
strength = float(focus_weights.get('strength', 0) or 0)
strength_endurance = float(focus_weights.get('strength_endurance', 0) or 0)
power = float(focus_weights.get('power', 0) or 0)
total_strength = strength + strength_endurance + power
# Endurance training
aerobic = focus_weights.get('aerobic_endurance', 0)
anaerobic = focus_weights.get('anaerobic_endurance', 0)
cardiovascular = focus_weights.get('cardiovascular_health', 0)
aerobic = float(focus_weights.get('aerobic_endurance', 0) or 0)
anaerobic = float(focus_weights.get('anaerobic_endurance', 0) or 0)
cardiovascular = float(focus_weights.get('cardiovascular_health', 0) or 0)
total_cardio = aerobic + anaerobic + cardiovascular
# Mobility/Coordination
flexibility = focus_weights.get('flexibility', 0)
mobility = focus_weights.get('mobility', 0)
balance = focus_weights.get('balance', 0)
reaction = focus_weights.get('reaction', 0)
rhythm = focus_weights.get('rhythm', 0)
coordination = focus_weights.get('coordination', 0)
flexibility = float(focus_weights.get('flexibility', 0) or 0)
mobility = float(focus_weights.get('mobility', 0) or 0)
balance = float(focus_weights.get('balance', 0) or 0)
reaction = float(focus_weights.get('reaction', 0) or 0)
rhythm = float(focus_weights.get('rhythm', 0) or 0)
coordination = float(focus_weights.get('coordination', 0) or 0)
total_ability = flexibility + mobility + balance + reaction + rhythm + coordination
total_activity_weight = total_strength + total_cardio + total_ability
@@ -671,9 +671,9 @@ def calculate_activity_score(profile_id: str, focus_weights: Optional[Dict] = No
if not components:
return None
# Weighted average
total_score = sum(score * weight for _, score, weight in components)
total_weight = sum(weight for _, _, weight in components)
# Weighted average (float: DB-Aggregate können Decimal sein)
total_score = sum(float(score) * float(weight) for _, score, weight in components)
total_weight = sum(float(weight) for _, _, weight in components)
return int(total_score / total_weight)
@@ -725,12 +725,13 @@ def _score_cardio_presence(profile_id: str) -> Optional[int]:
if not row:
return None
cardio_days = row['cardio_days']
cardio_minutes = row['cardio_minutes'] or 0
# psycopg2: SUM() → oft Decimal — vor Mix mit float konvertieren
cardio_days = int(row['cardio_days'] or 0)
cardio_minutes = float(row['cardio_minutes'] or 0)
# Target: 3-5 days/week, 150+ minutes
day_score = min(100, (cardio_days / 4) * 100)
minute_score = min(100, (cardio_minutes / 150) * 100)
day_score = min(100.0, (cardio_days / 4) * 100)
minute_score = min(100.0, (cardio_minutes / 150) * 100)
return int((day_score + minute_score) / 2)
@@ -904,3 +905,266 @@ def calculate_activity_data_quality(profile_id: str) -> Dict[str, any]:
"quality": int(quality_score)
}
}
def _session_sort_ts(row: Dict) -> datetime:
"""Einheitlicher Zeitstempel für Sortierung und Pausenberechnung."""
d = row["date"]
if isinstance(d, str):
d = datetime.strptime(d[:10], "%Y-%m-%d").date()
st = row.get("start_time")
if st is None:
t = time(12, 0, 0)
else:
t = st
return datetime.combine(d, t)
def get_training_frequency_by_type_data(
profile_id: str,
days: int = 28,
) -> Dict[str, Any]:
"""
Pro activity_type (Roh-Label aus Import/Anzeige): Häufigkeit & Intensitätskennzahlen.
Returns:
{
"days_analyzed": int,
"confidence": str,
"by_type": [
{
"activity_type": str,
"session_count": int,
"sessions_per_week": float,
"avg_duration_min": float | None,
"avg_kcal_active": float | None,
"avg_hr_avg": float | None,
"avg_hr_max": float | None,
"avg_rpe": float | None,
"avg_kcal_per_min": float | None, # grobe Intensität, wenn kcal & Dauer
},
...
],
}
"""
weeks = max(days / 7.0, 0.01)
with get_db() as conn:
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
cur.execute(
"""
SELECT
activity_type,
COUNT(*)::int AS session_count,
AVG(duration_min)::float AS avg_duration_min,
AVG(kcal_active)::float AS avg_kcal_active,
AVG(hr_avg)::float AS avg_hr_avg,
AVG(hr_max)::float AS avg_hr_max,
AVG(rpe)::float AS avg_rpe,
SUM(COALESCE(duration_min, 0))::float AS sum_duration,
SUM(COALESCE(kcal_active, 0))::float AS sum_kcal
FROM activity_log
WHERE profile_id = %s AND date >= %s
GROUP BY activity_type
ORDER BY session_count DESC
""",
(profile_id, cutoff),
)
rows = [r2d(r) for r in cur.fetchall()]
if not rows:
return {
"days_analyzed": days,
"confidence": "insufficient",
"by_type": [],
}
by_type = []
for r in rows:
sc = int(r["session_count"])
sum_dur = float(r["sum_duration"] or 0)
sum_kcal = float(r["sum_kcal"] or 0)
kcal_per_min = (sum_kcal / sum_dur) if sum_dur > 0 else None
by_type.append(
{
"activity_type": r["activity_type"],
"session_count": sc,
"sessions_per_week": round(sc / weeks, 2),
"avg_duration_min": r["avg_duration_min"],
"avg_kcal_active": r["avg_kcal_active"],
"avg_hr_avg": r["avg_hr_avg"],
"avg_hr_max": r["avg_hr_max"],
"avg_rpe": r["avg_rpe"],
"avg_kcal_per_min": round(kcal_per_min, 2) if kcal_per_min is not None else None,
}
)
total_sessions = sum(x["session_count"] for x in by_type)
confidence = calculate_confidence(total_sessions, days, "general")
return {
"days_analyzed": days,
"confidence": confidence,
"by_type": by_type,
}
def get_training_inter_session_gap_data(
profile_id: str,
days: int = 28,
) -> Dict[str, Any]:
"""
Mittlere/median Pausen zwischen aufeinanderfolgenden Trainingseinheiten (Stunden).
Sortierung: Datum + start_time (fehlend → 12:00), dann created.
"""
with get_db() as conn:
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
cur.execute(
"""
SELECT date, start_time, created
FROM activity_log
WHERE profile_id = %s AND date >= %s
ORDER BY date ASC, start_time ASC NULLS LAST, created ASC
""",
(profile_id, cutoff),
)
rows = [r2d(r) for r in cur.fetchall()]
if len(rows) < 2:
return {
"days_analyzed": days,
"confidence": "insufficient",
"gap_hours_median": None,
"gap_hours_mean": None,
"gap_hours_min": None,
"gaps_count": 0,
}
gaps = []
prev_ts = None
for r in rows:
ts = _session_sort_ts(r)
if prev_ts is not None:
gaps.append((ts - prev_ts).total_seconds() / 3600.0)
prev_ts = ts
if not gaps:
return {
"days_analyzed": days,
"confidence": "insufficient",
"gap_hours_median": None,
"gap_hours_mean": None,
"gap_hours_min": None,
"gaps_count": 0,
}
gaps_sorted = sorted(gaps)
mid = len(gaps_sorted) // 2
median = (
gaps_sorted[mid]
if len(gaps_sorted) % 2
else (gaps_sorted[mid - 1] + gaps_sorted[mid]) / 2.0
)
confidence = calculate_confidence(len(rows), days, "general")
return {
"days_analyzed": days,
"confidence": confidence,
"gap_hours_median": round(median, 1),
"gap_hours_mean": round(statistics.mean(gaps), 1),
"gap_hours_min": round(min(gaps), 1),
"gaps_count": len(gaps),
}
def get_training_sessions_recent_weeks_data(
profile_id: str,
weeks: int = 4,
) -> Dict[str, Any]:
"""
Letzte Wochen mit Einzeltrainings für KI-Kontext (Dauer, kcal, HF, Typ).
weeks: Anzahl zurückliegender ISO-Kalenderwochen (Default 4).
"""
days = max(weeks * 7, 7)
with get_db() as conn:
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
cur.execute(
"""
SELECT
a.id,
a.date,
a.start_time,
a.activity_type,
a.training_category,
a.duration_min,
a.kcal_active,
a.hr_avg,
a.hr_max,
a.rpe,
tt.name_de AS training_type_name
FROM activity_log a
LEFT JOIN training_types tt ON tt.id = a.training_type_id
WHERE a.profile_id = %s AND a.date >= %s
ORDER BY a.date ASC, a.start_time ASC NULLS LAST, a.created ASC
""",
(profile_id, cutoff),
)
rows = [r2d(r) for r in cur.fetchall()]
if not rows:
return {
"weeks": [],
"meta": {
"weeks_requested": weeks,
"days_loaded": days,
"session_count": 0,
"confidence": "insufficient",
},
}
by_week: Dict[str, List[Dict]] = {}
for r in rows:
d = r["date"]
if isinstance(d, str):
d = datetime.strptime(d[:10], "%Y-%m-%d").date()
iso = d.isocalendar()
wk = f"{iso.year}-W{iso.week:02d}"
if wk not in by_week:
by_week[wk] = []
dur = r.get("duration_min")
dur_f = float(dur) if dur is not None else None
kcal = r.get("kcal_active")
kcal_f = float(kcal) if kcal is not None else None
hr_a = r.get("hr_avg")
hr_m = r.get("hr_max")
by_week[wk].append(
{
"date": d,
"start_time": str(r["start_time"]) if r.get("start_time") is not None else None,
"activity_type": r.get("activity_type"),
"training_category": r.get("training_category"),
"training_type_name": r.get("training_type_name"),
"duration_min": dur_f,
"kcal_active": kcal_f,
"hr_avg": int(hr_a) if hr_a is not None else None,
"hr_max": int(hr_m) if hr_m is not None else None,
"rpe": int(r["rpe"]) if r.get("rpe") is not None else None,
}
)
week_keys = sorted(by_week.keys())
weeks_out = [{"week_iso": wk, "sessions": by_week[wk]} for wk in week_keys]
confidence = calculate_confidence(len(rows), days, "general")
return serialize_dates(
{
"weeks": weeks_out,
"meta": {
"weeks_requested": weeks,
"days_loaded": days,
"session_count": len(rows),
"confidence": confidence,
},
}
)
+145 -24
View File
@@ -5,6 +5,9 @@ Provides structured data for body composition and measurements.
Functions:
- get_latest_weight_data(): Most recent weight entry
- get_bmi_data(): BMI from latest weight + profile height
- get_profile_goal_weight_data(): Zielgewicht (Profilfeld)
- get_profile_goal_bf_pct_data(): Ziel-KFA % (Profilfeld)
- get_weight_trend_data(): Weight trend with slope and direction
- get_body_composition_data(): Body fat percentage and lean mass
- get_circumference_summary_data(): Latest circumference measurements
@@ -68,6 +71,105 @@ def get_latest_weight_data(
}
def get_bmi_data(profile_id: str) -> Dict:
"""
BMI from latest weight_log entry and profiles.height (cm).
Returns:
{
"bmi": float | None,
"weight_kg": float | None,
"height_cm": float | None,
"confidence": "high" | "insufficient",
}
"""
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""
SELECT pr.height,
(SELECT wl.weight FROM weight_log wl
WHERE wl.profile_id = pr.id
ORDER BY wl.date DESC
LIMIT 1) AS weight
FROM profiles pr
WHERE pr.id = %s
""",
(profile_id,),
)
row = cur.fetchone()
if not row:
return {
"bmi": None,
"weight_kg": None,
"height_cm": None,
"confidence": "insufficient",
}
height_cm = row["height"]
weight = row["weight"]
if height_cm is None or weight is None:
return {
"bmi": None,
"weight_kg": safe_float(weight) if weight is not None else None,
"height_cm": safe_float(height_cm) if height_cm is not None else None,
"confidence": "insufficient",
}
h = safe_float(height_cm)
w = safe_float(weight)
if h <= 0:
return {
"bmi": None,
"weight_kg": w,
"height_cm": h,
"confidence": "insufficient",
}
height_m = h / 100.0
bmi = w / (height_m ** 2)
return {
"bmi": bmi,
"weight_kg": w,
"height_cm": h,
"confidence": "high",
}
def get_profile_goal_weight_data(profile_id: str) -> Dict:
"""Strategisches Zielgewicht aus profiles.goal_weight (kg), nicht goals-Tabelle."""
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"SELECT goal_weight FROM profiles WHERE id=%s",
(profile_id,),
)
row = cur.fetchone()
if not row or row.get("goal_weight") is None:
return {"goal_weight_kg": None, "confidence": "insufficient"}
return {
"goal_weight_kg": safe_float(row["goal_weight"]),
"confidence": "high",
}
def get_profile_goal_bf_pct_data(profile_id: str) -> Dict:
"""Strategisches Ziel-KFA aus profiles.goal_bf_pct (%), nicht goals-Tabelle."""
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"SELECT goal_bf_pct FROM profiles WHERE id=%s",
(profile_id,),
)
row = cur.fetchone()
if not row or row.get("goal_bf_pct") is None:
return {"goal_bf_pct": None, "confidence": "insufficient"}
return {
"goal_bf_pct": safe_float(row["goal_bf_pct"]),
"confidence": "high",
}
def get_weight_trend_data(
profile_id: str,
days: int = 28
@@ -89,7 +191,8 @@ def get_weight_trend_data(
"confidence": str,
"days_analyzed": int,
"first_date": date,
"last_date": date
"last_date": date,
"series": [{"date": date, "weight": float}, ...], # für Charts ohne zweites Query
}
Confidence Rules:
@@ -127,7 +230,8 @@ def get_weight_trend_data(
"delta": 0.0,
"direction": "unknown",
"first_date": None,
"last_date": None
"last_date": None,
"series": [],
}
# Extract values
@@ -152,7 +256,11 @@ def get_weight_trend_data(
"confidence": confidence,
"days_analyzed": days,
"first_date": rows[0]['date'],
"last_date": rows[-1]['date']
"last_date": rows[-1]['date'],
"series": [
{"date": r["date"], "weight": safe_float(r["weight"])}
for r in rows
],
}
@@ -337,12 +445,16 @@ def calculate_weight_7d_median(profile_id: str) -> Optional[float]:
ORDER BY date DESC
""", (profile_id,))
weights = [row['weight'] for row in cur.fetchall()]
weights = [
safe_float(row['weight'])
for row in cur.fetchall()
if row['weight'] is not None
]
if len(weights) < 4: # Need at least 4 measurements
return None
return round(statistics.median(weights), 1)
return round(float(statistics.median(weights)), 1)
def calculate_weight_28d_slope(profile_id: str) -> Optional[float]:
@@ -370,7 +482,11 @@ def _calculate_weight_slope(profile_id: str, days: int) -> Optional[float]:
ORDER BY date
""", (profile_id, days))
data = [(row['date'], row['weight']) for row in cur.fetchall()]
data = [
(row['date'], safe_float(row['weight']))
for row in cur.fetchall()
if row['weight'] is not None
]
# Need minimum data points based on period
min_points = max(18, int(days * 0.6)) # 60% coverage
@@ -380,21 +496,21 @@ def _calculate_weight_slope(profile_id: str, days: int) -> Optional[float]:
# Convert dates to days since start
start_date = data[0][0]
x_values = [(date - start_date).days for date, _ in data]
y_values = [weight for _, weight in data]
y_values = [w for _, w in data]
# Linear regression
# Linear regression (alles float: PostgreSQL numeric → Decimal in Python)
n = len(data)
x_mean = sum(x_values) / n
y_mean = sum(y_values) / n
x_mean = float(sum(x_values)) / n
y_mean = float(sum(y_values)) / n
numerator = sum((x - x_mean) * (y - y_mean) for x, y in zip(x_values, y_values))
denominator = sum((x - x_mean) ** 2 for x in x_values)
numerator = sum(float(x - x_mean) * float(y - y_mean) for x, y in zip(x_values, y_values))
denominator = float(sum((x - x_mean) ** 2 for x in x_values))
if denominator == 0:
return None
slope = numerator / denominator
return round(slope, 4) # kg/day
return round(float(slope), 4) # kg/day
def calculate_goal_projection_date(profile_id: str, goal_id: str) -> Optional[str]:
@@ -486,19 +602,24 @@ def _calculate_body_composition_change(profile_id: str, metric: str, days: int)
recent = data[0]
oldest = data[-1]
# Calculate FM and LBM
recent_fm = recent['weight'] * (recent['bf_pct'] / 100)
recent_lbm = recent['weight'] - recent_fm
# Calculate FM and LBM (DB numeric → Decimal; für Regression/Scores nur float)
rw = float(safe_float(recent['weight']) or 0)
ob = float(safe_float(recent['bf_pct']) or 0)
ow = float(safe_float(oldest['weight']) or 0)
obf = float(safe_float(oldest['bf_pct']) or 0)
oldest_fm = oldest['weight'] * (oldest['bf_pct'] / 100)
oldest_lbm = oldest['weight'] - oldest_fm
recent_fm = rw * (ob / 100)
recent_lbm = rw - recent_fm
oldest_fm = ow * (obf / 100)
oldest_lbm = ow - oldest_fm
if metric == 'fm':
change = recent_fm - oldest_fm
else:
change = recent_lbm - oldest_lbm
return round(change, 2)
return round(float(change), 2)
# ── Circumference Calculations ──────────────────────────────────────────────
@@ -623,9 +744,9 @@ def calculate_body_progress_score(profile_id: str, focus_weights: Optional[Dict]
from data_layer.scores import get_user_focus_weights
focus_weights = get_user_focus_weights(profile_id)
weight_loss = focus_weights.get('weight_loss', 0)
muscle_gain = focus_weights.get('muscle_gain', 0)
body_recomp = focus_weights.get('body_recomposition', 0)
weight_loss = float(focus_weights.get('weight_loss', 0) or 0)
muscle_gain = float(focus_weights.get('muscle_gain', 0) or 0)
body_recomp = float(focus_weights.get('body_recomposition', 0) or 0)
total_body_weight = weight_loss + muscle_gain + body_recomp
@@ -652,8 +773,8 @@ def calculate_body_progress_score(profile_id: str, focus_weights: Optional[Dict]
if not components:
return None
total_score = sum(score * weight for _, score, weight in components)
total_weight = sum(weight for _, _, weight in components)
total_score = sum(float(score) * float(weight) for _, score, weight in components)
total_weight = sum(float(weight) for _, _, weight in components)
return int(total_score / total_weight)
+288 -227
View File
@@ -25,6 +25,88 @@ from datetime import datetime, timedelta, date
from db import get_db, get_cursor, r2d
from data_layer.utils import calculate_confidence, safe_float, safe_int
# Fallback TDEE (kcal/day) when demographics for Mifflin–St Jeor are incomplete.
TDEE_KCAL_PER_KG_BODYWEIGHT = 32.5
# PAL applied to MSJ BMR when height, sex, dob and weight are available (moderate activity).
TDEE_PAL_MODERATE = 1.55
def _age_years_from_dob(dob) -> Optional[int]:
if dob is None:
return None
try:
if isinstance(dob, str):
birth = datetime.strptime(dob[:10], "%Y-%m-%d").date()
else:
birth = dob
today = date.today()
return today.year - birth.year - ((today.month, today.day) < (birth.month, birth.day))
except Exception:
return None
def _mifflin_st_jeor_bmr_kcal(
weight_kg: float, height_cm: float, age_years: int, sex_is_male: bool
) -> float:
if sex_is_male:
return 10.0 * weight_kg + 6.25 * height_cm - 5.0 * age_years + 5.0
return 10.0 * weight_kg + 6.25 * height_cm - 5.0 * age_years - 161.0
def estimate_tdee_kcal_from_latest_weight(profile_id: str) -> Optional[float]:
"""
Estimated TDEE (kcal/day).
Primary: Mifflin–St Jeor BMR × TDEE_PAL_MODERATE when latest weight plus
profiles.height, profiles.sex, profiles.dob are usable.
Fallback: latest weight (kg) × TDEE_KCAL_PER_KG_BODYWEIGHT (legacy heuristic).
Returns None if no weight on record.
"""
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""SELECT weight FROM weight_log
WHERE profile_id=%s ORDER BY date DESC LIMIT 1""",
(profile_id,),
)
wrow = cur.fetchone()
if not wrow or wrow["weight"] is None:
return None
weight_kg = float(wrow["weight"])
cur.execute(
"SELECT height, sex, dob FROM profiles WHERE id=%s",
(profile_id,),
)
prow = cur.fetchone()
if prow and prow.get("height") and prow.get("sex") is not None and prow.get("dob"):
height_cm = float(prow["height"])
age = _age_years_from_dob(prow["dob"])
if age is not None and 10 < age < 120 and height_cm > 50:
sex_raw = str(prow["sex"]).strip().lower()
sex_is_male = sex_raw in ("m", "male", "männlich", "mann")
bmr = _mifflin_st_jeor_bmr_kcal(weight_kg, height_cm, age, sex_is_male)
if bmr > 400:
return bmr * TDEE_PAL_MODERATE
return weight_kg * TDEE_KCAL_PER_KG_BODYWEIGHT
def _get_profile_goal_mode(profile_id: str) -> str:
"""Strategic goal_mode from profiles (Phase 0a); defaults to health."""
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT goal_mode FROM profiles WHERE id=%s", (profile_id,))
row = cur.fetchone()
if row and row.get("goal_mode"):
g = str(row["goal_mode"]).strip().lower()
if g:
return g
return "health"
def get_nutrition_average_data(
profile_id: str,
@@ -56,20 +138,29 @@ def get_nutrition_average_data(
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
# Mean over calendar days (per-day sums), not over raw log rows.
cur.execute(
"""SELECT
AVG(kcal) as kcal_avg,
AVG(protein_g) as protein_avg,
AVG(carbs_g) as carbs_avg,
AVG(fat_g) as fat_avg,
COUNT(*) as data_points
FROM nutrition_log
WHERE profile_id=%s AND date >= %s""",
(profile_id, cutoff)
AVG(daily_kcal) AS kcal_avg,
AVG(daily_protein) AS protein_avg,
AVG(daily_carbs) AS carbs_avg,
AVG(daily_fat) AS fat_avg,
COUNT(*)::int AS day_count
FROM (
SELECT date,
COALESCE(SUM(kcal), 0)::float AS daily_kcal,
COALESCE(SUM(protein_g), 0)::float AS daily_protein,
COALESCE(SUM(carbs_g), 0)::float AS daily_carbs,
COALESCE(SUM(fat_g), 0)::float AS daily_fat
FROM nutrition_log
WHERE profile_id=%s AND date >= %s
GROUP BY date
) AS daily""",
(profile_id, cutoff),
)
row = cur.fetchone()
if not row or row['data_points'] == 0:
if not row or row["day_count"] == 0:
return {
"kcal_avg": 0.0,
"protein_avg": 0.0,
@@ -80,7 +171,7 @@ def get_nutrition_average_data(
"days_analyzed": days
}
data_points = row['data_points']
data_points = row["day_count"]
confidence = calculate_confidence(data_points, days, "general")
return {
@@ -190,79 +281,73 @@ def get_energy_balance_data(
days: int = 7
) -> Dict:
"""
Calculate energy balance (intake - estimated expenditure).
Energy balance (intake - estimated expenditure), kcal/day.
Note: This is a simplified calculation.
For accurate TDEE, use profile-based calculations.
Args:
profile_id: User profile ID
days: Analysis window (default 7)
Returns:
{
"energy_balance": float, # kcal/day (negative = deficit)
"avg_intake": float,
"estimated_tdee": float,
"status": str, # "deficit" | "surplus" | "maintenance"
"confidence": str,
"days_analyzed": int,
"data_points": int
}
Intake: mean of daily total kcal (sum per calendar day).
TDEE: estimate_tdee_kcal_from_latest_weight (MSJ × PAL oder kg-Fallback).
"""
with get_db() as conn:
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
# Get average intake
cur.execute(
"""SELECT AVG(kcal) as avg_kcal, COUNT(*) as cnt
"""SELECT date, SUM(kcal)::float AS daily_kcal
FROM nutrition_log
WHERE profile_id=%s AND date >= %s AND kcal IS NOT NULL""",
(profile_id, cutoff)
WHERE profile_id=%s AND date >= %s AND kcal IS NOT NULL
GROUP BY date
ORDER BY date""",
(profile_id, cutoff),
)
row = cur.fetchone()
if not row or row['cnt'] == 0:
return {
"energy_balance": 0.0,
"avg_intake": 0.0,
"estimated_tdee": 0.0,
"status": "unknown",
"confidence": "insufficient",
"days_analyzed": days,
"data_points": 0
}
avg_intake = safe_float(row['avg_kcal'])
data_points = row['cnt']
# Simple TDEE estimation (this should be improved with profile data)
# For now, use a rough estimate: 2500 kcal for average adult
estimated_tdee = 2500.0 # TODO: Calculate from profile (weight, height, age, activity)
energy_balance = avg_intake - estimated_tdee
# Determine status
if energy_balance < -200:
status = "deficit"
elif energy_balance > 200:
status = "surplus"
else:
status = "maintenance"
confidence = calculate_confidence(data_points, days, "general")
daily_rows = cur.fetchall()
if not daily_rows:
return {
"energy_balance": energy_balance,
"energy_balance": 0.0,
"avg_intake": 0.0,
"estimated_tdee": 0.0,
"status": "unknown",
"confidence": "insufficient",
"days_analyzed": days,
"data_points": 0,
}
daily_totals = [safe_float(r["daily_kcal"]) for r in daily_rows]
avg_intake = sum(daily_totals) / len(daily_totals)
data_points = len(daily_totals)
estimated_tdee = estimate_tdee_kcal_from_latest_weight(profile_id)
if estimated_tdee is None:
return {
"energy_balance": 0.0,
"avg_intake": avg_intake,
"estimated_tdee": estimated_tdee,
"status": status,
"confidence": confidence,
"estimated_tdee": 0.0,
"status": "unknown",
"confidence": "insufficient",
"days_analyzed": days,
"data_points": data_points
}
energy_balance = avg_intake - estimated_tdee
if energy_balance < -200:
status = "deficit"
elif energy_balance > 200:
status = "surplus"
else:
status = "maintenance"
confidence = calculate_confidence(data_points, days, "general")
return {
"energy_balance": energy_balance,
"avg_intake": avg_intake,
"estimated_tdee": estimated_tdee,
"status": status,
"confidence": confidence,
"days_analyzed": days,
"data_points": data_points
}
def get_protein_adequacy_data(
profile_id: str,
@@ -291,7 +376,6 @@ def get_protein_adequacy_data(
"confidence": str
}
"""
# Get protein targets
targets = get_protein_targets_data(profile_id)
with get_db() as conn:
@@ -299,60 +383,55 @@ def get_protein_adequacy_data(
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
cur.execute(
"""SELECT
AVG(protein_g) as avg_protein,
COUNT(*) as cnt,
SUM(CASE WHEN protein_g >= %s AND protein_g <= %s THEN 1 ELSE 0 END) as days_in_target
"""SELECT COALESCE(SUM(protein_g), 0)::float AS daily_protein
FROM nutrition_log
WHERE profile_id=%s AND date >= %s AND protein_g IS NOT NULL""",
(targets['protein_target_low'], targets['protein_target_high'], profile_id, cutoff)
WHERE profile_id=%s AND date >= %s
GROUP BY date""",
(profile_id, cutoff),
)
row = cur.fetchone()
if not row or row['cnt'] == 0:
return {
"adequacy_score": 0,
"avg_protein_g": 0.0,
"target_protein_low": targets['protein_target_low'],
"target_protein_high": targets['protein_target_high'],
"protein_g_per_kg": 0.0,
"days_in_target": 0,
"days_with_data": 0,
"confidence": "insufficient"
}
avg_protein = safe_float(row['avg_protein'])
days_with_data = row['cnt']
days_in_target = row['days_in_target']
protein_g_per_kg = avg_protein / targets['current_weight'] if targets['current_weight'] > 0 else 0.0
# Calculate adequacy score
# 100 = always in target range
# Scale based on percentage of days in target + average relative to target
target_pct = (days_in_target / days_with_data * 100) if days_with_data > 0 else 0
# Bonus/penalty for average protein level
target_mid = (targets['protein_target_low'] + targets['protein_target_high']) / 2
avg_vs_target = (avg_protein / target_mid) if target_mid > 0 else 0
# Weighted score: 70% target days, 30% average level
adequacy_score = int(target_pct * 0.7 + min(avg_vs_target * 100, 100) * 0.3)
adequacy_score = max(0, min(100, adequacy_score)) # Clamp to 0-100
confidence = calculate_confidence(days_with_data, days, "general")
rows = cur.fetchall()
if not rows or targets.get("confidence") == "insufficient" or targets["current_weight"] <= 0:
return {
"adequacy_score": adequacy_score,
"avg_protein_g": avg_protein,
"adequacy_score": 0,
"avg_protein_g": 0.0,
"target_protein_low": targets['protein_target_low'],
"target_protein_high": targets['protein_target_high'],
"protein_g_per_kg": protein_g_per_kg,
"days_in_target": days_in_target,
"days_with_data": days_with_data,
"confidence": confidence
"protein_g_per_kg": 0.0,
"days_in_target": 0,
"days_with_data": 0,
"confidence": "insufficient"
}
daily_totals = [safe_float(r["daily_protein"]) for r in rows]
days_with_data = len(daily_totals)
low = targets["protein_target_low"]
high = targets["protein_target_high"]
days_in_target = sum(1 for d in daily_totals if low <= d <= high)
avg_protein = sum(daily_totals) / days_with_data
protein_g_per_kg = avg_protein / targets["current_weight"] if targets["current_weight"] > 0 else 0.0
target_pct = (days_in_target / days_with_data * 100) if days_with_data > 0 else 0
target_mid = (low + high) / 2
avg_vs_target = (avg_protein / target_mid) if target_mid > 0 else 0
adequacy_score = int(target_pct * 0.7 + min(avg_vs_target * 100, 100) * 0.3)
adequacy_score = max(0, min(100, adequacy_score))
confidence = calculate_confidence(days_with_data, days, "general")
return {
"adequacy_score": adequacy_score,
"avg_protein_g": avg_protein,
"target_protein_low": targets['protein_target_low'],
"target_protein_high": targets['protein_target_high'],
"protein_g_per_kg": protein_g_per_kg,
"days_in_target": days_in_target,
"days_with_data": days_with_data,
"confidence": confidence
}
def get_macro_consistency_data(
profile_id: str,
@@ -387,16 +466,18 @@ def get_macro_consistency_data(
cur.execute(
"""SELECT
protein_g, carbs_g, fat_g, kcal
COALESCE(SUM(kcal), 0)::float AS kcal,
COALESCE(SUM(protein_g), 0)::float AS protein_g,
COALESCE(SUM(carbs_g), 0)::float AS carbs_g,
COALESCE(SUM(fat_g), 0)::float AS fat_g
FROM nutrition_log
WHERE profile_id=%s
AND date >= %s
AND protein_g IS NOT NULL
AND carbs_g IS NOT NULL
AND fat_g IS NOT NULL
AND kcal > 0
ORDER BY date""",
(profile_id, cutoff)
WHERE profile_id=%s AND date >= %s
GROUP BY date
HAVING COALESCE(SUM(kcal), 0) > 0
AND COALESCE(SUM(protein_g), 0) > 0
AND COALESCE(SUM(carbs_g), 0) > 0
AND COALESCE(SUM(fat_g), 0) > 0""",
(profile_id, cutoff),
)
rows = cur.fetchall()
@@ -413,7 +494,6 @@ def get_macro_consistency_data(
"data_points": len(rows)
}
# Calculate macro percentages for each day
import statistics
protein_pcts = []
@@ -425,7 +505,6 @@ def get_macro_consistency_data(
if total_kcal == 0:
continue
# Convert grams to kcal (protein=4, carbs=4, fat=9)
protein_kcal = safe_float(row['protein_g']) * 4
carbs_kcal = safe_float(row['carbs_g']) * 4
fat_kcal = safe_float(row['fat_g']) * 9
@@ -491,50 +570,15 @@ def get_macro_consistency_data(
def calculate_energy_balance_7d(profile_id: str) -> Optional[float]:
"""
Calculate 7-day average energy balance (kcal/day)
Positive = surplus, Negative = deficit
Migration from Phase 0b:
Used by placeholders that need single balance value
7-day mean energy balance (kcal/day), same rules as get_energy_balance_data(..., 7).
"""
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("""
SELECT kcal
FROM nutrition_log
WHERE profile_id = %s
AND date >= CURRENT_DATE - INTERVAL '7 days'
ORDER BY date DESC
""", (profile_id,))
calories = [row['kcal'] for row in cur.fetchall()]
if len(calories) < 4: # Need at least 4 days
return None
avg_intake = float(sum(calories) / len(calories))
# Get estimated TDEE (simplified - could use Harris-Benedict)
# For now, use weight-based estimate
cur.execute("""
SELECT weight
FROM weight_log
WHERE profile_id = %s
ORDER BY date DESC
LIMIT 1
""", (profile_id,))
weight_row = cur.fetchone()
if not weight_row:
return None
# Simple TDEE estimate: bodyweight (kg) × 30-35
# TODO: Improve with activity level, age, gender
estimated_tdee = float(weight_row['weight']) * 32.5
balance = avg_intake - estimated_tdee
return round(balance, 0)
data = get_energy_balance_data(profile_id, 7)
if data["data_points"] < 4:
return None
tdee = data.get("estimated_tdee") or 0
if tdee <= 0:
return None
return round(float(data["energy_balance"]), 0)
def calculate_energy_deficit_surplus(profile_id: str, days: int = 7) -> Optional[str]:
@@ -654,15 +698,14 @@ def calculate_protein_days_in_target(profile_id: str, target_low: float = 1.6, t
def calculate_protein_adequacy_28d(profile_id: str) -> Optional[int]:
"""
Protein adequacy score 0-100 (last 28 days)
Based on consistency and target achievement
Protein adequacy score 0-100 (last 28 days).
Uses per-calendar-day total protein vs. average weight in the window (g/kg per day).
"""
import statistics
with get_db() as conn:
cur = get_cursor(conn)
# Get average weight (28d)
cur.execute("""
SELECT AVG(weight) as avg_weight
FROM weight_log
@@ -676,38 +719,29 @@ def calculate_protein_adequacy_28d(profile_id: str) -> Optional[int]:
weight = float(weight_row['avg_weight'])
# Get protein intake (28d)
cur.execute("""
SELECT protein_g
SELECT COALESCE(SUM(protein_g), 0)::float AS daily_protein
FROM nutrition_log
WHERE profile_id = %s
AND date >= CURRENT_DATE - INTERVAL '28 days'
AND protein_g IS NOT NULL
GROUP BY date
""", (profile_id,))
protein_values = [float(row['protein_g']) for row in cur.fetchall()]
daily_totals = [float(row['daily_protein']) for row in cur.fetchall()]
if len(protein_values) < 18: # 60% coverage
if len(daily_totals) < 18:
return None
# Calculate metrics
protein_per_kg_values = [p / weight for p in protein_values]
protein_per_kg_values = [p / weight for p in daily_totals]
avg_protein_per_kg = sum(protein_per_kg_values) / len(protein_per_kg_values)
# Target range: 1.6-2.2 g/kg for active individuals
target_mid = 1.9
# Score based on distance from target
if 1.6 <= avg_protein_per_kg <= 2.2:
base_score = 100
elif avg_protein_per_kg < 1.6:
# Below target
base_score = max(40, 100 - ((1.6 - avg_protein_per_kg) * 40))
else:
# Above target (less penalty)
base_score = max(80, 100 - ((avg_protein_per_kg - 2.2) * 10))
# Consistency bonus/penalty
std_dev = statistics.stdev(protein_per_kg_values)
if std_dev < 0.3:
consistency_bonus = 10
@@ -723,20 +757,24 @@ def calculate_protein_adequacy_28d(profile_id: str) -> Optional[int]:
def calculate_macro_consistency_score(profile_id: str) -> Optional[int]:
"""
Macro consistency score 0-100 (last 28 days)
Lower variability = higher score
Macro consistency score 0-100 (last 28 days).
CV of daily totals (kcal and macros), not raw log rows.
"""
import statistics
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("""
SELECT kcal, protein_g, fat_g, carbs_g
SELECT
COALESCE(SUM(kcal), 0)::float AS dk,
COALESCE(SUM(protein_g), 0)::float AS dp,
COALESCE(SUM(fat_g), 0)::float AS df,
COALESCE(SUM(carbs_g), 0)::float AS dc
FROM nutrition_log
WHERE profile_id = %s
AND date >= CURRENT_DATE - INTERVAL '28 days'
AND kcal IS NOT NULL
ORDER BY date DESC
GROUP BY date
HAVING COALESCE(SUM(kcal), 0) > 0
""", (profile_id,))
data = cur.fetchall()
@@ -744,9 +782,7 @@ def calculate_macro_consistency_score(profile_id: str) -> Optional[int]:
if len(data) < 18:
return None
# Calculate coefficient of variation for each macro
def cv(values):
"""Coefficient of variation (std_dev / mean)"""
if not values or len(values) < 2:
return None
mean = sum(values) / len(values)
@@ -755,10 +791,10 @@ def calculate_macro_consistency_score(profile_id: str) -> Optional[int]:
std_dev = statistics.stdev(values)
return std_dev / mean
calories_cv = cv([d['kcal'] for d in data])
protein_cv = cv([d['protein_g'] for d in data if d['protein_g']])
fat_cv = cv([d['fat_g'] for d in data if d['fat_g']])
carbs_cv = cv([d['carbs_g'] for d in data if d['carbs_g']])
calories_cv = cv([d['dk'] for d in data])
protein_cv = cv([d['dp'] for d in data if d['dp']])
fat_cv = cv([d['df'] for d in data if d['df']])
carbs_cv = cv([d['dc'] for d in data if d['dc']])
cv_values = [v for v in [calories_cv, protein_cv, fat_cv, carbs_cv] if v is not None]
@@ -767,9 +803,6 @@ def calculate_macro_consistency_score(profile_id: str) -> Optional[int]:
avg_cv = sum(cv_values) / len(cv_values)
# Score: lower CV = higher score
# CV < 0.2 = excellent consistency
# CV > 0.5 = poor consistency
if avg_cv < 0.2:
score = 100
elif avg_cv < 0.3:
@@ -811,14 +844,16 @@ def calculate_nutrition_score(profile_id: str, focus_weights: Optional[Dict] = N
from data_layer.scores import get_user_focus_weights
focus_weights = get_user_focus_weights(profile_id)
# Nutrition-related focus areas (English keys from DB)
protein_intake = focus_weights.get('protein_intake', 0)
calorie_balance = focus_weights.get('calorie_balance', 0)
macro_consistency = focus_weights.get('macro_consistency', 0)
meal_timing = focus_weights.get('meal_timing', 0)
hydration = focus_weights.get('hydration', 0)
# Nutrition-related focus areas (English keys from DB; Gewichte immer float)
protein_intake = float(focus_weights.get('protein_intake', 0) or 0)
calorie_balance = float(focus_weights.get('calorie_balance', 0) or 0)
macro_consistency = float(focus_weights.get('macro_consistency', 0) or 0)
meal_timing = float(focus_weights.get('meal_timing', 0) or 0)
hydration = float(focus_weights.get('hydration', 0) or 0)
total_nutrition_weight = protein_intake + calorie_balance + macro_consistency + meal_timing + hydration
total_nutrition_weight = (
protein_intake + calorie_balance + macro_consistency + meal_timing + hydration
)
if total_nutrition_weight == 0:
return None # No nutrition goals
@@ -853,40 +888,66 @@ def calculate_nutrition_score(profile_id: str, focus_weights: Optional[Dict] = N
if not components:
return None
# Weighted average
total_score = sum(score * weight for _, score, weight in components)
total_weight = sum(weight for _, _, weight in components)
# Weighted average (float: DB-Werte können Decimal sein)
total_score = sum(float(score) * float(weight) for _, score, weight in components)
total_weight = sum(float(weight) for _, _, weight in components)
return int(total_score / total_weight)
def _score_calorie_adherence(profile_id: str) -> Optional[int]:
"""Score calorie target adherence (0-100)"""
# Check for energy balance goal
# For now, use energy balance calculation
"""Score calorie target adherence (0–100) using 7d balance vs profiles.goal_mode."""
balance = calculate_energy_balance_7d(profile_id)
if balance is None:
return None
# Score based on whether deficit/surplus aligns with goal
# Simplified: assume weight loss goal = deficit is good
# TODO: Check actual goal type
mode = _get_profile_goal_mode(profile_id)
b = float(balance)
abs_balance = abs(balance)
def _weight_loss(x: float) -> int:
if -550 <= x <= -250:
return 100
if x > 450:
return 38
if -750 <= x < -550 or -250 < x <= 120:
return 82
if x < -1200:
return 52
if -950 <= x < -750 or 120 < x <= 350:
return 68
return 58
# Moderate deficit/surplus = good
if 200 <= abs_balance <= 500:
return 100
elif 100 <= abs_balance <= 700:
return 85
elif abs_balance <= 900:
return 70
elif abs_balance <= 1200:
return 55
else:
def _surplus_friendly(x: float) -> int:
if 80 <= x <= 480:
return 100
if -120 <= x < 80 or 480 < x <= 700:
return 86
if -380 <= x < -120:
return 68
if x > 850:
return 54
if x < -650:
return 44
return 72
def _maintenance(x: float) -> int:
a = abs(x)
if a <= 200:
return 100
if a <= 400:
return 84
if a <= 650:
return 70
if a <= 900:
return 55
return 40
if mode == "weight_loss":
return _weight_loss(b)
if mode in ("strength", "recomposition"):
return _surplus_friendly(b)
return _maintenance(b)
def _score_macro_balance(profile_id: str) -> Optional[int]:
"""Score macro balance (0-100)"""
+117 -29
View File
@@ -15,12 +15,50 @@ Phase 0c: Multi-Layer Architecture
Version: 1.0
"""
from typing import Dict, List, Optional
import json
from typing import Dict, List, Optional, Any
from datetime import datetime, timedelta, date
from db import get_db, get_cursor, r2d
from db import get_db, get_cursor
from data_layer.utils import calculate_confidence, safe_float, safe_int
def _parse_sleep_segments(raw: Any) -> Optional[List[dict]]:
"""JSONB kann dict/list/str sein; ungültig → None."""
if raw is None:
return None
if isinstance(raw, str):
try:
raw = json.loads(raw)
except (json.JSONDecodeError, TypeError):
return None
if not isinstance(raw, list):
return None
return raw
def _segment_minutes(seg: Any) -> int:
if not isinstance(seg, dict):
return 0
for key in ("duration_min", "duration_minutes", "minutes"):
v = seg.get(key)
if v is not None:
return max(0, safe_int(v))
return 0
def _normalize_sleep_phase(seg: dict) -> str:
"""Kleinbuchstaben; Apple „Core“-Schlaf wird wie light gewertet."""
if not isinstance(seg, dict):
return ""
p = seg.get("phase")
if p is None:
return ""
s = str(p).strip().lower()
if s in ("core", "asleep"):
return "light"
return s
def get_sleep_duration_data(
profile_id: str,
days: int = 7
@@ -51,7 +89,7 @@ def get_sleep_duration_data(
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
cur.execute(
"""SELECT sleep_segments FROM sleep_log
"""SELECT sleep_segments, duration_minutes FROM sleep_log
WHERE profile_id=%s AND date >= %s
ORDER BY date DESC""",
(profile_id, cutoff)
@@ -72,12 +110,17 @@ def get_sleep_duration_data(
nights_with_data = 0
for row in rows:
segments = row['sleep_segments']
night_minutes = 0
segments = _parse_sleep_segments(row.get("sleep_segments"))
if segments:
night_minutes = sum(seg.get('duration_min', 0) for seg in segments)
if night_minutes > 0:
total_minutes += night_minutes
nights_with_data += 1
night_minutes = sum(_segment_minutes(seg) for seg in segments)
if night_minutes <= 0:
dm = row.get("duration_minutes")
if dm is not None:
night_minutes = max(0, safe_int(dm))
if night_minutes > 0:
total_minutes += night_minutes
nights_with_data += 1
if nights_with_data == 0:
return {
@@ -136,7 +179,9 @@ def get_sleep_quality_data(
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
cur.execute(
"""SELECT sleep_segments FROM sleep_log
"""SELECT sleep_segments, duration_minutes, deep_minutes, rem_minutes,
light_minutes, awake_minutes
FROM sleep_log
WHERE profile_id=%s AND date >= %s
ORDER BY date DESC""",
(profile_id, cutoff)
@@ -163,15 +208,29 @@ def get_sleep_quality_data(
count = 0
for row in rows:
segments = row['sleep_segments']
if segments:
# Note: segments use 'phase' key, stored lowercase (deep, rem, light, awake)
deep_rem_min = sum(s.get('duration_min', 0) for s in segments if s.get('phase') in ['deep', 'rem'])
light_min = sum(s.get('duration_min', 0) for s in segments if s.get('phase') == 'light')
awake_min = sum(s.get('duration_min', 0) for s in segments if s.get('phase') == 'awake')
total_min = sum(s.get('duration_min', 0) for s in segments)
deep_rem_min = light_min = awake_min = 0
total_min = 0
used_segments = False
segments = _parse_sleep_segments(row.get("sleep_segments"))
if segments:
total_min = sum(_segment_minutes(s) for s in segments)
if total_min > 0:
deep_rem_min = sum(
_segment_minutes(s)
for s in segments
if _normalize_sleep_phase(s) in ("deep", "rem")
)
light_min = sum(
_segment_minutes(s)
for s in segments
if _normalize_sleep_phase(s) == "light"
)
awake_min = sum(
_segment_minutes(s)
for s in segments
if _normalize_sleep_phase(s) == "awake"
)
quality_pct = (deep_rem_min / total_min) * 100
total_quality += quality_pct
total_deep_rem += deep_rem_min
@@ -179,6 +238,28 @@ def get_sleep_quality_data(
total_awake += awake_min
total_all += total_min
count += 1
used_segments = True
if not used_segments:
d, r, l, a = (
row.get("deep_minutes"),
row.get("rem_minutes"),
row.get("light_minutes"),
row.get("awake_minutes"),
)
if d is not None or r is not None or l is not None:
di, ri, li = (d or 0), (r or 0), (l or 0)
phase_sum = di + ri + li
ai = (a or 0) if a is not None else 0
total_min = phase_sum + ai
if total_min > 0 and phase_sum > 0:
quality_pct = ((di + ri) / total_min) * 100
total_quality += quality_pct
total_deep_rem += di + ri
total_light += li
total_awake += ai
total_all += total_min
count += 1
if count == 0:
return {
@@ -351,8 +432,8 @@ def calculate_recovery_score_v2(profile_id: str) -> Optional[int]:
return None
# Weighted average
total_score = sum(score * weight for _, score, weight in components)
total_weight = sum(weight for _, _, weight in components)
total_score = sum(float(score) * float(weight) for _, score, weight in components)
total_weight = sum(float(weight) for _, _, weight in components)
final_score = int(total_score / total_weight)
@@ -783,17 +864,24 @@ def calculate_sleep_quality_7d(profile_id: str) -> Optional[int]:
quality_scores = []
for s in sleep_data:
if s['deep_minutes'] and s['rem_minutes']:
quality_pct = ((s['deep_minutes'] + s['rem_minutes']) / s['duration_minutes']) * 100
# 40-60% deep+REM is good
if quality_pct >= 45:
quality_scores.append(100)
elif quality_pct >= 35:
quality_scores.append(75)
elif quality_pct >= 25:
quality_scores.append(50)
else:
quality_scores.append(30)
dur = s["duration_minutes"]
if not dur or dur <= 0:
continue
d = s["deep_minutes"]
r = s["rem_minutes"]
if d is None and r is None:
continue
di, ri = (d or 0), (r or 0)
quality_pct = ((di + ri) / dur) * 100
# 40-60% deep+REM is good
if quality_pct >= 45:
quality_scores.append(100)
elif quality_pct >= 35:
quality_scores.append(75)
elif quality_pct >= 25:
quality_scores.append(50)
else:
quality_scores.append(30)
if not quality_scores:
return None
+30 -19
View File
@@ -202,29 +202,30 @@ def calculate_goal_progress_score(profile_id: str) -> Optional[int]:
total_weight = 0.0
for focus_area_id, weight in focus_weights.items():
w = float(weight)
component = focus_to_component.get(focus_area_id)
if component == 'body' and body_score is not None:
total_score += body_score * weight
total_weight += weight
total_score += float(body_score) * w
total_weight += w
elif component == 'nutrition' and nutrition_score is not None:
total_score += nutrition_score * weight
total_weight += weight
total_score += float(nutrition_score) * w
total_weight += w
elif component == 'activity' and activity_score is not None:
total_score += activity_score * weight
total_weight += weight
total_score += float(activity_score) * w
total_weight += w
elif component == 'recovery' and recovery_score is not None:
total_score += recovery_score * weight
total_weight += weight
total_score += float(recovery_score) * w
total_weight += w
elif component == 'health' and health_risk_score is not None:
total_score += health_risk_score * weight
total_weight += weight
total_score += float(health_risk_score) * w
total_weight += w
if total_weight == 0:
return None
# Normalize to 0-100
final_score = total_score / total_weight
# Normalize to 0-100 (Explizit float: Zwischensummen können Decimal aus DB sein)
final_score = float(total_score) / float(total_weight)
return int(final_score)
@@ -282,9 +283,9 @@ def calculate_health_stability_score(profile_id: str) -> Optional[int]:
activities = cur.fetchall()
if activities:
total_minutes = sum(a['duration_min'] for a in activities)
total_minutes = float(sum(float(a['duration_min'] or 0) for a in activities))
# WHO recommends 150-300 min/week moderate activity
movement_score = min(100, (total_minutes / 150) * 100)
movement_score = min(100.0, (total_minutes / 150) * 100)
components.append(('movement', movement_score, 20))
# 4. Waist circumference risk (15%)
@@ -328,8 +329,8 @@ def calculate_health_stability_score(profile_id: str) -> Optional[int]:
return None
# Weighted average
total_score = sum(score * weight for _, score, weight in components)
total_weight = sum(weight for _, _, weight in components)
total_score = sum(float(score) * float(weight) for _, score, weight in components)
total_weight = sum(float(weight) for _, _, weight in components)
return int(total_score / total_weight)
@@ -532,9 +533,19 @@ def calculate_focus_area_progress(profile_id: str, focus_area_id: str) -> Option
if not goals:
return None
# Weighted average by contribution_weight
total_progress = sum(g['progress_pct'] * g['contribution_weight'] for g in goals)
total_weight = sum(g['contribution_weight'] for g in goals)
# Weighted average; progress_pct darf NULL sein (Ziele ohne quantitative Berechnung)
parts: List[tuple] = []
for g in goals:
pct = g['progress_pct']
if pct is None:
continue
parts.append((float(pct), float(g['contribution_weight'])))
if not parts:
return None
total_progress = sum(p * w for p, w in parts)
total_weight = sum(w for _, w in parts)
return int(total_progress / total_weight) if total_weight > 0 else None
+7
View File
@@ -14,6 +14,10 @@ from slowapi.errors import RateLimitExceeded
from db import init_db
# Placeholder registry: load all register_placeholder() side-effects before any request
# so get_placeholder_catalog() and exports see consistent metadata (see Phase A plan).
import placeholder_registrations # noqa: F401
# Import routers
from routers import auth, profiles, weight, circumference, caliper
from routers import activity, nutrition, photos, insights, prompts
@@ -31,6 +35,7 @@ from routers import workflows # Phase 2 Workflow Engine - Execution
from routers import reference_values # Persönliche Referenzwerte (Profil)
from routers import admin_reference_value_types # Admin: Referenzwert-Typen
from routers import app_dashboard # Geschützter App-Bereich: Dashboard-Lab Layout
from routers import csv_import, admin_csv_templates # Issue #21 Universal CSV Parser
# ── App Configuration ─────────────────────────────────────────────────────────
DATA_DIR = Path(os.getenv("DATA_DIR", "./data"))
@@ -121,6 +126,8 @@ app.include_router(workflows.router) # /api/workflows/* (Phase 2 Exec
app.include_router(reference_values.router) # /api/reference-value-types, /api/profile-reference-values
app.include_router(admin_reference_value_types.router) # /api/admin/reference-value-types
app.include_router(app_dashboard.router) # /api/app/dashboard-layout
app.include_router(csv_import.router) # /api/csv/* (Issue #21)
app.include_router(admin_csv_templates.router) # /api/admin/csv-templates/* (Issue #21)
# ── Health Check ──────────────────────────────────────────────────────────────
@app.get("/")
@@ -0,0 +1,75 @@
-- Migration 042: Universal CSV Parser – Mapping-Registry & Import-Log (Issue #21)
-- Tabellen für System-Templates (profile_id NULL, is_system true) und User-Mappings.
CREATE TABLE IF NOT EXISTS csv_field_mappings (
id SERIAL PRIMARY KEY,
profile_id UUID REFERENCES profiles(id) ON DELETE CASCADE,
is_system BOOLEAN NOT NULL DEFAULT false,
module VARCHAR(50) NOT NULL,
mapping_name VARCHAR(100) NOT NULL,
description TEXT,
column_signature TEXT[] NOT NULL DEFAULT '{}',
delimiter VARCHAR(10) NOT NULL DEFAULT ',',
encoding VARCHAR(20) NOT NULL DEFAULT 'utf-8',
has_header BOOLEAN NOT NULL DEFAULT true,
field_mappings JSONB NOT NULL DEFAULT '{}',
type_conversions JSONB,
usage_count INTEGER NOT NULL DEFAULT 0,
last_used_at TIMESTAMPTZ,
success_rate REAL NOT NULL DEFAULT 1.0,
created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT csv_field_mappings_system_profile CHECK (
(is_system = true AND profile_id IS NULL)
OR (is_system = false AND profile_id IS NOT NULL)
)
);
COMMENT ON TABLE csv_field_mappings IS 'CSV Import: System-Templates + User-Mappings (Issue #21)';
COMMENT ON COLUMN csv_field_mappings.is_system IS 'true = globales Template (nur Admin pflegbar), false = User-Mapping';
CREATE UNIQUE INDEX IF NOT EXISTS idx_csv_field_mappings_system_module_name
ON csv_field_mappings (module, mapping_name)
WHERE is_system = true AND profile_id IS NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_csv_field_mappings_user_module_name
ON csv_field_mappings (profile_id, module, mapping_name)
WHERE is_system = false;
CREATE INDEX IF NOT EXISTS idx_csv_field_mappings_module_profile
ON csv_field_mappings (module, profile_id);
CREATE INDEX IF NOT EXISTS idx_csv_field_mappings_system_module
ON csv_field_mappings (module)
WHERE is_system = true;
CREATE TABLE IF NOT EXISTS csv_import_log (
id SERIAL PRIMARY KEY,
profile_id UUID NOT NULL REFERENCES profiles(id) ON DELETE CASCADE,
mapping_id INTEGER REFERENCES csv_field_mappings(id) ON DELETE SET NULL,
module VARCHAR(50) NOT NULL,
filename VARCHAR(255),
rows_total INTEGER,
rows_imported INTEGER,
rows_updated INTEGER,
rows_skipped INTEGER,
rows_errors INTEGER,
error_details JSONB,
started_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
finished_at TIMESTAMPTZ,
status VARCHAR(20) NOT NULL DEFAULT 'running',
affected_ids JSONB
);
CREATE INDEX IF NOT EXISTS idx_csv_import_log_profile_module
ON csv_import_log (profile_id, module DESC, started_at DESC);
COMMENT ON COLUMN csv_import_log.affected_ids IS 'Pro Import gesammelte Primärschlüssel je Tabelle (Rollback / Bereinigung)';
INSERT INTO system_config (key, value, updated_at)
VALUES (
'csv_import',
'{"max_rows_per_file": 50000, "max_file_bytes": 52428800}'::jsonb,
CURRENT_TIMESTAMP
)
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,315 @@
-- Migration 043: CSV Parser – System-Templates (Issue #21)
-- Idempotent: pro Template nur einfügen, wenn noch kein System-Eintrag für module+mapping_name existiert.
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL,
true,
'nutrition',
'FDDB Export (Standard)',
'Standard-Format für FDDB.de CSV-Exporte (Deutsch). Delimiter Semikolon, kJ → kcal Konvertierung.',
ARRAY['datum_tag_monat_jahr_stunde_minute', 'fett_g', 'kh_g', 'kj', 'protein_g']::TEXT[],
';',
'utf-8',
true,
'{
"datum_tag_monat_jahr_stunde_minute": "date",
"kj": "kcal",
"fett_g": "fat_g",
"kh_g": "carbs_g",
"protein_g": "protein_g"
}'::JSONB,
'{
"date": {
"type": "date",
"format": "dd.mm.yyyy HH:MM",
"extract": "date_only"
},
"kcal": {
"type": "float",
"source_unit": "kj",
"decimal_separator": ","
},
"fat_g": {"type": "float", "decimal_separator": ","},
"carbs_g": {"type": "float", "decimal_separator": ","},
"protein_g": {"type": "float", "decimal_separator": ","}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'nutrition' AND f.mapping_name = 'FDDB Export (Standard)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'nutrition', 'MyFitnessPal Export',
'Standard CSV export from MyFitnessPal (English)',
ARRAY['Carbohydrates (g)', 'Calories', 'Date', 'Fat (g)', 'Protein (g)']::TEXT[],
',', 'utf-8', true,
'{
"Date": "date",
"Calories": "kcal",
"Fat (g)": "fat_g",
"Carbohydrates (g)": "carbs_g",
"Protein (g)": "protein_g"
}'::JSONB,
'{
"date": {"type": "date", "format": "yyyy-mm-dd"},
"kcal": {"type": "float", "decimal_separator": "."},
"fat_g": {"type": "float", "decimal_separator": "."},
"carbs_g": {"type": "float", "decimal_separator": "."},
"protein_g": {"type": "float", "decimal_separator": "."}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'nutrition' AND f.mapping_name = 'MyFitnessPal Export'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'nutrition', 'Cronometer Export',
'Cronometer daily nutrition export (English)',
ARRAY['Day', 'Energy (kcal)', 'Fat (g)', 'Net Carbs (g)', 'Protein (g)']::TEXT[],
',', 'utf-8', true,
'{
"Day": "date",
"Energy (kcal)": "kcal",
"Fat (g)": "fat_g",
"Net Carbs (g)": "carbs_g",
"Protein (g)": "protein_g"
}'::JSONB,
'{
"date": {"type": "date", "format": "yyyy-mm-dd"},
"kcal": {"type": "float", "decimal_separator": "."},
"fat_g": {"type": "float", "decimal_separator": "."},
"carbs_g": {"type": "float", "decimal_separator": "."},
"protein_g": {"type": "float", "decimal_separator": "."}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'nutrition' AND f.mapping_name = 'Cronometer Export'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'activity', 'Apple Health Workout Export (English)',
'Apple Health CSV-Export für Workouts (English). Automatisches Training-Type-Mapping.',
ARRAY['Active Energy (kcal)', 'Distance (km)', 'Duration', 'End', 'Heart Rate Average (bpm)', 'Start', 'Workout Type']::TEXT[],
',', 'utf-8', true,
'{
"Workout Type": "activity_type",
"Start": "start_time",
"End": "end_time",
"Duration": "duration_min",
"Distance (km)": "distance_km",
"Active Energy (kcal)": "kcal_active",
"Heart Rate Average (bpm)": "hr_avg"
}'::JSONB,
'{
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "extract": "date_and_time", "flexible": true},
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": true},
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes"},
"distance_km": {"type": "float", "decimal_separator": "."},
"kcal_active": {"type": "float", "decimal_separator": "."},
"hr_avg": {"type": "int"}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'activity' AND f.mapping_name = 'Apple Health Workout Export (English)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'activity', 'Apple Health Workout Export (Deutsch)',
'Apple Health CSV-Export für Workouts (Deutsch). Automatisches Training-Type-Mapping.',
ARRAY['Aktive Energie (kJ)', 'Aktive Energie (kcal)', 'Dauer', 'Durchschnittliche Herzfrequenz (bpm)', 'Ende', 'Ruheeinträge (kJ)', 'Start', 'Strecke (km)', 'Trainingsart']::TEXT[],
',', 'utf-8', true,
'{
"Trainingsart": "activity_type",
"Start": "start_time",
"Ende": "end_time",
"Dauer": "duration_min",
"Strecke (km)": "distance_km",
"Aktive Energie (kcal)": "kcal_active",
"Aktive Energie (kJ)": "kcal_active",
"Ruheeinträge (kJ)": "kcal_resting",
"Durchschnittliche Herzfrequenz (bpm)": "hr_avg"
}'::JSONB,
'{
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "extract": "date_and_time", "flexible": true},
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": true},
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes"},
"distance_km": {"type": "float", "decimal_separator": ",", "flexible": true},
"kcal_active": {"type": "float", "decimal_separator": ".", "flexible": true, "source_unit": "kj"},
"kcal_resting": {"type": "float", "decimal_separator": ".", "flexible": true, "source_unit": "kj"},
"hr_avg": {"type": "int", "flexible": true}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'activity' AND f.mapping_name = 'Apple Health Workout Export (Deutsch)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'activity', 'Garmin Connect Export',
'Garmin Connect activity CSV export (English)',
ARRAY['Activity Type', 'Avg HR', 'Calories', 'Date', 'Distance', 'Duration', 'Time']::TEXT[],
',', 'utf-8', true,
'{
"Activity Type": "activity_type",
"Date": "date",
"Time": "start_time",
"Duration": "duration_min",
"Distance": "distance_km",
"Calories": "kcal_active",
"Avg HR": "hr_avg"
}'::JSONB,
'{
"date": {"type": "date", "format": "yyyy-mm-dd"},
"start_time": {"type": "time", "format": "HH:MM:SS"},
"duration_min": {"type": "duration", "format": "HH:MM:SS", "target_unit": "minutes"},
"distance_km": {"type": "float", "decimal_separator": "."},
"kcal_active": {"type": "float", "decimal_separator": "."},
"hr_avg": {"type": "int"}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'activity' AND f.mapping_name = 'Garmin Connect Export'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'blood_pressure', 'Omron Export (Deutsch)',
'Omron Blutdruckmessgerät CSV-Export (Deutsch)',
ARRAY['Datum', 'Diastolisch (mmHg)', 'Puls (bpm)', 'Systolisch (mmHg)', 'Zeit']::TEXT[],
',', 'utf-8', true,
'{
"Datum": "measured_date",
"Zeit": "measured_time",
"Systolisch (mmHg)": "systolic",
"Diastolisch (mmHg)": "diastolic",
"Puls (bpm)": "pulse"
}'::JSONB,
'{
"measured_date": {"type": "date", "format": "dd.mm.yyyy"},
"measured_time": {"type": "time", "format": "HH:MM"},
"systolic": {"type": "int"},
"diastolic": {"type": "int"},
"pulse": {"type": "int"}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'blood_pressure' AND f.mapping_name = 'Omron Export (Deutsch)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'blood_pressure', 'Omron Export (English)',
'Omron blood pressure monitor CSV export (English)',
ARRAY['Date', 'Diastolic (mmHg)', 'Pulse (bpm)', 'Systolic (mmHg)', 'Time']::TEXT[],
',', 'utf-8', true,
'{
"Date": "measured_date",
"Time": "measured_time",
"Systolic (mmHg)": "systolic",
"Diastolic (mmHg)": "diastolic",
"Pulse (bpm)": "pulse"
}'::JSONB,
'{
"measured_date": {"type": "date", "format": "mm/dd/yyyy"},
"measured_time": {"type": "time", "format": "HH:MM"},
"systolic": {"type": "int"},
"diastolic": {"type": "int"},
"pulse": {"type": "int"}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'blood_pressure' AND f.mapping_name = 'Omron Export (English)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'weight', 'Apple Health Weight Export',
'Apple Health body mass CSV export',
ARRAY['Body Mass (kg)', 'Start']::TEXT[],
',', 'utf-8', true,
'{
"Start": "date",
"Body Mass (kg)": "weight"
}'::JSONB,
'{
"date": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "extract": "date_only"},
"weight": {"type": "float", "decimal_separator": "."}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'weight' AND f.mapping_name = 'Apple Health Weight Export'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL, true, 'weight', 'Withings Export',
'Withings smart scale CSV export (weight, body fat, muscle mass)',
ARRAY['Body Fat (%)', 'Date', 'Muscle Mass (kg)', 'Weight (kg)']::TEXT[],
',', 'utf-8', true,
'{
"Date": "date",
"Weight (kg)": "weight"
}'::JSONB,
'{
"date": {"type": "date", "format": "yyyy-mm-dd"},
"weight": {"type": "float", "decimal_separator": "."}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'weight' AND f.mapping_name = 'Withings Export'
);
@@ -0,0 +1,155 @@
-- Migration 044: CSV Parser — System-Vorlagen Schlaf (Apple) + Vitalwerte Baseline (Issue #21)
-- Idempotent wie 043.
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL,
true,
'sleep',
'Apple Health Schlaf (Segment-Export)',
'Apple-Health-Schlaf-CSV mit Phasen-Zeilen (Start, End, Duration (hr), Value). Import läuft aggregiert pro Nacht.',
ARRAY['Start', 'End', 'Duration (hr)', 'Value']::TEXT[],
',',
'utf-8',
true,
'{
"Start": "-",
"End": "-",
"Duration (hr)": "-",
"Value": "-"
}'::JSONB,
'{}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'sleep' AND f.mapping_name = 'Apple Health Schlaf (Segment-Export)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL,
true,
'sleep',
'Apple Health Schlaf (Schlafanalyse / Nacht)',
'Apple-Health-Schlafanalyse mit Nacht-Zusammenfassung (u. a. Total Sleep (hr), Start, End).',
ARRAY['Start', 'End', 'Total Sleep (hr)']::TEXT[],
',',
'utf-8',
true,
'{
"Start": "-",
"End": "-",
"Total Sleep (hr)": "-"
}'::JSONB,
'{}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'sleep' AND f.mapping_name = 'Apple Health Schlaf (Schlafanalyse / Nacht)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL,
true,
'vitals_baseline',
'Apple Health Baseline Vitals (English)',
'Tägliche Baseline-Messungen aus Apple Health (Ruhepuls, HRV, VO2, SpO2, Atemfrequenz).',
ARRAY[
'Start',
'Resting Heart Rate',
'Heart Rate Variability',
'VO2 Max',
'Oxygen Saturation',
'Respiratory Rate'
]::TEXT[],
',',
'utf-8',
true,
'{
"Start": "date",
"Resting Heart Rate": "resting_hr",
"Heart Rate Variability": "hrv",
"VO2 Max": "vo2_max",
"Oxygen Saturation": "spo2",
"Respiratory Rate": "respiratory_rate"
}'::JSONB,
'{
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": true
},
"resting_hr": {"type": "int", "flexible": true},
"hrv": {"type": "int", "flexible": true},
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": true},
"spo2": {"type": "int", "flexible": true},
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": true}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'vitals_baseline' AND f.mapping_name = 'Apple Health Baseline Vitals (English)'
);
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions
)
SELECT
NULL,
true,
'vitals_baseline',
'Apple Health Baseline Vitals (Deutsch)',
'Tägliche Baseline-Messungen aus Apple Health (deutsche Spaltenbezeichnungen).',
ARRAY[
'Datum/Uhrzeit',
'Ruhepuls (count/min)',
'Herzfrequenzvariabilität (ms)',
'VO2 max (ml/(kg·min))',
'Blutsauerstoffsättigung (%)',
'Atemfrequenz (count/min)'
]::TEXT[],
',',
'utf-8',
true,
'{
"Datum/Uhrzeit": "date",
"Ruhepuls (count/min)": "resting_hr",
"Herzfrequenzvariabilität (ms)": "hrv",
"VO2 max (ml/(kg·min))": "vo2_max",
"Blutsauerstoffsättigung (%)": "spo2",
"Atemfrequenz (count/min)": "respiratory_rate"
}'::JSONB,
'{
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": true
},
"resting_hr": {"type": "int", "flexible": true},
"hrv": {"type": "int", "flexible": true},
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": true},
"spo2": {"type": "int", "flexible": true},
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": true}
}'::JSONB
WHERE NOT EXISTS (
SELECT 1 FROM csv_field_mappings f
WHERE f.is_system AND f.profile_id IS NULL
AND f.module = 'vitals_baseline' AND f.mapping_name = 'Apple Health Baseline Vitals (Deutsch)'
);
@@ -0,0 +1,27 @@
-- Migration 045: Schlaf-Systemvorlage — Signatur wie „Schlafanalyse“-Export (Date/Time, Sources, …)
-- Ergänzt die Vorlage aus 044 für besseres Matching; Import-Logik unverändert (Apple-Aggregat-Parser).
UPDATE csv_field_mappings
SET
column_signature = ARRAY[
'Date/Time',
'Start',
'End',
'Total Sleep (hr)',
'Asleep (Unspecified) (hr)',
'In Bed (hr)',
'Core (hr)',
'Deep (hr)',
'REM (hr)',
'Awake (hr)',
'Sources'
]::TEXT[],
description = COALESCE(
description,
'Apple-Health-Schlafanalyse (Nacht-Zusammenfassung): Spalten wie Date/Time, Start/End mit Zeitzone, Kern/Tief/REM etc.'
),
updated_at = CURRENT_TIMESTAMP
WHERE is_system = true
AND profile_id IS NULL
AND module = 'sleep'
AND mapping_name = 'Apple Health Schlaf (Schlafanalyse / Nacht)';
@@ -0,0 +1,19 @@
-- Migration 046: FDDB-Systemvorlage — kcal ohne legacy conversion_factor/target_unit (Issue #21, source_unit-Registry)
-- Doppelte Umrechnung (kj-Faktor * 0.239) wurde fälschlich gespeichert; target_unit ist nur für duration vorgesehen.
UPDATE csv_field_mappings
SET type_conversions = jsonb_set(
type_conversions,
'{kcal}',
((type_conversions->'kcal') - 'conversion_factor' - 'target_unit')
|| jsonb_build_object('source_unit', 'kj')
)
WHERE is_system = true
AND profile_id IS NULL
AND module = 'nutrition'
AND mapping_name = 'FDDB Export (Standard)'
AND (type_conversions->'kcal') IS NOT NULL
AND (
(type_conversions->'kcal') ? 'conversion_factor'
OR (type_conversions->'kcal') ? 'target_unit'
);
@@ -0,0 +1,7 @@
-- Migration 047: CSV-Vorlagen — optionale Zeilenaggregation (group_by + aggregates) vor DB-Schreiben
ALTER TABLE csv_field_mappings
ADD COLUMN IF NOT EXISTS import_row_processing JSONB;
COMMENT ON COLUMN csv_field_mappings.import_row_processing IS
'Optional: { "group_by": ["date"], "aggregates": { "kcal": "sum" } } — siehe csv_parser/import_row_processing.py';
@@ -0,0 +1,29 @@
-- Universal-CSV-Import setzt source = 'csv'. Alte CHECK-Constraints kennen das nicht.
-- Alle passenden CHECKs zu `source` droppen (PG speichert IN oder = ANY (ARRAY[...])).
DO $$
DECLARE
r RECORD;
BEGIN
FOR r IN
SELECT c.conname
FROM pg_constraint c
JOIN pg_class t ON c.conrelid = t.oid
JOIN pg_namespace n ON n.oid = t.relnamespace
WHERE n.nspname = 'public'
AND t.relname = 'vitals_baseline'
AND c.contype = 'c'
AND (
c.conname = 'vitals_baseline_source_check'
OR COALESCE(pg_get_constraintdef(c.oid), '') ILIKE '%source%'
)
LOOP
EXECUTE format('ALTER TABLE public.vitals_baseline DROP CONSTRAINT %I', r.conname);
END LOOP;
END $$;
ALTER TABLE public.vitals_baseline DROP CONSTRAINT IF EXISTS vitals_baseline_source_check;
ALTER TABLE public.vitals_baseline ADD CONSTRAINT vitals_baseline_source_check
CHECK (source IN ('manual', 'apple_health', 'garmin', 'withings', 'csv'));
COMMENT ON COLUMN vitals_baseline.source IS 'manual | apple_health | garmin | withings | csv (Universal-Import)';
@@ -0,0 +1,30 @@
-- Idempotent: gleicher Zielzustand wie 048, auch wenn die CHECK schon existiert (z. B. 048 DB-seitig
-- erfolgreich, aber nicht in schema_migrations). Ein einziges DO vermeidet Mehrfach-Statement-Probleme.
DO $$
DECLARE
r RECORD;
BEGIN
FOR r IN
SELECT c.conname
FROM pg_constraint c
JOIN pg_class t ON c.conrelid = t.oid
JOIN pg_namespace n ON n.oid = t.relnamespace
WHERE n.nspname = 'public'
AND t.relname = 'vitals_baseline'
AND c.contype = 'c'
AND (
c.conname = 'vitals_baseline_source_check'
OR COALESCE(pg_get_constraintdef(c.oid), '') ILIKE '%source%'
)
LOOP
EXECUTE format('ALTER TABLE public.vitals_baseline DROP CONSTRAINT %I', r.conname);
END LOOP;
EXECUTE $ddl$
ALTER TABLE public.vitals_baseline ADD CONSTRAINT vitals_baseline_source_check
CHECK (source IN ('manual', 'apple_health', 'garmin', 'withings', 'csv'))
$ddl$;
EXCEPTION
WHEN duplicate_object THEN
NULL;
END $$;
@@ -0,0 +1,26 @@
-- Apple Health Workout-CSV: Zeit oft ohne Sekunden (HH:MM); dateutil dayfirst bricht ISO YYYY-MM-DD.
-- type_converter: zusätzliche Patterns + ISO-reihenfolge in _dateutil_parse.
-- Bestehende System-Vorlagen: flexible für Start/End (idempotent).
UPDATE csv_field_mappings
SET type_conversions = jsonb_set(
jsonb_set(
COALESCE(type_conversions, '{}'::jsonb),
'{start_time}',
COALESCE(type_conversions->'start_time', '{}'::jsonb) || '{"flexible": true}'::jsonb,
true
),
'{end_time}',
COALESCE(type_conversions->'end_time', '{}'::jsonb) || '{"flexible": true}'::jsonb,
true
)
WHERE is_system = true
AND profile_id IS NULL
AND module = 'activity'
AND mapping_name IN (
'Apple Health Workout Export (English)',
'Apple Health Workout Export (Deutsch)'
)
AND type_conversions IS NOT NULL
AND type_conversions ? 'start_time'
AND type_conversions ? 'end_time';
@@ -0,0 +1,9 @@
-- Universal-CSV-Import schreibt source = 'csv' (siehe csv_parser/executor _import_blood_pressure).
ALTER TABLE blood_pressure_log DROP CONSTRAINT IF EXISTS blood_pressure_log_source_check;
ALTER TABLE blood_pressure_log ADD CONSTRAINT blood_pressure_log_source_check
CHECK (source IN ('manual', 'omron', 'apple_health', 'withings', 'csv'));
COMMENT ON COLUMN blood_pressure_log.source IS
'manual | omron | apple_health | withings | csv (Universal-CSV-Import)';
@@ -0,0 +1,8 @@
-- Apple-/Import-Werte: Energie oft >1000 (kJ oder kcal), HR-Felder NUMERIC(5,2) zu eng für Fehlzuordnungen.
-- Idempotent: Typ nur anheben, nie verkleinern.
ALTER TABLE activity_log
ALTER COLUMN kcal_active TYPE NUMERIC(12, 2),
ALTER COLUMN kcal_resting TYPE NUMERIC(12, 2),
ALTER COLUMN hr_avg TYPE NUMERIC(6, 2),
ALTER COLUMN hr_max TYPE NUMERIC(6, 2);
@@ -0,0 +1,26 @@
-- Apple Health Workout (Deutsch): „Aktive Energie (kJ)“ / „Ruheeinträge (kJ)“ → DB speichert kcal.
-- type_conversions.source_unit „kj“ nutzt field_units (1/4.184).
UPDATE csv_field_mappings
SET
field_mappings = COALESCE(field_mappings, '{}'::jsonb)
|| '{"Aktive Energie (kJ)": "kcal_active", "Ruheeinträge (kJ)": "kcal_resting"}'::jsonb,
type_conversions = jsonb_set(
jsonb_set(
COALESCE(type_conversions, '{}'::jsonb),
'{kcal_active}',
COALESCE(type_conversions->'kcal_active', '{}'::jsonb) || '{"source_unit": "kj"}'::jsonb,
true
),
'{kcal_resting}',
COALESCE(
type_conversions->'kcal_resting',
'{"type": "float", "decimal_separator": ",", "flexible": true}'::jsonb
) || '{"source_unit": "kj"}'::jsonb,
true
)
WHERE is_system = true
AND profile_id IS NULL
AND module = 'activity'
AND mapping_name = 'Apple Health Workout Export (Deutsch)'
AND type_conversions IS NOT NULL;
+29 -19
View File
@@ -12,30 +12,48 @@
-- ============================================================================
-- ============================================================================
-- 1. Rename csv_import to data_import
-- 1. csv_import → data_import (FK-sicher)
-- ============================================================================
UPDATE features
SET
id = 'data_import',
name = 'Daten importieren',
description = 'CSV-Import (FDDB, Apple Health) + ZIP-Backup-Import'
WHERE id = 'csv_import';
-- Zuerst Ziel-Feature-Zeile sichern, dann alle FKs umhängen, dann csv_import
-- entfernen. PK direkt per UPDATE ändern scheitert, solange tier_limits noch
-- feature_id = 'csv_import' referenziert (tier_limits_feature_id_fkey).
INSERT INTO features (id, name, description, category, limit_type, reset_period, default_limit, active)
VALUES ('data_import', 'Daten importieren', 'CSV-Import (FDDB, Apple Health) + ZIP-Backup-Import', 'import', 'count', 'monthly', 0, true)
ON CONFLICT (id) DO UPDATE SET
name = EXCLUDED.name,
description = EXCLUDED.description,
category = EXCLUDED.category,
limit_type = EXCLUDED.limit_type,
reset_period = EXCLUDED.reset_period;
-- Update tier_limits references
UPDATE tier_limits
SET feature_id = 'data_import'
WHERE feature_id = 'csv_import';
-- Update user_feature_restrictions references
UPDATE user_feature_restrictions
SET feature_id = 'data_import'
WHERE feature_id = 'csv_import';
-- Update user_feature_usage references
UPDATE user_feature_usage
SET feature_id = 'data_import'
WHERE feature_id = 'csv_import';
-- Widget-Gateway (Migration 041): sonst blockiert FK beim Löschen von csv_import
DO $$
BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'widget_feature_requirements'
) THEN
UPDATE widget_feature_requirements
SET feature_id = 'data_import'
WHERE feature_id = 'csv_import';
END IF;
END $$;
DELETE FROM features WHERE id = 'csv_import';
-- ============================================================================
-- 2. Remove old export_csv/json/zip features
-- ============================================================================
@@ -69,16 +87,8 @@ ON CONFLICT (id) DO UPDATE SET
reset_period = EXCLUDED.reset_period;
-- ============================================================================
-- 4. Ensure data_import exists and is properly configured
-- 4. data_import: in Schritt 1 angelegt; kein zweites INSERT nötig
-- ============================================================================
INSERT INTO features (id, name, description, category, limit_type, reset_period, default_limit, active)
VALUES ('data_import', 'Daten importieren', 'CSV-Import (FDDB, Apple Health) + ZIP-Backup-Import', 'import', 'count', 'monthly', 0, true)
ON CONFLICT (id) DO UPDATE SET
name = EXCLUDED.name,
description = EXCLUDED.description,
category = EXCLUDED.category,
limit_type = EXCLUDED.limit_type,
reset_period = EXCLUDED.reset_period;
-- ============================================================================
-- 5. Update tier_limits for data_export (consolidate from old features)
+7 -8
View File
@@ -1,11 +1,10 @@
"""
Complete Placeholder Metadata Definitions
Complete Placeholder Metadata Definitions (Legacy / Normativ v1)
This module contains manually curated, complete metadata for all 116 placeholders.
It combines automatic extraction with manual annotation to ensure 100% normative compliance.
IMPORTANT: This is the authoritative source for placeholder metadata.
All new placeholders MUST be added here with complete metadata.
Hinweis (2026-04): **Verbindliche Metadaten-Pflege** erfolgt über
`backend/placeholder_registrations/` + `placeholder_registry.py` (114 Keys, deckungsgleich
mit `PLACEHOLDER_MAP`). Dieses Modul bleibt für ältere Generator-/Export-Pfade und
Tests; neue Platzhalter hier nicht mehr duplizieren.
"""
from placeholder_metadata import (
PlaceholderMetadata,
@@ -28,7 +27,7 @@ from typing import List
def get_all_placeholder_metadata() -> List[PlaceholderMetadata]:
"""
Returns complete metadata for all 116 placeholders.
Returns complete metadata for all 114 placeholders (Registry ist maßgeblich).
This is the authoritative, manually curated source.
"""
@@ -476,7 +475,7 @@ def get_all_placeholder_metadata() -> List[PlaceholderMetadata]:
notes=["Quadrant-Logik basiert auf FM/LBM Delta-Vorzeichen"],
),
# NOTE: Continuing with all 116 placeholders would make this file very long.
# NOTE: Continuing with all 114 placeholders would make this file very long.
# For brevity, I'll create a separate generator that fills all remaining placeholders.
# The pattern is established above - each placeholder gets full metadata.
]
+14 -4
View File
@@ -29,13 +29,22 @@ def extract_value_raw(value_display: str, output_type: OutputType, placeholder_t
Returns: (raw_value, success)
"""
if not value_display or value_display in ['nicht verfügbar', 'nicht genug Daten']:
return None, True
s = (value_display or "").strip()
if (
not s
or s in ['nicht verfügbar', 'nicht genug Daten']
or s.startswith('nicht verfügbar —')
):
# V2 strict mode: missing/unavailable value is not a successful extraction
return None, False
# JSON output type
if output_type == OutputType.JSON:
try:
return json.loads(value_display), True
parsed = json.loads(value_display)
if isinstance(parsed, dict) and parsed.get('_available') is False:
return None, False
return parsed, True
except (json.JSONDecodeError, TypeError):
# Try to find JSON in string
json_match = re.search(r'(\{.*\}|\[.*\])', value_display, re.DOTALL)
@@ -111,7 +120,8 @@ def infer_unit_strict(key: str, description: str, output_type: OutputType, place
return 'kg'
# Circumferences/lengths
if any(x in key_lower for x in ['umfang', 'waist', 'hip', 'chest', 'arm', 'leg', 'delta']) and 'circumference' in desc_lower:
circumference_terms = ['umfang', 'waist', 'hip', 'chest', 'arm', 'leg', 'taill', 'hueft', 'brust', 'oberarm', 'oberschenkel']
if any(x in key_lower for x in circumference_terms) or any(x in desc_lower for x in ['circumference', 'umfang', 'taill', 'hüft', 'hueft', 'brust', 'oberarm', 'oberschenkel']):
return 'cm'
# Time durations
+25 -1
View File
@@ -8,7 +8,31 @@ Auto-imports all placeholder registrations to populate the global registry.
from . import nutrition_part_a
from . import nutrition_part_b
from . import nutrition_part_c
from . import nutrition_score
from . import body_metrics
from . import body_extras
from . import activity_metrics
from . import activity_session_insights
from . import schlaf_erholung
from . import vitalwerte
from . import profil_zeitraum
from . import phase_0b_meta_scores
from . import phase_0b_ziele_fokus
from . import korrelationen
__all__ = ['nutrition_part_a', 'nutrition_part_b', 'nutrition_part_c', 'body_metrics', 'activity_metrics']
__all__ = [
'nutrition_part_a',
'nutrition_part_b',
'nutrition_part_c',
'nutrition_score',
'body_metrics',
'body_extras',
'activity_metrics',
'activity_session_insights',
'schlaf_erholung',
'vitalwerte',
'profil_zeitraum',
'phase_0b_meta_scores',
'phase_0b_ziele_fokus',
'korrelationen',
]
@@ -0,0 +1,19 @@
"""Gemeinsames Evidence-Tagging für Registry-Einträge."""
from placeholder_registry import EvidenceType, PlaceholderMetadata
STANDARD_FIELDS = (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables", "semantic_contract",
"unit", "time_window", "output_type", "placeholder_type", "format_hint",
"example_output", "minimum_data_requirements", "confidence_logic",
"missing_value_policy", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
)
def tag_standard_evidence(meta: PlaceholderMetadata) -> None:
for field in STANDARD_FIELDS:
meta.set_evidence(field, EvidenceType.CODE_DERIVED)
meta.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
meta.set_evidence("known_limitations", EvidenceType.MIXED)
@@ -1,7 +1,7 @@
"""
Activity Metrics Placeholder Registrations
Registers all 17 activity-related placeholders in the central placeholder registry.
Registers 17 Aktivitäts-Platzhalter hier; 3 Session-/Erholungs-Keys in activity_session_insights.py (20 gesamt).
Evidence-based metadata with clear tagging of source.
@@ -10,6 +10,9 @@ Groups:
- Basic Metrics (7): training_minutes_week, training_frequency_7d, quality_sessions_pct,
proxy_internal_load_7d, monotony_score, strain_score, rest_day_compliance
- Advanced Metrics (7): ability_balance_*, vo2max_trend_28d, activity_score
Resolver: alle Keys gebündelt unter „Training / Aktivität“ in PLACEHOLDER_MAP;
activity_score nicht unter „Meta Scores“.
"""
from placeholder_registry import (
@@ -938,9 +941,9 @@ def register_activity_group_3():
description="VO2 Max Trend über 28 Tage",
category="Aktivität",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_vo2max_trend_28d",
resolver_function="_safe_float",
data_layer_module="backend/data_layer/activity_metrics.py",
data_layer_function="calculate_vo2max_trend",
data_layer_function="calculate_vo2max_trend_28d",
source_tables=["vitals_baseline"],
time_window="28d",
output_type=OutputType.NUMERIC,
@@ -977,8 +980,8 @@ def register_activity_group_3():
"EDGE CASE: Nur 1 Messung → kein Trend → missing_value. "
"EDGE CASE: Große Zeitlücken zwischen Messungen → Trend nicht aussagekräftig."
),
layer_1_decision="Data Layer (activity_metrics.calculate_vo2max_trend) - QUESTIONABLE",
layer_2a_decision="Placeholder Resolver (formatting only)",
layer_1_decision="Data Layer (activity_metrics.calculate_vo2max_trend_28d) — Kategorie diskutierbar",
layer_2a_decision="Placeholder Resolver (_safe_float)",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c Multi-Layer Architecture conform",
issue_53_alignment="Layer separation established"
@@ -1020,8 +1023,8 @@ def register_activity_group_3():
description="Gesamtaktivitäts-Score (gewichtet)",
category="Aktivität",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_activity_score",
data_layer_module="backend/data_layer/scores.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/activity_metrics.py",
data_layer_function="calculate_activity_score",
source_tables=["activity_log", "training_types", "rest_days", "vitals_baseline", "user_focus_area_weights"],
time_window="composite (7d, 14d, 28d mixed)",
@@ -1065,8 +1068,8 @@ def register_activity_group_3():
"QUESTIONABLE: Vermischt Metriken mit unterschiedlicher Verlässlichkeit "
"(z.B. quality_sessions_pct hat TO_VERIFY Issues)."
),
layer_1_decision="Data Layer (scores.calculate_activity_score)",
layer_2a_decision="Placeholder Resolver (formatting only)",
layer_1_decision="Data Layer (activity_metrics.calculate_activity_score)",
layer_2a_decision="Placeholder Resolver (_safe_int)",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0c Multi-Layer Architecture conform",
issue_53_alignment="Layer separation established"
@@ -0,0 +1,184 @@
"""
Registry: Trainings-Häufigkeit, Pausen zwischen Einheiten, wöchentliche Session-JSON (KI-Rohkontext).
"""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
EvidenceType,
OutputType,
PlaceholderType,
register_placeholder,
)
def _ev(meta: PlaceholderMetadata, field: str, et: EvidenceType = EvidenceType.CODE_DERIVED):
meta.set_evidence(field, et)
def register_activity_session_insights():
md_freq = PlaceholderMetadata(
key="training_frequency_by_type_md",
category="Aktivität",
description=(
"Markdown-Tabelle: pro Trainingsart (activity_type) Sessions, Ø/Woche, "
"Dauer, kcal, HF, RPE, kcal/min (Intensitätsproxy)"
),
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_training_frequency_by_type_md",
data_layer_module="backend/data_layer/activity_metrics.py",
data_layer_function="get_training_frequency_by_type_data",
source_tables=["activity_log"],
semantic_contract=(
"Aggregat über activity_log gruppiert nach activity_type (Roh-Label). "
"sessions_per_week = count / (days/7). avg_kcal_per_min = Summe kcal / Summe min."
),
business_meaning="KI: Häufigkeit & Belastung pro Sportart, Erholungs-/Überlastungs-Kontext",
unit="Markdown",
time_window="default 28 Tage",
output_type=OutputType.TEXT_SUMMARY,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="GitHub-Flavored Markdown-Tabelle",
example_output="| Art | n | Ø/Woche | … |",
minimum_data_requirements="Mindestens eine Session im Fenster",
quality_filter_policy=None,
confidence_logic="Wie calculate_confidence anhand Session-Anzahl",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="no_data",
legacy_display="Keine Trainingsdaten",
),
known_limitations=(
"Gruppierung nach activity_type-String (Import-Namen), nicht nur training_type_id. "
"HF/RPE oft NULL je nach Quelle. Pausen-Analyse separater Platzhalter."
),
layer_1_decision="activity_metrics.get_training_frequency_by_type_data",
layer_2a_decision="get_training_frequency_by_type_md",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
for f in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables", "semantic_contract",
"unit", "time_window", "output_type", "placeholder_type", "format_hint",
"example_output", "minimum_data_requirements", "confidence_logic",
"missing_value_policy", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
):
_ev(md_freq, f)
_ev(md_freq, "business_meaning", EvidenceType.DRAFT_DERIVED)
_ev(md_freq, "known_limitations", EvidenceType.MIXED)
register_placeholder(md_freq)
md_gap = PlaceholderMetadata(
key="training_inter_session_gap_md",
category="Aktivität",
description="Median/Mittel/Min der Stunden zwischen aufeinanderfolgenden Trainingseinheiten",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_training_inter_session_gap_md",
data_layer_module="backend/data_layer/activity_metrics.py",
data_layer_function="get_training_inter_session_gap_data",
source_tables=["activity_log"],
semantic_contract=(
"Sessions chronologisch; Zeitstempel = date + start_time oder 12:00. "
"Lücken in Stunden zwischen aufeinanderfolgenden Starts."
),
business_meaning="KI: ausreichend Erholung zwischen Belastungen? Doppelbelastung?",
unit="Markdown",
time_window="default 28 Tage",
output_type=OutputType.TEXT_SUMMARY,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Kurzer Markdown-Fließtext",
example_output="**Pause zwischen Trainings** …",
minimum_data_requirements="Mindestens 2 Sessions",
quality_filter_policy=None,
confidence_logic="calculate_confidence über Session-Anzahl",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="insufficient_data",
legacy_display="Zu wenige Trainings",
),
known_limitations=(
"Kein Unterscheidung aktiv/passiv außerhalb activity_log. "
"Fehlende Uhrzeit verzerrt Reihenfolge am selben Tag nicht (nur ein künstlicher Mittag)."
),
layer_1_decision="activity_metrics.get_training_inter_session_gap_data",
layer_2a_decision="get_training_inter_session_gap_md",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
for f in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables", "semantic_contract",
"unit", "time_window", "output_type", "placeholder_type", "format_hint",
"example_output", "minimum_data_requirements", "confidence_logic",
"missing_value_policy", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
):
_ev(md_gap, f)
_ev(md_gap, "business_meaning", EvidenceType.DRAFT_DERIVED)
_ev(md_gap, "known_limitations", EvidenceType.MIXED)
register_placeholder(md_gap)
pj = PlaceholderMetadata(
key="training_sessions_recent_json",
category="Aktivität",
description=(
"JSON: letzte ISO-Kalenderwochen mit Einheiten (Datum, Art, Dauer, kcal, HF Ø/max, RPE, Kategorie)"
),
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_json",
data_layer_module="backend/data_layer/activity_metrics.py",
data_layer_function="get_training_sessions_recent_weeks_data",
source_tables=["activity_log", "training_types"],
semantic_contract=(
"Struktur weeks[].week_iso, sessions[] mit Feldern für KI-Auswertung. "
"Default 4 ISO-Wochen zurück."
),
business_meaning="Rohkontext für wochenweise Auswertung (Erholung, Intensität) in der KI",
unit="JSON string",
time_window="4 ISO-Wochen (28 Tage Datenfenster)",
output_type=OutputType.JSON,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="JSON-Objekt als String",
example_output='{"weeks":[...],"meta":{...}}',
minimum_data_requirements="Optional Sessions; meta.confidence bei leer insufficient",
quality_filter_policy=None,
confidence_logic="meta.confidence aus Session-Anzahl",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="no_data",
legacy_display="{}",
),
known_limitations=(
"Token-Länge bei vielen Sessions beachten. training_type_name nur bei gesetztem training_type_id."
),
layer_1_decision="activity_metrics.get_training_sessions_recent_weeks_data",
layer_2a_decision="_safe_json('training_sessions_recent_json')",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
for f in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables", "semantic_contract",
"unit", "time_window", "output_type", "placeholder_type", "format_hint",
"example_output", "minimum_data_requirements", "confidence_logic",
"missing_value_policy", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
):
_ev(pj, f)
_ev(pj, "business_meaning", EvidenceType.DRAFT_DERIVED)
_ev(pj, "known_limitations", EvidenceType.MIXED)
register_placeholder(pj)
register_activity_session_insights()
@@ -0,0 +1,237 @@
"""
Registry: BMI, Profil-Ziele (goal_weight, goal_bf_pct), body_progress_score.
Profilfelder sind unabhängig von der goals-Tabelle; operative Ziele über andere Keys.
"""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
EvidenceType,
OutputType,
PlaceholderType,
register_placeholder,
)
def register_body_extras():
bmi = PlaceholderMetadata(
key="bmi",
category="Körper",
description="Body-Mass-Index aus letztem Gewicht und Profilgröße (cm)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="calculate_bmi",
data_layer_module="backend/data_layer/body_metrics.py",
data_layer_function="get_bmi_data",
source_tables=["profiles", "weight_log"],
semantic_contract=(
"BMI = Gewicht_kg / (Größe_m)² mit Größe_m = profiles.height / 100 "
"und Gewicht = jüngster Eintrag in weight_log."
),
business_meaning="Standard-Körpermaß für Coaching und Risiko-Kontext",
unit="kg/m²",
time_window="latest weight + aktuelle Profilgröße",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="Eine Dezimalstelle, ohne Einheit im String",
example_output="24.3",
minimum_data_requirements="Profil mit height > 0 und mindestens ein weight_log",
quality_filter_policy=None,
confidence_logic="high nur wenn BMI berechenbar; sonst insufficient / Anzeige nicht verfügbar",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="no_data",
legacy_display="nicht verfügbar",
),
known_limitations=(
"Keine ethnischen Referenzkurven; Profilgröße kann veraltet sein. "
"Unterscheidet nicht Muskelmasse vs. Fett."
),
layer_1_decision="body_metrics.get_bmi_data",
layer_2a_decision="placeholder_resolver.calculate_bmi (Format)",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1 als Quelle",
evidence={},
)
for field in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables",
"semantic_contract", "business_meaning", "unit", "time_window",
"output_type", "placeholder_type", "format_hint", "example_output",
"minimum_data_requirements", "confidence_logic", "missing_value_policy",
"known_limitations", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
):
bmi.set_evidence(field, EvidenceType.CODE_DERIVED)
bmi.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
bmi.set_evidence("known_limitations", EvidenceType.MIXED)
register_placeholder(bmi)
gw = PlaceholderMetadata(
key="goal_weight",
category="Körper",
description="Zielgewicht aus Profilfeld profiles.goal_weight (kg)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_goal_weight",
data_layer_module="backend/data_layer/body_metrics.py",
data_layer_function="get_profile_goal_weight_data",
source_tables=["profiles"],
semantic_contract=(
"Strategisches Soll-Gewicht im Profil; unabhängig von der goals-Tabelle "
"(dort detaillierte Ziele mit Fortschritt)."
),
business_meaning="Schneller Abgleich Prompt vs. Profil-Default-Zielgewicht",
unit="kg",
time_window="Profil-Snapshot",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="Eine Dezimalstelle oder Text „nicht gesetzt“",
example_output="82.0",
minimum_data_requirements="profiles.goal_weight IS NOT NULL",
quality_filter_policy=None,
confidence_logic="high wenn gesetzt",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="not_set",
legacy_display="nicht gesetzt",
),
known_limitations="Kann von aktiven goals.weight-Zielen abweichen.",
layer_1_decision="body_metrics.get_profile_goal_weight_data",
layer_2a_decision="placeholder_resolver.get_goal_weight",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1 als Quelle",
evidence={},
)
for field in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables",
"semantic_contract", "unit", "time_window", "output_type",
"placeholder_type", "format_hint", "example_output",
"minimum_data_requirements", "confidence_logic", "missing_value_policy",
"layer_1_decision", "layer_2a_decision", "layer_2b_reuse_possible",
"architecture_alignment", "issue_53_alignment",
):
gw.set_evidence(field, EvidenceType.CODE_DERIVED)
gw.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
gw.set_evidence("known_limitations", EvidenceType.MIXED)
register_placeholder(gw)
gbf = PlaceholderMetadata(
key="goal_bf_pct",
category="Körper",
description="Ziel-Körperfettanteil aus Profilfeld profiles.goal_bf_pct (%)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_goal_bf_pct",
data_layer_module="backend/data_layer/body_metrics.py",
data_layer_function="get_profile_goal_bf_pct_data",
source_tables=["profiles"],
semantic_contract="Strategisches Ziel-KFA im Profil.",
business_meaning="Prompt-Abgleich mit Profil-Ziel-KFA",
unit="%",
time_window="Profil-Snapshot",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="Eine Dezimalstelle oder Text „nicht gesetzt“",
example_output="15.0",
minimum_data_requirements="profiles.goal_bf_pct IS NOT NULL",
quality_filter_policy=None,
confidence_logic="high wenn gesetzt",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="not_set",
legacy_display="nicht gesetzt",
),
known_limitations="Kann von goals body_fat abweichen.",
layer_1_decision="body_metrics.get_profile_goal_bf_pct_data",
layer_2a_decision="placeholder_resolver.get_goal_bf_pct",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1 als Quelle",
evidence={},
)
for field in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables",
"semantic_contract", "unit", "time_window", "output_type",
"placeholder_type", "format_hint", "example_output",
"minimum_data_requirements", "confidence_logic", "missing_value_policy",
"layer_1_decision", "layer_2a_decision", "layer_2b_reuse_possible",
"architecture_alignment", "issue_53_alignment",
):
gbf.set_evidence(field, EvidenceType.CODE_DERIVED)
gbf.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
gbf.set_evidence("known_limitations", EvidenceType.MIXED)
register_placeholder(gbf)
bps = PlaceholderMetadata(
key="body_progress_score",
category="Körper",
description="Körper-Fortschritts-Score 0–100, gewichtet nach Focus (Abnehmen, Muskelaufbau, Recomp)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/body_metrics.py",
data_layer_function="calculate_body_progress_score",
source_tables=[
"user_focus_area_weights",
"focus_area_definitions",
"goals",
"weight_log",
"caliper_log",
"circumference_log",
],
semantic_contract=(
"Gewichteter Mittelwert aus bis zu drei Komponenten: Trend vs. Gewichtsziel, "
"Körperzusammensetzung (FM/LBM/Recomp-Quadrant), Taille-Trend. "
"Komponenten nur aktiv, wenn passende Focus-Gewichte > 0."
),
business_meaning="Meta-KPI: passt dokumentierter Körperfortschritt zur gewichteten Körper-Priorität?",
unit="Score (0–100)",
time_window="composite (u. a. 28d Deltas, Ziel-Fortschritt)",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl oder „nicht verfügbar“",
example_output="72",
minimum_data_requirements=(
"Summe der Körper-Focus-Gewichte (weight_loss + muscle_gain + body_recomposition) > 0 "
"und mindestens eine bewertbare Komponente mit Daten."
),
quality_filter_policy=None,
confidence_logic="Kein separates Confidence-Feld; None wenn keine Körper-Gewichtung oder keine Teilscores.",
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="not_applicable",
legacy_display="nicht verfügbar",
),
known_limitations=(
"Abhängig von user_focus_area_weights und aktiven weight-goals für Gewichts-Teilscore. "
"Taille-Score wird mit festem Basisgewicht 20+ eingemischt und kann dominieren."
),
layer_1_decision="body_metrics.calculate_body_progress_score",
layer_2a_decision="placeholder_resolver._safe_int('body_progress_score', …)",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1 als Quelle",
evidence={},
)
for field in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables",
"semantic_contract", "unit", "time_window", "output_type",
"placeholder_type", "format_hint", "example_output",
"minimum_data_requirements", "confidence_logic", "missing_value_policy",
"layer_1_decision", "layer_2a_decision", "layer_2b_reuse_possible",
"architecture_alignment", "issue_53_alignment",
):
bps.set_evidence(field, EvidenceType.CODE_DERIVED)
bps.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
bps.set_evidence("known_limitations", EvidenceType.MIXED)
register_placeholder(bps)
register_body_extras()
@@ -1,7 +1,8 @@
"""
Body Metrics Placeholder Registrations
Registers 17 body composition and measurement placeholders:
Registers 17 Körper-Metriken in diesem Modul; insgesamt 21 Körper-Keys in der Registry
(zusätzlich body_extras.py: bmi, goal_weight, goal_bf_pct, body_progress_score).
Weight & Trends (7):
- weight_aktuell
@@ -29,7 +30,7 @@ Summaries (2):
- circ_summary
Evidence-based metadata with comprehensive formula documentation.
Code inspection: backend/data_layer/body_metrics.py (830 lines)
Siehe backend/data_layer/body_metrics.py als Layer-1-Implementierung.
"""
from placeholder_registry import (
@@ -0,0 +1,96 @@
"""Registry: Korrelations- und Treiber-Metriken (Data Layer correlations)."""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
OutputType,
PlaceholderType,
register_placeholder,
)
from ._evidence import tag_standard_evidence
CAT = "Korrelationen"
MVP = lambda reason, disp: MissingValuePolicy(
available=False, value_raw=None, missing_reason=reason, legacy_display=disp
)
def register_korrelationen():
for key, dl_fn, desc, tables, sem in [
(
"correlation_energy_weight_lag",
"calculate_lag_correlation",
"JSON: Lag-Korrelation Energiebilanz ↔ Gewicht",
["nutrition_log", "weight_log"],
"correlations.calculate_lag_correlation(pid, 'energy', 'weight')",
),
(
"correlation_protein_lbm",
"calculate_lag_correlation",
"JSON: Lag-Korrelation Protein ↔ Magermasse",
["nutrition_log", "weight_log", "caliper_log"],
"correlations.calculate_lag_correlation(pid, 'protein', 'lbm')",
),
(
"correlation_load_hrv",
"calculate_lag_correlation",
"JSON: Lag-Korrelation Trainingslast ↔ HRV",
["activity_log", "vitals_baseline"],
"correlations.calculate_lag_correlation(pid, 'training_load', 'hrv')",
),
(
"correlation_load_rhr",
"calculate_lag_correlation",
"JSON: Lag-Korrelation Trainingslast ↔ Ruhepuls",
["activity_log", "vitals_baseline"],
"correlations.calculate_lag_correlation(pid, 'training_load', 'rhr')",
),
(
"plateau_detected",
"calculate_plateau_detected",
"JSON: Platten-Erkennung (Gewicht/Körper)",
["weight_log", "caliper_log"],
"correlations.calculate_plateau_detected",
),
(
"top_drivers",
"calculate_top_drivers",
"JSON: Top Treiber für Ziel-/Score-Variablen",
["weight_log", "nutrition_log", "activity_log", "vitals_baseline", "sleep_log"],
"correlations.calculate_top_drivers",
),
]:
m = PlaceholderMetadata(
key=key,
category=CAT,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_json",
data_layer_module="backend/data_layer/correlations.py",
data_layer_function=dl_fn,
source_tables=tables,
semantic_contract=sem,
business_meaning="Strukturierte Korrelationsausgabe für KI",
unit="JSON",
time_window="funktionsintern",
output_type=OutputType.JSON,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="JSON-String",
example_output="{}",
minimum_data_requirements="Ausreichend gekoppelte Zeitreihen",
quality_filter_policy=None,
confidence_logic="Wie correlations.*",
missing_value_policy=MVP("insufficient_data", "{}"),
known_limitations="Bei wenigen Daten leer oder wenig robust",
layer_1_decision=f"correlations.{dl_fn}",
layer_2a_decision="_safe_json",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
register_korrelationen()
@@ -53,6 +53,13 @@ def register_nutrition_part_a():
"layer_1_decision": "Data Layer (nutrition_metrics.get_nutrition_average_data)",
"layer_2a_decision": "Placeholder Resolver (formatting only)",
"architecture_alignment": "Phase 0c Multi-Layer Architecture conform",
"minimum_data_requirements": (
"Mind. ein Kalendertag mit nutrition_log im Fenster; Mittelwerte aus täglicher Aggregation. "
"Confidence über calculate_confidence(day_count, days) in get_nutrition_average_data."
),
"quality_filter_policy": (
"Kein Outlier-Filter auf Tagesaggregaten; leere Tage fehlen in der Aggregation (kein Imputing)."
),
}
# Common evidence for shared fields
@@ -73,8 +80,8 @@ def register_nutrition_part_a():
"layer_2b_reuse_possible": EvidenceType.TO_VERIFY, # not verified in charts
"architecture_alignment": EvidenceType.CODE_DERIVED, # imports from data_layer
"issue_53_alignment": EvidenceType.MIXED, # layer separation visible, issue conformity derived
"minimum_data_requirements": EvidenceType.UNRESOLVED, # not explicit in code
"quality_filter_policy": EvidenceType.UNRESOLVED, # not implemented
"minimum_data_requirements": EvidenceType.CODE_DERIVED,
"quality_filter_policy": EvidenceType.CODE_DERIVED,
}
# ── kcal_avg ──────────────────────────────────────────────────────────────
@@ -94,8 +101,6 @@ def register_nutrition_part_a():
known_limitations="nur Intake, kein Bedarf; sagt allein nichts über Zielpassung",
layer_2b_reuse_possible=None, # to_verify - not checked in chart code
issue_53_alignment="Layer separation established",
minimum_data_requirements=None, # unresolved
quality_filter_policy=None, # unresolved
**common_metadata
)
@@ -131,8 +136,6 @@ def register_nutrition_part_a():
),
layer_2b_reuse_possible=None,
issue_53_alignment="Layer separation established",
minimum_data_requirements=None,
quality_filter_policy=None,
**common_metadata
)
@@ -165,8 +168,6 @@ def register_nutrition_part_a():
),
layer_2b_reuse_possible=None,
issue_53_alignment="Layer separation established",
minimum_data_requirements=None,
quality_filter_policy=None,
**common_metadata
)
@@ -196,8 +197,6 @@ def register_nutrition_part_a():
known_limitations="meist im Gesamtkontext der Makroverteilung relevant",
layer_2b_reuse_possible=None,
issue_53_alignment="Layer separation established",
minimum_data_requirements=None,
quality_filter_policy=None,
**common_metadata
)
@@ -1,7 +1,7 @@
"""
Placeholder Registrations - Nutrition Part C
Registers 5 nutrition-related placeholders with complete metadata:
Registers 5 nutrition-related placeholders in this file (nutrition_score: siehe nutrition_score.py):
- macro_consistency_score
- energy_balance_7d
- energy_deficit_surplus
@@ -113,7 +113,7 @@ energy_balance_metadata = PlaceholderMetadata(
resolver_function="_safe_float('energy_balance_7d', pid, decimals=0)",
data_layer_module="backend/data_layer/nutrition_metrics.py",
data_layer_function="calculate_energy_balance_7d",
source_tables=["nutrition_log", "weight_log"],
source_tables=["nutrition_log", "weight_log", "profiles"],
# Semantic
semantic_contract="Liefert die geschätzte Energiebilanz über 7 Tage als Differenz zwischen durchschnittlicher Energieaufnahme und geschätztem TDEE (Total Daily Energy Expenditure). Positiver Wert = Überschuss, Negativer Wert = Defizit.",
@@ -127,11 +127,14 @@ energy_balance_metadata = PlaceholderMetadata(
# Quality
minimum_data_requirements="Mindestens 4 Tage mit Kalorienerfassung in 7-Tage-Fenster. Aktuelles Gewicht aus weight_log erforderlich.",
quality_filter_policy="Unvollständige Intake-Daten und fehlende Gewichtsmessung reduzieren Verlässlichkeit. TDEE-Schätzung ist vereinfacht (weight_kg × 32.5).",
quality_filter_policy=(
"Unvollständige Intake-Daten und fehlende Gewichtsmessung reduzieren Verlässlichkeit. "
"TDEE: Mifflin–St Jeor × PAL 1.55 wenn Höhe, Geschlecht, DOB und Gewicht vorhanden, sonst kg×32.5."
),
confidence_logic=(
"Kombiniert Intake-Abdeckung und Robustheit des Verbrauchsmodells. "
"Niedrigere Confidence bei <7 Tagen Daten oder fehlendem Gewicht. "
"TDEE-Modell ist vereinfacht → inherent uncertainty."
"PAL=1.55 ist ein Festwert (moderate Aktivität), kein individuelles Aktivitätslogging."
),
missing_value_policy=MissingValuePolicy(
available=False,
@@ -140,11 +143,10 @@ energy_balance_metadata = PlaceholderMetadata(
legacy_display="nicht verfügbar"
),
known_limitations=(
"TDEE-MODELL: Vereinfacht als bodyweight_kg × 32.5 (mittlerer Multiplikator). "
"NICHT berücksichtigt: Aktivitätslevel, Alter, Geschlecht, Stoffwechselanpassungen. "
"TODO in Code: Harris-Benedict oder Mifflin-St Jeor für präzisere TDEE-Schätzung. "
"ACHTUNG: Energiebilanz ist modellbasiert, nicht direkt gemessen. "
"Einheit ist kcal/Tag (daily average), NICHT 7d-Total."
"TDEE: Bei vollständigem Profil (Größe, Geschlecht, DOB, Gewicht) Mifflin–St Jeor BMR × 1.55; "
"sonst Fallback kg×32.5. PAL ist nicht nutzerkonfigurierbar. "
"Energiebilanz ist modellbasiert, nicht gemessen. "
"Einheit kcal/Tag (Tagesmittel), nicht 7-Tage-Summe."
),
# Architecture
@@ -435,8 +437,9 @@ Part C Registration Complete:
Total Nutrition Cluster:
- Part A: 4 placeholders (kcal_avg, protein_avg, carb_avg, fat_avg)
- Part B: 5 placeholders (protein targets + adequacy)
- Part C: 5 placeholders (consistency + balance + meta)
→ 14 nutrition placeholders total
- Part C: 5 placeholders in dieser Datei (consistency + balance + meta)
- nutrition_score: eigenes Modul nutrition_score.py
→ 15 Ernährungs-Platzhalter gesamt (A+B+C+nutrition_score)
All registrations follow Phase 0c Multi-Layer Architecture:
- Layer 1 (Data Layer): Calculations
@@ -0,0 +1,102 @@
"""
Placeholder registration: nutrition_score
Focus-gewichteter Ernährungs-Meta-Score (separates Modul, um nutrition_part_c schlank zu halten).
"""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
EvidenceType,
OutputType,
PlaceholderType,
register_placeholder,
)
nutrition_score_metadata = PlaceholderMetadata(
key="nutrition_score",
category="Ernährung",
description="Ernährungs-Score (0–100), gewichtet nach Focus Areas",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/nutrition_metrics.py",
data_layer_function="calculate_nutrition_score",
source_tables=[
"nutrition_log",
"weight_log",
"user_focus_area_weights",
"focus_area_definitions",
],
semantic_contract=(
"Gewichteter Score 0–100 aus Komponenten, die nur einfließen, wenn der Nutzer "
"passende Ernährungs-Focus-Gewichte gesetzt hat (z. B. protein_intake, "
"calorie_balance, macro_consistency). Nutzt u. a. Protein-Adequacy, "
"Makro-Konsistenz, Kalorien-Adhärenz (über Energiebilanz) und Makro-Balance."
),
business_meaning=(
"Verdichteter KPI für Prompts: passt die dokumentierte Ernährung zur "
"gewichteten strategischen Priorität des Nutzers?"
),
unit="score (0-100)",
time_window="composite (7d / 28d je Komponente)",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl; bei fehlender Ernährungs-Gewichtung oft nicht verfügbar",
example_output="72",
minimum_data_requirements=(
"Mindestens eine Ernährungs-Focus-Komponente mit Gewicht > 0; "
"sowie je nach Komponente ausreichende nutrition_log-/weight_log-Abdeckung."
),
quality_filter_policy=None,
confidence_logic=(
"Kein separates Confidence-Feld im Resolver; fehlende Komponenten werden "
"aus der Gewichtung ausgeschlossen. total_nutrition_weight == 0 ergibt keinen Score."
),
missing_value_policy=MissingValuePolicy(
available=False,
value_raw=None,
missing_reason="not_applicable",
legacy_display="nicht verfügbar",
),
known_limitations=(
"Abhängig von user_focus_area_weights; ohne Ernährungs-Fokus liefert die "
"Funktion None. Kalorien-Adhärenz nutzt 7d-Energiebilanz vs. profiles.goal_mode "
"(weight_loss / strength+recomposition / sonst maintenance). "
"_score_macro_balance nutzt zeilenbasierte 28d-Abfrage (langfristig an "
"Tagesaggregation angleichen)."
),
layer_1_decision="Data Layer (nutrition_metrics.calculate_nutrition_score)",
layer_2a_decision="Placeholder Resolver (_safe_int)",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c: Berechnung in nutrition_metrics",
issue_53_alignment="Layer 1 als Quelle; Komponenten nutzen weitere Layer-1-Funktionen",
evidence={},
)
nutrition_score_metadata.set_evidence("key", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("category", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("description", EvidenceType.MIXED)
nutrition_score_metadata.set_evidence("resolver_module", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("resolver_function", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("data_layer_module", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("data_layer_function", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("source_tables", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("semantic_contract", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
nutrition_score_metadata.set_evidence("unit", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("time_window", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("output_type", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("placeholder_type", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("format_hint", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("example_output", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("minimum_data_requirements", EvidenceType.MIXED)
nutrition_score_metadata.set_evidence("confidence_logic", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("missing_value_policy", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("known_limitations", EvidenceType.MIXED)
nutrition_score_metadata.set_evidence("layer_1_decision", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("layer_2a_decision", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("layer_2b_reuse_possible", EvidenceType.TO_VERIFY)
nutrition_score_metadata.set_evidence("architecture_alignment", EvidenceType.CODE_DERIVED)
nutrition_score_metadata.set_evidence("issue_53_alignment", EvidenceType.MIXED)
register_placeholder(nutrition_score_metadata)
@@ -0,0 +1,66 @@
"""Registry: Meta-Scores (Phase 0b) — Ziel-Fortschritt und Datenqualität."""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
OutputType,
PlaceholderType,
register_placeholder,
)
from ._evidence import tag_standard_evidence
CAT = "Scores (Phase 0b)"
MVP = lambda reason, disp: MissingValuePolicy(
available=False, value_raw=None, missing_reason=reason, legacy_display=disp
)
def register_phase_0b_meta_scores():
for key, dl_fn, desc, unit in [
(
"goal_progress_score",
"calculate_goal_progress_score",
"Aggregierter Ziel-Fortschritt 0–100",
"0–100",
),
(
"data_quality_score",
"calculate_data_quality_score",
"Datenqualitäts-Score 0–100",
"0–100",
),
]:
m = PlaceholderMetadata(
key=key,
category=CAT,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/scores.py",
data_layer_function=dl_fn,
source_tables=["goals", "weight_log", "nutrition_log", "activity_log", "profiles"],
semantic_contract=f"scores.{dl_fn} (siehe Data Layer).",
business_meaning="Meta-KPI für Prompt-Gewichtung",
unit=unit,
time_window="composite",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl als String",
example_output="72",
minimum_data_requirements="Abhängig von Score-Implementierung",
quality_filter_policy=None,
confidence_logic="Wie calculate_* in scores.py",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations="Bei dünnen Daten weniger aussagekräftig",
layer_1_decision=f"scores.{dl_fn}",
layer_2a_decision="_safe_int",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
register_phase_0b_meta_scores()
@@ -0,0 +1,392 @@
"""Registry: Ziele, Fokusbereiche, Kategorie-Scores und formatierte Listen (Phase 0b)."""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
OutputType,
PlaceholderType,
register_placeholder,
)
from ._evidence import tag_standard_evidence
CAT = "Ziele & Fokus (Phase 0b)"
MVP = lambda reason, disp: MissingValuePolicy(
available=False, value_raw=None, missing_reason=reason, legacy_display=disp
)
def register_phase_0b_ziele_fokus():
# Top-Ziel / Top-Fokusbereich
m = PlaceholderMetadata(
key="top_goal_name",
category=CAT,
description="Name/Typ des höchstpriorisierten Ziels",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_str",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="get_top_priority_goal",
source_tables=["goals"],
semantic_contract="Feld name oder goal_type aus get_top_priority_goal",
business_meaning="Priorisierung für KI-Empfehlungen",
unit="text",
time_window="aktuell",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Kurztext",
example_output="Gewicht 80kg",
minimum_data_requirements="Mindestens ein aktives Ziel",
quality_filter_policy=None,
confidence_logic="scores.get_top_priority_goal",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.get_top_priority_goal",
layer_2a_decision="_safe_str('top_goal_name')",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="top_goal_progress_pct",
category=CAT,
description="Fortschritt Top-Ziel (%)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="get_top_priority_goal",
source_tables=["goals"],
semantic_contract="progress_pct aus get_top_priority_goal",
business_meaning="Priorisierung für KI-Empfehlungen",
unit="%",
time_window="aktuell",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl",
example_output="65",
minimum_data_requirements="Mindestens ein aktives Ziel",
quality_filter_policy=None,
confidence_logic="scores.get_top_priority_goal",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.get_top_priority_goal",
layer_2a_decision="_safe_int('top_goal_progress_pct')",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="top_goal_status",
category=CAT,
description="Status-Label Top-Ziel",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_str",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="get_top_priority_goal",
source_tables=["goals"],
semantic_contract="status aus get_top_priority_goal",
business_meaning="Priorisierung für KI-Empfehlungen",
unit="text",
time_window="aktuell",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Kurztext",
example_output="active",
minimum_data_requirements="Mindestens ein aktives Ziel",
quality_filter_policy=None,
confidence_logic="scores.get_top_priority_goal",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.get_top_priority_goal",
layer_2a_decision="_safe_str('top_goal_status')",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="top_focus_area_name",
category=CAT,
description="Bezeichnung des gewichtet stärksten Fokusbereichs",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_str",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="get_top_focus_area",
source_tables=["user_focus_area_weights", "focus_area_definitions"],
semantic_contract="label aus get_top_focus_area",
business_meaning="Priorisierung für KI-Empfehlungen",
unit="text",
time_window="aktuell",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Kurztext",
example_output="Kraft",
minimum_data_requirements="Gewichtete Fokusbereiche",
quality_filter_policy=None,
confidence_logic="scores.get_top_focus_area",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.get_top_focus_area",
layer_2a_decision="_safe_str('top_focus_area_name')",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="top_focus_area_progress",
category=CAT,
description="Fortschritt Top-Fokusbereich (%)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="get_top_focus_area",
source_tables=["user_focus_area_weights", "focus_area_definitions", "goals"],
semantic_contract="progress aus get_top_focus_area",
business_meaning="Priorisierung für KI-Empfehlungen",
unit="%",
time_window="aktuell",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl",
example_output="58",
minimum_data_requirements="Gewichtete Fokusbereiche",
quality_filter_policy=None,
confidence_logic="scores.get_top_focus_area",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.get_top_focus_area",
layer_2a_decision="_safe_int('top_focus_area_progress')",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
# Kategorie Progress / Weight (7 Kategorien)
for slug in (
"körper",
"ernährung",
"aktivität",
"recovery",
"vitalwerte",
"mental",
"lebensstil",
):
key_p = f"focus_cat_{slug}_progress"
key_w = f"focus_cat_{slug}_weight"
m_p = PlaceholderMetadata(
key=key_p,
category=CAT,
description=f"Aggregierter Fortschritt Kategorie „{slug}“ (%)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="calculate_category_progress",
source_tables=["goals", "focus_area_definitions", "user_focus_area_weights"],
semantic_contract=f"scores.calculate_category_progress(pid, '{slug}')",
business_meaning="Focus-Area-Kategorie-Score",
unit="%",
time_window="aktuell",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl",
example_output="55",
minimum_data_requirements="Gewichtete Bereiche in Kategorie",
quality_filter_policy=None,
confidence_logic="scores.calculate_category_progress",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.calculate_category_progress",
layer_2a_decision="_safe_int",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m_p)
register_placeholder(m_p)
m_w = PlaceholderMetadata(
key=key_w,
category=CAT,
description=f"Nutzer-Gewichtung Kategorie „{slug}“ (Anteil 0–1)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_float",
data_layer_module="backend/data_layer/scores.py",
data_layer_function="calculate_category_weight",
source_tables=["user_focus_area_weights", "focus_area_definitions"],
semantic_contract=f"scores.calculate_category_weight(pid, '{slug}')",
business_meaning="Kategorie-Gewichtung im Fokusmodell",
unit="0–1",
time_window="aktuell",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Dezimal",
example_output="0.25",
minimum_data_requirements="user_focus_area_weights",
quality_filter_policy=None,
confidence_logic="scores.calculate_category_weight",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="scores.calculate_category_weight",
layer_2a_decision="_safe_float",
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m_w)
register_placeholder(m_w)
# Strukturierte Ziele / Fokus
for key, res_fn, dl_mod, dl_fn, desc, out, ptype in [
(
"active_goals_json",
"_safe_json",
"backend/goal_utils.py",
"get_active_goals",
"Aktive Ziele als JSON",
OutputType.JSON,
PlaceholderType.RAW_DATA,
),
(
"active_goals_md",
"_safe_str",
"backend/placeholder_resolver.py",
"_format_goals_as_markdown",
"Aktive Ziele als Markdown-Tabelle",
OutputType.TEXT_SUMMARY,
PlaceholderType.INTERPRETED,
),
(
"focus_areas_weighted_json",
"_safe_json",
"backend/placeholder_resolver.py",
"_get_focus_areas_weighted_json",
"Gewichtete Fokusbereiche mit Namen (JSON)",
OutputType.JSON,
PlaceholderType.RAW_DATA,
),
(
"focus_areas_weighted_md",
"_safe_str",
"backend/placeholder_resolver.py",
"_format_focus_areas_as_markdown",
"Gewichtete Fokusbereiche als Markdown",
OutputType.TEXT_SUMMARY,
PlaceholderType.INTERPRETED,
),
(
"focus_area_weights_json",
"_safe_json",
"backend/data_layer/scores.py",
"get_user_focus_weights",
"Rohe Gewichtungen key→Anteil (JSON)",
OutputType.JSON,
PlaceholderType.RAW_DATA,
),
]:
m = PlaceholderMetadata(
key=key,
category=CAT,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function=res_fn,
data_layer_module=dl_mod,
data_layer_function=dl_fn,
source_tables=["goals", "focus_area_definitions", "user_focus_area_weights"],
semantic_contract=f"{dl_fn} (siehe Modul {dl_mod})",
business_meaning="Strukturierte Übersicht für Prompts",
unit="JSON" if out == OutputType.JSON else "markdown",
time_window="aktuell",
output_type=out,
placeholder_type=ptype,
format_hint="String aus Resolver",
example_output="[]" if out == OutputType.JSON else "—",
minimum_data_requirements="Ziele bzw. Fokusgewichte",
quality_filter_policy=None,
confidence_logic="Resolver + goal_utils / scores",
missing_value_policy=MVP("insufficient_data", "[]" if out == OutputType.JSON else "nicht verfügbar"),
known_limitations=None,
layer_1_decision=dl_fn,
layer_2a_decision=res_fn,
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 1",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
for key, res_fn, dl_fn, desc, ex in [
(
"top_3_focus_areas",
"_safe_str",
"_format_top_focus_areas",
"Top-3 Fokusbereiche als formatierter Text",
"1. Kraft …",
),
(
"top_3_goals_behind_schedule",
"_safe_str",
"_format_goals_behind",
"Bis zu drei Ziele hinter Zeitplan",
"—",
),
(
"top_3_goals_on_track",
"_safe_str",
"_format_goals_on_track",
"Bis zu drei Ziele im Plan",
"—",
),
]:
m = PlaceholderMetadata(
key=key,
category=CAT,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function=res_fn,
data_layer_module="backend/goal_utils.py",
data_layer_function="get_active_goals",
source_tables=["goals", "focus_area_definitions"],
semantic_contract=f"Resolver {dl_fn}",
business_meaning="Kurzlisten für Coaching-Prompts",
unit="text",
time_window="aktuell",
output_type=OutputType.TEXT_SUMMARY,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Freitext / Aufzählung",
example_output=ex,
minimum_data_requirements="Ziele / Fokusdaten",
quality_filter_policy=None,
confidence_logic=dl_fn,
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision="goals + focus aggregation",
layer_2a_decision=dl_fn,
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Layer 2a",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
register_phase_0b_ziele_fokus()
@@ -0,0 +1,139 @@
"""
Registry: Profil-Stammdaten und statische Zeitraum-Labels für Prompts.
"""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
OutputType,
PlaceholderType,
register_placeholder,
)
from ._evidence import tag_standard_evidence
MVP = lambda reason, disp: MissingValuePolicy(
available=False, value_raw=None, missing_reason=reason, legacy_display=disp
)
def register_profil_zeitraum():
cat_profil = "Profil"
for key, desc, res_fn, unit, ptype, out, hint, ex, sem in [
(
"name",
"Anzeigename aus profiles.name",
"get_profile_name",
"text",
PlaceholderType.ATOMIC,
OutputType.STRING,
"Kurzname",
"Max",
"profiles.name, Fallback „Nutzer“.",
),
(
"age",
"Alter in Jahren aus profiles.dob",
"get_profile_age_display",
"Jahre",
PlaceholderType.ATOMIC,
OutputType.STRING,
"Ganzzahl oder unbekannt",
"42",
"Berechnung aus Geburtsdatum; PostgreSQL date oder ISO-String.",
),
(
"height",
"Körpergröße (cm) aus profiles.height",
"get_profile_height_display",
"cm",
PlaceholderType.ATOMIC,
OutputType.STRING,
"Zahl oder unbekannt",
"180",
"profiles.height.",
),
(
"geschlecht",
"Geschlecht (männlich/weiblich) aus profiles.sex",
"get_profile_geschlecht_display",
"Kategorie",
PlaceholderType.ATOMIC,
OutputType.STRING,
"m/w-Mapping",
"männlich",
"sex == 'm' → männlich, sonst weiblich.",
),
]:
m = PlaceholderMetadata(
key=key,
category=cat_profil,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function=res_fn,
data_layer_module=None,
data_layer_function=None,
source_tables=["profiles"],
semantic_contract=sem,
business_meaning="Profil-Kontext für KI-Prompts",
unit=unit,
time_window="latest profile row",
output_type=out,
placeholder_type=ptype,
format_hint=hint,
example_output=ex,
minimum_data_requirements="Profilzeile",
quality_filter_policy=None,
confidence_logic="Row vorhanden",
missing_value_policy=MVP("no_data", "unbekannt" if key != "name" else "Nutzer"),
known_limitations="Keine diversen Geschlechtsoptionen im Platzhalter",
layer_1_decision="profiles",
layer_2a_decision=res_fn,
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Resolver",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
cat_zeit = "Zeitraum"
for key, desc, res_fn, sem, ex_out in [
("datum_heute", "Heutiges Datum (lokal)", "get_datum_heute", "datetime.now, Format dd.mm.yyyy", "11.04.2026"),
("zeitraum_7d", "Label „letzte 7 Tage“", "get_zeitraum_label_7d", "Statisches UI/Prompt-Label", "letzte 7 Tage"),
("zeitraum_30d", "Label „letzte 30 Tage“", "get_zeitraum_label_30d", "Statisches UI/Prompt-Label", "letzte 30 Tage"),
("zeitraum_90d", "Label „letzte 90 Tage“", "get_zeitraum_label_90d", "Statisches UI/Prompt-Label", "letzte 90 Tage"),
]:
m = PlaceholderMetadata(
key=key,
category=cat_zeit,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function=res_fn,
data_layer_module=None,
data_layer_function=None,
source_tables=[],
semantic_contract=sem,
business_meaning="Zeitlicher Bezug im Prompt ohne Datenabfrage",
unit="label",
time_window="n/a",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.META,
format_hint="Kurzdeutsch",
example_output=ex_out,
minimum_data_requirements=None,
quality_filter_policy=None,
confidence_logic="Immer verfügbar",
missing_value_policy=None,
known_limitations="Kein kalender-basierter Datenfilter allein durch Platzhalter",
layer_1_decision="n/a",
layer_2a_decision=res_fn,
layer_2b_reuse_possible=False,
architecture_alignment="Phase 0b",
issue_53_alignment="Resolver",
evidence={},
)
tag_standard_evidence(m)
register_placeholder(m)
register_profil_zeitraum()
@@ -0,0 +1,236 @@
"""
Registry: Schlaf, Ruhetage, Recovery-Score, Schlaf-Metriken, Schlaf-Erholungs-Korrelation.
"""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
EvidenceType,
OutputType,
PlaceholderType,
register_placeholder,
)
CAT = "Schlaf & Erholung"
MVP = lambda reason, disp: MissingValuePolicy(
available=False, value_raw=None, missing_reason=reason, legacy_display=disp
)
def _tag(m: PlaceholderMetadata):
for f in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables", "semantic_contract",
"unit", "time_window", "output_type", "placeholder_type", "format_hint",
"example_output", "minimum_data_requirements", "confidence_logic",
"missing_value_policy", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
):
m.set_evidence(f, EvidenceType.CODE_DERIVED)
m.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
m.set_evidence("known_limitations", EvidenceType.MIXED)
def register_schlaf_erholung():
# ── formatierte Schlaf-/Ruhetage-Snapshots ───────────────────────────────
m = PlaceholderMetadata(
key="sleep_avg_duration",
category=CAT,
description="Durchschnittliche Schlafdauer (Stunden), formatiert",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_sleep_avg_duration",
data_layer_module="backend/data_layer/recovery_metrics.py",
data_layer_function="get_sleep_duration_data",
source_tables=["sleep_log"],
semantic_contract="Mittel aus Schlafphasen im Fenster (siehe get_sleep_duration_data).",
business_meaning="KI-Kontext Schlafdauer",
unit="h (Anzeige mit Einheit)",
time_window="7d default im Resolver",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="z. B. 7.2h",
example_output="7.2h",
minimum_data_requirements="sleep_log im Fenster",
quality_filter_policy=None,
confidence_logic="data['confidence'] im Layer1",
missing_value_policy=MVP("no_data", "nicht verfügbar"),
known_limitations="Abhängig von Import/Qualität der Phasen",
layer_1_decision="recovery_metrics.get_sleep_duration_data",
layer_2a_decision="get_sleep_avg_duration",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="sleep_avg_quality",
category=CAT,
description="Schlafqualität (Deep+REM %), formatiert",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_sleep_avg_quality",
data_layer_module="backend/data_layer/recovery_metrics.py",
data_layer_function="get_sleep_quality_data",
source_tables=["sleep_log"],
semantic_contract="Anteil Deep+REM aus Segmenten (siehe get_sleep_quality_data).",
business_meaning="KI-Kontext Schlafqualität",
unit="%",
time_window="7d default",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="Prozent oder nicht verfügbar",
example_output="24%",
minimum_data_requirements="sleep_log mit Phasen",
quality_filter_policy=None,
confidence_logic="Layer-1-Confidence",
missing_value_policy=MVP("no_data", "nicht verfügbar"),
known_limitations="Segment-Schreibweise case-sensitiv normalisiert",
layer_1_decision="recovery_metrics.get_sleep_quality_data",
layer_2a_decision="get_sleep_avg_quality",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="rest_days_count",
category=CAT,
description="Anzahl dokumentierter Ruhetage (30d default)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_rest_days_count",
data_layer_module="backend/data_layer/recovery_metrics.py",
data_layer_function="get_rest_days_data",
source_tables=["rest_days"],
semantic_contract="Count rest_days im Zeitraum",
business_meaning="Aktive/passive Erholungstags-Übersicht",
unit="count",
time_window="30d default",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="z. B. 2 Ruhetage",
example_output="2 Ruhetage",
minimum_data_requirements="rest_days",
quality_filter_policy=None,
confidence_logic="Immer Zählung, 0 möglich",
missing_value_policy=MVP("no_data", "0 Ruhetage"),
known_limitations="Nur explizit erfasste Ruhetage",
layer_1_decision="recovery_metrics.get_rest_days_data",
layer_2a_decision="get_rest_days_count",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="recovery_score",
category=CAT,
description="Recovery-Score 0–100 (v2, komposit)",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_int",
data_layer_module="backend/data_layer/recovery_metrics.py",
data_layer_function="calculate_recovery_score_v2",
source_tables=["sleep_log", "vitals_baseline", "activity_log"],
semantic_contract="Gewichteter Score aus Schlaf, Vitaltrends, optional Load (siehe Implementierung).",
business_meaning="Gesamt-Recovery-KPI für Prompts",
unit="0–100",
time_window="composite",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.SCORE,
format_hint="Ganzzahl-String",
example_output="72",
minimum_data_requirements="Teilkomponenten je nach Gewichtung",
quality_filter_policy=None,
confidence_logic="Wie calculate_recovery_score_v2",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations="Abhängig von Datenabdeckung HF/HRV/Schlaf",
layer_1_decision="recovery_metrics.calculate_recovery_score_v2",
layer_2a_decision="_safe_int('recovery_score_v2')",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
for key, dl_fn, desc, unit, tbls, res_fn in [
("sleep_avg_duration_7d", "calculate_sleep_avg_duration_7d", "Durchschnittliche Schlafdauer 7d (h)", "h", ["sleep_log"], "_safe_float"),
("sleep_debt_hours", "calculate_sleep_debt_hours", "Kumulative Schlafschuld (h)", "h", ["sleep_log"], "_safe_float"),
("sleep_regularity_proxy", "calculate_sleep_regularity_proxy", "Schlaf-Regularität (Proxy)", "min", ["sleep_log"], "_safe_float"),
("recent_load_balance_3d", "calculate_recent_load_balance_3d", "Load-Balance 3d (Score)", "score", ["activity_log"], "_safe_int"),
("sleep_quality_7d", "calculate_sleep_quality_7d", "Schlafqualität 7d (0–100)", "0-100", ["sleep_log"], "_safe_int"),
]:
m = PlaceholderMetadata(
key=key,
category=CAT,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function=res_fn,
data_layer_module="backend/data_layer/recovery_metrics.py",
data_layer_function=dl_fn,
source_tables=tbls,
semantic_contract=f"Berechnung {dl_fn} in recovery_metrics.",
business_meaning="Erholungs-Detailmetrik",
unit=unit,
time_window="siehe Funktion",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="numerischer String",
example_output="1.0",
minimum_data_requirements="wie Funktion",
quality_filter_policy=None,
confidence_logic="Funktionsintern",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations=None,
layer_1_decision=f"recovery_metrics.{dl_fn}",
layer_2a_decision="Resolver _safe_float/_safe_int",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="correlation_sleep_recovery",
category=CAT,
description="JSON: Korrelation Schlaf ↔ Recovery-Indikatoren",
resolver_module="backend/placeholder_resolver.py",
resolver_function="_safe_json",
data_layer_module="backend/data_layer/correlations.py",
data_layer_function="calculate_correlation_sleep_recovery",
source_tables=["sleep_log", "vitals_baseline", "activity_log"],
semantic_contract="Strukturierte Korrelationsauswertung (siehe correlations).",
business_meaning="KI: Zusammenhänge Schlaf und Erholung",
unit="JSON",
time_window="funktionsabhängig",
output_type=OutputType.JSON,
placeholder_type=PlaceholderType.RAW_DATA,
format_hint="JSON-String",
example_output="{}",
minimum_data_requirements="Ausreichend gekoppelte Datenpunkte",
quality_filter_policy=None,
confidence_logic="Wie correlation_metrics",
missing_value_policy=MVP("insufficient_data", "{}"),
known_limitations="Bei wenig Daten leer oder schwach",
layer_1_decision="correlations.calculate_correlation_sleep_recovery",
layer_2a_decision="_safe_json",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
register_schlaf_erholung()
@@ -0,0 +1,180 @@
"""
Registry: Baseline-Vitals (Ruhepuls, HRV, VO2 Max) und Abweichung vs. persönlicher Baseline.
"""
from placeholder_registry import (
PlaceholderMetadata,
MissingValuePolicy,
EvidenceType,
OutputType,
PlaceholderType,
register_placeholder,
)
CAT = "Vitalwerte"
MVP = lambda reason, disp: MissingValuePolicy(
available=False, value_raw=None, missing_reason=reason, legacy_display=disp
)
def _tag(m: PlaceholderMetadata):
for f in (
"key", "category", "description", "resolver_module", "resolver_function",
"data_layer_module", "data_layer_function", "source_tables", "semantic_contract",
"unit", "time_window", "output_type", "placeholder_type", "format_hint",
"example_output", "minimum_data_requirements", "confidence_logic",
"missing_value_policy", "layer_1_decision", "layer_2a_decision",
"layer_2b_reuse_possible", "architecture_alignment", "issue_53_alignment",
):
m.set_evidence(f, EvidenceType.CODE_DERIVED)
m.set_evidence("business_meaning", EvidenceType.DRAFT_DERIVED)
m.set_evidence("known_limitations", EvidenceType.MIXED)
def register_vitalwerte():
m = PlaceholderMetadata(
key="vitals_avg_hr",
category=CAT,
description="Durchschnittlicher Ruhepuls (7d), formatiert",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_vitals_avg_hr",
data_layer_module="backend/data_layer/health_metrics.py",
data_layer_function="get_resting_heart_rate_data",
source_tables=["vitals_baseline"],
semantic_contract="Mittel RHR aus vitals_baseline im Fenster (siehe health_metrics).",
business_meaning="KI-Kontext kardiovaskuläre Ruhelage",
unit="bpm (Anzeige mit Einheit)",
time_window="7d default im Resolver",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="z. B. 58 bpm",
example_output="58 bpm",
minimum_data_requirements="vitals_baseline im Fenster",
quality_filter_policy=None,
confidence_logic="data['confidence'] im Layer1",
missing_value_policy=MVP("no_data", "nicht verfügbar"),
known_limitations="Nur erfasste Morgen-Baseline-Messungen",
layer_1_decision="health_metrics.get_resting_heart_rate_data",
layer_2a_decision="get_vitals_avg_hr",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="vitals_avg_hrv",
category=CAT,
description="Durchschnittliche HRV (7d), formatiert",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_vitals_avg_hrv",
data_layer_module="backend/data_layer/health_metrics.py",
data_layer_function="get_heart_rate_variability_data",
source_tables=["vitals_baseline"],
semantic_contract="Mittel HRV aus vitals_baseline im Fenster.",
business_meaning="KI-Kontext autonome Regulation / Erholung",
unit="ms (Anzeige mit Einheit)",
time_window="7d default",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="z. B. 45 ms",
example_output="45 ms",
minimum_data_requirements="vitals_baseline mit HRV",
quality_filter_policy=None,
confidence_logic="data['confidence'] im Layer1",
missing_value_policy=MVP("no_data", "nicht verfügbar"),
known_limitations="Geräte-/Messprotokoll kann streuen",
layer_1_decision="health_metrics.get_heart_rate_variability_data",
layer_2a_decision="get_vitals_avg_hrv",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
m = PlaceholderMetadata(
key="vitals_vo2_max",
category=CAT,
description="Aktueller VO2 Max (letzte Messung), formatiert",
resolver_module="backend/placeholder_resolver.py",
resolver_function="get_vitals_vo2_max",
data_layer_module="backend/data_layer/health_metrics.py",
data_layer_function="get_vo2_max_data",
source_tables=["vitals_baseline"],
semantic_contract="Jüngster vo2_max aus vitals_baseline.",
business_meaning="Ausdauer-/Fitness-Kontext",
unit="ml/kg/min",
time_window="latest",
output_type=OutputType.STRING,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="eine Dezimalstelle + Einheit",
example_output="42.0 ml/kg/min",
minimum_data_requirements="mindestens eine VO2-Messung",
quality_filter_policy=None,
confidence_logic="data['confidence'] im Layer1",
missing_value_policy=MVP("no_data", "nicht verfügbar"),
known_limitations="Schätzung vs. Labortest je nach Quelle",
layer_1_decision="health_metrics.get_vo2_max_data",
layer_2a_decision="get_vitals_vo2_max",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
for key, dl_fn, desc, unit, res_fn in [
(
"hrv_vs_baseline_pct",
"calculate_hrv_vs_baseline_pct",
"HRV vs. persönlicher Baseline (%)",
"%",
"_safe_float",
),
(
"rhr_vs_baseline_pct",
"calculate_rhr_vs_baseline_pct",
"Ruhepuls vs. persönlicher Baseline (%)",
"%",
"_safe_float",
),
]:
m = PlaceholderMetadata(
key=key,
category=CAT,
description=desc,
resolver_module="backend/placeholder_resolver.py",
resolver_function=res_fn,
data_layer_module="backend/data_layer/recovery_metrics.py",
data_layer_function=dl_fn,
source_tables=["vitals_baseline"],
semantic_contract=f"Vergleich aktueller Wert zu Baseline (siehe {dl_fn}).",
business_meaning="Erholungs- und Belastungsindikator relativ zur Norm des Nutzers",
unit=unit,
time_window="funktionsintern",
output_type=OutputType.NUMERIC,
placeholder_type=PlaceholderType.INTERPRETED,
format_hint="numerischer Prozent-String",
example_output="5.2",
minimum_data_requirements="Ausreichend Baseline-Historie",
quality_filter_policy=None,
confidence_logic="Funktionsintern",
missing_value_policy=MVP("insufficient_data", "nicht verfügbar"),
known_limitations="Baseline braucht ausreichend Vorlauf",
layer_1_decision=f"recovery_metrics.{dl_fn}",
layer_2a_decision=f"Resolver {res_fn}",
layer_2b_reuse_possible=True,
architecture_alignment="Phase 0c",
issue_53_alignment="Layer 1",
evidence={},
)
_tag(m)
register_placeholder(m)
register_vitalwerte()
+36
View File
@@ -258,6 +258,42 @@ class PlaceholderRegistry:
return metadata._resolver_func(profile_id)
def build_ai_placeholder_caption(metadata: PlaceholderMetadata, max_len: int = 400) -> str:
"""
Kurzerklärung / Einordnung für {{key|x}} und Exportfeld ``ai_caption`` (ohne Wert, ohne Einheit).
Inhalt: business_meaning oder gekürzter semantic_contract; bei SCORE-Zeilen die 0–100-Skala.
Nicht enthalten: description (die nur bei {{key|d}} angehängt wird) und keine „Technischer Bezug: …“-Zeile.
"""
desc = (metadata.description or "").strip()
bm = (metadata.business_meaning or "").strip()
sc = (metadata.semantic_contract or "").strip()
chunks: List[str] = []
interpret = bm
if not interpret and sc:
interpret = sc if len(sc) <= max_len else sc[: max_len - 1] + "…"
if interpret:
il = interpret.lower()
redundant = bool(
desc
and len(desc) >= 10
and desc.lower() in il
)
if not redundant:
chunks.append(interpret)
if metadata.placeholder_type == PlaceholderType.SCORE:
chunks.append("Skala 0–100: höher = im Modell günstiger / besser abgestimmt.")
out = " ".join(c for c in chunks if c).strip()
if len(out) > max_len + 120:
out = out[: max_len + 60] + "…"
return out
# Global registry instance
_global_registry = PlaceholderRegistry()
File diff suppressed because it is too large. Load diff
+78 -44
View File
@@ -12,14 +12,17 @@ import re
from typing import Dict, Any, Optional
from db import get_db, get_cursor, r2d
from fastapi import HTTPException
from placeholder_resolver import get_catalog_row_for_key
def resolve_placeholders(template: str, variables: Dict[str, Any], debug_info: Optional[Dict] = None, catalog: Optional[Dict] = None) -> str:
"""
Replace {{placeholder}} with values from variables dict.
Supports modifiers:
- {{key|d}} - Include description in parentheses (requires catalog)
Modifiers (Katalog aus get_placeholder_catalog empfohlen):
- {{key|d}} — Wert — description (kurz)
- {{key|x}} — nur Erklärung (Katalogfeld ai_caption), ohne Zahlenwert
- {{key|d,x}} — Wert — description — Erklärung
Args:
template: String with {{key}} or {{key|modifiers}} placeholders
@@ -40,46 +43,66 @@ def resolve_placeholders(template: str, variables: Dict[str, Any], debug_info: O
parts = full_placeholder.split('|')
key = parts[0].strip()
modifiers = parts[1].strip() if len(parts) > 1 else ''
mods = {x.strip().lower() for x in modifiers.split(",") if x.strip()}
want_d = "d" in mods
want_x = "x" in mods
if key in variables:
value = variables[key]
# Convert dict/list to JSON string
if isinstance(value, (dict, list)):
resolved_value = json.dumps(value, ensure_ascii=False)
else:
resolved_value = str(value)
# Apply modifiers
if 'd' in modifiers:
if catalog:
# Add description from catalog
description = None
for cat_items in catalog.values():
matching = [item for item in cat_items if item['key'] == key]
if matching:
description = matching[0].get('description', '')
break
if description:
resolved_value = f"{resolved_value} ({description})"
else:
# Catalog not available - log warning in debug
if debug_info is not None:
if 'warnings' not in debug_info:
debug_info['warnings'] = []
debug_info['warnings'].append(f"Modifier |d used but catalog not available for {key}")
# Track resolution for debug
def _warn(msg: str):
if debug_info is not None:
resolved[key] = resolved_value[:100] + ('...' if len(resolved_value) > 100 else '')
debug_info.setdefault("warnings", []).append(msg)
return resolved_value
else:
# Keep placeholder if no value found
row = get_catalog_row_for_key(catalog, key) if catalog else None
if want_x and not want_d:
if key not in variables:
if debug_info is not None:
unresolved.append(key)
return match.group(0)
expl = (row.get("ai_caption") or "").strip() if row else ""
if not expl and catalog is None:
_warn(f"Modifier |x für {key}: Katalog fehlt (ai_caption).")
out = expl
if debug_info is not None:
resolved[key] = out[:100] + ("..." if len(out) > 100 else "")
return out
if key not in variables:
if debug_info is not None:
unresolved.append(key)
return match.group(0)
value = variables[key]
if isinstance(value, (dict, list)):
resolved_value = json.dumps(value, ensure_ascii=False)
else:
resolved_value = str(value)
if not want_d and not want_x:
out = resolved_value
if debug_info is not None:
resolved[key] = out[:100] + ("..." if len(out) > 100 else "")
return out
parts = [resolved_value]
if want_d:
if row:
desc = (row.get("description") or "").strip()
if desc:
parts.append(desc)
else:
_warn(f"Modifier |d für {key}: Katalog fehlt (description).")
if want_x:
expl = (row.get("ai_caption") or "").strip() if row else ""
if expl:
parts.append(expl)
elif catalog is not None:
_warn(f"Modifier |x (mit |d) für {key}: ai_caption leer.")
out = " — ".join(parts)
if debug_info is not None:
resolved[key] = out[:100] + ("..." if len(out) > 100 else "")
return out
result = re.sub(r'\{\{([^}]+)\}\}', replacer, template)
# Store debug info
@@ -144,7 +167,8 @@ async def execute_prompt(
prompt_slug: str,
variables: Dict[str, Any],
openrouter_call_func,
enable_debug: bool = False
enable_debug: bool = False,
progress_callback = None # NEW: Optional callback für SSE Progress-Updates
) -> Dict[str, Any]:
"""
Execute a single prompt (base or pipeline type).
@@ -194,7 +218,7 @@ async def execute_prompt(
elif prompt_type == 'workflow':
# Workflow prompt: graph-based execution (Phase 0: Foundation)
return await execute_workflow_prompt(prompt, variables, openrouter_call_func, enable_debug, catalog)
return await execute_workflow_prompt(prompt, variables, openrouter_call_func, enable_debug, catalog, progress_callback)
else:
raise HTTPException(400, f"Unknown prompt type: {prompt_type}")
@@ -255,7 +279,11 @@ async def execute_base_prompt(
if enable_debug:
debug_info['template'] = template
debug_info['final_prompt'] = prompt_text[:500] + ('...' if len(prompt_text) > 500 else '')
# Volltext für Test-UI (Admin); sehr große Prompts nur weich begrenzen
_max = 512 * 1024
debug_info['final_prompt'] = (
prompt_text if len(prompt_text) <= _max else prompt_text[:_max] + "\n… [gekürzt, >512KB]"
)
debug_info['available_variables'] = list(variables.keys())
# Call AI
@@ -374,7 +402,10 @@ async def execute_pipeline_prompt(
if enable_debug:
prompt_debug['source'] = 'inline'
prompt_debug['template'] = template
prompt_debug['final_prompt'] = prompt_text[:500] + ('...' if len(prompt_text) > 500 else '')
_max = 512 * 1024
prompt_debug['final_prompt'] = (
prompt_text if len(prompt_text) <= _max else prompt_text[:_max] + "\n… [gekürzt, >512KB]"
)
prompt_debug.update(placeholder_debug)
response = await openrouter_call_func(prompt_text)
@@ -439,7 +470,8 @@ async def execute_prompt_with_data(
modules: Optional[Dict[str, bool]] = None,
timeframes: Optional[Dict[str, int]] = None,
openrouter_call_func = None,
enable_debug: bool = False
enable_debug: bool = False,
progress_callback = None # NEW: Optional callback für SSE Progress-Updates
) -> Dict[str, Any]:
"""
Execute prompt with data loaded from database.
@@ -464,7 +496,7 @@ async def execute_prompt_with_data(
'today': datetime.now().strftime('%Y-%m-%d')
}
# Load placeholder catalog for |d modifier support
# Load placeholder catalog for |d / |x Modifier
try:
catalog = get_placeholder_catalog(profile_id)
except Exception as e:
@@ -575,7 +607,7 @@ async def execute_prompt_with_data(
variables['goals_data'] = []
# Execute prompt
return await execute_prompt(prompt_slug, variables, openrouter_call_func, enable_debug)
return await execute_prompt(prompt_slug, variables, openrouter_call_func, enable_debug, progress_callback)
async def execute_workflow_prompt(
@@ -583,7 +615,8 @@ async def execute_workflow_prompt(
variables: Dict[str, Any],
openrouter_call_func,
enable_debug: bool = False,
catalog: Optional[Dict] = None
catalog: Optional[Dict] = None,
progress_callback = None # NEW: Optional callback für SSE Progress-Updates
) -> Dict[str, Any]:
"""
Execute a workflow-type prompt (graph-based execution).
@@ -622,7 +655,8 @@ async def execute_workflow_prompt(
profile_id=variables.get('profile_id', 'unknown'), # From context
variables=variables,
openrouter_call_func=openrouter_call_func,
enable_debug=enable_debug
enable_debug=enable_debug,
progress_callback=progress_callback # NEW: Progress-Callbacks durchreichen
)
# Convert ExecutionResult to dict for API response
+9
View File
@@ -0,0 +1,9 @@
[pytest]
testpaths = tests
python_files = test_*.py
python_functions = test_*
addopts = -q --tb=short
markers =
smoke: Fast smoke tests for core regression checks.
integration: Integration tests across modules/db/api behavior.
slow: Long-running or heavy tests.
+6
View File
@@ -0,0 +1,6 @@
# Entwicklung / CI: nach requirements.txt installieren
# pip install -r requirements.txt -r requirements-dev.txt
#
# Empfohlene Smoke-Suite (CSV-Import + Parser, ohne DB/Workflow-Integration):
# python -m pytest tests/test_csv_parser_core.py tests/test_csv_import_executor.py tests/test_mapping_suggest.py
pytest>=8.0
+2 -1
View File
@@ -158,7 +158,8 @@ def parse_decision_questions(section_text: str) -> Dict[str, str]:
for pattern in patterns:
matches = re.finditer(pattern, section_text, re.MULTILINE | re.IGNORECASE)
for match in matches:
question_type = match.group(1).strip().lower()
# Preserve original case for question IDs (e.g., "qAnalyst" not "qanalyst")
question_type = match.group(1).strip()
answer = match.group(2).strip()
# Entferne Klammern und Whitespace
+38 -27
View File
@@ -198,6 +198,43 @@ def activity_stats(x_profile_id: Optional[str]=Header(default=None), session: di
return {"count":len(rows),"total_kcal":round(total_kcal),"total_min":round(total_min),"by_type":by_type}
def get_training_type_for_activity_with_cursor(cur, activity_type: str, profile_id: str | None = None):
"""
Wie get_training_type_for_activity, aber mit bestehendem Cursor (z. B. Universal-CSV-Import).
Vermeidet verschachteltes get_db() — bei maxconn=1 sonst Deadlock auf dem Connection-Pool.
"""
if profile_id:
cur.execute(
"""
SELECT m.training_type_id, t.category, t.subcategory
FROM activity_type_mappings m
JOIN training_types t ON m.training_type_id = t.id
WHERE m.activity_type = %s AND m.profile_id = %s
LIMIT 1
""",
(activity_type, profile_id),
)
row = cur.fetchone()
if row:
return (row["training_type_id"], row["category"], row["subcategory"])
cur.execute(
"""
SELECT m.training_type_id, t.category, t.subcategory
FROM activity_type_mappings m
JOIN training_types t ON m.training_type_id = t.id
WHERE m.activity_type = %s AND m.profile_id IS NULL
LIMIT 1
""",
(activity_type,),
)
row = cur.fetchone()
if row:
return (row["training_type_id"], row["category"], row["subcategory"])
return (None, None, None)
def get_training_type_for_activity(activity_type: str, profile_id: str = None):
"""
Map activity_type to training_type_id using database mappings.
@@ -211,33 +248,7 @@ def get_training_type_for_activity(activity_type: str, profile_id: str = None):
"""
with get_db() as conn:
cur = get_cursor(conn)
# Try user-specific mapping first
if profile_id:
cur.execute("""
SELECT m.training_type_id, t.category, t.subcategory
FROM activity_type_mappings m
JOIN training_types t ON m.training_type_id = t.id
WHERE m.activity_type = %s AND m.profile_id = %s
LIMIT 1
""", (activity_type, profile_id))
row = cur.fetchone()
if row:
return (row['training_type_id'], row['category'], row['subcategory'])
# Try global mapping
cur.execute("""
SELECT m.training_type_id, t.category, t.subcategory
FROM activity_type_mappings m
JOIN training_types t ON m.training_type_id = t.id
WHERE m.activity_type = %s AND m.profile_id IS NULL
LIMIT 1
""", (activity_type,))
row = cur.fetchone()
if row:
return (row['training_type_id'], row['category'], row['subcategory'])
return (None, None, None)
return get_training_type_for_activity_with_cursor(cur, activity_type, profile_id)
@router.get("/uncategorized")
+430
View File
@@ -0,0 +1,430 @@
"""
Admin: System-CSV-Templates (csv_field_mappings, is_system=true) pflegen (Issue #21).
"""
from __future__ import annotations
from typing import Any, List, Optional
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
from pydantic import BaseModel, Field
from psycopg2.extras import Json
from auth import require_admin
from db import get_db, get_cursor, r2d
from csv_parser.core import (
column_signature,
decode_raw_bytes,
get_csv_import_limits,
headers_signature_rank_metrics,
normalize_header_for_signature,
parse_csv_sample,
)
from csv_parser.mapping_suggest import build_type_conversions_for_mapping, suggest_field_mappings
from csv_parser.import_row_processing import (
validate_import_row_processing as validate_import_row_processing_spec,
)
from csv_parser.module_registry import get_module_definition
from csv_parser.template_validator import validate_csv_template
router = APIRouter(prefix="/api/admin/csv-templates", tags=["admin", "csv-import"])
class CsvSystemTemplateCreate(BaseModel):
module: str
mapping_name: str = Field(..., min_length=1, max_length=100)
description: Optional[str] = None
column_signature: List[str] = Field(default_factory=list)
delimiter: str = ","
encoding: str = "utf-8"
has_header: bool = True
field_mappings: dict = Field(default_factory=dict)
type_conversions: Optional[dict] = None
import_row_processing: Optional[dict] = None
class CsvSystemTemplateUpdate(BaseModel):
mapping_name: Optional[str] = Field(default=None, min_length=1, max_length=100)
description: Optional[str] = None
column_signature: Optional[List[str]] = None
delimiter: Optional[str] = None
encoding: Optional[str] = None
has_header: Optional[bool] = None
field_mappings: Optional[dict] = None
type_conversions: Optional[dict] = None
import_row_processing: Optional[dict] = None
class CsvImportLimitsBody(BaseModel):
max_rows_per_file: int = Field(..., ge=100, le=2_000_000)
max_file_bytes: int = Field(..., ge=10_000, le=2_147_483_648)
class CsvTemplateValidateBody(BaseModel):
"""Formatprüfung ohne Speichern (field_mappings + type_conversions + optional row_processing)."""
module: str
field_mappings: dict = Field(default_factory=dict)
type_conversions: Optional[dict] = None
import_row_processing: Optional[dict] = None
column_signature: Optional[List[str]] = None
def _row_full(m: dict) -> dict:
return {
"id": m["id"],
"module": m["module"],
"mapping_name": m["mapping_name"],
"description": m.get("description"),
"column_signature": list(m["column_signature"]) if m.get("column_signature") else [],
"delimiter": m["delimiter"],
"encoding": m["encoding"],
"has_header": m["has_header"],
"field_mappings": m["field_mappings"],
"type_conversions": m.get("type_conversions"),
"import_row_processing": m.get("import_row_processing"),
"usage_count": m.get("usage_count"),
"success_rate": m.get("success_rate"),
"last_used_at": m.get("last_used_at"),
"created_at": m.get("created_at"),
"updated_at": m.get("updated_at"),
"is_system": m["is_system"],
}
@router.get("/import-limits")
def admin_get_csv_import_limits(session: dict = Depends(require_admin)):
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT value FROM system_config WHERE key = %s", ("csv_import",))
row = cur.fetchone()
return get_csv_import_limits(r2d(row) if row else None)
@router.put("/import-limits")
def admin_put_csv_import_limits(body: CsvImportLimitsBody, session: dict = Depends(require_admin)):
payload = {"max_rows_per_file": body.max_rows_per_file, "max_file_bytes": body.max_file_bytes}
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""
INSERT INTO system_config (key, value, updated_at)
VALUES ('csv_import', %s, CURRENT_TIMESTAMP)
ON CONFLICT (key) DO UPDATE
SET value = EXCLUDED.value, updated_at = CURRENT_TIMESTAMP
""",
(Json(payload),),
)
return payload
@router.get("")
def list_system_templates(
module: Optional[str] = None,
session: dict = Depends(require_admin),
):
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""
SELECT * FROM csv_field_mappings
WHERE is_system = true AND profile_id IS NULL
AND (%s::text IS NULL OR module = %s)
ORDER BY module, mapping_name
""",
(module, module),
)
rows = [r2d(r) for r in cur.fetchall()]
return {"templates": [_row_full(m) for m in rows]}
@router.post("/analyze-upload")
async def admin_analyze_csv_for_template(
file: UploadFile = File(...),
module: str = Form(...),
delimiter: Optional[str] = Form(default=None),
seed_template_id: Optional[int] = Form(default=None),
session: dict = Depends(require_admin),
):
"""
CSV hochladen wie im Nutzer-Import: Spalten + Vorschau + Vorschläge für field_mappings
und type_conversions. Optional Seed-Vorlage (ID) oder beste Systemvorlage (Abdeckung, dann Jaccard).
"""
_ = session
if not get_module_definition(module):
raise HTTPException(400, f"Unbekanntes Modul: {module}")
raw = await file.read()
limits = _admin_csv_limits()
max_bytes = limits.get("max_file_bytes", 52_428_800)
if len(raw) > max_bytes:
raise HTTPException(
413,
f"Datei zu groß (max. {max_bytes} Bytes laut Systemkonfiguration)",
)
text = decode_raw_bytes(raw)
if not text.strip():
raise HTTPException(400, "Leere Datei")
max_rows = limits.get("max_rows_per_file", 50_000)
if text.count("\n") > max_rows + 5:
raise HTTPException(
413,
f"Zu viele Zeilen (>{max_rows}) laut Systemkonfiguration",
)
delim = delimiter if delimiter in (",", ";", "\t") else None
headers, sample_rows, used_delim = parse_csv_sample(text, delimiter=delim, max_data_rows=5)
if not headers:
raise HTTPException(400, "Keine Kopfzeile oder leeres CSV")
sig = column_signature(headers)
seed_row: dict | None = None
with get_db() as conn:
cur = get_cursor(conn)
if seed_template_id is not None:
cur.execute(
"""
SELECT * FROM csv_field_mappings
WHERE id = %s AND is_system = true AND profile_id IS NULL AND module = %s
""",
(seed_template_id, module),
)
seed_row = r2d(cur.fetchone())
if not seed_row:
raise HTTPException(404, "Seed-Vorlage nicht gefunden oder falsches Modul")
else:
cur.execute(
"""
SELECT * FROM csv_field_mappings
WHERE is_system = true AND profile_id IS NULL AND module = %s
""",
(module,),
)
rows = [r2d(r) for r in cur.fetchall()]
best: dict | None = None
best_key: tuple[float, int, float] = (-1.0, -1, -1.0)
for t in rows:
t_sig = list(t.get("column_signature") or [])
m = headers_signature_rank_metrics(sig, t_sig)
key = (m["confidence"], m["columns_matched"], m["jaccard"])
if key > best_key:
best_key = key
best = t
if best and best_key[0] > 0:
seed_row = best
seed_fm = (seed_row or {}).get("field_mappings") or {}
if isinstance(seed_fm, str):
seed_fm = {}
seed_tc = (seed_row or {}).get("type_conversions")
if not isinstance(seed_tc, dict):
seed_tc = {}
field_mappings = suggest_field_mappings(headers, module, seed_fm if seed_fm else None)
type_conversions = build_type_conversions_for_mapping(module, field_mappings, seed_tc if seed_tc else None)
seed_meta = None
if seed_row:
t_sig = list(seed_row.get("column_signature") or [])
sm = headers_signature_rank_metrics(sig, t_sig)
seed_meta = {
"id": seed_row["id"],
"mapping_name": seed_row.get("mapping_name"),
"confidence": sm["confidence"],
"template_recall": sm["template_recall"],
"jaccard": sm["jaccard"],
"columns_matched": sm["columns_matched"],
"columns_in_template": sm["columns_in_template"],
"columns_in_csv": sm["columns_in_csv"],
}
return {
"filename": file.filename,
"module": module,
"delimiter": used_delim,
"encoding": "utf-8",
"columns": headers,
"column_signature_normalized": sig,
"sample_rows": sample_rows,
"seed_template": seed_meta,
"field_mappings": field_mappings,
"type_conversions": type_conversions,
}
def _admin_csv_limits() -> dict[str, int]:
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT value FROM system_config WHERE key = %s", ("csv_import",))
row = cur.fetchone()
return get_csv_import_limits(r2d(row) if row else None)
@router.post("/validate")
def validate_system_template_dry_run(body: CsvTemplateValidateBody, session: dict = Depends(require_admin)):
"""
Validatorlauf für eine Vorlagen-Konfiguration (ohne DB-Schreiben).
Nutzbar aus dem Admin-Editor vor dem Speichern.
"""
if not get_module_definition(body.module):
raise HTTPException(400, f"Unbekanntes Modul: {body.module}")
return validate_csv_template(
body.module,
body.field_mappings,
body.type_conversions,
body.import_row_processing,
body.column_signature,
)
@router.get("/{template_id}")
def get_system_template(template_id: int, session: dict = Depends(require_admin)):
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"SELECT * FROM csv_field_mappings WHERE id = %s AND is_system = true AND profile_id IS NULL",
(template_id,),
)
m = r2d(cur.fetchone())
if not m:
raise HTTPException(404, "System-Template nicht gefunden")
return _row_full(m)
@router.post("")
def create_system_template(body: CsvSystemTemplateCreate, session: dict = Depends(require_admin)):
if not get_module_definition(body.module):
raise HTTPException(400, f"Unbekanntes Modul: {body.module}")
report = validate_csv_template(
body.module,
body.field_mappings,
body.type_conversions,
body.import_row_processing,
body.column_signature,
)
if not report["valid"]:
raise HTTPException(status_code=422, detail=report)
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions, import_row_processing
) VALUES (
NULL, true, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s
) RETURNING id
""",
(
body.module,
body.mapping_name,
body.description,
body.column_signature,
body.delimiter,
body.encoding,
body.has_header,
Json(body.field_mappings),
Json(body.type_conversions) if body.type_conversions is not None else None,
Json(body.import_row_processing) if body.import_row_processing is not None else None,
),
)
new_id = cur.fetchone()["id"]
return {"id": new_id, "validation": report}
@router.put("/{template_id}")
def update_system_template(
template_id: int,
body: CsvSystemTemplateUpdate,
session: dict = Depends(require_admin),
):
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"SELECT * FROM csv_field_mappings WHERE id = %s AND is_system = true AND profile_id IS NULL",
(template_id,),
)
existing = r2d(cur.fetchone())
if not existing:
raise HTTPException(404, "System-Template nicht gefunden")
patch: dict[str, Any] = body.model_dump(exclude_unset=True)
if not patch:
return _row_full(existing)
fm = patch.get("field_mappings", existing["field_mappings"])
tc_eff = patch.get("type_conversions", existing.get("type_conversions"))
irp_eff = patch.get("import_row_processing", existing.get("import_row_processing"))
col_eff = patch.get("column_signature", existing.get("column_signature"))
report = validate_csv_template(
existing["module"],
fm,
tc_eff,
irp_eff,
col_eff if isinstance(col_eff, list) else None,
)
if not report["valid"]:
raise HTTPException(status_code=422, detail=report)
fields_sql = []
vals: list = []
if "mapping_name" in patch:
fields_sql.append("mapping_name = %s")
vals.append(patch["mapping_name"])
if "description" in patch:
fields_sql.append("description = %s")
vals.append(patch["description"])
if "column_signature" in patch:
fields_sql.append("column_signature = %s")
vals.append(patch["column_signature"])
if "delimiter" in patch:
fields_sql.append("delimiter = %s")
vals.append(patch["delimiter"])
if "encoding" in patch:
fields_sql.append("encoding = %s")
vals.append(patch["encoding"])
if "has_header" in patch:
fields_sql.append("has_header = %s")
vals.append(patch["has_header"])
if "field_mappings" in patch:
fields_sql.append("field_mappings = %s")
vals.append(Json(patch["field_mappings"]))
if "type_conversions" in patch:
fields_sql.append("type_conversions = %s")
tc = patch["type_conversions"]
vals.append(Json(tc) if tc is not None else None)
if "import_row_processing" in patch:
irp = patch["import_row_processing"]
fields_sql.append("import_row_processing = %s")
vals.append(Json(irp) if irp is not None else None)
fields_sql.append("updated_at = CURRENT_TIMESTAMP")
vals.append(template_id)
cur.execute(
f"UPDATE csv_field_mappings SET {', '.join(fields_sql)} WHERE id = %s",
tuple(vals),
)
cur.execute("SELECT * FROM csv_field_mappings WHERE id = %s", (template_id,))
m = r2d(cur.fetchone())
return {**_row_full(m), "validation": report}
@router.delete("/{template_id}")
def delete_system_template(template_id: int, session: dict = Depends(require_admin)):
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"DELETE FROM csv_field_mappings WHERE id = %s AND is_system = true AND profile_id IS NULL RETURNING id",
(template_id,),
)
row = cur.fetchone()
if not row:
raise HTTPException(404, "System-Template nicht gefunden")
return {"deleted": template_id}
+56 -46
View File
@@ -35,7 +35,8 @@ from data_layer.nutrition_metrics import (
get_nutrition_average_data,
get_protein_targets_data,
get_protein_adequacy_data,
get_macro_consistency_data
get_macro_consistency_data,
get_energy_balance_data,
)
from data_layer.activity_metrics import (
get_activity_summary_data,
@@ -118,7 +119,7 @@ def get_weight_trend_chart(
"""
profile_id = session['profile_id']
# Get structured data from data layer
# Get structured data from data layer (includes series — no second weight_log query)
trend_data = get_weight_trend_data(profile_id, days)
# Early return if insufficient data
@@ -136,22 +137,12 @@ def get_weight_trend_chart(
}
}
# Get raw data points for chart
from db import get_db, get_cursor
with get_db() as conn:
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
cur.execute(
"""SELECT date, weight FROM weight_log
WHERE profile_id=%s AND date >= %s
ORDER BY date""",
(profile_id, cutoff)
)
rows = cur.fetchall()
# Format for Chart.js
labels = [row['date'].isoformat() for row in rows]
values = [float(row['weight']) for row in rows]
series = trend_data.get("series") or []
labels = [
pt["date"].isoformat() if hasattr(pt["date"], "isoformat") else str(pt["date"])
for pt in series
]
values = [pt["weight"] for pt in series]
return {
"chart_type": "line",
@@ -346,17 +337,20 @@ def get_energy_balance_chart(
"""
profile_id = session['profile_id']
balance_meta = get_energy_balance_data(profile_id, days)
from db import get_db, get_cursor
with get_db() as conn:
cur = get_cursor(conn)
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
cur.execute(
"""SELECT date, kcal
"""SELECT date, SUM(kcal)::float AS kcal
FROM nutrition_log
WHERE profile_id=%s AND date >= %s AND kcal IS NOT NULL
GROUP BY date
ORDER BY date""",
(profile_id, cutoff)
(profile_id, cutoff),
)
rows = cur.fetchall()
@@ -374,7 +368,21 @@ def get_energy_balance_chart(
}
}
# Prepare data
estimated_tdee = balance_meta.get("estimated_tdee") or 0
if estimated_tdee <= 0:
return {
"chart_type": "line",
"data": {
"labels": [],
"datasets": []
},
"metadata": {
"confidence": "insufficient",
"data_points": len(rows),
"message": "Kein Gewicht für TDEE-Schätzung (weight_log erforderlich)"
}
}
labels = []
daily_values = []
avg_7d = []
@@ -384,23 +392,19 @@ def get_energy_balance_chart(
labels.append(row['date'].isoformat())
daily_values.append(safe_float(row['kcal']))
# 7d rolling average
start_7d = max(0, i - 6)
window_7d = [safe_float(rows[j]['kcal']) for j in range(start_7d, i + 1)]
avg_7d.append(round(sum(window_7d) / len(window_7d), 1) if window_7d else None)
# 14d rolling average
start_14d = max(0, i - 13)
window_14d = [safe_float(rows[j]['kcal']) for j in range(start_14d, i + 1)]
avg_14d.append(round(sum(window_14d) / len(window_14d), 1) if window_14d else None)
# Calculate TDEE (estimated, should come from profile)
# TODO: Calculate from profile (weight, height, age, activity level)
estimated_tdee = 2500.0
# Calculate deficit/surplus
avg_intake = sum(daily_values) / len(daily_values) if daily_values else 0
energy_balance = avg_intake - estimated_tdee
avg_intake = float(balance_meta.get("avg_intake") or (sum(daily_values) / len(daily_values) if daily_values else 0))
energy_balance = float(balance_meta.get("energy_balance") or (avg_intake - estimated_tdee))
balance_status = balance_meta.get("status") or (
"deficit" if energy_balance < -200 else "surplus" if energy_balance > 200 else "maintenance"
)
datasets = [
{
@@ -443,8 +447,7 @@ def get_energy_balance_chart(
}
]
from data_layer.utils import calculate_confidence
confidence = calculate_confidence(len(rows), days, "general")
confidence = balance_meta.get("confidence") or "low"
return {
"chart_type": "line",
@@ -458,7 +461,7 @@ def get_energy_balance_chart(
"avg_kcal": round(avg_intake, 1),
"estimated_tdee": estimated_tdee,
"energy_balance": round(energy_balance, 1),
"balance_status": "deficit" if energy_balance < -200 else "surplus" if energy_balance > 200 else "maintenance",
"balance_status": balance_status,
"first_date": rows[0]['date'],
"last_date": rows[-1]['date']
})
@@ -587,11 +590,12 @@ def get_protein_adequacy_chart(
cutoff = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
cur.execute(
"""SELECT date, protein_g
"""SELECT date, SUM(protein_g)::float AS protein_g
FROM nutrition_log
WHERE profile_id=%s AND date >= %s AND protein_g IS NOT NULL
GROUP BY date
ORDER BY date""",
(profile_id, cutoff)
(profile_id, cutoff),
)
rows = cur.fetchall()
@@ -687,7 +691,6 @@ def get_protein_adequacy_chart(
from data_layer.utils import calculate_confidence
confidence = calculate_confidence(len(rows), days, "general")
# Count days in target
days_in_target = sum(1 for v in daily_values if target_low <= v <= target_high)
return {
@@ -978,16 +981,23 @@ def get_nutrition_adherence_score(
# Get nutrition data
cur.execute(
"""SELECT COUNT(*) as cnt,
AVG(kcal) as avg_kcal,
STDDEV(kcal) as std_kcal,
AVG(protein_g) as avg_protein,
AVG(carbs_g) as avg_carbs,
AVG(fat_g) as avg_fat
FROM nutrition_log
WHERE profile_id=%s AND date >= %s
AND kcal IS NOT NULL""",
(profile_id, cutoff)
"""WITH daily AS (
SELECT date,
COALESCE(SUM(kcal), 0)::float AS dk,
COALESCE(SUM(protein_g), 0)::float AS dp,
COALESCE(SUM(carbs_g), 0)::float AS dc,
COALESCE(SUM(fat_g), 0)::float AS df FROM nutrition_log
WHERE profile_id=%s AND date >= %s AND kcal IS NOT NULL
GROUP BY date
)
SELECT COUNT(*)::int AS cnt,
AVG(dk) AS avg_kcal,
STDDEV(dk) AS std_kcal,
AVG(dp) AS avg_protein,
AVG(dc) AS avg_carbs,
AVG(df) AS avg_fat
FROM daily""",
(profile_id, cutoff),
)
stats = cur.fetchone()
+646
View File
@@ -0,0 +1,646 @@
"""
CSV-Import: Nutzer-Endpunkte für Analyse, Mappings, Limits (Issue #21).
"""
from __future__ import annotations
import logging
from typing import Any, Optional
from fastapi import APIRouter, Depends, File, Form, Header, HTTPException, UploadFile
from pydantic import BaseModel
from psycopg2.extras import Json
from auth import require_auth, check_feature_access, increment_feature_usage
from feature_logger import log_feature_usage
from db import get_db, get_cursor, r2d
from routers.profiles import get_pid
from csv_parser.executor import (
diagnose_activity_row,
diagnose_blood_pressure_row,
diagnose_vitals_row,
run_universal_csv_import,
)
from csv_parser.core import (
decode_raw_bytes,
column_signature,
get_csv_import_limits,
headers_signature_rank_metrics,
iter_csv_dict_rows,
normalize_header_for_signature,
parse_csv_sample,
)
from csv_parser.type_converter import build_row_after_mapping, diagnose_row_mapping
from csv_parser.field_units import source_unit_choices_for_field
from csv_parser.import_errors import enrich_row_error
from csv_parser.module_registry import get_module_definition, list_modules, validate_field_mappings
from csv_parser.sleep_apple_import import detect_apple_sleep_csv_format
router = APIRouter(prefix="/api/csv", tags=["csv-import"])
logger = logging.getLogger(__name__)
def _load_import_limits() -> dict[str, int]:
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT value FROM system_config WHERE key = %s", ("csv_import",))
row = cur.fetchone()
return get_csv_import_limits(r2d(row) if row else None)
def _mapping_to_summary(m: dict) -> dict:
return {
"id": m["id"],
"module": m["module"],
"name": m["mapping_name"],
"description": m.get("description"),
"is_system": m["is_system"],
"usage_count": m.get("usage_count"),
"success_rate": m.get("success_rate"),
"last_used_at": m.get("last_used_at"),
"created_at": m.get("created_at"),
}
@router.get("/modules")
def csv_modules(session: dict = Depends(require_auth)):
"""Unterstützte Import-Module und Felddefinitionen."""
out = []
for mid in list_modules():
d = get_module_definition(mid)
if d:
fields_out = {}
for fname, finfo in (d.get("fields") or {}).items():
fd = dict(finfo)
opts = source_unit_choices_for_field(mid, fname)
if opts:
fd["source_unit_options"] = opts
fields_out[fname] = fd
out.append(
{
"id": mid,
"table": d["table"],
"fields": fields_out,
"import_mode": d.get("import_mode"),
"import_row_processing_default": d.get("import_row_processing_default"),
}
)
return {"modules": out}
@router.get("/limits")
def csv_limits(session: dict = Depends(require_auth)):
"""Admin-konfigurierbare Import-Limits (system_config.csv_import)."""
return _load_import_limits()
@router.get("/mappings")
def list_csv_mappings(
module: Optional[str] = None,
session: dict = Depends(require_auth),
):
"""System-Templates + eigene User-Mappings."""
pid = str(session["profile_id"])
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""
SELECT id, module, mapping_name, description, is_system, profile_id,
usage_count, success_rate, last_used_at, created_at
FROM csv_field_mappings
WHERE is_system = true
AND (%s::text IS NULL OR module = %s)
ORDER BY usage_count DESC NULLS LAST, mapping_name
""",
(module, module),
)
system_rows = [r2d(r) for r in cur.fetchall()]
cur.execute(
"""
SELECT id, module, mapping_name, description, is_system, profile_id,
usage_count, success_rate, last_used_at, created_at
FROM csv_field_mappings
WHERE is_system = false AND profile_id = %s::uuid
AND (%s::text IS NULL OR module = %s)
ORDER BY last_used_at DESC NULLS LAST, mapping_name
""",
(pid, module, module),
)
user_rows = [r2d(r) for r in cur.fetchall()]
return {
"system_templates": [_mapping_to_summary(m) for m in system_rows],
"user_mappings": [_mapping_to_summary(m) for m in user_rows],
}
class CopyMappingBody(BaseModel):
name: Optional[str] = None
@router.post("/mappings/{mapping_id}/copy")
def copy_csv_mapping(
mapping_id: int,
body: CopyMappingBody | None = None,
session: dict = Depends(require_auth),
):
"""System- oder eigenes Mapping als neues User-Mapping kopieren."""
pid = str(session["profile_id"])
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""
SELECT * FROM csv_field_mappings WHERE id = %s
""",
(mapping_id,),
)
src = r2d(cur.fetchone())
if not src:
raise HTTPException(404, "Mapping nicht gefunden")
if not src["is_system"] and str(src.get("profile_id")) != pid:
raise HTTPException(403, "Kein Zugriff auf dieses Mapping")
base_name = (body.name if body and body.name else None) or f"{src['mapping_name']} (Kopie)"
name = base_name
n = 1
while True:
cur.execute(
"""
SELECT 1 FROM csv_field_mappings
WHERE profile_id = %s::uuid AND module = %s AND mapping_name = %s
""",
(pid, src["module"], name),
)
if not cur.fetchone():
break
n += 1
name = f"{base_name} {n}"
cur.execute(
"""
INSERT INTO csv_field_mappings (
profile_id, is_system, module, mapping_name, description,
column_signature, delimiter, encoding, has_header,
field_mappings, type_conversions, import_row_processing,
usage_count, success_rate
) VALUES (
%s::uuid, false, %s, %s, %s,
%s, %s, %s, %s, %s, %s, %s, 0, 1.0
) RETURNING id
""",
(
pid,
src["module"],
name,
src.get("description"),
src["column_signature"],
src["delimiter"],
src["encoding"],
src["has_header"],
Json(src["field_mappings"]),
Json(src["type_conversions"]) if src.get("type_conversions") is not None else None,
Json(src["import_row_processing"])
if src.get("import_row_processing") is not None
else None,
),
)
new_id = cur.fetchone()["id"]
return {"new_mapping_id": new_id, "mapping_name": name}
@router.post("/analyze")
async def analyze_csv(
file: UploadFile = File(...),
module: Optional[str] = Form(default=None),
delimiter: Optional[str] = Form(default=None),
session: dict = Depends(require_auth),
):
"""
Erste Zeilen parsen, Signatur bilden, Vorlagen nach Ähnlichkeit ranken.
Ohne `module`: alle System- + eigene User-Vorlagen (echte Format-Erkennung über Spalten-Signatur).
Mit `module`: nur Vorlagen dieses Moduls (Abwärtskompatibilität).
"""
if module and not get_module_definition(module):
raise HTTPException(400, f"Unbekanntes Modul: {module}")
pid = str(session["profile_id"])
raw = await file.read()
limits = _load_import_limits()
max_bytes = limits.get("max_file_bytes", 52_428_800)
if len(raw) > max_bytes:
raise HTTPException(
413,
f"Datei zu groß (max. {max_bytes} Bytes laut Systemkonfiguration)",
)
text = decode_raw_bytes(raw)
max_rows = limits.get("max_rows_per_file", 50_000)
if text.count("\n") > max_rows + 5:
raise HTTPException(
413,
f"Zu viele Zeilen (>{max_rows}) laut Systemkonfiguration csv_import.max_rows_per_file",
)
delim = delimiter if delimiter in (",", ";", "\t") else None
headers, sample_rows, used_delim = parse_csv_sample(text, delimiter=delim, max_data_rows=5)
sig = column_signature(headers)
apple_sleep_csv = False
try:
detect_apple_sleep_csv_format(headers)
apple_sleep_csv = True
except ValueError:
pass
mod_def = get_module_definition(module) if module else None
available_fields = mod_def["fields"] if mod_def else None
with get_db() as conn:
cur = get_cursor(conn)
if module:
cur.execute(
"""
SELECT id, module, mapping_name, description, column_signature,
delimiter, encoding, has_header, field_mappings, type_conversions, is_system
FROM csv_field_mappings
WHERE module = %s
AND (is_system = true OR (is_system = false AND profile_id = %s::uuid))
""",
(module, pid),
)
else:
cur.execute(
"""
SELECT id, module, mapping_name, description, column_signature,
delimiter, encoding, has_header, field_mappings, type_conversions, is_system
FROM csv_field_mappings
WHERE is_system = true OR (is_system = false AND profile_id = %s::uuid)
""",
(pid,),
)
templates = [r2d(r) for r in cur.fetchall()]
ranked = []
for t in templates:
t_sig = list(t["column_signature"]) if t["column_signature"] else []
metrics = headers_signature_rank_metrics(sig, t_sig)
conf = float(metrics["confidence"] or 0)
if apple_sleep_csv and t.get("module") == "sleep":
conf = max(conf, 1.0)
ranked.append(
{
"mapping_id": t["id"],
"module": t["module"],
"mapping_name": t["mapping_name"],
"is_system": bool(t.get("is_system")),
"confidence": round(conf, 4),
"template_recall": metrics["template_recall"],
"jaccard": metrics["jaccard"],
"columns_matched": metrics["columns_matched"],
"columns_in_template": metrics["columns_in_template"],
"columns_in_csv": metrics["columns_in_csv"],
"match_type": "template_recall",
}
)
ranked.sort(
key=lambda x: (
-(x.get("confidence") or 0),
-(x.get("columns_matched") or 0),
-(x.get("jaccard") or 0),
),
)
top = ranked[:25]
recommended = top[0] if top and (top[0]["confidence"] or 0) > 0 else None
warnings: list[str] = []
if apple_sleep_csv and not any(t.get("module") == "sleep" for t in templates):
warnings.append(
"Diese Datei ist ein Apple-Schlafexport, aber es fehlt eine Vorlage für das Modul «Schlaf» "
"(z. B. Migration 044 / Admin → CSV-Vorlagen)."
)
if (
apple_sleep_csv
and recommended
and recommended.get("module") != "sleep"
and any(t.get("module") == "sleep" for t in templates)
):
warnings.append(
"Apple-Schlaf-CSV erkannt — bitte eine Vorlage unter Modul «Schlaf» wählen, nicht «"
+ str(recommended.get("module") or "")
+ "».",
)
return {
"module_filter": module,
"filename": file.filename,
"encoding_note": "utf-8/latin-1 mit BOM-Strip",
"delimiter": used_delim,
"columns": headers,
"column_signature_normalized": sig,
"sample_rows": sample_rows,
"detected_mappings": top,
"recommended": recommended,
"available_fields": available_fields,
"format_detection": {"apple_sleep": apple_sleep_csv},
"warnings": warnings,
}
@router.post("/import-diagnose")
async def csv_import_diagnose(
file: UploadFile = File(...),
mapping_id: int = Form(...),
module: Optional[str] = Form(default=None),
x_profile_id: Optional[str] = Header(default=None),
session: dict = Depends(require_auth),
):
"""
Kein Import, keine Quota: gleiche Datei + gewählte Vorlage wie beim Import,
aber nur die ersten Zeilen durch Mapping/Typkonvertierung (und Vital-Checks).
Zum Debugging wenn «Datum fehlt» o. Ä. ohne DB-Zugriff.
"""
pid = get_pid(x_profile_id)
raw = await file.read()
limits = _load_import_limits()
max_bytes = limits.get("max_file_bytes", 52_428_800)
if len(raw) > max_bytes:
raise HTTPException(413, f"Datei zu groß (max. {max_bytes} Bytes)")
text = decode_raw_bytes(raw)
if not text.strip():
raise HTTPException(400, "Leere Datei")
with get_db() as conn:
cur = get_cursor(conn)
m = _fetch_mapping_row(cur, mapping_id, pid, module)
if not get_module_definition(m["module"]):
raise HTTPException(400, f"Modul nicht unterstützt: {m['module']}")
fm = m.get("field_mappings") or {}
if isinstance(fm, str):
fm = {}
tc = m.get("type_conversions")
if not isinstance(tc, dict):
tc = {}
delim = str(m.get("delimiter") or ",")
exec_module = str(m["module"])
rows_out: list[dict[str, Any]] = []
for i, row in enumerate(iter_csv_dict_rows(text, delim, has_header=True)):
if i >= 5:
break
typed = build_row_after_mapping(row, fm, tc, module=exec_module)
dm = diagnose_row_mapping(row, fm, tc, module=exec_module, mapped_typed=typed)
entry: dict[str, Any] = {
"row_index": i + 1,
**dm,
}
if exec_module == "vitals_baseline":
entry["vitals"] = diagnose_vitals_row(typed)
elif exec_module == "blood_pressure":
entry["blood_pressure"] = diagnose_blood_pressure_row(typed)
elif exec_module == "activity":
entry["activity"] = diagnose_activity_row(typed)
rows_out.append(entry)
return {
"mapping_id": mapping_id,
"mapping_name": m.get("mapping_name"),
"module": exec_module,
"delimiter_used": delim,
"has_header": bool(m.get("has_header", True)),
"rows_diagnosed": len(rows_out),
"rows": rows_out,
}
def _fetch_mapping_row(
cur,
mapping_id: int,
profile_id: str,
module: Optional[str] = None,
) -> dict:
cur.execute(
"""
SELECT * FROM csv_field_mappings WHERE id = %s
""",
(mapping_id,),
)
m = r2d(cur.fetchone())
if not m:
raise HTTPException(404, "Mapping nicht gefunden")
if module is not None and m.get("module") != module:
raise HTTPException(400, "Mapping gehört zu einem anderen Modul")
if not m.get("is_system"):
if str(m.get("profile_id") or "") != profile_id:
raise HTTPException(403, "Kein Zugriff auf dieses Mapping")
return m
def _check_module_feature_access(pid: str, module: str) -> None:
if module == "nutrition":
access = check_feature_access(pid, "nutrition_entries")
log_feature_usage(pid, "nutrition_entries", access, "csv_universal_import")
if not access["allowed"]:
raise HTTPException(
403,
f"Limit erreicht (Ernährungseinträge): {access.get('used')}/{access.get('limit')}",
)
elif module == "weight":
access = check_feature_access(pid, "weight_entries")
log_feature_usage(pid, "weight_entries", access, "csv_universal_import")
if not access["allowed"]:
raise HTTPException(
403,
f"Limit erreicht (Gewichtseinträge): {access.get('used')}/{access.get('limit')}",
)
elif module == "activity":
access = check_feature_access(pid, "activity_entries")
log_feature_usage(pid, "activity_entries", access, "csv_universal_import")
if not access["allowed"]:
raise HTTPException(
403,
f"Limit erreicht (Aktivitätseinträge): {access.get('used')}/{access.get('limit')}",
)
@router.post("/import")
async def csv_import_execute(
file: UploadFile = File(...),
mapping_id: int = Form(...),
module: Optional[str] = Form(default=None),
x_profile_id: Optional[str] = Header(default=None),
session: dict = Depends(require_auth),
):
"""
Universal-CSV-Import: Zielmodul kommt aus der gewählten Vorlage (`mapping_id`).
Optional `module` dient nur der Absicherung (muss mit Vorlage übereinstimmen).
"""
pid = get_pid(x_profile_id)
access_di = check_feature_access(pid, "data_import")
log_feature_usage(pid, "data_import", access_di, "csv_universal_import")
if not access_di["allowed"]:
raise HTTPException(
403,
"Limit erreicht (Daten importieren): "
f"{access_di.get('used')}/{access_di.get('limit')}",
)
raw = await file.read()
limits = _load_import_limits()
max_bytes = limits.get("max_file_bytes", 52_428_800)
if len(raw) > max_bytes:
raise HTTPException(
413,
f"Datei zu groß (max. {max_bytes} Bytes laut Systemkonfiguration)",
)
text = decode_raw_bytes(raw)
if not text.strip():
raise HTTPException(400, "Leere Datei")
max_rows = limits.get("max_rows_per_file", 50_000)
if text.count("\n") > max_rows + 5:
raise HTTPException(
413,
f"Zu viele Zeilen (>{max_rows}) laut Systemkonfiguration",
)
log_id: int | None = None
err_response: HTTPException | None = None
result: dict | None = None
resolved_module: str | None = None
try:
with get_db() as conn:
cur = get_cursor(conn)
m = _fetch_mapping_row(cur, mapping_id, pid, module)
exec_module = m["module"]
resolved_module = exec_module
if not get_module_definition(exec_module):
raise HTTPException(
400,
f"Modul der Vorlage wird vom Importer noch nicht unterstützt: {exec_module}",
)
_check_module_feature_access(pid, exec_module)
cur.execute(
"""
INSERT INTO csv_import_log (
profile_id, mapping_id, module, filename,
rows_total, rows_imported, rows_updated, rows_skipped, rows_errors,
status, error_details, affected_ids
) VALUES (
%s::uuid, %s, %s, %s,
0, 0, 0, 0, 0,
'running', NULL, NULL
) RETURNING id
""",
(pid, mapping_id, exec_module, file.filename or "upload.csv"),
)
log_id = cur.fetchone()["id"]
cur.execute("SAVEPOINT csv_import_exec")
try:
result = run_universal_csv_import(
cur,
pid,
exec_module,
text,
file.filename or "upload.csv",
m,
)
except Exception as exec_err:
logger.exception("Universal-CSV-Import fehlgeschlagen: %s", exec_err)
cur.execute("ROLLBACK TO SAVEPOINT csv_import_exec")
err_payload = {"error": str(exec_err), **enrich_row_error(str(exec_err), exec_module)}
cur.execute(
"""
UPDATE csv_import_log SET
finished_at = CURRENT_TIMESTAMP,
status = 'failed',
error_details = %s
WHERE id = %s
""",
(Json([err_payload]), log_id),
)
hint = err_payload.get("hint")
msg = f"Import fehlgeschlagen: {exec_err}"
if hint:
msg = f"{msg} ({hint})"
err_response = HTTPException(500, msg)
else:
cur.execute("RELEASE SAVEPOINT csv_import_exec")
cur.execute(
"""
UPDATE csv_import_log SET
finished_at = CURRENT_TIMESTAMP,
status = 'success',
rows_total = %s,
rows_imported = %s,
rows_updated = %s,
rows_skipped = %s,
rows_errors = %s,
error_details = %s,
affected_ids = %s
WHERE id = %s
""",
(
result["rows_total"],
result["rows_imported"],
result["rows_updated"],
result["rows_skipped"],
result["rows_errors"],
Json(result["error_details"]),
Json(result["affected_ids"]),
log_id,
),
)
cur.execute(
"""
UPDATE csv_field_mappings SET
usage_count = usage_count + 1,
last_used_at = CURRENT_TIMESTAMP,
updated_at = CURRENT_TIMESTAMP
WHERE id = %s
""",
(mapping_id,),
)
except HTTPException:
raise
if err_response:
raise err_response
assert result is not None
increment_feature_usage(pid, "data_import")
ne = result.get("new_entries", result["rows_imported"])
if resolved_module == "nutrition":
for _ in range(ne):
increment_feature_usage(pid, "nutrition_entries")
elif resolved_module == "weight":
for _ in range(ne):
increment_feature_usage(pid, "weight_entries")
elif resolved_module == "activity":
for _ in range(ne):
increment_feature_usage(pid, "activity_entries")
return {
"success": True,
"import_log_id": log_id,
"module": resolved_module,
"stats": {
"total_rows": result["rows_total"],
"imported": result["rows_imported"],
"updated": result["rows_updated"],
"skipped": result["rows_skipped"],
"errors": result["rows_errors"],
},
"error_details": result["error_details"],
}
+6
View File
@@ -699,6 +699,12 @@ def get_goals_grouped(session: dict = Depends(require_auth)):
grouped[cat] = []
goal_dict = r2d(goal)
# Keep in sync with GET /goals/list: derive current_value & progress from live data
try:
_update_goal_progress(conn, pid, goal_dict)
except Exception as e:
print(f"[ERROR] Failed to update progress for goal {goal_dict.get('id')}: {e}")
goal_dict['focus_contributions'] = focus_map.get(goal['id'], [])
grouped[cat].append(goal_dict)
+466 -165
View File
@@ -17,10 +17,11 @@ from models import (
PromptCreate, PromptUpdate, PromptGenerateRequest,
PipelineConfigCreate, PipelineConfigUpdate
)
from prompt_executor import resolve_placeholders as resolve_prompt_placeholders
from placeholder_resolver import (
resolve_placeholders,
get_unknown_placeholders,
get_placeholder_example_values,
format_value_with_d_modifier,
get_available_placeholders,
get_placeholder_catalog
)
@@ -31,6 +32,42 @@ OPENROUTER_MODEL = os.getenv("OPENROUTER_MODEL", "anthropic/claude-sonnet-4")
router = APIRouter(prefix="/api/prompts", tags=["prompts"])
# Metadaten-Schlüssel in workflow aggregate_results (nicht als „einziger“ Nutzer-Output)
_WORKFLOW_AGG_META_KEYS = frozenset({
"combined_analysis",
"all_signals",
"total_nodes",
"executed_nodes",
"failed_nodes",
"skipped_nodes",
})
def _workflow_user_facing_content(agg: object) -> str:
"""
Nutzer-sichtbarer Text wie im Admin WorkflowResultViewer („Final Output“):
primär aggregated_result['analysis_core'], nicht das gesamte JSON.
"""
if agg is None:
return ""
if isinstance(agg, str):
return agg
if not isinstance(agg, dict):
return json.dumps(agg, ensure_ascii=False)
core = agg.get("analysis_core")
if isinstance(core, str) and core.strip():
return core
combined = agg.get("combined_analysis")
if isinstance(combined, str) and combined.strip():
return combined
non_meta = [k for k in agg.keys() if k not in _WORKFLOW_AGG_META_KEYS]
if len(non_meta) == 1:
v = agg[non_meta[0]]
if isinstance(v, str):
return v
return json.dumps(v, ensure_ascii=False)
return json.dumps(agg, ensure_ascii=False)
@router.get("")
def list_prompts(session: dict=Depends(require_auth)):
@@ -67,6 +104,156 @@ def list_placeholders_endpoint(session: dict=Depends(require_auth)):
return get_placeholder_catalog(profile_id)
@router.get("/export-all")
def export_all_prompts(session: dict = Depends(require_admin)):
"""
Export all prompts as JSON array.
Admin only. Used for backup and dev→prod sync.
IMPORTANT: Must be defined BEFORE /{prompt_id} route to avoid routing conflict.
"""
from datetime import datetime
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT * FROM ai_prompts ORDER BY sort_order, slug")
prompts = [r2d(row) for row in cur.fetchall()]
# Convert to export format (clean up DB-specific fields)
export_data = []
for p in prompts:
export_item = {
'slug': p['slug'],
'name': p['name'],
'display_name': p.get('display_name'),
'description': p.get('description'),
'type': p.get('type', 'pipeline'),
'category': p.get('category', 'ganzheitlich'),
'template': p.get('template'),
'stages': p.get('stages'),
'output_format': p.get('output_format', 'text'),
'output_schema': p.get('output_schema'),
'question_augmentations': p.get('question_augmentations'),
'graph_data': p.get('graph_data'),
'active': p.get('active', True),
'sort_order': p.get('sort_order', 0)
}
export_data.append(export_item)
return {
'export_date': datetime.now().isoformat(),
'count': len(export_data),
'prompts': export_data
}
@router.post("/import")
def import_prompts(
data: dict,
overwrite: bool = False,
session: dict = Depends(require_admin)
):
"""
Import prompts from JSON export.
Args:
data: Export data from /export-all endpoint
overwrite: If true, update existing prompts. If false, skip duplicates.
Returns:
Summary of import results (created, updated, skipped)
IMPORTANT: Must be defined BEFORE /{prompt_id} route to avoid routing conflict.
"""
if 'prompts' not in data:
raise HTTPException(400, "Invalid import data: missing 'prompts' key")
prompts = data['prompts']
created = 0
updated = 0
skipped = 0
errors = []
with get_db() as conn:
cur = get_cursor(conn)
for p in prompts:
slug = p.get('slug')
if not slug:
errors.append('Prompt without slug skipped')
continue
# Check if exists
cur.execute("SELECT id FROM ai_prompts WHERE slug=%s", (slug,))
existing = cur.fetchone()
if existing and not overwrite:
skipped += 1
continue
# Prepare JSON fields if present
stages_json = None
if p.get('stages'):
stages_json = json.dumps(p['stages']) if isinstance(p['stages'], list) else p['stages']
output_schema_json = None
if p.get('output_schema'):
output_schema_json = json.dumps(p['output_schema']) if isinstance(p['output_schema'], dict) else p['output_schema']
question_aug_json = None
if p.get('question_augmentations'):
question_aug_json = json.dumps(p['question_augmentations']) if isinstance(p['question_augmentations'], (dict, list)) else p['question_augmentations']
graph_data_json = None
if p.get('graph_data'):
graph_data_json = json.dumps(p['graph_data']) if isinstance(p['graph_data'], dict) else p['graph_data']
if existing:
# Update existing
cur.execute("""
UPDATE ai_prompts SET
name=%s, display_name=%s, description=%s, type=%s,
category=%s, template=%s, stages=%s, output_format=%s,
output_schema=%s, question_augmentations=%s, graph_data=%s,
active=%s, sort_order=%s,
updated=CURRENT_TIMESTAMP
WHERE slug=%s
""", (
p.get('name'), p.get('display_name'), p.get('description'),
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
p.get('template'), stages_json, p.get('output_format', 'text'),
output_schema_json, question_aug_json, graph_data_json,
p.get('active', True), p.get('sort_order', 0), slug
))
updated += 1
else:
# Create new
cur.execute("""
INSERT INTO ai_prompts (
slug, name, display_name, description, type, category,
template, stages, output_format, output_schema,
question_augmentations, graph_data,
active, sort_order, created, updated
) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP)
""", (
slug, p.get('name'), p.get('display_name'), p.get('description'),
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
p.get('template'), stages_json, p.get('output_format', 'text'),
output_schema_json, question_aug_json, graph_data_json,
p.get('active', True), p.get('sort_order', 0)
))
created += 1
conn.commit()
return {
'success': True,
'created': created,
'updated': updated,
'skipped': skipped,
'errors': errors if errors else None
}
@router.get("/{prompt_id}")
def get_prompt(prompt_id: str, session: dict=Depends(require_auth)):
"""Get single AI prompt by ID (UUID)."""
@@ -178,11 +365,31 @@ def duplicate_prompt(prompt_id: str, session: dict=Depends(require_admin)):
new_display_name = f"{original.get('display_name') or original['name']} (Kopie)"
# Prepare JSONB fields (convert dict/list to JSON string if needed)
stages_json = json.dumps(original['stages']) if original.get('stages') else None
output_schema_json = json.dumps(original['output_schema']) if original.get('output_schema') else None
question_aug_json = json.dumps(original['question_augmentations']) if original.get('question_augmentations') else None
graph_data_json = json.dumps(original['graph_data']) if original.get('graph_data') else None
cur.execute(
"""INSERT INTO ai_prompts (id, name, slug, display_name, description, template, category, active, sort_order, created, updated)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)""",
(new_id, new_name, new_slug, new_display_name, original['description'], original['template'],
original.get('category', 'ganzheitlich'), original['active'], original['sort_order'])
"""INSERT INTO ai_prompts (
id, name, slug, display_name, description, template, category,
type, stages, output_format, output_schema,
question_augmentations, graph_data,
active, sort_order, created, updated
) VALUES (
%s, %s, %s, %s, %s, %s, %s,
%s, %s, %s, %s,
%s, %s,
%s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
)""",
(new_id, new_name, new_slug, new_display_name,
original.get('description'), original.get('template'),
original.get('category', 'ganzheitlich'),
original.get('type', 'pipeline'), stages_json,
original.get('output_format', 'text'), output_schema_json,
question_aug_json, graph_data_json,
original.get('active', True), original.get('sort_order', 0))
)
return {"id": new_id, "slug": new_slug, "name": new_name}
@@ -225,7 +432,13 @@ def preview_prompt(data: dict, session: dict=Depends(require_auth)):
template = data.get('template', '')
profile_id = session['profile_id']
resolved = resolve_placeholders(template, profile_id)
catalog = get_placeholder_catalog(profile_id)
processed = get_placeholder_example_values(profile_id)
variables = {
k.replace('{{', '').replace('}}', ''): v
for k, v in processed.items()
}
resolved = resolve_prompt_placeholders(template, variables, None, catalog)
unknown = get_unknown_placeholders(template)
return {
@@ -251,8 +464,8 @@ def export_placeholder_values(session: dict = Depends(require_auth)):
"""
Export all available placeholders with their current resolved values.
Returns JSON export suitable for download with all placeholders
resolved for the current user's profile.
Pro Zeile: value = {{key}}, example = Vorschau {{key|d}} (Wert — description),
ai_caption = Text für {{key|x}} (Erklärung ohne Wert). JSON für das aktive Profil.
"""
from datetime import datetime
profile_id = session['profile_id']
@@ -280,12 +493,16 @@ def export_placeholder_values(session: dict = Depends(require_auth)):
export_data['placeholders_by_category'][category] = []
for item in items:
key = item['key'].replace('{{', '').replace('}}', '')
export_data['placeholders_by_category'][category].append({
raw_val = cleaned_values.get(key, 'nicht verfügbar')
row = {
'key': item['key'],
'description': item['description'],
'value': cleaned_values.get(key, 'nicht verfügbar'),
'example': item.get('example')
})
'value': raw_val,
'example': format_value_with_d_modifier(str(raw_val), item),
}
if item.get('ai_caption'):
row['ai_caption'] = item['ai_caption']
export_data['placeholders_by_category'][category].append(row)
# Also include flat list for easy access
export_data['all_placeholders'] = cleaned_values
@@ -377,8 +594,14 @@ def export_placeholder_values_extended(
if 'value_raw' not in metadata.unresolved_fields:
metadata.unresolved_fields.append('value_raw')
# Check availability
if value in ['nicht verfügbar', 'nicht genug Daten', '[Fehler:', '[Nicht']:
# Check availability (Resolver liefert oft „nicht verfügbar — <Grund>“)
sv = str(value)
if (
sv in ['nicht verfügbar', 'nicht genug Daten']
or sv.startswith('nicht verfügbar —')
or sv.startswith('[Fehler:')
or sv.startswith('[Nicht')
):
metadata.available = False
metadata.missing_reason = value
else:
@@ -447,11 +670,12 @@ def export_placeholder_values_extended(
export_data['legacy']['placeholders_by_category'][category] = []
for item in items:
key = item['key'].replace('{{', '').replace('}}', '')
raw_val = cleaned_values.get(key, 'nicht verfügbar')
export_data['legacy']['placeholders_by_category'][category].append({
'key': item['key'],
'description': item['description'],
'value': cleaned_values.get(key, 'nicht verfügbar'),
'example': item.get('example')
'value': raw_val,
'example': format_value_with_d_modifier(str(raw_val), item),
})
# Fill metadata flat
@@ -713,27 +937,60 @@ def export_placeholder_catalog_zip(
# ── KI-Assisted Prompt Engineering ───────────────────────────────────────────
async def call_openrouter(prompt: str, max_tokens: int = 1500) -> str:
async def call_openrouter(prompt: str, max_tokens: int = 4096) -> str:
"""Call OpenRouter API to get AI response."""
if not OPENROUTER_KEY:
raise HTTPException(status_code=500, detail="OpenRouter API key not configured")
async with httpx.AsyncClient() as client:
resp = await client.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {OPENROUTER_KEY}"},
json={
"model": OPENROUTER_MODEL,
"messages": [{"role": "user", "content": prompt}],
"max_tokens": max_tokens
},
timeout=60.0
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
"https://openrouter.ai/api/v1/chat/completions",
headers={"Authorization": f"Bearer {OPENROUTER_KEY}"},
json={
"model": OPENROUTER_MODEL,
"messages": [{"role": "user", "content": prompt}],
"max_tokens": max_tokens
},
timeout=120.0
)
except httpx.TimeoutException as e:
raise HTTPException(status_code=504, detail=f"OpenRouter-Zeitüberschreitung: {e}") from e
except httpx.RequestError as e:
raise HTTPException(status_code=502, detail=f"OpenRouter-Verbindungsfehler: {e}") from e
if resp.status_code != 200:
raise HTTPException(
status_code=resp.status_code,
detail=f"OpenRouter API error ({resp.status_code}): {resp.text[:2000]}",
)
if resp.status_code != 200:
raise HTTPException(status_code=resp.status_code, detail=f"OpenRouter API error: {resp.text}")
try:
data = resp.json()
except json.JSONDecodeError:
raise HTTPException(
status_code=502,
detail=f"OpenRouter lieferte kein JSON: {resp.text[:800]}",
)
return resp.json()['choices'][0]['message']['content'].strip()
choices = data.get("choices") or []
if not choices:
err_snip = json.dumps(data.get("error") or data, ensure_ascii=False)[:1200]
raise HTTPException(
status_code=502,
detail=f"OpenRouter ohne choices in der Antwort: {err_snip}",
)
message = choices[0].get("message") or {}
content = message.get("content")
if content is None:
raise HTTPException(
status_code=502,
detail="OpenRouter-Antwort ohne message.content (z. B. Tool-Calls oder abweichendes Schema).",
)
if not isinstance(content, str):
content = str(content)
return content.strip()
def collect_example_data(profile_id: str, data_categories: list[str]) -> dict:
@@ -1188,13 +1445,176 @@ from prompt_executor import execute_prompt_with_data
from models import UnifiedPromptCreate, UnifiedPromptUpdate
@router.post("/execute")
async def execute_unified_prompt(
prompt_slug: str,
@router.post("/execute-stream")
async def execute_unified_prompt_stream(
prompt_slug: str = Query(..., description="Slug of prompt to execute"),
token: Optional[str] = Query(None, description="Auth token (temporary solution for SSE)"),
modules: Optional[dict] = None,
timeframes: Optional[dict] = None,
debug: bool = False,
save: bool = False,
debug: bool = Query(False, description="Include debug information (node_states, etc.)"),
save: bool = Query(False, description="Save result to ai_insights")
):
"""
Execute a unified prompt with Server-Sent Events (SSE) streaming.
Returns live progress updates during workflow execution:
- execution_started: Workflow has begun
- node_complete: Each node completes
- execution_complete: Final result ready
- execution_failed: Error occurred
Use this endpoint for long-running workflows (>30s) to avoid gateway timeouts.
"""
# Manual auth: verify token and get profile_id
if not token:
raise HTTPException(401, "Missing auth token")
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT profile_id FROM sessions WHERE token = %s", (token,))
row = cur.fetchone()
if not row:
raise HTTPException(401, "Invalid or expired token")
profile_id = row['profile_id']
# Use default modules/timeframes if not provided
if not modules:
modules = {
'körper': True,
'ernährung': True,
'training': True,
'schlaf': True,
'vitalwerte': True
}
if not timeframes:
timeframes = {
'körper': 30,
'ernährung': 30,
'training': 14,
'schlaf': 14,
'vitalwerte': 7
}
# Wrapper function for OpenRouter calls
async def workflow_llm_call(prompt: str, model: str = None) -> str:
return await call_openrouter(prompt)
# SSE Event Generator
async def event_stream():
"""Generate Server-Sent Events during workflow execution."""
import asyncio
from asyncio import Queue
# Event queue for progress updates
event_queue = Queue()
# Flag to track execution completion
execution_complete = False
# Define progress callback for streaming updates
async def progress_callback(event_type: str, data: dict):
"""Queue SSE event for streaming to client."""
event_data = {
"type": event_type,
**data
}
await event_queue.put(event_data)
# Start workflow execution in background task
async def execute_workflow_async():
nonlocal execution_complete
try:
# Execute workflow with progress callbacks
result = await execute_prompt_with_data(
prompt_slug=prompt_slug,
profile_id=profile_id,
modules=modules,
timeframes=timeframes,
openrouter_call_func=workflow_llm_call,
enable_debug=debug or save,
progress_callback=progress_callback
)
# Save to ai_insights if requested (same logic as /execute)
if save:
if result['type'] == 'pipeline':
final_output = result.get('output', {})
if isinstance(final_output, dict) and len(final_output) == 1:
content = list(final_output.values())[0]
else:
content = json.dumps(final_output, ensure_ascii=False)
elif result['type'] == 'workflow':
content = _workflow_user_facing_content(result.get('aggregated_result'))
else:
content = result.get('output', '')
if isinstance(content, dict):
content = json.dumps(content, ensure_ascii=False)
# Save to database (minimal metadata for now)
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"""INSERT INTO ai_insights (profile_id, scope, content, metadata, created)
VALUES (%s, %s, %s, %s, CURRENT_TIMESTAMP)""",
(profile_id, prompt_slug, content, json.dumps({"prompt_type": result['type']}))
)
conn.commit()
except Exception as e:
# Queue error event
await event_queue.put({
"type": "execution_failed",
"error": str(e)
})
finally:
execution_complete = True
# Start workflow execution in background
import asyncio
execution_task = asyncio.create_task(execute_workflow_async())
# Stream events from queue
try:
while not execution_complete or not event_queue.empty():
try:
# Wait for event with timeout
event = await asyncio.wait_for(event_queue.get(), timeout=0.5)
yield f"data: {json.dumps(event, ensure_ascii=False)}\n\n"
except asyncio.TimeoutError:
# Send keepalive ping
yield f": keepalive\n\n"
continue
# Wait for execution task to complete
await execution_task
except Exception as e:
# Send final error event
error_event = {
"type": "execution_failed",
"error": str(e)
}
yield f"data: {json.dumps(error_event, ensure_ascii=False)}\n\n"
return StreamingResponse(
event_stream(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no" # Disable nginx buffering
}
)
@router.post("/execute")
async def execute_unified_prompt(
prompt_slug: str = Query(..., description="Slug of prompt to execute"),
modules: Optional[dict] = None,
timeframes: Optional[dict] = None,
debug: bool = Query(False, description="Include debug information (node_states, etc.)"),
save: bool = Query(False, description="Save result to ai_insights"),
session: dict = Depends(require_auth)
):
"""
@@ -1231,6 +1651,14 @@ async def execute_unified_prompt(
'vitalwerte': 7
}
# Wrapper function to match workflow_executor's expected signature: (prompt, model) -> str
# workflow_executor calls: openrouter_call_func(prompt, "anthropic/claude-sonnet-4")
# but call_openrouter expects: call_openrouter(prompt, max_tokens=4096)
async def workflow_llm_call(prompt: str, model: str = None) -> str:
# Ignore model parameter (already set in OPENROUTER_MODEL env var)
# Use default max_tokens=4096 from call_openrouter
return await call_openrouter(prompt)
# Execute with prompt_executor
# Always enable debug when saving to collect metadata for value table
result = await execute_prompt_with_data(
@@ -1238,7 +1666,7 @@ async def execute_unified_prompt(
profile_id=profile_id,
modules=modules,
timeframes=timeframes,
openrouter_call_func=call_openrouter,
openrouter_call_func=workflow_llm_call, # Use wrapper with correct signature
enable_debug=debug or save # Enable debug if saving for metadata collection
)
@@ -1253,6 +1681,8 @@ async def execute_unified_prompt(
content = list(final_output.values())[0]
else:
content = json.dumps(final_output, ensure_ascii=False)
elif result['type'] == 'workflow':
content = _workflow_user_facing_content(result.get('aggregated_result'))
else:
# For base prompts, use output directly
content = result.get('output', '')
@@ -1569,132 +1999,3 @@ def update_unified_prompt(prompt_id: str, p: UnifiedPromptUpdate, session: dict
)
return {"ok": True}
@router.get("/export-all")
def export_all_prompts(session: dict = Depends(require_admin)):
"""
Export all prompts as JSON array.
Admin only. Used for backup and dev→prod sync.
"""
from datetime import datetime
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT * FROM ai_prompts ORDER BY sort_order, slug")
prompts = [r2d(row) for row in cur.fetchall()]
# Convert to export format (clean up DB-specific fields)
export_data = []
for p in prompts:
export_item = {
'slug': p['slug'],
'name': p['name'],
'display_name': p.get('display_name'),
'description': p.get('description'),
'type': p.get('type', 'pipeline'),
'category': p.get('category', 'ganzheitlich'),
'template': p.get('template'),
'stages': p.get('stages'),
'output_format': p.get('output_format', 'text'),
'output_schema': p.get('output_schema'),
'active': p.get('active', True),
'sort_order': p.get('sort_order', 0)
}
export_data.append(export_item)
return {
'export_date': datetime.now().isoformat(),
'count': len(export_data),
'prompts': export_data
}
@router.post("/import")
def import_prompts(
data: dict,
overwrite: bool = False,
session: dict = Depends(require_admin)
):
"""
Import prompts from JSON export.
Args:
data: Export data from /export-all endpoint
overwrite: If true, update existing prompts. If false, skip duplicates.
Returns:
Summary of import results (created, updated, skipped)
"""
if 'prompts' not in data:
raise HTTPException(400, "Invalid import data: missing 'prompts' key")
prompts = data['prompts']
created = 0
updated = 0
skipped = 0
errors = []
with get_db() as conn:
cur = get_cursor(conn)
for p in prompts:
slug = p.get('slug')
if not slug:
errors.append('Prompt without slug skipped')
continue
# Check if exists
cur.execute("SELECT id FROM ai_prompts WHERE slug=%s", (slug,))
existing = cur.fetchone()
if existing and not overwrite:
skipped += 1
continue
# Prepare stages JSON if present
stages_json = None
if p.get('stages'):
stages_json = json.dumps(p['stages']) if isinstance(p['stages'], list) else p['stages']
if existing:
# Update existing
cur.execute("""
UPDATE ai_prompts SET
name=%s, display_name=%s, description=%s, type=%s,
category=%s, template=%s, stages=%s, output_format=%s,
output_schema=%s, active=%s, sort_order=%s,
updated=CURRENT_TIMESTAMP
WHERE slug=%s
""", (
p.get('name'), p.get('display_name'), p.get('description'),
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
p.get('template'), stages_json, p.get('output_format', 'text'),
p.get('output_schema'), p.get('active', True),
p.get('sort_order', 0), slug
))
updated += 1
else:
# Create new
cur.execute("""
INSERT INTO ai_prompts (
slug, name, display_name, description, type, category,
template, stages, output_format, output_schema,
active, sort_order, created, updated
) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP)
""", (
slug, p.get('name'), p.get('display_name'), p.get('description'),
p.get('type', 'pipeline'), p.get('category', 'ganzheitlich'),
p.get('template'), stages_json, p.get('output_format', 'text'),
p.get('output_schema'), p.get('active', True), p.get('sort_order', 0)
))
created += 1
conn.commit()
return {
'success': True,
'created': created,
'updated': updated,
'skipped': skipped,
'errors': errors if errors else None
}
+15 -299
View File
@@ -8,186 +8,16 @@ Endpoints:
"""
from fastapi import APIRouter, Depends, HTTPException, UploadFile, File
from pydantic import BaseModel
from typing import Literal
from datetime import datetime, timedelta, date
from decimal import Decimal, InvalidOperation
import csv
import io
import json
from typing import Literal
from auth import require_auth
from csv_parser.sleep_apple_import import import_apple_sleep_nights
from db import get_db, get_cursor
router = APIRouter(prefix="/api/sleep", tags=["sleep"])
def _strip_row_keys(row: dict) -> dict:
return {(k or "").strip(): (v.strip() if isinstance(v, str) else v) for k, v in row.items()}
def _safe_float(value) -> float | None:
if value is None or value == "":
return None
if isinstance(value, (int, float)):
return float(value)
if isinstance(value, Decimal):
return float(value)
try:
s = str(value).strip().replace(",", ".")
return float(s)
except (ValueError, InvalidOperation):
return None
def _parse_apple_sleep_datetime(value: str) -> datetime:
raw = (value or "").strip()
if not raw:
raise ValueError("empty datetime")
fmts = (
"%Y-%m-%d %H:%M:%S %z",
"%d.%m.%y %H:%M:%S",
"%d.%m.%Y %H:%M:%S",
"%Y-%m-%d %H:%M:%S",
)
last_err = None
for fmt in fmts:
try:
return datetime.strptime(raw, fmt)
except ValueError as e:
last_err = e
raise ValueError(f"Unbekanntes Datumsformat: {raw!r}") from last_err
def _hr_to_minutes(hours: float | None) -> int:
if hours is None:
return 0
return int(round(float(hours) * 60))
def _detect_apple_sleep_csv_format(fieldnames: list[str] | None) -> Literal["segments", "summary"]:
if not fieldnames:
raise HTTPException(
status_code=400,
detail="CSV enthält keine Spaltenüberschriften.",
)
fn = {(f or "").strip() for f in fieldnames}
if {"Start", "End", "Duration (hr)", "Value"}.issubset(fn):
return "segments"
if "Start" in fn and "End" in fn and "Total Sleep (hr)" in fn:
return "summary"
raise HTTPException(
status_code=400,
detail=(
"Unbekanntes Apple-Health-Schlaf-CSV. "
"Erwartet wird entweder der Segment-Export (Spalten Start, End, Duration (hr), Value) "
"oder die Schlafanalyse mit Nacht-Zusammenfassung (u. a. Total Sleep (hr), Core/Light, Deep, REM)."
),
)
def _build_nights_from_apple_summary(reader: csv.DictReader) -> dict[date, dict]:
nights_dict: dict[date, dict] = {}
for raw in reader:
row = _strip_row_keys(raw)
start_s = row.get("Start") or ""
end_s = row.get("End") or ""
if not start_s or not end_s:
continue
try:
start_dt = _parse_apple_sleep_datetime(start_s)
end_dt = _parse_apple_sleep_datetime(end_s)
except ValueError:
continue
dt_key = (row.get("Date/Time") or row.get("Datum/Uhrzeit") or "").strip()
if dt_key:
try:
wake_d = datetime.strptime(dt_key[:10], "%Y-%m-%d").date()
except ValueError:
wake_d = end_dt.date()
else:
wake_d = end_dt.date()
core_hr = _safe_float(row.get("Core (hr)"))
if core_hr is None:
core_hr = _safe_float(row.get("Light (hr)")) or 0.0
deep_min = _hr_to_minutes(_safe_float(row.get("Deep (hr)")))
rem_min = _hr_to_minutes(_safe_float(row.get("REM (hr)")))
light_min = _hr_to_minutes(core_hr)
awake_min = _hr_to_minutes(_safe_float(row.get("Awake (hr)")))
nights_dict[wake_d] = {
"bedtime": start_dt,
"wake_time": end_dt,
"segments": [],
"deep_minutes": deep_min,
"rem_minutes": rem_min,
"light_minutes": light_min,
"awake_minutes": awake_min,
}
return nights_dict
def _build_nights_from_apple_segments(reader: csv.DictReader, phase_map: dict) -> dict[date, dict]:
segments = []
for raw in reader:
row = _strip_row_keys(raw)
phase_key = (row.get("Value") or "").strip()
phase_en = phase_map.get(phase_key)
if phase_en is None:
continue
try:
start_dt = _parse_apple_sleep_datetime(row.get("Start") or "")
end_dt = _parse_apple_sleep_datetime(row.get("End") or "")
duration_hr = _safe_float(row.get("Duration (hr)"))
if duration_hr is None:
continue
except (ValueError, TypeError):
continue
duration_min = int(duration_hr * 60)
segments.append({
"start": start_dt,
"end": end_dt,
"duration_min": duration_min,
"phase": phase_en,
})
segments.sort(key=lambda s: s["start"])
nights = []
current_night = None
for seg in segments:
if current_night is None or (seg["start"] - current_night["wake_time"]).total_seconds() > 7200:
current_night = {
"bedtime": seg["start"],
"wake_time": seg["end"],
"segments": [],
"deep_minutes": 0,
"rem_minutes": 0,
"light_minutes": 0,
"awake_minutes": 0,
}
nights.append(current_night)
current_night["segments"].append(seg)
current_night["wake_time"] = max(current_night["wake_time"], seg["end"])
current_night["bedtime"] = min(current_night["bedtime"], seg["start"])
if seg["phase"] == "deep":
current_night["deep_minutes"] += seg["duration_min"]
elif seg["phase"] == "rem":
current_night["rem_minutes"] += seg["duration_min"]
elif seg["phase"] == "light":
current_night["light_minutes"] += seg["duration_min"]
elif seg["phase"] == "awake":
current_night["awake_minutes"] += seg["duration_min"]
nights_dict: dict[date, dict] = {}
for night in nights:
wake_date = night["wake_time"].date()
nights_dict[wake_date] = night
return nights_dict
# ── Models ────────────────────────────────────────────────────────────────────
class SleepCreate(BaseModel):
@@ -641,139 +471,25 @@ async def import_apple_health_sleep(
pid = session['profile_id']
content = await file.read()
csv_text = content.decode('utf-8-sig')
reader = csv.DictReader(io.StringIO(csv_text))
fmt = _detect_apple_sleep_csv_format(reader.fieldnames)
phase_map = {
"Kern": "light",
"Core": "light",
"Light": "light",
"REM": "rem",
"Tief": "deep",
"Deep": "deep",
"Wach": "awake",
"Awake": "awake",
"Schlafend": None,
"Asleep": None,
"In Bed": None,
}
if fmt == "summary":
nights_dict = _build_nights_from_apple_summary(reader)
else:
nights_dict = _build_nights_from_apple_segments(reader, phase_map)
if not nights_dict:
raise HTTPException(
status_code=400,
detail="Keine importierbaren Schlafzeilen gefunden (prüfe Start/Ende und Format).",
)
# Insert nights
imported = 0
skipped = 0
try:
csv_text = content.decode("utf-8-sig")
except UnicodeDecodeError:
csv_text = content.decode("latin-1")
with get_db() as conn:
cur = get_cursor(conn)
for date, night in nights_dict.items():
# Calculate sleep duration (deep + rem + light, WITHOUT awake)
# Note: awake_minutes tracked separately, not part of sleep duration
duration_minutes = (
night['deep_minutes'] +
night['rem_minutes'] +
night['light_minutes']
)
# Calculate wake_count (number of awake segments)
wake_count = sum(1 for seg in night['segments'] if seg['phase'] == 'awake')
# Prepare JSONB segments with full datetime
sleep_segments = [
{
'phase': seg['phase'],
'start': seg['start'].isoformat(), # Full datetime: 2026-03-21T22:30:00
'end': seg['end'].isoformat(), # Full datetime: 2026-03-21T23:15:00
'duration_min': seg['duration_min']
}
for seg in night['segments']
]
# Check if manual entry exists - do NOT overwrite
cur.execute("""
SELECT id, source FROM sleep_log
WHERE profile_id = %s AND date = %s
""", (pid, date))
existing = cur.fetchone()
if existing and existing['source'] == 'manual':
skipped += 1
continue # Skip - don't overwrite manual entries
# Upsert (only if not manual)
# If entry exists and is NOT manual → update
# If entry doesn't exist → insert
if existing:
# Update existing non-manual entry
cur.execute("""
UPDATE sleep_log SET
bedtime = %s,
wake_time = %s,
duration_minutes = %s,
wake_count = %s,
deep_minutes = %s,
rem_minutes = %s,
light_minutes = %s,
awake_minutes = %s,
sleep_segments = %s,
source = 'apple_health',
updated_at = CURRENT_TIMESTAMP
WHERE id = %s AND profile_id = %s
""", (
night['bedtime'].time(),
night['wake_time'].time(),
duration_minutes,
wake_count,
night['deep_minutes'],
night['rem_minutes'],
night['light_minutes'],
night['awake_minutes'],
json.dumps(sleep_segments),
existing['id'],
pid
))
else:
# Insert new entry
cur.execute("""
INSERT INTO sleep_log (
profile_id, date, bedtime, wake_time, duration_minutes,
wake_count, deep_minutes, rem_minutes, light_minutes, awake_minutes,
sleep_segments, source, created_at, updated_at
) VALUES (
%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, 'apple_health', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
)
""", (
pid,
date,
night['bedtime'].time(),
night['wake_time'].time(),
duration_minutes,
wake_count,
night['deep_minutes'],
night['rem_minutes'],
night['light_minutes'],
night['awake_minutes'],
json.dumps(sleep_segments)
))
imported += 1
try:
r = import_apple_sleep_nights(cur, pid, csv_text)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
conn.commit()
imported = r["inserted"] + r["updated"]
skipped = r["skipped"]
total = r["rows_total"]
return {
"imported": imported,
"skipped": skipped,
"total_nights": len(nights_dict),
"message": f"{imported} Nächte importiert, {skipped} übersprungen (manuelle Einträge)"
"total_nights": total,
"message": f"{imported} Nächte importiert, {skipped} übersprungen (manuelle Einträge)",
}
+9 -1
View File
@@ -86,11 +86,19 @@ async def execute_workflow_endpoint(
"type": "workflow"
}
# Wrapper function to match workflow_executor's expected signature: (prompt, model) -> str
# workflow_executor calls: openrouter_call_func(prompt, "anthropic/claude-sonnet-4")
# but call_openrouter expects: call_openrouter(prompt, max_tokens=4096)
async def workflow_llm_call(prompt: str, model: str) -> str:
# Ignore model parameter (already set in OPENROUTER_MODEL env var)
# Use default max_tokens=4096 from call_openrouter
return await call_openrouter(prompt)
try:
result = await execute_workflow_prompt(
prompt=workflow_prompt,
variables=variables,
openrouter_call_func=call_openrouter,
openrouter_call_func=workflow_llm_call, # Use wrapper with correct signature
enable_debug=request.enable_debug
)
return result
+4 -4
View File
@@ -160,10 +160,10 @@ CREATE TABLE IF NOT EXISTS activity_log (
end_time TIME,
activity_type VARCHAR(50) NOT NULL,
duration_min NUMERIC(6,2),
kcal_active NUMERIC(7,2),
kcal_resting NUMERIC(7,2),
hr_avg NUMERIC(5,2),
hr_max NUMERIC(5,2),
kcal_active NUMERIC(12,2),
kcal_resting NUMERIC(12,2),
hr_avg NUMERIC(6,2),
hr_max NUMERIC(6,2),
distance_km NUMERIC(7,2),
rpe INTEGER CHECK (rpe >= 1 AND rpe <= 10),
source VARCHAR(20) DEFAULT 'manual',
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""Test export-all endpoint"""
import sys
sys.path.insert(0, '/app')
from datetime import datetime
from db import get_db, get_cursor, r2d
print("Testing export-all logic...")
with get_db() as conn:
cur = get_cursor(conn)
cur.execute("SELECT * FROM ai_prompts ORDER BY sort_order, slug")
prompts = [r2d(row) for row in cur.fetchall()]
print(f"Found {len(prompts)} prompts")
# Convert to export format (clean up DB-specific fields)
export_data = []
for idx, p in enumerate(prompts):
print(f"\nProcessing prompt {idx+1}: {p.get('slug')}")
try:
export_item = {
'slug': p['slug'],
'name': p['name'],
'display_name': p.get('display_name'),
'description': p.get('description'),
'type': p.get('type', 'pipeline'),
'category': p.get('category', 'ganzheitlich'),
'template': p.get('template'),
'stages': p.get('stages'),
'output_format': p.get('output_format', 'text'),
'output_schema': p.get('output_schema'),
'question_augmentations': p.get('question_augmentations'),
'graph_data': p.get('graph_data'),
'active': p.get('active', True),
'sort_order': p.get('sort_order', 0)
}
export_data.append(export_item)
print(f" ✓ OK")
except Exception as e:
print(f" ✗ ERROR: {type(e).__name__}: {e}")
import traceback
traceback.print_exc()
break
print(f"\n\nSuccessfully processed {len(export_data)} prompts")
# Try to create the response dict
try:
result = {
'export_date': datetime.now().isoformat(),
'count': len(export_data),
'prompts': export_data
}
print("✓ Result dict created successfully")
# Try JSON serialization
import json
json_str = json.dumps(result)
print(f"✓ JSON serialization OK, length: {len(json_str)}")
except Exception as e:
print(f"✗ ERROR creating result: {type(e).__name__}: {e}")
import traceback
traceback.print_exc()
+385
View File
@@ -0,0 +1,385 @@
"""
Smoke-Tests für Universal-CSV-Import (Executor + Apple-Schlaf-Parser).
Nutzt einen minimalen Fake-Cursor (kein PostgreSQL), damit die Pipelines bei jedem
pytest-Lauf mitlaufen.
"""
from __future__ import annotations
import datetime as dt
import uuid
import pytest
from csv_parser.executor import diagnose_vitals_row, run_universal_csv_import
from csv_parser.sleep_apple_import import detect_apple_sleep_csv_format
from csv_parser.type_converter import build_row_after_mapping, diagnose_row_mapping
class _SeqCursor:
"""Minimaler Cursor: execute protokolliert; fetchone liefert vorgegebene Sequenz."""
def __init__(self, fetch_sequence: list) -> None:
self.executes: list[tuple[str, tuple | None]] = []
self._fetch = list(fetch_sequence)
def execute(self, sql: str, params=None) -> None:
self.executes.append((sql, params))
def fetchone(self):
if self._fetch:
return self._fetch.pop(0)
return None
PID = "00000000-0000-0000-0000-000000000001"
def test_detect_apple_sleep_summary_vs_segment():
assert detect_apple_sleep_csv_format(["Start", "End", "Total Sleep (hr)", "Core (hr)"]) == "summary"
assert detect_apple_sleep_csv_format(["Start", "End", "Duration (hr)", "Value"]) == "segments"
def test_run_universal_import_sleep_one_night_inserts(monkeypatch):
"""Eine Summary-Zeile → INSERT; SELECT vorher ohne Treffer."""
text = (
"Start,End,Total Sleep (hr),Core (hr),Deep (hr),REM (hr),Awake (hr)\n"
"2024-01-15 22:00:00,2024-01-16 06:00:00,8.0,5.0,1.0,1.5,0.5\n"
)
cur = _SeqCursor(
[
None,
{"id": 101},
]
)
out = run_universal_csv_import(cur, PID, "sleep", text, "sleep.csv", {})
assert out["rows_total"] >= 1
assert out["rows_imported"] >= 1
assert any("INSERT INTO sleep_log" in q[0] for q in cur.executes)
def test_run_universal_import_activity_insert(monkeypatch):
monkeypatch.setattr(
"csv_parser.executor._resolve_training_type_for_activity",
lambda *_a, **_k: (None, None, None),
)
text = (
"Workout Type,Start,End,Duration,Distance (km),Active Energy (kcal)\n"
"Running,2024-01-15 08:00:00,2024-01-15 09:00:00,1:00:00,10.0,500\n"
)
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {
"Workout Type": "activity_type",
"Start": "start_time",
"End": "end_time",
"Duration": "duration_min",
"Distance (km)": "distance_km",
"Active Energy (kcal)": "kcal_active",
},
"type_conversions": {
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
"end_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
"duration_min": {
"type": "duration",
"format": "HH:MM:SS",
"target_unit": "minutes",
"flexible": True,
},
"distance_km": {"type": "float", "decimal_separator": ".", "flexible": True},
"kcal_active": {"type": "float", "decimal_separator": ".", "flexible": True},
},
}
new_id = str(uuid.uuid4())
cur = _SeqCursor([None, {"id": new_id}])
out = run_universal_csv_import(cur, PID, "activity", text, "act.csv", mapping)
assert out["rows_imported"] == 1
assert out["new_entries"] == 1
assert any("INSERT INTO activity_log" in q[0] for q in cur.executes)
def test_run_universal_import_vitals_baseline_upsert_insert_path():
text = (
"Start,Resting Heart Rate,Heart Rate Variability,VO2 Max\n"
"2024-01-15 07:00:00,55,45,42.5\n"
)
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {
"Start": "date",
"Resting Heart Rate": "resting_hr",
"Heart Rate Variability": "hrv",
"VO2 Max": "vo2_max",
},
"type_conversions": {
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": True,
},
"resting_hr": {"type": "int", "flexible": True},
"hrv": {"type": "int", "flexible": True},
"vo2_max": {"type": "float", "decimal_separator": ".", "flexible": True},
},
}
cur = _SeqCursor([{"inserted": True, "id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"}])
out = run_universal_csv_import(cur, PID, "vitals_baseline", text, "v.csv", mapping)
assert out["rows_imported"] == 1
assert any("INSERT INTO vitals_baseline" in q[0] for q in cur.executes)
def test_run_universal_import_wide_german_vitals_with_english_template_slots():
"""Breiter Apple-DE-Export, aber nur englische Vorlagen-Spalten → über Aliase erkennbar."""
text = (
"Datum/Uhrzeit,Aktive Energie (kJ),Ruhepuls (count/min),Atemfrequenz (count/min)"
",Blutsauerstoffsättigung (%),Herzfrequenzvariabilität (ms),VO2 max (ml/(kg·min))\n"
"2026-04-03 00:00:00,,53,15.61,95.22,37.26,\n"
)
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {
"Start": "date",
"Resting Heart Rate": "resting_hr",
"Heart Rate Variability": "hrv",
"VO2 Max": "vo2_max",
"Oxygen Saturation": "spo2",
"Respiratory Rate": "respiratory_rate",
},
"type_conversions": {
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": True,
},
"resting_hr": {"type": "int", "flexible": True},
"hrv": {"type": "int", "flexible": True},
"vo2_max": {"type": "float", "decimal_separator": "auto", "flexible": True},
"spo2": {"type": "int", "flexible": True},
"respiratory_rate": {"type": "float", "decimal_separator": "auto", "flexible": True},
},
}
cur = _SeqCursor([{"inserted": True, "id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"}])
out = run_universal_csv_import(cur, PID, "vitals_baseline", text, "wide.csv", mapping)
assert out["rows_errors"] == 0
assert out["rows_imported"] == 1
def test_run_universal_import_vitals_baseline_two_rows_same_day_averages():
text = (
"Start,Resting Heart Rate,Heart Rate Variability,VO2 Max\n"
"2024-01-15 07:00:00,50,40,42.0\n"
"2024-01-15 18:00:00,60,50,43.0\n"
)
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {
"Start": "date",
"Resting Heart Rate": "resting_hr",
"Heart Rate Variability": "hrv",
"VO2 Max": "vo2_max",
},
"type_conversions": {
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": True,
},
"resting_hr": {"type": "int", "flexible": True},
"hrv": {"type": "int", "flexible": True},
"vo2_max": {"type": "float", "decimal_separator": ".", "flexible": True},
},
}
cur = _SeqCursor([{"inserted": True, "id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"}])
out = run_universal_csv_import(cur, PID, "vitals_baseline", text, "v2.csv", mapping)
assert out["rows_total"] == 2
assert out["rows_imported"] == 1
upsert = [q for q in cur.executes if "INSERT INTO vitals_baseline" in q[0]]
assert len(upsert) == 1
params = upsert[0][1]
assert params[2] == 55
assert params[3] == 45
assert params[4] == 42.5
def test_run_universal_import_activity_garmin_time_plus_date_columns(monkeypatch):
"""Datum in eigener Spalte, Uhrzeit wie bei Garmin nur als Uhrzeit."""
monkeypatch.setattr(
"csv_parser.executor._resolve_training_type_for_activity",
lambda *_a, **_k: (None, None, None),
)
text = (
"Activity Type,Date,Time,Duration,Distance,Calories,Avg HR\n"
"Run,2024-01-20,08:30:00,0:45:00,8.0,400,140\n"
)
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {
"Activity Type": "activity_type",
"Date": "date",
"Time": "start_time",
"Duration": "duration_min",
"Distance": "distance_km",
"Calories": "kcal_active",
"Avg HR": "hr_avg",
},
"type_conversions": {
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
"start_time": {"type": "time", "format": "HH:MM:SS", "flexible": True},
"duration_min": {
"type": "duration",
"format": "HH:MM:SS",
"target_unit": "minutes",
"flexible": True,
},
"distance_km": {"type": "float", "decimal_separator": ".", "flexible": True},
"kcal_active": {"type": "float", "decimal_separator": ".", "flexible": True},
"hr_avg": {"type": "int", "flexible": True},
},
}
new_id = str(uuid.uuid4())
cur = _SeqCursor([None, {"id": new_id}])
out = run_universal_csv_import(cur, PID, "activity", text, "garmin.csv", mapping)
assert out["rows_imported"] == 1
# Duplicate-Key muss Datum + kombinierte Startzeit enthalten
assert any(
params and "2024-01-20 08:30:00" in str(params)
for _sql, params in cur.executes
if params
)
def test_run_universal_import_nutrition_two_rows_same_day_aggregates_to_one_row():
"""Modul-Default: mehrere CSV-Zeilen pro Tag → Summe, ein nutrition_log-Eintrag."""
text = (
"Date,Kalorien,Protein,Fett,KH\n"
"2024-01-15,500,10,20,30\n"
"2024-01-15,300,5,10,15\n"
)
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {
"Date": "date",
"Kalorien": "kcal",
"Protein": "protein_g",
"Fett": "fat_g",
"KH": "carbs_g",
},
"type_conversions": {
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
"kcal": {"type": "float", "decimal_separator": ".", "flexible": True},
"protein_g": {"type": "float", "decimal_separator": ".", "flexible": True},
"fat_g": {"type": "float", "decimal_separator": ".", "flexible": True},
"carbs_g": {"type": "float", "decimal_separator": ".", "flexible": True},
},
}
cur = _SeqCursor([None])
out = run_universal_csv_import(cur, PID, "nutrition", text, "n.csv", mapping)
assert out["rows_total"] == 2
assert out["rows_imported"] == 1
insert_sqls = [q for q in cur.executes if "INSERT INTO nutrition_log" in q[0]]
assert len(insert_sqls) == 1
params = insert_sqls[0][1]
# (eid, profile_id, iso, kcal, prot, fat, carbs)
assert params[3] == 800.0
assert params[4] == 15.0
assert params[5] == 30.0
assert params[6] == 45.0
def test_run_universal_import_weight_two_rows_same_day_last_value():
"""Mehrere Gewichtszeilen pro Tag → Standard: letzter Wert in der Datei."""
text = "Date,Weight\n2024-01-15,85.0\n2024-01-15,83.5\n"
mapping = {
"delimiter": ",",
"has_header": True,
"field_mappings": {"Date": "date", "Weight": "weight"},
"type_conversions": {
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
"weight": {"type": "float", "decimal_separator": ".", "flexible": True},
},
}
cur = _SeqCursor([None])
out = run_universal_csv_import(cur, PID, "weight", text, "w.csv", mapping)
assert out["rows_total"] == 2
assert out["rows_imported"] == 1
insert_sqls = [q for q in cur.executes if "INSERT INTO weight_log" in q[0]]
assert len(insert_sqls) == 1
params = insert_sqls[0][1]
assert params[3] == 83.5
def test_activity_alias_maps_german_workout_wide_columns():
from csv_parser.type_converter import build_row_after_mapping
row = {
"Datum/Uhrzeit": "2026-04-03 08:00:00",
"Trainingsart": "Laufen",
"Dauer": "0:45:00",
}
fm = {"Workout Type": "activity_type"}
tc = {
"activity_type": {"type": "string"},
"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True},
"duration_min": {
"type": "duration",
"format": "HH:MM:SS",
"target_unit": "minutes",
"flexible": True,
},
}
out = build_row_after_mapping(row, fm, tc, module="activity")
assert str(out.get("activity_type")) == "Laufen"
assert isinstance(out.get("start_time"), dt.datetime)
assert out.get("duration_min") == 45.0
def test_blood_pressure_alias_combined_datetime_column():
from csv_parser.type_converter import build_row_after_mapping
row = {
"Datum/Uhrzeit": "2026-04-03 10:30:00",
"Systolisch (mmHg)": "120",
"Diastolisch (mmHg)": "80",
}
fm = {"Date": "measured_date"}
tc = {
"systolic": {"type": "int", "flexible": True},
"diastolic": {"type": "int", "flexible": True},
}
out = build_row_after_mapping(row, fm, tc, module="blood_pressure")
assert isinstance(out.get("start_time"), dt.datetime)
assert int(out.get("systolic")) == 120
def test_diagnose_vitals_row_and_mapping_smoke():
fm = {
"Datum/Uhrzeit": "date",
"Ruhepuls (count/min)": "resting_hr",
}
tc = {
"date": {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_only",
"flexible": True,
},
"resting_hr": {"type": "int", "flexible": True},
}
row = {"Datum/Uhrzeit": "2026-04-03 00:00:00", "Ruhepuls (count/min)": "53"}
typed = build_row_after_mapping(row, fm, tc, module="vitals_baseline")
d = diagnose_vitals_row(typed)
assert d["date_coerced_iso"] == "2026-04-03"
assert d["would_pass_prefilter"] is True
dm = diagnose_row_mapping(row, fm, tc, module="vitals_baseline", mapped_typed=typed)
assert str(dm["mapped"]["date"]).startswith("2026-04-03")
assert any(c["csv_column"] == "Datum/Uhrzeit" and c["source"] == "template" for c in dm["per_column"])
+303
View File
@@ -0,0 +1,303 @@
"""Tests für CSV-Parser Foundation (Issue #21)."""
import pytest
from csv_parser.core import (
decode_raw_bytes,
sniff_delimiter,
parse_csv_sample,
column_signature,
headers_signature_match_score,
headers_signature_rank_metrics,
get_csv_import_limits,
iter_csv_dict_rows,
)
from csv_parser.field_units import source_unit_choices_for_field
from csv_parser.mapping_suggest import build_type_conversions_for_mapping
from csv_parser.type_converter import convert_value, build_row_after_mapping
def test_decode_bom_utf8():
raw = "\ufeffa;b;c\n1;2;3".encode("utf-8-sig")
t = decode_raw_bytes(raw)
assert not t.startswith("\ufeff")
assert "a;b;c" in t
def test_sniff_delimiter():
assert sniff_delimiter("a;b;c;d") == ";"
assert sniff_delimiter("a,b,c") == ","
def test_parse_csv_sample_header():
text = "Date;kcal\n2024-01-01;2000\n"
headers, rows, delim = parse_csv_sample(text, delimiter=";", max_data_rows=3)
assert headers == ["Date", "kcal"]
assert delim == ";"
assert rows[0]["Date"] == "2024-01-01"
assert rows[0]["kcal"] == "2000"
def test_column_signature_sorted_unique():
sig = column_signature(["B", "a", "a"])
assert sig == ["a", "b"]
def test_jaccard():
s1 = column_signature(["Date", "Calories"])
s2 = column_signature(["Date", "Calories", "Fat"])
assert headers_signature_match_score(s1, s2) == pytest.approx(2 / 3)
def test_template_recall_full_when_csv_has_extra_columns():
"""Alle Template-Spalten in der CSV → Recall 1.0; Jaccard niedriger bei vielen Zusatzspalten."""
csv_sig = column_signature(
["D", "E", "F", "Extra1", "Extra2", "Extra3", "Extra4", "Extra5"]
)
tmpl_sig = column_signature(["d", "e", "f"])
m = headers_signature_rank_metrics(csv_sig, tmpl_sig)
assert m["confidence"] == 1.0
assert m["template_recall"] == 1.0
assert m["columns_matched"] == 3
assert m["columns_in_template"] == 3
assert m["jaccard"] == pytest.approx(3 / 8)
def test_get_csv_import_limits_default():
assert get_csv_import_limits(None)["max_rows_per_file"] == 50_000
def test_convert_date_and_kcal_factor():
d = convert_value("15.01.2024", "date", {"type": "date", "format": "dd.mm.yyyy"})
assert d.year == 2024 and d.month == 1 and d.day == 15
k = convert_value("8000", "kcal", {"type": "float", "conversion_factor": 0.239, "decimal_separator": "."})
assert abs(k - 8000 * 0.239) < 0.01
def test_convert_kcal_via_source_unit_kj():
spec = {"type": "float", "source_unit": "kj", "decimal_separator": "."}
k = convert_value("4184", "kcal", spec, module="nutrition")
assert abs(k - 1000.0) < 0.05
def test_apple_health_long_decimal_dot_preserved():
"""Apple: Mittel-HF u. Energie mit vielen Nachkommastellen; Punkt ist Dezimaltrenner."""
hr_spec = {"type": "int", "flexible": True}
r = convert_value("96.8749374730629", "hr_avg", hr_spec, module="activity")
assert r == 97
rest_spec = {"type": "float", "decimal_separator": ".", "flexible": True, "source_unit": "kj"}
kcal = convert_value("596.6689047323086", "kcal_resting", rest_spec, module="activity")
assert 140.0 < kcal < 145.0
def test_parse_float_auto_us_thousands_comma():
"""«12,345» ohne Dezimalpunkt weiter als Tausender möglich."""
v = convert_value("12345", "x", {"type": "float", "decimal_separator": "auto"})
assert v == 12345.0
v2 = convert_value("12,345", "x", {"type": "float", "decimal_separator": "auto"})
assert abs(v2 - 12345.0) < 0.01
def test_convert_protein_kg_to_g():
spec = {"type": "float", "source_unit": "kg", "decimal_separator": "."}
g = convert_value("0.1", "protein_g", spec, module="nutrition")
assert abs(g - 100.0) < 0.001
def test_convert_custom_source_unit_only_conversion_factor():
"""Nicht vordefinierte Umrechnung: conversion_factor (optional mit source_unit: custom)."""
spec = {"type": "float", "source_unit": "custom", "conversion_factor": 2.5, "decimal_separator": "."}
k = convert_value("100", "kcal", spec, module="nutrition")
assert abs(k - 250.0) < 0.001
def test_convert_unknown_source_unit_uses_conversion_factor_only():
spec = {"type": "float", "source_unit": "exotic_unit", "conversion_factor": 0.5, "decimal_separator": "."}
k = convert_value("200", "kcal", spec, module="nutrition")
assert abs(k - 100.0) < 0.001
def test_build_row_source_unit_without_module_no_factor():
"""Ohne module bleibt source_unit wirkungslos (Abwärtskompatibilität)."""
spec = {"type": "float", "source_unit": "kj", "decimal_separator": "."}
k = convert_value("4184", "kcal", spec, module=None)
assert abs(k - 4184.0) < 0.01
def test_iter_csv_dict_rows_full_file():
text = "a;b\n1;2\n3;4\n"
rows = list(iter_csv_dict_rows(text, ";", has_header=True))
assert rows == [{"a": "1", "b": "2"}, {"a": "3", "b": "4"}]
def test_build_row_after_mapping():
csv_row = {"Datum": "01.01.2024", "kj": "4200"}
fm = {"Datum": "date", "kj": "kcal"}
tc = {
"date": {"type": "date", "format": "dd.mm.yyyy"},
"kcal": {"type": "float", "conversion_factor": 0.239, "decimal_separator": "."},
}
out = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
assert out["date"].month == 1
assert out["kcal"] is not None
assert abs(float(out["kcal"]) - 4200 * 0.239) < 0.02
def test_build_type_conversions_kj_header_sets_source_unit():
fm = {"kJ": "kcal", "Datum": "date"}
tc = build_type_conversions_for_mapping("nutrition", fm, None)
assert tc["kcal"].get("source_unit") == "kj"
def test_build_row_fddb_raw_header_keys_match_normalized_template():
"""FDDB: DictReader liefert deutsche Überschrift, Seed nutzt normalisierten Key."""
csv_row = {
"Datum Tag Monat Jahr Stunde Minute": "01.01.2024 8:30",
"kJ": "42000",
"Fett (g)": "50",
"KH (g)": "200",
"Protein (g)": "100",
}
fm = {
"datum_tag_monat_jahr_stunde_minute": "date",
"kj": "kcal",
"fett_g": "fat_g",
"kh_g": "carbs_g",
"protein_g": "protein_g",
}
tc = {
"date": {"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only"},
"kcal": {
"type": "float",
"source_unit": "kj",
"decimal_separator": ",",
},
"fat_g": {"type": "float", "decimal_separator": ","},
"carbs_g": {"type": "float", "decimal_separator": ","},
"protein_g": {"type": "float", "decimal_separator": ","},
}
out = build_row_after_mapping(csv_row, fm, tc, module="nutrition")
assert out["date"].year == 2024 and out["date"].month == 1 and out["date"].day == 1
assert out["kcal"] is not None and abs(float(out["kcal"]) - (42000 / 4.184)) < 0.1
def test_convert_date_ddmm_with_seconds():
d = convert_value(
"15.01.2024 14:30:00",
"date",
{"type": "date", "format": "dd.mm.yyyy HH:MM", "extract": "date_only"},
)
assert d.month == 1 and d.day == 15
def test_float_decimal_separator_auto_eu_us():
assert abs(convert_value("1.234,56", "x", {"type": "float", "decimal_separator": "auto"}) - 1234.56) < 1e-9
assert abs(convert_value("1,234.56", "x", {"type": "float", "decimal_separator": "auto"}) - 1234.56) < 1e-9
assert abs(convert_value("1234,5", "x", {"type": "float", "decimal_separator": "auto"}) - 1234.5) < 1e-9
def test_float_flexible_falls_back_to_auto():
spec = {"type": "float", "decimal_separator": ",", "flexible": True}
assert abs(convert_value("1234.56", "x", spec) - 1234.56) < 1e-9
def test_date_flexible_iso_while_primary_ddmm():
spec = {
"type": "date",
"format": "dd.mm.yyyy",
"flexible": True,
"extract": "date_only",
}
d1 = convert_value("2024-03-15", "d", spec)
d2 = convert_value("15.03.2024", "d", spec)
assert d1 == d2
def test_date_extra_formats_without_days_in_name():
spec = {
"type": "date",
"format": "yyyy-mm-dd",
"formats": ["%d.%m.%Y"],
"extract": "date_only",
}
assert convert_value("08.04.2026", "d", spec).day == 8
def test_int_flexible_thousands():
assert convert_value("1.234", "n", {"type": "int", "flexible": True}) == 1234
def test_build_row_after_mapping_column_order_independent():
fm = {"Spalte B": "resting_hr", "Spalte A": "date"}
tc = {
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
"resting_hr": {"type": "int", "flexible": True},
}
r1 = build_row_after_mapping(
{"Spalte A": "2026-01-15", "Spalte B": "58"}, fm, tc, module="vitals_baseline"
)
r2 = build_row_after_mapping(
{"Spalte B": "58", "Spalte A": "2026-01-15"}, fm, tc, module="vitals_baseline"
)
assert r1 == r2
assert r1["resting_hr"] == 58
def test_omron_report_date_formats_without_flexible_flag():
"""Omron «Bericht»-Export: engl. Month abbrev + deutscher Monatsname; Vorlage oft nur dd.mm.yyyy."""
spec = {"type": "date", "format": "dd.mm.yyyy"}
assert convert_value("10 Apr. 2026", "measured_date", spec).isoformat() == "2026-04-10"
assert convert_value("31 März 2026", "measured_date", spec).isoformat() == "2026-03-31"
assert convert_value("11 März 2026", "measured_date", spec).isoformat() == "2026-03-11"
def test_int_flexible_german_decimal_rounds():
"""Apple-DE: HRV/SpO2 als «37,26» / «95,22» — nicht 3726 aus Ziffern konkatenieren."""
spec = {"type": "int", "flexible": True}
assert convert_value("37,26", "hrv", spec) == 37
assert convert_value("95,22", "spo2", spec) == 95
def test_datetime_flexible():
spec = {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True}
dtv = convert_value("15.01.2024 14:30:00", "t", spec)
assert dtv.month == 1 and dtv.day == 15 and dtv.hour == 14
def test_apple_workout_datetime_without_seconds_iso_not_swapped():
"""Apple Export: 2026-04-09 16:48 — ohne :SS; kein dayfirst-Fehlparser (09↔04)."""
spec = {
"type": "datetime",
"format": "yyyy-mm-dd HH:MM:SS",
"extract": "date_and_time",
}
dtv = convert_value("2026-04-09 16:48", "start_time", spec, module="activity")
assert dtv.year == 2026 and dtv.month == 4 and dtv.day == 9
assert dtv.hour == 16 and dtv.minute == 48
def test_iso_yyyy_mm_dd_dateutil_fallback_not_dayfirst_swapped():
"""Nur dateutil: ISO-Datum darf mit Default-dayfirst nicht vertauscht werden."""
spec = {"type": "date", "format": "dd.mm.yyyy", "flexible": True}
d = convert_value("2026-04-09", "d", spec)
assert d.month == 4 and d.day == 9
def test_activity_kcal_active_kj_source_unit_to_kcal():
"""Apple-DE: Aktive Energie als kJ; DB-Feld ist kcal."""
spec = {
"type": "float",
"decimal_separator": ".",
"flexible": True,
"source_unit": "kj",
}
v = convert_value("4184", "kcal_active", spec, module="activity")
assert abs(v - 1000.0) < 0.02
def test_source_unit_choices_include_custom_at_end():
opts = source_unit_choices_for_field("nutrition", "protein_g")
assert opts[-1]["id"] == "custom"
assert any(o["id"] == "mg" for o in opts)
+10
View File
@@ -0,0 +1,10 @@
from csv_parser.import_errors import enrich_row_error
def test_enrich_numeric_overflow():
d = enrich_row_error(
"numeric field overflow\nDETAIL: A field with precision 5, scale 2\n",
module="activity",
)
assert d["code"] == "db_numeric_overflow"
assert d["hint"] and "kj" in d["hint"].lower()
+155
View File
@@ -0,0 +1,155 @@
"""Tests für CSV-Zeilenaggregation (import_row_processing)."""
from __future__ import annotations
import datetime as dt
import pytest
from csv_parser.import_row_processing import (
aggregate_mapped_rows,
resolve_import_row_processing,
validate_import_row_processing,
)
def test_validate_rejects_unknown_aggregate():
with pytest.raises(ValueError, match="ungültige Operation"):
validate_import_row_processing(
"nutrition",
{"group_by": ["date"], "aggregates": {"kcal": "bogus"}},
{"Kal": "date", "E": "kcal"},
)
def test_validate_group_by_must_be_mapped():
with pytest.raises(ValueError, match="keiner CSV-Spalte zugeordnet"):
validate_import_row_processing(
"nutrition",
{"group_by": ["date"], "aggregates": {"kcal": "sum"}},
{"Kal": "kcal"}, # date nicht gemappt
)
def test_aggregate_mapped_rows_sums_same_group():
d = dt.date(2024, 1, 15)
rows = [
{"date": d, "kcal": 500.0, "protein_g": 20},
{"date": d, "kcal": 300.0, "protein_g": 15},
]
spec = {"group_by": ["date"], "aggregates": {"kcal": "sum", "protein_g": "sum"}}
out, err = aggregate_mapped_rows(rows, spec)
assert err == []
assert len(out) == 1
assert out[0]["kcal"] == 800.0
assert out[0]["protein_g"] == 35
def test_aggregate_mapped_rows_reject_second_group():
d = dt.date(2024, 1, 15)
rows = [
{"date": d, "kcal": 100.0},
{"date": d, "kcal": 200.0},
]
spec = {
"group_by": ["date"],
"aggregates": {"kcal": "sum"},
"multi_row_policy": "reject",
}
out, err = aggregate_mapped_rows(rows, spec)
assert out == []
assert len(err) == 1
assert err[0].get("error") == "mehrere_zeilen_pro_schluessel"
assert err[0].get("rows_in_group") == 2
def test_aggregate_mapped_rows_first_row_no_merge():
d = dt.date(2024, 1, 15)
rows = [
{"date": d, "kcal": 100.0},
{"date": d, "kcal": 999.0},
]
spec = {
"group_by": ["date"],
"aggregates": {"kcal": "sum"},
"multi_row_policy": "first_row",
}
out, err = aggregate_mapped_rows(rows, spec)
assert err == []
assert len(out) == 1
assert out[0]["kcal"] == 100.0
def test_dedupe_identical_rows_before_group():
d = dt.date(2024, 1, 15)
rows = [
{"date": d, "kcal": 50.0},
{"date": d, "kcal": 50.0},
{"date": d, "kcal": 50.0},
]
spec = {
"group_by": ["date"],
"aggregates": {"kcal": "sum"},
"dedupe_identical_rows": True,
}
out, err = aggregate_mapped_rows(rows, spec)
assert err == []
assert len(out) == 1
assert out[0]["kcal"] == 50.0
def test_validate_multi_row_policy():
with pytest.raises(ValueError, match="multi_row_policy"):
validate_import_row_processing(
"nutrition",
{
"group_by": ["date"],
"aggregates": {"kcal": "sum"},
"multi_row_policy": "nope",
},
{"D": "date", "K": "kcal"},
)
def test_resolve_explicit_overrides_default():
m = {
"import_row_processing": {"group_by": ["date"], "aggregates": {"kcal": "mean"}},
}
spec = resolve_import_row_processing("nutrition", m)
assert spec is not None
assert spec["aggregates"]["kcal"] == "mean"
def test_resolve_empty_dict_falls_back_to_module_default():
m: dict = {"import_row_processing": {}}
spec = resolve_import_row_processing("nutrition", m)
assert spec is not None
assert spec["group_by"] == ["date"]
assert spec["aggregates"]["kcal"] == "sum"
def test_resolve_none_uses_nutrition_default():
spec = resolve_import_row_processing("nutrition", {})
assert spec is not None
assert "date" in (spec.get("group_by") or [])
def test_resolve_weight_default_uses_last():
spec = resolve_import_row_processing("weight", {})
assert spec is not None
assert spec.get("aggregates", {}).get("weight") == "last"
assert spec.get("aggregates", {}).get("note") == "last"
def test_validate_weight_import_row_processing_ok():
validate_import_row_processing(
"weight",
{"group_by": ["date"], "aggregates": {"weight": "mean"}},
{"D": "date", "W": "weight"},
)
def test_resolve_vitals_baseline_default_uses_mean():
spec = resolve_import_row_processing("vitals_baseline", {})
assert spec is not None
assert spec.get("aggregates", {}).get("resting_hr") == "mean"
+42
View File
@@ -0,0 +1,42 @@
"""Tests für CSV mapping_suggest (Issue #21)."""
from csv_parser.mapping_suggest import (
build_type_conversions_for_mapping,
suggest_field_mappings,
)
def test_suggest_from_fddb_seed():
headers = ["Datum Tag Monat Jahr Stunde Minute", "kJ", "Fett (g)", "KH (g)", "Protein (g)"]
seed = {
"datum_tag_monat_jahr_stunde_minute": "date",
"kj": "kcal",
"fett_g": "fat_g",
"kh_g": "carbs_g",
"protein_g": "protein_g",
}
fm = suggest_field_mappings(headers, "nutrition", seed)
assert fm["Datum Tag Monat Jahr Stunde Minute"] == "date"
assert fm["kJ"] == "kcal"
def test_suggest_aliases_without_seed():
fm = suggest_field_mappings(
["Date", "Calories", "Protein (g)", "Fat (g)", "Carbs"],
"nutrition",
None,
)
assert fm["Date"] == "date"
assert fm["Calories"] == "kcal"
assert fm["Protein (g)"] == "protein_g"
def test_type_conversions_merge():
fm = {"Col A": "date", "Col B": "kcal"}
seed_tc = {
"kcal": {"type": "float", "conversion_factor": 0.239, "decimal_separator": "auto"},
}
tc = build_type_conversions_for_mapping("nutrition", fm, seed_tc)
assert "kcal" in tc
assert tc["kcal"].get("conversion_factor") == 0.239
assert "date" in tc
@@ -44,6 +44,15 @@ def test_value_raw_json():
assert not success
assert val is None
# Resolver-Fehlerhülle (kein verwertbares JSON für Charts)
val, success = extract_value_raw(
'{"_available": false, "_reason": "test"}',
OutputType.JSON,
PlaceholderType.RAW_DATA,
)
assert not success
assert val is None
def test_value_raw_number():
"""Numeric outputs must extract numbers without units."""
@@ -66,6 +75,13 @@ def test_value_raw_number():
val, success = extract_value_raw('nicht verfügbar', OutputType.NUMBER, PlaceholderType.ATOMIC)
assert not success
val, success = extract_value_raw(
'nicht verfügbar — Keine Messungen (detail)',
OutputType.NUMBER,
PlaceholderType.ATOMIC,
)
assert not success
def test_value_raw_markdown():
"""Markdown outputs keep as string."""
@@ -0,0 +1,104 @@
"""Tests für {{key|d}}, {{key|x}}, ai_caption und Unbekannt-Erkennung."""
from placeholder_registry import (
PlaceholderMetadata,
PlaceholderType,
OutputType,
build_ai_placeholder_caption,
)
import placeholder_resolver as pr
from placeholder_resolver import format_value_with_d_modifier
from prompt_executor import resolve_placeholders
def test_build_ai_caption_is_explanation_only():
m = PlaceholderMetadata(
key="test_x",
category="Test",
description="Kurzbeschreibung",
resolver_module="m",
resolver_function="f",
semantic_contract="Lang Vertrag " * 50,
business_meaning="Kernbedeutung für die KI.",
unit="g/day",
placeholder_type=PlaceholderType.INTERPRETED,
output_type=OutputType.NUMERIC,
)
cap = build_ai_placeholder_caption(m)
assert "Kernbedeutung" in cap
assert "Kurzbeschreibung" not in cap
def test_build_ai_caption_protein_avg_no_description_prefix():
m = PlaceholderMetadata(
key="protein_avg",
category="Ernährung",
description="Durchschn. Protein in g (30d)",
resolver_module="m",
resolver_function="f",
business_meaning="Zentraler Placeholder für Muskelerhalt.",
unit="g/day",
placeholder_type=PlaceholderType.INTERPRETED,
output_type=OutputType.NUMERIC,
)
cap = build_ai_placeholder_caption(m)
assert cap.startswith("Zentraler Placeholder")
assert "Durchschn. Protein" not in cap
assert "Technischer Bezug" not in cap
def test_build_ai_caption_score_adds_scale():
m = PlaceholderMetadata(
key="test_score",
category="Test",
description="Score",
resolver_module="m",
resolver_function="f",
business_meaning="Gewichteter Gesamtscore.",
unit="Score (0-100)",
placeholder_type=PlaceholderType.SCORE,
output_type=OutputType.NUMERIC,
)
cap = build_ai_placeholder_caption(m)
assert "0–100" in cap or "0-100" in cap
assert "Gewichteter" in cap
def test_placeholder_token_regex_optional_modifier():
m0 = pr._PLACEHOLDER_TOKEN_RE.search("{{fat_avg}}")
assert m0 and m0.group(1) == "fat_avg" and m0.group(2) is None
m1 = pr._PLACEHOLDER_TOKEN_RE.search("{{fat_avg|d}}")
assert m1 and m1.group(1) == "fat_avg" and m1.group(2).strip() == "d"
m2 = pr._PLACEHOLDER_TOKEN_RE.search("{{ protein_avg | d }}")
assert m2 and m2.group(1) == "protein_avg" and m2.group(2).strip() == "d"
m3 = pr._PLACEHOLDER_TOKEN_RE.search("{{k|d,x}}")
assert m3 and m3.group(1) == "k" and m3.group(2).strip() == "d,x"
def test_get_unknown_placeholders_strips_modifier():
unk = pr.get_unknown_placeholders("{{not_a_real_key|d}}")
assert set(unk) == {"not_a_real_key"}
def test_format_value_with_d_modifier_uses_description_only():
row = {
"key": "protein_avg",
"description": "Durchschn. Protein in g (30d)",
"ai_caption": "Nur für |x",
}
out = format_value_with_d_modifier("119g/Tag", row)
assert out == "119g/Tag — Durchschn. Protein in g (30d)"
assert "Nur für |x" not in out
def test_format_value_with_d_modifier_falls_back_to_description():
row = {"description": "Nur Beschreibung", "key": "x"}
assert format_value_with_d_modifier("42", row) == "42 — Nur Beschreibung"
def test_prompt_executor_modifiers_d_x_combined():
catalog = {"E": [{"key": "p", "description": "Desc", "ai_caption": "Expl"}]}
v = {"p": "99"}
assert resolve_placeholders("{{p|d}}", v, None, catalog) == "99 — Desc"
assert resolve_placeholders("{{p|x}}", v, None, catalog) == "Expl"
assert resolve_placeholders("{{p|d,x}}", v, None, catalog) == "99 — Desc — Expl"
assert resolve_placeholders("{{p}}", v, None, catalog) == "99"
+70
View File
@@ -0,0 +1,70 @@
"""Formatprüfung CSV-Vorlagen (template_validator)."""
from __future__ import annotations
from csv_parser.template_validator import validate_csv_template
def test_validate_kj_column_warns_without_source_unit():
r = validate_csv_template(
"activity",
{"Aktive Energie (kJ)": "kcal_active", "Start": "start_time", "Trainingsart": "activity_type"},
{"start_time": {"type": "datetime", "format": "yyyy-mm-dd HH:MM:SS", "flexible": True}},
None,
None,
)
assert r["valid"] is True
codes = {w["code"] for w in r["warnings"]}
assert "energy_kj_without_source_unit" in codes
def test_validate_invalid_target_error():
r = validate_csv_template(
"activity",
{"X": "not_a_field"},
{},
None,
None,
)
assert r["valid"] is False
assert any(e["code"] == "invalid_field_mapping" for e in r["errors"])
def test_signature_normalized_vs_raw_mapping_no_false_warning():
"""Analyse liefert normalisierte Signatur; field_mappings nutzt Original-Header wie im Seed."""
r = validate_csv_template(
"blood_pressure",
{
"Datum": "measured_date",
"Zeit": "measured_time",
"Systolisch (mmHg)": "systolic",
"Diastolisch (mmHg)": "diastolic",
"Puls (bpm)": "pulse",
},
{
"measured_date": {"type": "date", "format": "dd.mm.yyyy"},
"measured_time": {"type": "time", "format": "HH:MM"},
"systolic": {"type": "int"},
"diastolic": {"type": "int"},
"pulse": {"type": "int"},
},
None,
["datum", "diastolisch_mmhg", "puls_bpm", "systolisch_mmhg", "zeit"],
)
assert r["valid"] is True
assert not any(w["code"] == "signature_vs_mappings_mismatch" for w in r["warnings"])
def test_validate_duplicate_target_warning():
r = validate_csv_template(
"weight",
{"A": "weight", "B": "weight", "Tag": "date"},
{
"weight": {"type": "float", "decimal_separator": "."},
"date": {"type": "date", "format": "yyyy-mm-dd", "flexible": True},
},
None,
None,
)
assert r["valid"] is True
assert any(w["code"] == "duplicate_target_columns" for w in r["warnings"])
+32 -4
View File
@@ -7,9 +7,9 @@ Semantic Versioning: MAJOR.MINOR.PATCH
- PATCH: Bugfix, kleine Änderung, Refactor
"""
APP_VERSION = "0.9p"
BUILD_DATE = "2026-04-09"
DB_SCHEMA_VERSION = "20260406e" # Migration 041
APP_VERSION = "0.9q"
BUILD_DATE = "2026-04-11"
DB_SCHEMA_VERSION = "20260409c" # 048/049 vitals_baseline.source csv + SAVEPOINT Import
MODULE_VERSIONS = {
"auth": "1.2.0",
@@ -29,11 +29,39 @@ MODULE_VERSIONS = {
"exportdata": "1.1.0",
"importdata": "1.0.0",
"membership": "2.1.0",
"workflow": "0.6.0", # Phase 4: End Node Template Engine
"workflow": "0.7.0", # Part 3: Inline Prompts (reference + inline mode)
"app_dashboard": "1.11.0", # Entitlements: DB-Override widget→features (AND), sonst Katalog
"csv_import": "0.3.2", # Import-Fehler: enrich_row_error / freundlichere 500-Hinweise
"admin_csv_templates": "0.3.0", # POST /validate + Speichern nur bei valid (422 + warnings in Response)
}
CHANGELOG = [
{
"version": "0.9q",
"date": "2026-04-11",
"changes": [
"Workflow Engine Part 3: Inline Prompts",
"Frontend: Radio Buttons (Reference/Inline), InlineTemplateEditor Component",
"Frontend: Placeholder Picker für Inline-Templates, Cursor-Position Tracking",
"Backend: load_prompt_template() unterstützt inline_template",
"Backend: WorkflowNode.inline_template Feld hinzugefügt",
"Serialization: inline_template speichern/laden in graph_data",
"Validation: Prüft dass entweder prompt_slug ODER inline_template gesetzt",
],
},
{
"version": "0.9p",
"date": "2026-04-09",
"changes": [
"Issue #21 Phase 1: Migration 042/043 (csv_field_mappings, csv_import_log, Seeds)",
"csv_parser: core (Decode/Delimiter/Sample), module_registry, type_converter, permissions",
"API /api/csv: modules, limits, mappings, analyze, copy",
"API /api/admin/csv-templates: CRUD System-Templates, import-limits (system_config)",
"Issue #21: POST /api/csv/import + executor (nutrition Aggregat/Tag, weight, Blutdruck); activity 501",
"Issue #21: GET-Analyse über alle Vorlagen; Import nur mapping_id; GUI ohne Modul-Dropdown",
"v9c_cleanup_features.sql: FK-sichere csv_import→data_import Reihenfolge",
],
},
{
"version": "0.9n",
"date": "2026-04-06",
+258 -35
View File
@@ -15,10 +15,10 @@ from datetime import datetime
import uuid
import logging
import json
from jinja2 import Template, TemplateError
from jinja2 import Environment, ChainableUndefined, TemplateError
from workflow_models import (
WorkflowGraph, NodeExecutionState, ExecutionResult,
WorkflowGraph, WorkflowNode, NodeExecutionState, ExecutionResult,
NodeStatus, NormalizedSignal, FallbackStrategy, SignalStatus,
EndNodeOutputMode
)
@@ -42,7 +42,8 @@ async def execute_workflow(
profile_id: str = None,
variables: Dict[str, Any] = None,
openrouter_call_func = None, # Callback für LLM-Calls: async (prompt, model) -> str
enable_debug: bool = False
enable_debug: bool = False,
progress_callback = None # NEW: Optional callback für Progress-Updates: async (event_type, data) -> None
) -> ExecutionResult:
"""
Führt einen Workflow aus (mit conditional branching und path consolidation).
@@ -76,6 +77,13 @@ async def execute_workflow(
logger.info(f"Starting workflow execution: {execution_id}")
# NEW: Progress-Callback für Start
if progress_callback:
await progress_callback("execution_started", {
"execution_id": execution_id,
"started_at": started_at
})
try:
# 1. Lade Workflow-Definition
if graph_data:
@@ -158,9 +166,29 @@ async def execute_workflow(
enable_debug=enable_debug
)
# Add human-readable label to node_state for debug UI
# Priority: node.label (user-defined) > prompt_slug > node_type-id
node_label = node.label if hasattr(node, 'label') and node.label else (
node.prompt_slug if hasattr(node, 'prompt_slug') and node.prompt_slug else f"{node.type.value}-{node_id[:8]}"
)
if not node_state.debug_prompt_slug:
node_state.debug_prompt_slug = node_label
node_states.append(node_state)
context["node_results"][node_id] = node_state
# NEW: Progress-Callback aufrufen (für SSE Streaming)
if progress_callback:
await progress_callback("node_complete", {
"node_id": node_id,
"node_type": node.type,
"node_label": node_label,
"status": node_state.status.value,
"total_nodes": len(graph.nodes),
"completed_nodes": len([ns for ns in node_states if ns.status in [NodeStatus.EXECUTED, NodeStatus.SKIPPED]]),
"error": node_state.error if node_state.status == NodeStatus.FAILED else None
})
# Füge Nachfolger zur Queue hinzu
outgoing_edges = [e for e in graph.edges if e.from_node == node_id]
for edge in outgoing_edges:
@@ -185,6 +213,19 @@ async def execute_workflow(
logger.info(f"Workflow execution completed: {execution_id}")
# NEW: Progress-Callback für erfolgreiche Fertigstellung
if progress_callback:
await progress_callback("execution_complete", {
"execution_id": execution_id,
"status": "completed",
"aggregated_result": aggregated,
"total_nodes": len(node_states),
"completed_nodes": len([ns for ns in node_states if ns.status == NodeStatus.EXECUTED]),
"skipped_nodes": len([ns for ns in node_states if ns.status == NodeStatus.SKIPPED]),
"failed_nodes": len([ns for ns in node_states if ns.status == NodeStatus.FAILED]),
"completed_at": completed_at
})
return ExecutionResult(
execution_id=execution_id,
workflow_id=workflow_id or "N/A", # Placeholder when graph_data is used directly
@@ -198,6 +239,15 @@ async def execute_workflow(
except Exception as e:
logger.error(f"Workflow execution failed: {e}", exc_info=True)
# NEW: Progress-Callback für Fehler
if progress_callback:
await progress_callback("execution_failed", {
"execution_id": execution_id,
"status": "failed",
"error": str(e),
"completed_at": datetime.utcnow().isoformat()
})
# Speichere Failed State
completed_at = datetime.utcnow().isoformat()
save_execution_state(
@@ -270,17 +320,18 @@ async def execute_node(
# Logic Nodes (Phase 3)
if node.type == "logic":
return execute_logic_node(node, context, graph)
return execute_logic_node(node, context, graph, enable_debug)
# Join Nodes (Phase 4)
if node.type == "join":
return execute_join_node(node, context, graph)
return execute_join_node(node, context, graph, enable_debug)
# Analysis Nodes
if node.type == "analysis":
# 1. Lade Prompt
prompt_template = await load_prompt_template(node.prompt_slug, context)
logger.debug(f"Node {node.id}: Loaded prompt '{node.prompt_slug}'")
# 1. Lade Prompt (Part 3: inline_template support)
prompt_template = await load_prompt_template(node, context)
source_type = "inline" if node.inline_template else "reference"
logger.debug(f"Node {node.id}: Loaded prompt from {source_type}")
# 2. Parse question_augmentations
questions = []
@@ -353,6 +404,11 @@ async def execute_node(
decision_signals=parsed["decision_signals"],
normalized_signals=normalized_signals,
reasoning_anchors=parsed.get("reasoning_anchors"),
# Debug information (nur wenn enable_debug=True)
debug_prompt=augmented_prompt if enable_debug else None,
debug_raw_response=llm_response if enable_debug else None,
debug_node_type="analysis",
debug_prompt_slug=node.prompt_slug if enable_debug else None,
started_at=started_at,
completed_at=datetime.utcnow().isoformat()
)
@@ -374,7 +430,8 @@ async def execute_node(
def execute_logic_node(
node,
context: Dict[str, Any],
graph: WorkflowGraph
graph: WorkflowGraph,
enable_debug: bool = False
) -> NodeExecutionState:
"""
Führt Logic Node aus (Phase 3).
@@ -398,10 +455,49 @@ def execute_logic_node(
try:
if not node.condition:
raise ValueError(f"Logic node {node.id} has no condition")
error_msg = f"Logic node {node.id} has no condition defined"
logger.error(error_msg)
return NodeExecutionState(
node_id=node.id,
status=NodeStatus.FAILED,
error=error_msg,
started_at=started_at,
completed_at=datetime.utcnow().isoformat()
)
# Handle both formats (thanks to Union[LogicExpression, Condition] type):
# 1. Direct LogicExpression (UI format): node.condition is LogicExpression
# 2. Wrapped in Condition (legacy): node.condition is Condition with .expression
from workflow_models import LogicExpression, Condition
expression = None
if isinstance(node.condition, LogicExpression):
# UI format: direct LogicExpression
expression = node.condition
elif isinstance(node.condition, Condition):
# Legacy format: wrapped in Condition
expression = node.condition.expression
else:
# Fallback: try to detect format manually
if hasattr(node.condition, 'operator') and hasattr(node.condition, 'operands'):
expression = node.condition # Looks like LogicExpression
elif hasattr(node.condition, 'expression'):
expression = node.condition.expression # Looks like Condition
if expression is None:
error_msg = f"Logic node {node.id} has no valid condition/expression defined"
logger.error(error_msg)
return NodeExecutionState(
node_id=node.id,
status=NodeStatus.FAILED,
error=error_msg,
started_at=started_at,
completed_at=datetime.utcnow().isoformat()
)
# 1. Evaluiere Bedingung
result, error = evaluate_logic_expression(node.condition.expression, context)
result, error = evaluate_logic_expression(expression, context)
if error:
# Fehler bei Evaluation → Fallback anwenden
@@ -435,7 +531,15 @@ def execute_logic_node(
"evaluation_result": result if not error else None,
"error": error,
"activated_edges": activated_edges
})
}),
# Debug information
debug_prompt=json.dumps(expression.model_dump() if hasattr(expression, 'model_dump') else str(expression), ensure_ascii=False) if enable_debug and expression else None,
debug_raw_response=json.dumps({
"result": result,
"error": error,
"activated_edges": activated_edges
}, ensure_ascii=False) if enable_debug else None,
debug_node_type="logic"
)
except Exception as e:
@@ -452,7 +556,8 @@ def execute_logic_node(
def execute_join_node(
node,
context: Dict[str, Any],
graph: WorkflowGraph
graph: WorkflowGraph,
enable_debug: bool = False
) -> NodeExecutionState:
"""
Führt Join Node aus (Phase 4).
@@ -524,7 +629,16 @@ def execute_join_node(
normalized_signals=consolidated_signals_list,
metadata=join_result.metadata,
started_at=started_at,
completed_at=datetime.utcnow().isoformat()
completed_at=datetime.utcnow().isoformat(),
# Debug information
debug_prompt=f"Join Strategy: {node.join_strategy.value if node.join_strategy else 'wait_all'}\nPaths: {total_count}\nMinimum Required: {node.min_paths if node.min_paths else 'all'}" if enable_debug else None,
debug_raw_response=json.dumps({
"executed_paths": executed_count,
"total_paths": total_count,
"strategy": node.join_strategy.value if node.join_strategy else 'wait_all',
"consolidated_nodes": list(join_result.consolidated_analysis_core.keys())
}, ensure_ascii=False) if enable_debug else None,
debug_node_type="join"
)
except Exception as e:
@@ -593,10 +707,20 @@ def execute_end_node(
# AUTO mode: Concatenate all analysis_core values
logger.debug(f"End node {node.id}: Using AUTO output mode")
# Get graph from context for node label lookup
graph = context.get("graph")
combined_analysis = []
for node_id, node_state in context.get("node_results", {}).items():
if node_state.status == NodeStatus.EXECUTED and node_state.analysis_core:
combined_analysis.append(f"## {node_id}\n{node_state.analysis_core}")
# Get node label from graph (fallback to node_id if not found)
node_label = node_id # default
if graph:
node_obj = next((n for n in graph.nodes if n.id == node_id), None)
if node_obj and hasattr(node_obj, 'data') and node_obj.data:
node_label = node_obj.data.get('label', node_id)
combined_analysis.append(f"## {node_label}\n{node_state.analysis_core}")
final_output = "\n\n".join(combined_analysis) if combined_analysis else "[No analysis generated]"
@@ -618,6 +742,11 @@ def execute_end_node(
"reasoning_anchors": node_state.reasoning_anchors or "",
"status": node_state.status.value if node_state.status else "unknown",
}
# Backward-compatible shortcut:
# allow {{node_id.relevanz}} in addition to {{node_id.decision_signals.relevanz}}
if node_state.decision_signals:
for signal_key, signal_value in node_state.decision_signals.items():
node_context[signal_key] = signal_value
# Add normalized signals as {{node_id.signal_ID}}
# NOTE: question_type now IS the ID (not the type!)
@@ -660,7 +789,10 @@ def execute_end_node(
# Render template
try:
jinja_template = Template(node.template)
# ChainableUndefined keeps missing nested attributes renderable
# so Jinja's default filter can handle them.
jinja_env = Environment(undefined=ChainableUndefined)
jinja_template = jinja_env.from_string(node.template)
final_output = jinja_template.render(template_context)
logger.info(f"End node {node.id}: Template rendered successfully ({len(final_output)} chars)")
except TemplateError as te:
@@ -758,17 +890,31 @@ def _get_edges_by_label(node_id: str, label: str, graph: WorkflowGraph) -> List[
"""
Findet alle ausgehenden Edges mit bestimmtem Label.
Unterstützt beide Formate:
- Legacy: e.label == label (z.B. "then", "else")
- UI: e.source_handle == label (z.B. "true", "false")
Args:
node_id: Node-ID
label: Edge-Label (z.B. "then", "else", "uncertainty")
label: Edge-Label oder sourceHandle (z.B. "then"/"true", "else"/"false")
graph: WorkflowGraph
Returns:
Liste von Edge-IDs
"""
# Map label to sourceHandle equivalents
label_to_handle = {
"then": "true",
"else": "false"
}
handle_equivalent = label_to_handle.get(label, label)
matching_edges = [
e.id for e in graph.edges
if e.from_node == node_id and e.label == label
if e.from_node == node_id and (
e.label == label or # Legacy format
(hasattr(e, 'source_handle') and e.source_handle == handle_equivalent) # UI format
)
]
return matching_edges
@@ -804,39 +950,64 @@ def _has_active_incoming_edge(node, graph: WorkflowGraph, context: Dict[str, Any
return False
async def load_prompt_template(prompt_slug: str, context: Dict[str, Any]) -> str:
async def load_prompt_template(node: WorkflowNode, context: Dict[str, Any]) -> str:
"""
Lädt Prompt-Template aus DB und resolved Platzhalter.
Lädt Prompt-Template aus DB (reference mode) oder direkt vom Node (inline mode).
Part 3: Inline Prompts - Unterstützt zwei Modi:
- Reference Mode: prompt_slug → Template aus ai_prompts Tabelle
- Inline Mode: inline_template → Template direkt vom Node
Args:
prompt_slug: Slug des Prompts (z.B. "pipeline_body")
node: WorkflowNode mit prompt_slug ODER inline_template
context: {"variables": {"name": "Lars", ...}, "profile_id": "..."}
Returns:
Resolved prompt template
Raises:
HTTPException: Wenn weder prompt_slug noch inline_template gesetzt
Beispiel:
>>> template = await load_prompt_template("pipeline_body", {"profile_id": "123"})
>>> node = WorkflowNode(id="n1", prompt_slug="pipeline_body")
>>> template = await load_prompt_template(node, {"profile_id": "123"})
>>> "{{name}}" not in template
True
"""
from placeholder_resolver import get_placeholder_example_values, get_placeholder_catalog
from prompt_executor import resolve_placeholders
from fastapi import HTTPException
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"SELECT template FROM ai_prompts WHERE slug = %s AND active = true",
(prompt_slug,)
# Mode 1: Inline Template (NEU)
if node.inline_template:
logger.debug(f"Node {node.id}: Using inline template ({len(node.inline_template)} chars)")
template = node.inline_template
# Mode 2: Reference (bestehend)
elif node.prompt_slug:
logger.debug(f"Node {node.id}: Loading prompt '{node.prompt_slug}' from DB")
with get_db() as conn:
cur = get_cursor(conn)
cur.execute(
"SELECT template FROM ai_prompts WHERE slug = %s AND active = true",
(node.prompt_slug,)
)
row = cur.fetchone()
if not row:
raise HTTPException(status_code=404, detail=f"Prompt not found: {node.prompt_slug}")
template = row['template']
# Mode 3: Error - weder inline noch reference
else:
raise HTTPException(
status_code=400,
detail=f"Node {node.id}: Either prompt_slug or inline_template required"
)
row = cur.fetchone()
if not row:
raise ValueError(f"Prompt not found: {prompt_slug}")
template = row['template']
# Resolve Placeholders using modern prompt_executor method
profile_id = context.get("profile_id")
if not profile_id:
raise HTTPException(status_code=400, detail="profile_id required in context")
# Build variables dict with ALL registered placeholders
variables = {}
@@ -844,14 +1015,51 @@ async def load_prompt_template(prompt_slug: str, context: Dict[str, Any]) -> str
try:
# Get all placeholder values from registry
processed_placeholders = get_placeholder_example_values(profile_id)
logger.info(f"🔍 DEBUG: Loaded {len(processed_placeholders)} placeholders from registry")
logger.info(f"🔍 DEBUG: Sample keys (first 3): {list(processed_placeholders.keys())[:3]}")
# Remove {{ }} from keys (placeholder_resolver returns them with wrappers)
cleaned_placeholders = {
key.replace('{{', '').replace('}}', ''): value
key.replace('{{', '').replace('}}', '').strip(): value
for key, value in processed_placeholders.items()
}
logger.info(f"🔍 DEBUG: Cleaned keys (first 3): {list(cleaned_placeholders.keys())[:3]}")
logger.info(f"🔍 DEBUG: Sample values: name={cleaned_placeholders.get('name')}, age={cleaned_placeholders.get('age')}, geschlecht={cleaned_placeholders.get('geschlecht')}")
variables.update(cleaned_placeholders)
except Exception as e:
logger.warning(f"Failed to load placeholders for workflow: {e}")
logger.error(f"❌ CRITICAL: Failed to load placeholders for workflow: {e}", exc_info=True)
# Add workflow node outputs as placeholders (Part 3: Inline Prompts)
# Format: node_id.analysis_core, node_id.signal_xyz, node_id.question_xyz
node_results = context.get("node_results", {})
if node_results:
logger.info(f"🔍 DEBUG: Adding {len(node_results)} node outputs as placeholders")
for node_id, node_state in node_results.items():
# analysis_core
if hasattr(node_state, 'analysis_core') and node_state.analysis_core:
key = f"{node_id}.analysis_core"
variables[key] = node_state.analysis_core
logger.debug(f" Added placeholder: {key} = {node_state.analysis_core[:50]}...")
# decision_signals (keyed by question ID)
if hasattr(node_state, 'decision_signals') and node_state.decision_signals:
for signal_id, signal_value in node_state.decision_signals.items():
# Signal placeholder: node_id.signal_question_id
signal_key = f"{node_id}.signal_{signal_id}"
variables[signal_key] = signal_value
logger.debug(f" Added placeholder: {signal_key} = {signal_value}")
# Question texts (from graph metadata if available)
# NOTE: Question text placeholders are populated from graph in PlaceholderPicker
# Here we only add if available in node_state metadata
if hasattr(node_state, 'metadata') and isinstance(node_state.metadata, dict):
questions = node_state.metadata.get('questions', [])
for q in questions:
if isinstance(q, dict) and 'id' in q and 'question' in q:
question_key = f"{node_id}.question_{q['id']}"
variables[question_key] = q['question']
logger.debug(f" Added placeholder: {question_key}")
# Load catalog for |d modifier support
try:
@@ -860,13 +1068,22 @@ async def load_prompt_template(prompt_slug: str, context: Dict[str, Any]) -> str
catalog = None
logger.warning(f"Failed to load placeholder catalog for workflow: {e}")
logger.info(f"🔍 DEBUG: Template before resolution:\n{template[:200]}...")
logger.info(f"🔍 DEBUG: Variables dict has {len(variables)} entries")
# Resolve with modern executor
debug_info = {}
resolved = resolve_placeholders(
template=template,
variables=variables,
debug_info=debug_info,
catalog=catalog
)
logger.info(f"🔍 DEBUG: Resolved placeholders: {debug_info.get('resolved_placeholders', {})}")
logger.info(f"🔍 DEBUG: Unresolved placeholders: {debug_info.get('unresolved_placeholders', [])}")
logger.info(f"🔍 DEBUG: Template after resolution:\n{resolved[:200]}...")
return resolved
@@ -919,7 +1136,13 @@ def aggregate_results(node_states: List[NodeExecutionState], graph) -> Dict[str,
final_output = state.analysis_core
else:
# Regular node - add to combined analysis
combined_analysis.append(f"## {state.node_id}\n{state.analysis_core}")
# Get node label from graph (fallback to node_id)
node_label = state.node_id # default
node_obj = next((n for n in graph.nodes if n.id == state.node_id), None)
if node_obj and hasattr(node_obj, 'data') and node_obj.data:
node_label = node_obj.data.get('label', state.node_id)
combined_analysis.append(f"## {node_label}\n{state.analysis_core}")
if state.normalized_signals:
all_signals.extend([s.model_dump() for s in state.normalized_signals])
+29 -4
View File
@@ -6,7 +6,7 @@ Data validation schemas for Workflow-Graph, Knoten, Kanten, Bedingungen.
Konzept-Basis: konzept_workflow_engine_konsolidated.md
Anforderungsanalyse: anforderungsanalyse_umsetzungsplan.md
"""
from typing import Optional, List, Dict, Any
from typing import Optional, List, Dict, Any, Union
from pydantic import BaseModel, Field
from enum import Enum
@@ -148,18 +148,27 @@ class LogicExpression(BaseModel):
}
"""
operator: LogicOperator = Field(..., description="Logischer Operator (and, or, not) oder Vergleichsoperator")
operands: Optional[List[Any]] = Field(None, description="Liste von Operanden (LogicOperand oder verschachtelte LogicExpression)")
operands: Optional[List['LogicExpression']] = Field(None, description="Liste von Operanden (LogicOperand oder verschachtelte LogicExpression)")
# Bei einfachem Vergleich:
ref: Optional[str] = Field(None, description="Signal-Referenz (nur bei Vergleichsoperatoren)")
value: Optional[Any] = Field(None, description="Vergleichswert (nur bei Vergleichsoperatoren)")
# Enable forward reference resolution for recursive model
LogicExpression.model_rebuild()
class Condition(BaseModel):
"""
Bedingung für einen Logik-Knoten.
Unterstützt if/else-if/else-Logik.
Note: Uses extra='forbid' to ensure proper Union resolution with LogicExpression.
If unknown fields are present (like 'operator', 'operands'), deserialization fails
and Pydantic tries LogicExpression instead.
"""
model_config = {'extra': 'forbid'}
type: str = Field(default="if", description="Bedingungstyp: if, else-if, else")
expression: Optional[LogicExpression] = Field(None, description="Logischer Ausdruck (null bei 'else')")
then_path: Optional[str] = Field(None, description="Edge-ID für 'then'-Pfad")
@@ -187,14 +196,17 @@ class WorkflowNode(BaseModel):
"""
id: str = Field(..., description="Eindeutige Knoten-ID")
type: NodeType = Field(..., description="Knotentyp")
label: Optional[str] = Field(None, description="Node-Label (vom Editor, z.B. 'Qualitätseinschätzung')")
position: Optional[Position] = Field(None, description="Position im visuellen Editor")
# ANALYSIS-Knoten
prompt_slug: Optional[str] = Field(None, description="Slug des auszuführenden Prompts")
prompt_slug: Optional[str] = Field(None, description="Slug des auszuführenden Prompts (reference mode)")
inline_template: Optional[str] = Field(None, description="Inline-Prompt-Template (inline mode, Part 3)")
question_augmentations: Optional[List[QuestionAugmentation]] = Field(None, description="Fragenergänzungen (knotengebunden, überschreiben Prompt-Defaults)")
# LOGIC-Knoten
condition: Optional[Condition] = Field(None, description="Bedingung für Pfad-Routing")
# Support both formats: direct LogicExpression (UI) or wrapped in Condition (legacy)
condition: Optional[Union[LogicExpression, Condition]] = Field(None, description="Bedingung für Pfad-Routing")
fallback: Optional[FallbackConfig] = Field(None, description="Fallback-Konfiguration")
# JOIN-Knoten (Phase 4)
@@ -219,6 +231,10 @@ class WorkflowEdge(BaseModel):
to_node: str = Field(..., alias="to", description="Ziel-Knoten-ID")
label: Optional[str] = Field(None, description="Label für visuelle Darstellung (z.B. 'then', 'else')")
# UI-Format fields (React Flow)
source_handle: Optional[str] = Field(None, alias="sourceHandle", description="Source handle ID (UI format: 'true', 'false', 'out')")
target_handle: Optional[str] = Field(None, alias="targetHandle", description="Target handle ID (UI format: 'in', 'path_1', etc.)")
class WorkflowGraph(BaseModel):
"""
@@ -334,6 +350,15 @@ class NodeExecutionState(BaseModel):
normalized_signals: List[NormalizedSignal] = Field(default_factory=list, description="Normalisierte Signale (Phase 2)")
reasoning_anchors: Optional[str] = Field(None, description="Begründungsanker aus ## Begründung")
# Debug Information (nur wenn enable_debug=True)
debug_prompt: Optional[str] = Field(None, description="Vollständiger Prompt der an die KI gesendet wurde")
debug_raw_response: Optional[str] = Field(None, description="Rohe KI-Antwort (ungeparst)")
debug_node_type: Optional[str] = Field(None, description="Node-Typ (analysis, logic, join, etc.)")
debug_prompt_slug: Optional[str] = Field(None, description="Verwendeter Prompt-Slug (bei ANALYSIS nodes)")
# Metadata (für Join Nodes und andere Zusatzinfos)
metadata: Optional[Dict[str, Any]] = Field(None, description="Zusätzliche Metadaten (z.B. Join-Statistiken)")
# Error & Timing
error: Optional[str] = Field(None, description="Fehlermeldung bei failed")
started_at: Optional[str] = Field(None, description="Start-Timestamp (ISO)")
+1 -1
View File
@@ -18,7 +18,7 @@ This document establishes **mandatory governance rules** for placeholder managem
## 2. Scope
These guidelines apply to:
- All 116 existing placeholders
- All 114 existing placeholders (canonical: `PLACEHOLDER_MAP`)
- All new placeholders
- All modifications to existing placeholders
- All placeholder deprecations
@@ -79,7 +79,7 @@ curl -s -H "X-Auth-Token: $TOKEN" \
**Expected response:**
```json
{
"total_placeholders": 116,
"total_placeholders": 114,
"available": 98,
"missing": 18,
"by_type": {
@@ -9,12 +9,12 @@
## Executive Summary
This document summarizes the complete implementation of the normative placeholder metadata system for Mitai Jinkendo. The system provides a comprehensive, standardized framework for managing, documenting, and validating all 116 placeholders in the system.
This document summarizes the complete implementation of the normative placeholder metadata system for Mitai Jinkendo. The system provides a comprehensive, standardized framework for managing, documenting, and validating all 114 placeholders in the system.
**Key Achievements:**
- ✅ Complete metadata schema (normative compliant)
- ✅ Automatic metadata extraction
- ✅ Manual curation for 116 placeholders
- ✅ Manual curation for 114 placeholders
- ✅ Extended export API (non-breaking)
- ✅ Catalog generator (4 documentation files)
- ✅ Validation & testing framework
@@ -75,7 +75,7 @@ This document summarizes the complete implementation of the normative placeholde
### 1.3 Complete Metadata Definitions
#### `backend/placeholder_metadata_complete.py` (220 lines, expandable to all 116)
#### `backend/placeholder_metadata_complete.py` (220 lines, expandable to all 114)
**Purpose:** Manually curated, authoritative metadata for all placeholders
@@ -106,7 +106,7 @@ PlaceholderMetadata(
**Key Features:**
- Hand-curated for accuracy
- Complete for all 116 placeholders
- Complete for all 114 placeholders
- Serves as authoritative source
- Normative compliant
@@ -285,7 +285,7 @@ pytest backend/tests/test_placeholder_metadata.py -v
v
┌─────────────────────────────────────────────────────────────┐
│ Complete Registry │
│ (116 placeholders with full metadata) │
│ (114 placeholders with full metadata) │
└──────────┬──────────────────────────────────────────────────┘
│
├──> Generation Scripts (generate_*.py)
@@ -309,7 +309,7 @@ pytest backend/tests/test_placeholder_metadata.py -v
### 3.1 Metadata Extraction Flow
```
1. PLACEHOLDER_MAP (116 entries)
1. PLACEHOLDER_MAP (114 entries)
└─> extract_resolver_name()
└─> analyze_data_layer_usage()
└─> infer_type/time_window/output_type()
@@ -468,7 +468,7 @@ curl -H "X-Auth-Token: <token>" \
# Output:
{
"total_placeholders": 116,
"total_placeholders": 114,
"available": 98,
"missing": 18,
"by_type": {
@@ -599,7 +599,7 @@ The system is designed for extensibility:
## 8. Compliance Checklist
✅ **Normative Standard Compliance:**
- All 116 placeholders inventoried
- All 114 placeholders inventoried
- Complete metadata schema implemented
- Validation framework in place
- Non-breaking export API
+1
View File
@@ -29,6 +29,7 @@ Dieser Ordner ist **immer mit Git versioniert**. Er ergänzt **`.claude/docs/`**
| `issue-53-phase-0c-multi-layer-architecture.md` |
| `issue-54-dynamic-placeholder-system.md` |
| `issue-55-dynamic-aggregation-methods.md` |
| `issue-76-training-quality-goal-list-filter.md` |
| `PHASE_PLAN_RESPONSIVE_UI.md` |
| `REVIEW_OPEN_ISSUES_2026-04-04.md` |
| `UMSETZUNGSPLAN_TRAININGSPROFILE_SPORTSPEZIFISCH_2026-04-05.md` |
@@ -14,6 +14,8 @@
- ✅ Commits: 7 systematic commits (6 module migrations + 1 chart expansion)
- ✅ charts.py: 329 → 2246 lines (+1917 lines)
**Import vs. Auswertung:** Leitlinie „was beim Ingest erlaubt ist“ und Einordnung bestehender Import-Pfade: `.claude/rules/ARCHITECTURE.md` Abschnitt **8. CSV-Import vs. Data Layer**.
---
## Executive Summary
@@ -0,0 +1,49 @@
# Issue #76: Trainings-Qualität – Zielbezug + Listen-Filter
**Gitea:** http://192.168.2.144:3000/Lars/mitai-jinkendo/issues/76
**Status:** OFFEN
**Erstellt:** 11.04.2026
**Typ:** Enhancement (Backend, Frontend, Produkt)
**Verwandt:** #31 (globaler Quality-Filter), #15 (Quality-Filter KI/Charts – historisch)
---
## Kurzfassung
Die heutige **`quality_label` / `quality_filter_level`**-Logik soll überdacht werden: nicht nur **Akzeptanz** (formales Label), sondern langfristig **Nutzen für die Zielerreichung**. Parallel: der **globale Profil-Filter** in der **Aktivitätsliste** versteckt u. a. **fehlende Einstufungen** (`NULL`) – das erschwert Datenpflege und Transparenz.
---
## Ist-Zustand (technisch)
| Bereich | Filter nach `quality_filter_level`? |
|---------|--------------------------------------|
| `GET` Aktivitätsliste (`routers/activity.py`) | Ja (`get_quality_filter_sql`) |
| KI-Rohkontext (`routers/insights.py`, `_get_profile_data`) | Ja (letzte 90 Zeilen `activity_log`) |
| Data-Layer / Platzhalter / Chart-Aggregate (`data_layer/activity_metrics.py`) | Nein – alle Einträge im Zeitfenster |
Bei Stufen ≠ `all` schließt SQL-`IN` typischerweise **`NULL`** und nicht erlaubte Labels aus.
---
## Ziele
1. **Konzept „Qualität“:** Richtung Ziel-/Focus-/Phasenbezug; Umsetzung kann in Folge-Issues zerlegt werden.
2. **Listendarstellung:** Standard soll **alle** relevanten Einträge zeigen; **optionale lokale** Filter (inkl. „ohne Einstufung“), kein unbemerktes globales Ausblenden.
3. **KI/Insights:** Klarstellen, ob gefilterte vs. vollständige Aktivitätsmenge; ggf. konfigurierbar oder im Prompt kenntlich.
4. **Dokumentation:** Kurz festhalten, wo `get_quality_filter_sql` greift und wie das zu den Aggregationen passt.
---
## Code-Referenzen
- `backend/quality_filter.py`
- `backend/routers/activity.py` (`list_activity`)
- `backend/routers/insights.py` (`_get_profile_data`)
- `backend/data_layer/activity_metrics.py`
---
## Akzeptanz (aus Gitea, Checkliste)
Siehe Issue-Body in Gitea #76; bei Umsetzung dort Häkchen setzen und diese Datei ggf. um **Umsetzungsnotizen / Abnahme** ergänzen.
+6
View File
@@ -53,6 +53,9 @@ import RestDaysPage from './pages/RestDaysPage'
import VitalsPage from './pages/VitalsPage'
import GoalsPage from './pages/GoalsPage'
import CustomGoalsPage from './pages/CustomGoalsPage'
import UniversalCsvImportPage from './pages/UniversalCsvImportPage'
import AdminCsvTemplatesPage from './pages/AdminCsvTemplatesPage'
import AdminCsvTemplateEditorPage from './pages/AdminCsvTemplateEditorPage'
import WorkflowEditorPage from './pages/WorkflowEditorPage'
import DesktopSidebar from './components/DesktopSidebar'
import { getMainNavItems } from './config/appNav'
@@ -225,6 +228,7 @@ function AppShell() {
<Route path="/vitals" element={<VitalsPage />} />
<Route path="/custom-goals" element={<CustomGoalsPage />} />
<Route path="/nutrition" element={<NutritionPage />} />
<Route path="/csv-import" element={<UniversalCsvImportPage />} />
<Route path="/activity" element={<ActivityPage />} />
<Route path="/guide" element={<GuidePage />} />
</Route>
@@ -256,6 +260,8 @@ function AppShell() {
<Route path="goal-types" element={<AdminGoalTypesPage/>}/>
<Route path="focus-areas" element={<AdminFocusAreasPage/>}/>
<Route path="reference-value-types" element={<AdminReferenceValueTypesPage/>}/>
<Route path="csv-templates" element={<AdminCsvTemplatesPage />} />
<Route path="csv-templates/:id" element={<AdminCsvTemplateEditorPage />} />
</Route>
</Route>
<Route path="/workflow-editor/:id" element={<WorkflowEditorPage/>}/>
+13 -3
View File
@@ -18,6 +18,8 @@
--header-h: 52px;
--font: system-ui, -apple-system, 'Segoe UI', sans-serif;
--capture-content-max: 800px;
/* Admin: nutzt volle Hauptspalte bis zu dieser Obergrenze (siehe .app-main:has(.admin-shell)) */
--admin-main-max: min(1560px, calc(100vw - 220px));
}
@media (prefers-color-scheme: dark) {
:root {
@@ -619,13 +621,16 @@ a.analysis-split__nav-item {
.admin-page {
width: 100%;
min-width: 0;
}
/* Desktop: volle Breite der Admin-Spalte (nicht wie Erfassung 800px); Lesegröße leicht skaliert */
@media (min-width: 1024px) {
.admin-page {
max-width: var(--capture-content-max);
margin-left: auto;
margin-right: auto;
max-width: 100%;
margin-left: 0;
margin-right: 0;
font-size: clamp(15px, 0.88rem + 0.25vw, 18px);
}
}
@@ -841,6 +846,11 @@ a.analysis-split__nav-item {
box-sizing: border-box;
}
/* Admin: mehr horizontaler Raum für Tabellen auf großen Screens (:has ~2022+, sonst bleibt 1200px) */
.app-main:has(.admin-shell) {
max-width: var(--admin-main-max);
}
/* Dashboard (P3): Begrüßung + Kennzahlen-Zeile */
.dashboard-greeting {
display: flex;
+106
View File
@@ -0,0 +1,106 @@
/**
* ConfirmDialog Component
*
* Modal confirmation dialog
*
* Props:
* - message: string
* - onConfirm: callback when confirmed
* - onCancel: callback when cancelled
* - confirmText: string (default: 'OK')
* - cancelText: string (default: 'Abbrechen')
* - type: 'warning' | 'danger' | 'info' (default: 'warning')
*/
export function ConfirmDialog({
message,
onConfirm,
onCancel,
confirmText = 'OK',
cancelText = 'Abbrechen',
type = 'warning'
}) {
const colors = {
warning: '#FFC107',
danger: 'var(--danger)',
info: 'var(--accent)'
}
return (
<div
style={{
position: 'fixed',
top: 0,
left: 0,
right: 0,
bottom: 0,
background: 'rgba(0,0,0,0.5)',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
zIndex: 20000
}}
onClick={onCancel}
>
<div
style={{
background: 'var(--surface)',
borderRadius: '12px',
padding: '24px',
maxWidth: '500px',
width: '90%',
boxShadow: '0 8px 24px rgba(0,0,0,0.3)'
}}
onClick={(e) => e.stopPropagation()}
>
<div
style={{
fontSize: '16px',
marginBottom: '24px',
color: 'var(--text1)',
lineHeight: '1.5'
}}
>
{message}
</div>
<div
style={{
display: 'flex',
gap: '12px',
justifyContent: 'flex-end'
}}
>
<button
onClick={onCancel}
style={{
padding: '8px 16px',
border: '1px solid var(--border)',
borderRadius: '6px',
background: 'var(--surface2)',
color: 'var(--text1)',
cursor: 'pointer',
fontSize: '14px'
}}
>
{cancelText}
</button>
<button
onClick={onConfirm}
style={{
padding: '8px 16px',
border: 'none',
borderRadius: '6px',
background: colors[type],
color: type === 'warning' ? '#000' : 'white',
cursor: 'pointer',
fontSize: '14px',
fontWeight: 500
}}
>
{confirmText}
</button>
</div>
</div>
</div>
)
}
Loaded 100 of 132 files, more files were not shown because too many files have changed in this diff. Show more