- Updated the `validate_csv_template` function to normalize both the column signature and field mappings for accurate comparison, preventing false warnings about mismatches.
- Enhanced warning messages to provide clearer guidance on the relationship between normalized signatures and raw field mappings.
- Added a new test to ensure that normalized signatures do not trigger false warnings when compared to raw mappings.
- Introduced a new endpoint for validating CSV templates without saving, allowing users to check field mappings and type conversions.
- Updated the `create_system_template` and `update_system_template` functions to include validation reports in responses.
- Enhanced error handling in CSV import processes by integrating `enrich_row_error` for more informative error messages.
- Improved the AdminCsvTemplateEditorPage to support format checking and display validation results, enhancing user experience.
- Incremented version numbers for `csv_import` and `admin_csv_templates` to reflect these updates.
- Increased precision for `kcal_active`, `kcal_resting`, `hr_avg`, and `hr_max` fields in the activity log schema.
- Added a new function `_activity_hr_bpm` to validate heart rate values during CSV import, ensuring they fall within plausible ranges.
- Updated the CSV parser to utilize the new heart rate validation function for improved data integrity.
- Enhanced the type converter to accommodate additional aliases for energy fields in CSV imports.
- Added a test to verify conversion of active energy from kJ to kcal, ensuring accurate data handling.
- Added SAVEPOINT management to the `_import_activity` function to improve error handling during CSV imports.
- Moved the training type resolution logic to occur within a transaction block, ensuring that errors can be rolled back without affecting the entire import process.
- Enhanced error logging to capture evaluation failures, providing better insights during CSV import operations.
- Modified `_resolve_training_type_for_activity` to accept a database cursor, improving efficiency and avoiding potential deadlocks during CSV imports.
- Introduced `get_training_type_for_activity_with_cursor` to handle training type resolution with an existing cursor, streamlining database interactions.
- Updated related calls in the activity import logic to utilize the new function, ensuring consistent behavior across the application.
- Introduced `diagnose_blood_pressure_row` and `diagnose_activity_row` functions to validate and analyze blood pressure and activity data from CSV imports.
- Updated the CSV import logic to handle combined datetime columns for blood pressure and activity, improving data integrity during import.
- Enhanced type conversion specifications to include `start_time` for blood pressure and activity, ensuring accurate data mapping.
- Added tests to validate the new diagnosis functions and their integration with existing import processes, ensuring robustness and reliability.
- Updated frontend messages to provide clearer guidance on blood pressure and activity data handling during CSV imports.
- Implemented a new endpoint for diagnosing CSV imports without writing to the database, allowing users to validate mappings and type conversions.
- Introduced the `diagnose_vitals_row` function to analyze vital metrics and provide detailed feedback on data validity.
- Enhanced the CSV import logic to include alias handling for vital fields, improving compatibility with different CSV formats.
- Updated the frontend to support the new diagnosis feature, including UI elements for displaying diagnosis results and error details.
- Added tests to ensure the correctness of the new diagnosis functionality and its integration with existing import processes.
- Updated the CSV import logic to include SAVEPOINT management, allowing for better error handling during the vitals baseline import process.
- Enhanced the SQL migration script to drop existing CHECK constraints related to the 'source' field, ensuring compatibility with the new universal CSV import.
- Incremented DB_SCHEMA_VERSION to "20260409c" to reflect these changes and improve the import process reliability.
- Updated the aggregate_mapped_rows function to support multiple row policies, allowing for flexible handling of duplicate keys during CSV imports.
- Introduced deduplication of identical rows before aggregation, improving data integrity.
- Enhanced validation for multi_row_policy and dedupe_identical_rows in import_row_processing specifications.
- Updated the AdminCsvTemplateEditorPage to include options for multi-row policies and deduplication settings, improving user experience in template management.
- Added comprehensive tests to validate new aggregation behaviors and ensure correct error handling for multiple rows.
- Updated the CSV import logic to support new row processing specifications for weight and vitals baseline, allowing for better data aggregation and validation.
- Implemented handling for multiple rows on the same day, enabling aggregation of values such as averages for vitals and last values for weight.
- Enhanced test coverage for the new import functionalities, ensuring correct behavior during data processing and validation.
- Refactored the module registry to include default import row processing options for better flexibility in handling CSV data.
- Updated the nutrition import logic to utilize a new row processing specification, improving data aggregation and validation.
- Refactored the template rendering process in the workflow executor to use Jinja2's Environment with ChainableUndefined for better handling of missing attributes.
- Added backward-compatible shortcuts for accessing decision signals in node contexts, enhancing flexibility in template usage.
- Introduced import row processing options in CSV templates, allowing for more customizable data handling during imports.
- Updated the source_unit_choices_for_field function to include a custom option for user-defined conversion factors, improving flexibility in unit conversions.
- Modified the AdminCsvTemplateEditorPage to support custom conversion factors, allowing users to input specific scaling factors for their data.
- Added tests to ensure the custom option is correctly included in the source unit choices and functions as expected in the template editor.
- Added workflow_run triggers for "Deploy Development" and "Deploy Production" to ensure tests run only after successful deployments.
- Updated Python version in CI from 3.12 to 3.11 for better compatibility with the Debian 12 ARM64 runner.
- Enhanced job conditions to skip tests on failed workflow runs.
- Improved frontend build process by updating Node.js setup and ensuring correct directory navigation.
- Refined CSV parsing logic to handle custom and unknown source units, enhancing conversion flexibility.
- Added new tests for custom source unit handling in CSV conversions, ensuring accurate processing.
- Updated the CSV import architecture to clarify the distinction between import and data layer responsibilities, as outlined in the new section of ARCHITECTURE.md.
- Enhanced the build_row_after_mapping function to include module-specific context for improved data processing.
- Introduced source unit options in the admin CSV template editor to facilitate user-defined conversions, improving flexibility in handling various data formats.
- Added new tests to validate the handling of source units and ensure accurate conversions during CSV imports.
- Updated module definitions to include unit specifications for nutritional and activity data fields, enhancing data integrity.
- Integrated date parsing improvements using dateutil for better handling of various date formats in sleep data.
- Added total sleep hours to the nights dictionary for comprehensive sleep analysis.
- Updated the import logic to handle cases where sleep duration is zero, providing appropriate warnings.
- Enhanced the CSV import interface to detect Apple sleep CSV format and provide user feedback on template selection.
- Improved the admin CSV template editor to accommodate new sleep import requirements and clarify usage instructions.
- Added support for new CSV import modules: sleep and vitals_baseline, expanding the import capabilities.
- Implemented backend logic for handling CSV imports related to sleep and vitals baseline, including error handling and data processing.
- Updated frontend components to include new modules in the CSV import interface, improving user experience.
- Introduced unit tests for the new import functionalities to ensure reliability and correctness.
- Enhanced existing CSV analysis features to accommodate the new modules, ensuring consistent behavior across the application.
- Added new functions for calculating header signature recall and ranking metrics, improving the analysis of CSV templates.
- Updated existing CSV analysis endpoints to utilize the new ranking metrics, enhancing the accuracy of template matching.
- Refactored related code to replace Jaccard score calculations with the new metrics, providing a more comprehensive evaluation of CSV structure.
- Improved documentation for new functions to clarify their purpose and usage in the context of CSV template analysis.
- Added a new endpoint for analyzing uploaded CSV files, providing suggestions for field mappings and type conversions.
- Implemented validation for required field targets to ensure all mandatory fields are mapped correctly.
- Enhanced the admin CSV templates interface with new routes and navigation options in the frontend.
- Updated API utility functions to support the new CSV analysis functionality.
- Improved error handling for CSV uploads, including file size and row count checks.
- Added a new function to strip header keys of unwanted characters, improving CSV import consistency.
- Updated CSV row iteration to utilize the new header normalization function, ensuring cleaner data processing.
- Enhanced date parsing capabilities to support flexible formats, accommodating various date representations in CSV files.
- Introduced additional tests to validate the new header normalization and date parsing functionalities.
- Updated version for csv_import to 0.2.0, reflecting new features.
- Implemented a new POST endpoint for universal CSV import, supporting nutrition, weight, and blood pressure modules.
- Added CSV parsing function to yield rows as dictionaries for easier data handling.
- Enhanced error handling and logging for import operations.
- Introduced tests for the new CSV parsing functionality to ensure reliability.
- Added permissions for editing and deleting CSV field mappings.
- Created type converter for CSV cells to handle various data types.
- Implemented database migrations for CSV field mappings and import logs.
- Seeded initial system templates for nutrition and activity data imports.
- Developed admin endpoints for managing system CSV templates.
- Introduced user endpoints for CSV import analysis and mapping retrieval.
- Added tests for core CSV parser functionalities, including delimiter detection and value conversion.
