273c4c6919
Update default COLLECTION_PREFIX to "mindnet" for production environments, requiring explicit setting of COLLECTION_PREFIX=mindnet_dev in .env for development. This change enhances clarity and ensures proper environment configuration.
Lars2026-01-12 15:49:44 +01:00
2ed4488cf6
Enhance timeout handling and diagnostics in runtime service verification
Lars2026-01-12 15:37:12 +01:00
36490425c5
Implement runtime check for EdgeDTO version support in health service
Lars2026-01-12 15:34:56 +01:00
b8cb8bb89b
Add runtime check for EdgeDTO version support in health endpoint
Lars2026-01-12 15:31:38 +01:00
6d268d9dfb
Enhance .env loading mechanism and EdgeDTO creation with error handling
Lars2026-01-12 15:27:23 +01:00
df5f9b3fe4
angleichen der Default prefix für die Collections
Lars2026-01-12 15:02:23 +01:00
5e67cd470c
Merge pull request 'Update deterministic sorting of semantic_groups in build_edges_for_note to handle None values correctly. Introduced a custom sort function to ensure consistent edge extraction across batches, preventing variance in edge counts.' (#23) from WP24c_BugFix into main
Lars2026-01-12 11:42:01 +01:00
0b2a1f1a63
Update deterministic sorting of semantic_groups in build_edges_for_note to handle None values correctly. Introduced a custom sort function to ensure consistent edge extraction across batches, preventing variance in edge counts.
Lars2026-01-12 11:31:20 +01:00
d0012355b9
Merge pull request 'WP24c - Agentic Edge Validation & Chunk-Aware Multigraph-System (v4.5.8)' (#22) from WP24c into main
v3.2.0
Lars2026-01-12 10:53:19 +01:00
1056078e6a
Refactor ID collision logging in ingestion_processor.py for improved clarity and structure
Lars2026-01-12 10:07:24 +01:00
c42a76b3d7
Add dedicated logging for ID collisions in ingestion_processor.py
Lars2026-01-12 09:04:36 +01:00
ec9b3c68af
Implement ID collision detection and enhance logging in ingestion_processor.py
Lars2026-01-12 08:56:28 +01:00
f9118a36f8
Enhance logging in ingestion_processor.py to include normalized file path and note title
Lars2026-01-12 08:33:11 +01:00
e52eed40ca
Refactor hash input handling in ingestion_processor.py to use dictionary format
Lars2026-01-12 08:21:21 +01:00
43641441ef
Refactor hash input and body/frontmatter handling in ingestion_processor.py for improved accuracy
Lars2026-01-12 08:19:43 +01:00
c613d81846
Enhance logging in ingestion_processor.py for detailed change detection diagnostics
Lars2026-01-12 08:16:03 +01:00
de5db09b51
Update logging levels in ingestion_processor.py and import_markdown.py for improved visibility
Lars2026-01-12 08:13:26 +01:00
7cb8fd6602
Enhance logging in ingestion_processor.py for improved change detection diagnostics
Lars2026-01-12 08:08:29 +01:00
6047e94964
Refactor edge processing in graph_derive_edges.py and ingestion_processor.py for consistency and efficiency
Lars2026-01-12 08:04:28 +01:00
78fbc9b31b
Enhance ingestion_processor.py with path normalization and strict change detection
Lars2026-01-12 07:53:03 +01:00
742792770c
Implement Phase 3 Agentic Edge Validation in ingestion_processor.py and related documentation updates
Lars2026-01-12 07:45:54 +01:00
b19f91c3ee
Refactor edge validation process in ingestion_processor.py
Lars2026-01-11 21:47:11 +01:00
9b0d8c18cb
Implement LLM validation for candidate edges in ingestion_processor.py
Lars2026-01-11 21:27:07 +01:00
f2a2f4d2df
Refine LLM validation zone handling in graph_derive_edges.py
Lars2026-01-11 21:09:07 +01:00
ea0fd951f2
Enhance LLM validation zone extraction in graph_derive_edges.py
Lars2026-01-11 20:58:33 +01:00
c8c828c8a8
Add LLM validation zone extraction and configuration support in graph_derive_edges.py
Lars2026-01-11 20:19:12 +01:00
716a063849
Enhance decision_engine.py to support context reuse during compression failures. Implement error handling to return original content when compression fails, ensuring robust fallback mechanisms without re-retrieval. Update logging for better traceability of compression and fallback processes, improving overall reliability in stream handling.
Lars2026-01-11 19:14:15 +01:00
3dc81ade0f
Update logging in decision_engine.py and retriever.py to use node_id as chunk_id and total_score instead of score for improved accuracy in debug statements. This change aligns with the new data structure introduced in version 4.5.4, enhancing traceability in retrieval processes.
Lars2026-01-11 18:55:13 +01:00
1df89205ac
Update EdgeDTO to support extended provenance values and modify explanation building in retriever.py to accommodate new provenance types. This enhances the handling of edge data for improved accuracy in retrieval processes.
Lars2026-01-11 17:54:33 +01:00
2445f7cb2b
Implement chunk-aware graph traversal in hybrid_retrieve: Extract both note_id and chunk_id from hits to enhance seed coverage for edge retrieval. Combine direct and additional chunk IDs for improved accuracy in subgraph expansion. Update debug logging to reflect the new seed and chunk ID handling, ensuring better traceability in graph retrieval processes.
Lars2026-01-11 17:48:30 +01:00
47fdcf8eed
Update logging in retriever.py for version 4.5.1: Modify edge count logging to utilize the adjacency list instead of the non-existent .edges attribute in the subgraph, enhancing accuracy in debug statements related to graph retrieval processes.
Lars2026-01-11 17:44:20 +01:00
3e27c72b80
Enhance logging capabilities across multiple modules for version 4.5.0: Introduce detailed debug statements in decision_engine.py, retriever_scoring.py, retriever.py, and logging_setup.py to improve traceability during retrieval processes. Implement dynamic log level configuration based on environment variables, allowing for more flexible debugging and monitoring of application behavior.
Lars2026-01-11 17:30:34 +01:00
2d87f9d816
Enhance compatibility in chunking and edge processing for version 4.4.1: Harmonize handling of "to" and "target_id" across chunking_processor.py, graph_derive_edges.py, and ingestion_processor.py. Ensure consistent validation and processing of explicit callouts, improving integration and reliability in edge candidate handling.
Lars2026-01-11 15:39:03 +01:00
d7d6155203
Refactor logging in graph_derive_edges.py for version 4.4.0: Move logger initialization to module level for improved accessibility across functions. This change enhances debugging capabilities and maintains consistency in logging practices.
Lars2026-01-11 15:28:14 +01:00
f8506c0bb2
Refactor logging in graph_derive_edges.py and ingestion_chunk_payload.py: Remove redundant logging import and ensure consistent logger initialization for improved debugging capabilities. This change enhances traceability in edge processing and chunk ingestion.
Lars2026-01-11 15:25:57 +01:00
c91910ee9f
Enhance logging and debugging in chunking_processor.py, graph_derive_edges.py, and ingestion_chunk_payload.py for version 4.4.0: Introduce detailed debug statements to trace chunk extraction, global scan comparisons, and payload transfers. Improve visibility into candidate pool handling and decision-making processes for callout edges, ensuring better traceability and debugging capabilities.
Lars2026-01-11 15:21:46 +01:00
ee91583614
Update graph_derive_edges.py to version 4.3.1: Introduce precision prioritization for chunk scope, ensuring chunk candidates are favored over note scope. Adjust confidence values for explicit callouts and enhance key generation for consistent deduplication. Improve edge processing logic to reinforce the precedence of chunk scope in decision-making.
Lars2026-01-11 15:08:08 +01:00
3a17b646e1
Update graph_derive_edges.py and ingestion_chunk_payload.py for version 4.3.0: Introduce debug logging for data transfer audits and candidate pool handling to address potential data loss. Ensure candidate_pool is explicitly retained for accurate chunk attribution, enhancing traceability and reliability in edge processing.
Lars2026-01-11 14:51:38 +01:00
727de50290
Refine edge parsing and chunk attribution in chunking_parser.py and graph_derive_edges.py for version 4.2.9: Ensure current_edge_type persists across empty lines in callout blocks for accurate link processing. Implement two-phase synchronization for chunk authority, collecting explicit callout keys before the global scan to prevent duplicates. Enhance callout extraction logic to respect existing chunk callouts, improving deduplication and processing efficiency.
Lars2026-01-11 14:30:16 +01:00
a780104b3c
Enhance edge processing in graph_derive_edges.py for version 4.2.9: Finalize chunk attribution with synchronization to "Semantic First" signal. Collect callout keys from candidate pool before text scan to prevent duplicates. Update callout extraction logic to ensure strict adherence to existing chunk callouts, improving deduplication and processing efficiency.
Lars2026-01-11 14:07:16 +01:00
f51e1cb2c4
Fix regex pattern in parse_edges_robust to support multiple leading '>' characters for edge callouts, enhancing flexibility in edge parsing.
Lars2026-01-11 12:03:36 +01:00
20fb1e92e2
Enhance chunking functionality in version 4.2.8: Update callout pattern to support additional syntax for edge and abstract callouts. Modify get_chunk_config to allow fallback to chunk_profile if chunking_profile is not present. Ensure explicit passing of chunk_profile in make_chunk_payloads for improved payload handling. Update type hints in chunking_parser for better clarity.
Lars2026-01-11 11:49:16 +01:00
1d66ca0649
Update chunking_utils.py to include Optional type hint: Add Optional to the import statement for improved type annotations, enhancing code clarity and maintainability.
Lars2026-01-11 11:16:30 +01:00
55b64c331a
Enhance chunking system with WP-24c v4.2.6 and v4.2.7 updates: Introduce is_meta_content flag for callouts in RawBlock, ensuring they are chunked but later removed for clean context. Update parse_blocks and propagate_section_edges to handle callout edges with explicit provenance for chunk attribution. Implement clean-context logic to remove callout syntax post-processing, maintaining chunk integrity. Adjust get_chunk_config to prioritize frontmatter overrides for chunking profiles. Update documentation to reflect these changes.
Lars2026-01-11 11:14:31 +01:00
4d43cc526e
Update ingestion_processor.py to version 4.2.4: Implement hash-based change detection for content integrity verification. Restore iterative matching based on content hashes, enhancing the accuracy of change detection. Update documentation to reflect changes in the processing logic and versioning.
Lars2026-01-11 08:08:30 +01:00
6131b315d7
Update graph_derive_edges.py to version 4.2.2: Implement semantic de-duplication with improved scope decision-making. Enhance edge ID calculation by prioritizing semantic grouping before scope assignment, ensuring accurate edge representation across different contexts. Update documentation to reflect changes in edge processing logic and prioritization strategy.
Lars2026-01-10 22:20:13 +01:00
dfff46e45c
Update graph_derive_edges.py to version 4.2.1: Implement Clean-Context enhancements, including consolidated callout extraction and smart scope prioritization. Refactor callout handling to avoid duplicates and improve processing efficiency. Update documentation to reflect changes in edge extraction logic and prioritization strategy.
Lars2026-01-10 22:17:03 +01:00
003a270548
Implement WP-24c v4.2.0: Introduce configurable header names and levels for LLM validation and Note-Scope zones in the chunking system. Update chunking models, parser, and processor to support exclusion of edge zones during chunking. Enhance documentation and configuration files to reflect new environment variables for improved flexibility in Markdown processing.
Lars2026-01-10 21:46:51 +01:00
39fd15b565
Update graph_db_adapter.py, graph_derive_edges.py, graph_subgraph.py, graph_utils.py, ingestion_processor.py, and retriever.py to version 4.1.0: Introduce Scope-Awareness and Section-Filtering features, enhancing edge retrieval and processing. Implement Note-Scope Zones extraction from Markdown, improve edge ID generation with target_section, and prioritize Note-Scope Links during de-duplication. Update documentation for clarity and consistency across modules.
Lars2026-01-10 19:55:51 +01:00
be2bed9927
Update qdrant_points.py, ingestion_processor.py, and import_markdown.py to version 4.1.0: Enhance edge ID generation by incorporating target_section for improved multigraph support and symmetry integrity. Update documentation and logging for clarity, ensuring consistent ID generation across phases and compatibility with the ingestion workflow.
Lars2026-01-10 17:03:44 +01:00
2da98e8e37
Update graph_derive_edges.py and graph_utils.py to version 4.1.0: Enhance edge ID generation by incorporating target_section into the ID calculation, allowing for distinct edges across different sections. Update documentation to reflect changes in ID structure and improve clarity on edge handling during de-duplication.
Lars2026-01-10 15:45:26 +01:00
a852975811
Update qdrant_points.py, graph_utils.py, graph_derive_edges.py, and ingestion_processor.py to version 4.0.0: Implement GOLD-STANDARD identity with strict 4-parameter ID generation, eliminating rule_id and variant from ID calculations. Enhance documentation for clarity and consistency across modules, addressing ID drift and ensuring compatibility in the ingestion workflow.
Lars2026-01-10 15:19:46 +01:00
8fd7ef804d
Update ingestion_processor.py to version 3.4.3: Remove incompatible edge_registry initialization, maintain strict two-phase strategy, and fix ID generation issues. Enhance logging and comments for clarity, ensuring compatibility and improved functionality in the ingestion workflow.
Lars2026-01-10 14:02:10 +01:00
b0f4309a29
Update qdrant_points.py, graph_utils.py, ingestion_processor.py, and import_markdown.py: Enhance ID generation and error handling, centralize identity logic to prevent ID drift, and improve documentation clarity. Update versioning to reflect changes in functionality and maintain compatibility across modules.
Lars2026-01-10 14:00:12 +01:00
c33b1c644a
Update graph_utils.py to version 1.6.1: Restore '_edge' function to address ImportError, revert to UUIDv5 for Qdrant compatibility, and maintain section logic in ID generation. Enhance documentation for clarity and refine edge ID generation process.
Lars2026-01-10 10:58:44 +01:00
7cc823e2f4
NEUSTART von vorne mit frischer Codebasis
Lars2026-01-10 10:56:47 +01:00
7e00344b84
Update ingestion_processor.py to version 3.3.8: Address Ghost-ID issues, enhance Pydantic safety, and improve logging clarity. Refine symmetry injection logic and ensure strict phase separation for authority checks. Adjust comments for better understanding and maintainability.
Lars2026-01-10 08:32:59 +01:00
ec89d83916
Update ingestion_db.py, ingestion_processor.py, and import_markdown.py: Enhance documentation and logging clarity, improve artifact purging and symmetry injection logic, and implement stricter authority checks. Update versioning to 2.6.0 and 3.3.7 to reflect changes in functionality and maintain compatibility with the ingestion service.
Lars2026-01-10 08:06:07 +01:00
57656bbaaf
Refactor ingestion_db.py and ingestion_processor.py: Enhance documentation and logging clarity, integrate cloud resilience and error handling, and improve artifact purging logic. Update versioning to 3.3.6 to reflect changes in functionality, including strict phase separation and authority checks for explicit edges.
Lars2026-01-10 07:45:43 +01:00
7953acf3ee
Update import_markdown.py to version 2.5.0: Implement global two-phase write strategy, enhance folder filtering to exclude system directories, and refine logging for improved clarity. Adjusted processing phases for better organization and error handling during markdown ingestion.
Lars2026-01-10 07:35:50 +01:00
3f528f2184
Refactor ingestion_db.py and ingestion_processor.py: Enhance documentation for clarity, improve symmetry injection logic, and refine artifact purging process. Update versioning to 3.3.5 to reflect changes in functionality and maintainability, ensuring robust handling of explicit edges and authority checks.
Lars2026-01-10 07:25:43 +01:00
29e334625e
Refactor ingestion_db.py and ingestion_processor.py: Simplify comments and documentation for clarity, enhance artifact purging logic to protect against accidental deletions, and improve symmetry injection process descriptions. Update versioning to reflect changes in functionality and maintainability.
Lars2026-01-10 06:54:11 +01:00
114cea80de
Update ingestion_processor.py to version 3.3.2: Implement two-phase write strategy and API compatibility fix, ensuring data authority for explicit edges. Enhance logging clarity and adjust batch import process to maintain compatibility with importer script. Refine comments for improved understanding and maintainability.
Lars2026-01-10 06:43:31 +01:00
981b0cba1f
Update ingestion_db.py and ingestion_processor.py to version 3.3.1: Enhance documentation for clarity, refine edge validation logic, and improve logging mechanisms. Implement strict separation of explicit writes and symmetry validation in the two-phase ingestion workflow, ensuring data authority and integrity. Adjust comments for better understanding and maintainability.
Lars2026-01-09 23:29:41 +01:00
e2c40666d1
Enhance ingestion_db.py and ingestion_processor.py: Integrate authority checks for Point-IDs and improve edge validation logic. Update logging mechanisms and refine batch import process with two-phase writing strategy. Adjust documentation for clarity and accuracy, reflecting version updates to 2.2.0 and 3.3.0 respectively.
Lars2026-01-09 23:25:57 +01:00
c9ae58725c
Update ingestion_processor.py to version 3.3.0: Integrate global authority mapping and enhance two-pass ingestion workflow. Improve logging mechanisms and edge validation logic, ensuring robust handling of explicit edges and authority protection. Adjust documentation for clarity and accuracy.
Lars2026-01-09 23:04:19 +01:00