Commit Graph

563 Commits

Author SHA1 Message Date
7e00344b84 Update ingestion_processor.py to version 3.3.8: Address Ghost-ID issues, enhance Pydantic safety, and improve logging clarity. Refine symmetry injection logic and ensure strict phase separation for authority checks. Adjust comments for better understanding and maintainability. 2026-01-10 08:32:59 +01:00
ec89d83916 Update ingestion_db.py, ingestion_processor.py, and import_markdown.py: Enhance documentation and logging clarity, improve artifact purging and symmetry injection logic, and implement stricter authority checks. Update versioning to 2.6.0 and 3.3.7 to reflect changes in functionality and maintain compatibility with the ingestion service. 2026-01-10 08:06:07 +01:00
57656bbaaf Refactor ingestion_db.py and ingestion_processor.py: Enhance documentation and logging clarity, integrate cloud resilience and error handling, and improve artifact purging logic. Update versioning to 3.3.6 to reflect changes in functionality, including strict phase separation and authority checks for explicit edges. 2026-01-10 07:45:43 +01:00
3f528f2184 Refactor ingestion_db.py and ingestion_processor.py: Enhance documentation for clarity, improve symmetry injection logic, and refine artifact purging process. Update versioning to 3.3.5 to reflect changes in functionality and maintainability, ensuring robust handling of explicit edges and authority checks. 2026-01-10 07:25:43 +01:00
29e334625e Refactor ingestion_db.py and ingestion_processor.py: Simplify comments and documentation for clarity, enhance artifact purging logic to protect against accidental deletions, and improve symmetry injection process descriptions. Update versioning to reflect changes in functionality and maintainability. 2026-01-10 06:54:11 +01:00
114cea80de Update ingestion_processor.py to version 3.3.2: Implement two-phase write strategy and API compatibility fix, ensuring data authority for explicit edges. Enhance logging clarity and adjust batch import process to maintain compatibility with importer script. Refine comments for improved understanding and maintainability. 2026-01-10 06:43:31 +01:00
981b0cba1f Update ingestion_db.py and ingestion_processor.py to version 3.3.1: Enhance documentation for clarity, refine edge validation logic, and improve logging mechanisms. Implement strict separation of explicit writes and symmetry validation in the two-phase ingestion workflow, ensuring data authority and integrity. Adjust comments for better understanding and maintainability. 2026-01-09 23:29:41 +01:00
e2c40666d1 Enhance ingestion_db.py and ingestion_processor.py: Integrate authority checks for Point-IDs and improve edge validation logic. Update logging mechanisms and refine batch import process with two-phase writing strategy. Adjust documentation for clarity and accuracy, reflecting version updates to 2.2.0 and 3.3.0 respectively. 2026-01-09 23:25:57 +01:00
c9ae58725c Update ingestion_processor.py to version 3.3.0: Integrate global authority mapping and enhance two-pass ingestion workflow. Improve logging mechanisms and edge validation logic, ensuring robust handling of explicit edges and authority protection. Adjust documentation for clarity and accuracy. 2026-01-09 23:04:19 +01:00
4318395c83 Update ingestion_db.py and ingestion_processor.py: Refine documentation and enhance logging mechanisms. Improve edge validation logic with robust ID resolution and clarify comments for better understanding. Version updates to 2.2.1 and 3.2.1 respectively. 2026-01-09 22:35:04 +01:00
00264a9653 Refactor ingestion_processor.py for version 3.2.0: Integrate Mixture of Experts architecture, enhance logging stability, and improve edge validation. Update batch import process with symmetry memory and modularized schema logic. Adjust documentation for clarity and robustness. 2026-01-09 22:23:10 +01:00
7e4ea670b1 Update ingestion_processor.py to version 3.2.0: Enhance logging stability and improve edge validation by addressing KeyError risks. Implement batch import with symmetry memory and modularized schema logic for explicit edge handling. Adjust documentation and versioning for improved clarity and robustness. 2026-01-09 22:15:14 +01:00
008a470f02 Refactor graph_utils.py and ingestion_processor.py: Update documentation for deterministic UUIDs to enhance Qdrant compatibility. Improve logging and ID validation in ingestion_processor.py, including adjustments to edge processing logic and batch import handling for better clarity and robustness. Version updates to 1.2.0 and 3.1.9 respectively. 2026-01-09 22:05:50 +01:00
7ed82ad82e Update graph_utils.py and ingestion_processor.py to versions 1.2.0 and 3.1.9 respectively: Transition to deterministic UUIDs for edge ID generation to ensure Qdrant compatibility and prevent HTTP 400 errors. Enhance ID validation and streamline edge processing logic to improve robustness and prevent collisions with known system types. Adjust versioning and documentation accordingly. 2026-01-09 21:46:47 +01:00
72cf71fa87 Update ingestion_processor.py to version 3.1.8: Enhance ID validation to prevent HTTP 400 errors and improve edge generation robustness by excluding known system types. Refactor edge processing logic to ensure valid note IDs and streamline database interactions. Adjust versioning and documentation accordingly. 2026-01-09 21:41:53 +01:00
9cb08777fa Update ingestion_processor.py to version 3.1.7: Enhance authority enforcement for explicit edges by implementing runtime ID protection and database checks to prevent overwriting. Refactor edge generation logic to ensure strict authority compliance and improve symmetry handling. Adjust versioning and documentation accordingly. 2026-01-09 21:31:44 +01:00
2c18f8b3de Update ingestion_db.py and ingestion_processor.py to version 2.2.0 and 3.1.6 respectively: Integrate authority checks for Point-IDs and enhance edge validation logic to prevent overwriting explicit edges by virtual symmetries. Introduce new function to verify explicit edge presence in the database, ensuring improved integrity in edge generation. Adjust versioning and documentation accordingly. 2026-01-09 21:07:02 +01:00
d5d6987ce2 Update ingestion_processor.py to version 3.1.5: Implement database-aware redundancy checks to prevent overwriting explicit edges by virtual symmetries. Enhance edge validation logic to include real-time database queries, ensuring improved integrity in edge generation. Adjust versioning and documentation accordingly. 2026-01-09 20:27:45 +01:00
61a319a049 Update ingestion_processor.py to version 3.1.4: Implement semantic cross-note redundancy checks to enhance edge generation logic. Refactor redundancy validation to distinguish between local and cross-note redundancies, ensuring improved bidirectional graph integrity. Adjust versioning and documentation accordingly. 2026-01-09 18:41:05 +01:00
a392dc2786 Update type_registry, graph_utils, ingestion_note_payload, and discovery services for dynamic edge handling: Integrate EdgeRegistry for improved edge defaults and topology management (WP-24c). Enhance type loading and edge resolution logic to ensure backward compatibility while transitioning to a more robust architecture. Version bumps to 1.1.0 for type_registry, 1.1.0 for graph_utils, 2.5.0 for ingestion_note_payload, and 1.1.0 for discovery service. 2026-01-09 15:20:12 +01:00
5e2a074019 Implement origin-based purge logic in ingestion_db.py to prevent accidental deletion of inverse edges during re-imports. Enhance logging for error handling and artifact checks. Update ingestion_processor.py to support redundancy checks and improve symmetry logic for edge generation, ensuring bidirectional graph integrity. Version bump to 3.1.2. 2026-01-09 14:41:50 +01:00
9b3fd7723e Update ingestion processor to version 3.1.0: Fix bidirectional edge injection for Qdrant, streamline edge validation by removing symmetry logic from the validation step, and enhance inverse edge generation in the processing pipeline. Improve logging for symmetry creation in edge payloads. 2026-01-09 14:25:46 +01:00
4802eba27b Integrate symmetric edge logic and discovery API: Update ingestion processor and validation to support automatic inverse edge generation. Enhance edge registry for dual vocabulary and schema management. Introduce new discovery endpoint for proactive edge suggestions, improving graph topology and edge validation processes. 2026-01-09 13:57:10 +01:00
d17c966301 Enhance callout extraction in graph_extractors.py: Update regex to support nested [!edge] callouts and improve handling of indentation levels. This allows for more flexible parsing of callout structures in the input text.
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2026-01-06 10:20:55 +01:00
8505538b34 Refactor ingestion validation and decision engine error handling: Differentiate between transient and permanent validation errors in ingestion validation to improve data integrity. Enhance decision engine configuration loading with schema validation and error handling for missing keys and YAML syntax errors. Update fallback synthesis prompt handling in LLMService for improved error recovery. Add new fallback synthesis prompts to prompts.yaml for better context-based responses. 2026-01-02 22:09:16 +01:00
a9d0874fe9 Enhance prompt retrieval in LLMService: Implement detailed trace-logging for prompt lookup hierarchy, improving traceability of model-specific, provider, and global fallback matches. This update refines the logging mechanism to provide clearer insights during prompt resolution. 2026-01-02 21:47:47 +01:00
1563ebbdf9 Update Decision Engine to version 1.3.2: Implement ultra-robust intent parsing using regex, restore prepend_instruction logic, and enhance logging for configuration loading. Improve fallback mechanisms for response generation to ensure reliability. 2026-01-02 21:42:09 +01:00
38fac89f73 Update Decision Engine for WP-25b: Enhance intent processing with robust intent cleaning and lazy loading. Improve strategy determination by validating against known strategies and streamline response generation. Bump version to 1.3.1 to reflect these optimizations. 2026-01-02 21:35:02 +01:00
7026fc4fed Update components for WP-25b: Implement Lazy-Prompt-Orchestration across ingestion, decision engine, chat interface, and LLM service. Enhance prompt management with hierarchical model support and streamline response generation by removing manual formatting. Bump versions to reflect new features and optimizations. 2026-01-02 20:43:31 +01:00
9b906bbabf Update FastAPI application and related services for WP-25a: Enhance lifespan management with Mixture of Experts (MoE) integrity checks, improve logging and error handling in LLMService, and integrate profile-driven orchestration across components. Bump versions for main application, ingestion services, and LLM profiles to reflect new features and optimizations. 2026-01-02 08:57:29 +01:00
9a98093e70 Enhance Decision Engine configuration: Add 'router_profile' setting to decision_engine.yaml and update the DecisionEngine class to utilize this profile when generating responses, improving flexibility in strategy determination. 2026-01-02 07:45:34 +01:00
f62983b08f Enhance logging in LLMService: Update log messages for MoE dispatch and default provider usage, and add detailed logging before OpenRouter calls for improved traceability and debugging. 2026-01-02 07:26:24 +01:00
d0eae8e43c Update Decision Engine and related components for WP-25a: Bump version to 1.2.0, enhance multi-stream retrieval with pre-synthesis compression, and integrate Mixture of Experts (MoE) profile support. Refactor chat interface to utilize new compression logic and llm_profiles for improved synthesis. Maintain compatibility with existing methods and ensure robust error handling across services. 2026-01-02 07:04:43 +01:00
ea38743a2a Update Decision Engine and DTOs for WP-25: Bump version to 1.0.3 and 0.7.1 respectively. Introduce stream tracing support by adding 'stream_origin' to QueryHit model. Enhance robustness with pre-initialization of stream variables and improve logging for unknown strategies. Refactor prompts for clarity and consistency in multi-stream synthesis. 2026-01-01 12:38:32 +01:00
ed3f3e5588 Update Decision Engine for WP-25: Bump version to 1.0.2, enhance robustness by pre-initializing stream variables to prevent KeyErrors, and fix template mismatches in strategy definitions. Ensure compatibility with updated YAML configuration for multi-stream retrieval strategies. 2026-01-01 08:51:33 +01:00
bb6959a090 Update LLMService for WP-25: Enhance stability with improved response handling, including safeguards against empty responses and adjustments for short input validations. Maintain compatibility with previous logic for rate limits and retries. Version bump to 3.4.2. 2026-01-01 08:31:15 +01:00
d49d509451 Initialize logging setup in main application prior to app creation for improved error tracking and management. 2026-01-01 08:09:20 +01:00
008167268f Update main application and services for WP-25 release, introducing Agentic Multi-Stream RAG capabilities. Enhance lifespan management, global error handling, and integrate LLMService with DecisionEngine for improved retrieval and synthesis. Update dependencies and versioning across modules, ensuring compatibility with new multi-stream architecture. Refactor chat router to support new intent classification and retrieval strategies, while maintaining stability and performance improvements. 2026-01-01 07:52:41 +01:00
d35bdc64b9 Implement WP-15c enhancements across graph and retrieval modules, including full metadata support for Super-Edge aggregation and Note-Level Diversity Pooling. Update scoring logic to reflect new edge handling and improve retrieval accuracy. Version updates to reflect these changes. 2025-12-30 21:47:18 +01:00
4327fc939c zrück zur Vorversion zum Test der LLM checks 2025-12-30 09:40:30 +01:00
ef1046c6f5 Enhance callout relation extraction by ensuring correct termination on new headers. Update regex for simple kinds to support hyphens. Refactor block processing logic for improved clarity and functionality. 2025-12-30 09:26:38 +01:00
ef8cf719f2 Update ingestion processor to version 2.13.12, synchronizing profile resolution with registry defaults. Refactor profile retrieval logic to utilize the profile determined in make_note_payload, ensuring consistency in chunk configuration. 2025-12-30 09:11:55 +01:00
6aa6b32a6c Update chunking system to version 3.9.9, synchronizing parameters with the orchestrator and enhancing edge detection. Implement robust parsing to prevent duplicate edges in section propagation. Adjust comments for clarity and consistency across the codebase. 2025-12-30 08:40:19 +01:00
33dff04d47 Fix v3.3.5: Prevent duplicate Wikilink targets in text by checking for existing references before injecting section edges. Update comments for clarity and maintain consistency in the code structure. 2025-12-30 08:22:17 +01:00
65d697b7be Aktualisierung der atomaren Sektions-Chunking-Strategie auf Version 3.9.8 mit verbesserten Implementierungen des 'Pack-and-Carry-Over' Verfahrens. Einführung von Look-Ahead zur strikten Einhaltung von Sektionsgrenzen und Vermeidung redundanter Kanten-Injektionen. Anpassungen an der Chunk-Erstellung und Optimierung der Handhabung von leeren Überschriften. 2025-12-30 07:54:54 +01:00
06fc42ed37 Aktualisierung des Chunking-Parsers zur Einführung der Funktion parse_edges_robust zur Extraktion von Kanten-Kandidaten aus Wikilinks und Callouts. Verbesserung der Satzverarbeitung durch die Implementierung der Funktion split_sentences. Aktualisierung der Sektions-Chunking-Strategie auf Version 3.9.6 mit optimierter Handhabung von leeren Überschriften und Carry-Over Logik zur besseren Chunk-Erstellung. 2025-12-30 07:44:30 +01:00
3c5c567077 Aktualisierung der atomaren Sektions-Chunking-Strategie auf Version 3.9.5 mit Implementierung des 'Pack-and-Carry-Over' Verfahrens. Einführung neuer Konfigurationsoptionen für Smart-Edge und strikte Überschriftenteilung. Verbesserte Handhabung von leeren Überschriften und Anpassungen an der Warteschlangen-Verarbeitung zur Optimierung der Chunk-Erstellung. 2025-12-30 07:41:30 +01:00
8f65e550c8 Optimierung des Chunking-Parsers zur Unterstützung atomarer Blöcke und Verbesserung der Satzverarbeitung. Aktualisierung der Sektions-Chunking-Strategie auf Version 3.9.0 mit regelkonformer Implementierung und Anpassungen an der Warteschlangen-Verarbeitung für Carry-Over. Verbesserte Handhabung von Überschriften und Metadaten zur Gewährleistung der strukturellen Integrität. 2025-12-29 22:16:12 +01:00
6b83879741 Aktualisierung des Chunking-Parsers zur Verbesserung der Satzverarbeitung und Blocktrennung. Einführung des 'Pack-and-Carry-Over' Verfahrens in der Sektions-Chunking-Strategie zur Optimierung der Handhabung von großen Sektionen und Gewährleistung der Sektionsintegrität. Anpassungen an der Token-Schätzung und Verbesserung der Metadatenverarbeitung. 2025-12-29 22:04:23 +01:00
be265e9cc0 Verbesserung des Chunking-Parsers zur Unterstützung atomarer Blöcke und Gewährleistung der strukturellen Integrität von Callouts. Aktualisierung der Beschreibung und Optimierung der Satz- und Blockverarbeitung, einschließlich präziserer Handhabung von H1-Überschriften und Trennern. 2025-12-29 21:48:54 +01:00
680c36ab59 Aktualisierung des Chunking-Parsers zur Verbesserung der Blockverarbeitung und Beschreibung. Anpassungen an der atomaren Sektions-Chunking-Strategie mit optimierter Token-Schätzung und neuen Hilfsfunktionen zur besseren Handhabung von großen Sektionen. Einführung einer präziseren Schätzung für deutsche Texte und Anpassungen an der Logik zur Handhabung von Sektionen. 2025-12-29 21:45:14 +01:00
96b4f65cd1 Aktualisierung des Chunking-Parsers zur Verbesserung der Blockverarbeitung und Beschreibung. Anpassungen an der atomaren Sektions-Chunking-Strategie mit optimierter Token-Schätzung und neuen Hilfsfunktionen zur besseren Handhabung von großen Sektionen. 2025-12-29 21:37:11 +01:00
b1a897e51c Verbesserung des Chunking-Parsers zur Unterstützung aller Überschriften (H1-H6) und Optimierung der Block-Trennung für atomares Sektions-Chunking. Aktualisierung der Sektions-Chunking-Strategie mit striktem Look-Ahead und präziserer Token-Schätzung für eine verbesserte Handhabung von großen Blöcken. 2025-12-29 21:26:05 +01:00
e5a34efee9 Verbesserung des Chunking-Parsers zur Gewährleistung der Integrität von Callouts und Listen sowie Anpassungen an der Blockverarbeitung. Aktualisierung der atomaren Sektions-Chunking-Strategie mit Block-Aware-Flushing und optimierter Token-Schätzung für eine präzisere Handhabung von großen Blöcken. 2025-12-29 21:15:03 +01:00
f9ac4e4dbf Verbesserung der atomaren Sektions-Chunking-Strategie durch Einführung strikter Look-Ahead-Logik und präventiven Flush zur Gewährleistung von Sektionsgrenzen. Anpassungen an der Token-Schätzung und Umbenennung von Funktionen zur besseren Lesbarkeit. 2025-12-29 21:05:42 +01:00
1b40e29f40 Optimierung des Chunking-Parsers zur Unterstützung atomares Chunking und verbesserte Block-Trennung. Anpassungen an der Sektions-Chunking-Strategie zur Wahrung von Sektionsgrenzen und Vermeidung von Überhängen. 2025-12-29 20:57:07 +01:00
7eba1fb487 Aktualisierung des Chunking-Parsers zur Unterstützung aller Überschriften im Stream und Verbesserung der Metadatenverarbeitung. Anpassungen an der atomaren Sektions-Chunking-Strategie zur besseren Handhabung von Blockinhalten und Token-Schätzungen. 2025-12-29 20:45:04 +01:00
838083b909 Verbesserung des Chunking-Parsers zur Unterstützung von H1-Überschriften und Anpassung der Metadatenlogik. Implementierung einer atomaren Sektions-Chunking-Strategie, die Überschriften und deren Inhalte zusammenhält. 2025-12-29 20:33:43 +01:00
8f5eb36b5f neuer Chunking parser, der Headings mitführt und nicht mitten im Abschnitt schneidet 2025-12-29 20:16:23 +01:00
b7d1bcce3d Rücksprung zur Vorwersion, in der 2 Kantentypen angelegt wurden 2025-12-29 18:04:14 +01:00
03d3173ca6 neu deduplizierung für callout-edges 2025-12-29 12:42:26 +01:00
38a61d7b50 Fix: Semantische Deduplizierung in graph_derive_edges.py 2025-12-29 12:21:57 +01:00
0a429e1f7b anpassungen Kantenvergeleich 2025-12-29 11:45:25 +01:00
857ba953e3 bug fix 2025-12-29 11:00:00 +01:00
e180018c99 Anpassung gui 2025-12-29 10:31:51 +01:00
ac9956bf00 Index und Anlage neues Feld in qdrant 2025-12-29 10:16:51 +01:00
62b5a8bf65 Anpassung payload für neues Feld in edges 2025-12-29 08:40:05 +01:00
303efefcb7 bug fix 2025-12-29 08:19:40 +01:00
feeb7c2d92 Initial WP4d 2025-12-29 07:58:20 +01:00
ea9a54421a ui_fraph.old Version 2025-12-29 07:51:07 +01:00
fdf99b2bb0 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 18:53:11 +01:00
c7cd641f89 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 18:51:44 +01:00
18b90c8df3 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 18:16:29 +01:00
8d3bc1c2e2 next try
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 12:00:31 +01:00
079d988034 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2025-12-28 11:57:49 +01:00
aa9d388337 ui_update
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:54:44 +01:00
92bd3d9a47 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2025-12-28 11:52:49 +01:00
53058d1504 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:49:41 +01:00
3fe8463a03 bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:44:04 +01:00
5c4ce5d727 neuer test
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2025-12-28 11:41:47 +01:00
459193e7b1 test ui
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:38:51 +01:00
98f21323fb bug fix
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2025-12-28 11:35:18 +01:00
515248d438 kanten mit #Abschnitt finden
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:30:22 +01:00
b0c69ad3e0 UI test
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:23:46 +01:00
c5f29ab4ae Erweiterung der Kanten um Abschnittsinformationen
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2025-12-28 11:20:30 +01:00
876ee898d8 Ui_Update_cytos
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 3s
2025-12-28 11:15:11 +01:00
e93bab6ea7 Fassadenauflösung unter app/core
All checks were successful
Deploy mindnet to llm-node / deploy (push) Successful in 4s
2025-12-28 11:04:40 +01:00
7fa9ce81bd letzte anpassungen 2025-12-27 20:30:24 +01:00
8490911958 modularisierung 2025-12-27 20:26:00 +01:00
19d899b277 Große Modularisierung WP19b 2025-12-27 19:47:23 +01:00
37ec8b614e bug fix 2025-12-27 19:12:14 +01:00
e045371969 Anpassung der Textausgabe zur Filterung der Steuerzeichen 2025-12-27 18:59:38 +01:00
cd5383432e Parametrisierung der wesentliche Einstellwerte in der types.yaml 2025-12-27 18:45:15 +01:00
8b8baa27b3 W19b flexible Level Überschriften 2025-12-27 18:31:00 +01:00
386fa3ef0c WP15b vollständieg chunking strategien 2025-12-27 18:17:13 +01:00
19c96fd00f graph refacturiert 2025-12-27 14:44:44 +01:00
ecb35fb869 parser refactured WP15b 2025-12-27 14:31:12 +01:00
21cda0072a refacturing parser 2025-12-27 14:26:42 +01:00
e3858e8bc3 aufräumen und löschen von Alt-Scripten WP19b 2025-12-27 14:15:22 +01:00
f08a331bc6 herstellung vollständiger Kompaitibilität 2025-12-27 13:20:37 +01:00