Enhance chunking system with WP-24c v4.2.6 and v4.2.7 updates: Introduce is_meta_content flag for callouts in RawBlock, ensuring they are chunked but later removed for clean context. Update parse_blocks and propagate_section_edges to handle callout edges with explicit provenance for chunk attribution. Implement clean-context logic to remove callout syntax post-processing, maintaining chunk integrity. Adjust get_chunk_config to prioritize frontmatter overrides for chunking profiles. Update documentation to reflect these changes.
This commit is contained in:
1 parent
4d43cc526e
commit
55b64c331a
8 files changed
+231
-43
No files matched your search
@@ -22,11 +22,13 @@ def propagate_section_edges(chunks: List[Chunk]) -> List[Chunk]:
|
||||
continue
|
||||
|
||||
# Nutzt den robusten Parser aus dem Package
|
||||
edges = parse_edges_robust(ch.text)
|
||||
if edges:
|
||||
# WP-24c v4.2.7: parse_edges_robust gibt jetzt Liste von Dicts zurück
|
||||
edge_infos = parse_edges_robust(ch.text)
|
||||
if edge_infos:
|
||||
if ch.section_path not in section_map:
|
||||
section_map[ch.section_path] = set()
|
||||
section_map[ch.section_path].update(edges)
|
||||
for edge_info in edge_infos:
|
||||
section_map[ch.section_path].add(edge_info["edge"])
|
||||
|
||||
# 2. Injizieren: Kanten in jeden Chunk der Sektion zurückschreiben (Broadcasting)
|
||||
for ch in chunks:
|
||||
@@ -37,7 +39,9 @@ def propagate_section_edges(chunks: List[Chunk]) -> List[Chunk]:
|
||||
|
||||
# Vorhandene Kanten (Typ:Ziel) in DIESEM Chunk ermitteln,
|
||||
# um Dopplungen (z.B. durch Callouts) zu vermeiden.
|
||||
existing_edges = parse_edges_robust(ch.text)
|
||||
# WP-24c v4.2.7: parse_edges_robust gibt jetzt Liste von Dicts zurück
|
||||
existing_edge_infos = parse_edges_robust(ch.text)
|
||||
existing_edges = {ei["edge"] for ei in existing_edge_infos}
|
||||
|
||||
injections = []
|
||||
# Sortierung für deterministische Ergebnisse
|
||||
|
||||
Reference in new issue
Block a user