Enhance chunking system with WP-24c v4.2.6 and v4.2.7 updates: Introduce is_meta_content flag for callouts in RawBlock, ensuring they are chunked but later removed for clean context. Update parse_blocks and propagate_section_edges to handle callout edges with explicit provenance for chunk attribution. Implement clean-context logic to remove callout syntax post-processing, maintaining chunk integrity. Adjust get_chunk_config to prioritize frontmatter overrides for chunking profiles. Update documentation to reflect these changes.
This commit is contained in:
1 parent
4d43cc526e
commit
55b64c331a
8 files changed
+231
-43
No files matched your search
@@ -209,6 +209,7 @@ def build_edges_for_note(
|
||||
"""
|
||||
Erzeugt und aggregiert alle Kanten für eine Note.
|
||||
WP-24c v4.2.0: Unterstützt Note-Scope Extraktions-Zonen.
|
||||
WP-24c v4.2.7: Chunk-Attribution für Callouts über candidate_pool mit explicit:callout Provenance.
|
||||
|
||||
Args:
|
||||
note_id: ID der Note
|
||||
@@ -313,11 +314,17 @@ def build_edges_for_note(
|
||||
edges.append(_edge(k, "chunk", cid, t, note_id, payload))
|
||||
|
||||
# B. Candidate Pool (WP-15b Validierte KI-Kanten)
|
||||
# WP-24c v4.2.7: Sammle Callout-Keys für Chunk-Attribution
|
||||
pool = ch.get("candidate_pool") or ch.get("candidate_edges") or []
|
||||
for cand in pool:
|
||||
raw_t, k, p = cand.get("to"), cand.get("kind", "related_to"), cand.get("provenance", "semantic_ai")
|
||||
t, sec = parse_link_target(raw_t, note_id)
|
||||
if t:
|
||||
# WP-24c v4.2.7: Wenn Provenance explicit:callout, füge zu all_chunk_callout_keys hinzu
|
||||
# Dadurch weiß die globale Extraktion, dass diese Kante bereits auf Chunk-Ebene versorgt ist
|
||||
if p == "explicit:callout":
|
||||
all_chunk_callout_keys.add((k, t, sec))
|
||||
|
||||
# WP-24c v4.1.0: target_section fließt nun fest in die ID-Generierung ein
|
||||
payload = {
|
||||
"chunk_id": cid,
|
||||
|
||||
@@ -24,6 +24,7 @@ PROVENANCE_PRIORITY = {
|
||||
"explicit:wikilink": 1.00,
|
||||
"inline:rel": 0.95,
|
||||
"callout:edge": 0.90,
|
||||
"explicit:callout": 0.90, # WP-24c v4.2.7: Callout-Kanten aus candidate_pool
|
||||
"semantic_ai": 0.90, # Validierte KI-Kanten
|
||||
"structure:belongs_to": 1.00,
|
||||
"structure:order": 0.95, # next/prev
|
||||
|
||||
Reference in new issue
Block a user