Enhance Slot Evaluation and Scoring Mechanisms
Deploy Development / deploy (push) Successful in 46s
Test Suite / pytest-backend (push) Successful in 44s
Test Suite / lint-backend (push) Successful in 0s
Test Suite / build-frontend (push) Successful in 14s
Test Suite / k6 /health Baseline (push) Successful in 33s
Test Suite / playwright-tests (push) Successful in 1m26s

- Introduced new functions `_off_topic_semantic_scores_by_slot` and `_score_exercise_stage_fit_for_spec` to improve the evaluation of off-topic steps and exercise stage fit, enhancing the quality assessment process.
- Updated `_run_unified_slot_improvement_review` to incorporate off-topic scores and exercise stage fit scoring, refining the decision-making process for slot suggestions.
- Enhanced existing logic to streamline the handling of slot scores and improve the overall robustness of slot management in path evaluations.
This commit is contained in:
Lars committed 2026-06-13 12:33:16 +02:00
1 parent e9bf5bd1a5
commit cd457e3ea0
5 files changed
+654 -437

No files matched your search

@@ -2,6 +2,7 @@
from planning_exercise_path_builder import (
_parse_slot_refs_from_text,
_problematic_slots_from_path_qa,
_slot_auto_select_library,
_slot_suggestion_accepted,
)
from planning_progression_roadmap import StageSpecArtifact
@@ -95,3 +96,18 @@ def test_problematic_slots_from_llm_schritt_text():
specs = [_spec(7)]
problems = _problematic_slots_from_path_qa(qa, steps, specs)
assert 7 in problems
def test_slot_auto_select_requires_higher_score():
assert _slot_auto_select_library(
baseline_slot_score=0.5,
proposed_slot_score=0.51,
baseline_exercise_id=1,
proposed_exercise_id=2,
)
assert not _slot_auto_select_library(
baseline_slot_score=0.5,
proposed_slot_score=0.5,
baseline_exercise_id=1,
proposed_exercise_id=2,
)