Enhance Slot Evaluation and Scoring Mechanisms
Deploy Development / deploy (push) Successful in 46s
Test Suite / pytest-backend (push) Successful in 44s
Test Suite / lint-backend (push) Successful in 0s
Test Suite / build-frontend (push) Successful in 14s
Test Suite / k6 /health Baseline (push) Successful in 33s
Test Suite / playwright-tests (push) Successful in 1m26s
Deploy Development / deploy (push) Successful in 46s
Test Suite / pytest-backend (push) Successful in 44s
Test Suite / lint-backend (push) Successful in 0s
Test Suite / build-frontend (push) Successful in 14s
Test Suite / k6 /health Baseline (push) Successful in 33s
Test Suite / playwright-tests (push) Successful in 1m26s
- Introduced new functions `_off_topic_semantic_scores_by_slot` and `_score_exercise_stage_fit_for_spec` to improve the evaluation of off-topic steps and exercise stage fit, enhancing the quality assessment process. - Updated `_run_unified_slot_improvement_review` to incorporate off-topic scores and exercise stage fit scoring, refining the decision-making process for slot suggestions. - Enhanced existing logic to streamline the handling of slot scores and improve the overall robustness of slot management in path evaluations.
This commit is contained in:
1 parent
e9bf5bd1a5
commit
cd457e3ea0
5 files changed
+654
-437
No files matched your search
@@ -2,6 +2,7 @@
|
||||
from planning_exercise_path_builder import (
|
||||
_parse_slot_refs_from_text,
|
||||
_problematic_slots_from_path_qa,
|
||||
_slot_auto_select_library,
|
||||
_slot_suggestion_accepted,
|
||||
)
|
||||
from planning_progression_roadmap import StageSpecArtifact
|
||||
@@ -95,3 +96,18 @@ def test_problematic_slots_from_llm_schritt_text():
|
||||
specs = [_spec(7)]
|
||||
problems = _problematic_slots_from_path_qa(qa, steps, specs)
|
||||
assert 7 in problems
|
||||
|
||||
|
||||
def test_slot_auto_select_requires_higher_score():
|
||||
assert _slot_auto_select_library(
|
||||
baseline_slot_score=0.5,
|
||||
proposed_slot_score=0.51,
|
||||
baseline_exercise_id=1,
|
||||
proposed_exercise_id=2,
|
||||
)
|
||||
assert not _slot_auto_select_library(
|
||||
baseline_slot_score=0.5,
|
||||
proposed_slot_score=0.5,
|
||||
baseline_exercise_id=1,
|
||||
proposed_exercise_id=2,
|
||||
)
|
||||
Reference in new issue
Block a user