- Introduced `_normalize_mw_category` function to clean category names for API calls, ensuring consistent handling of category prefixes.
- Updated `SmwClient` methods to utilize normalized category names, improving data retrieval accuracy.
- Added `_wiki_category_or_default` function to provide default categories based on import type, enhancing user experience during imports.
- Integrated new fields `karate_relevance` and `relevance_level` into various admin components, allowing for better skill management.
- Incremented app version to 0.8.145 and updated changelog to reflect these changes.
- Added `karate_relevance` and `relevance_level` fields to the SkillCreate and SkillResponse models, allowing for more detailed skill attributes.
- Updated the SMW property mapping to include these new fields, facilitating their integration during data import.
- Implemented parsing logic for relevance levels from Wiki data, ensuring proper handling of values between 1 and 3.
- Modified the upsert and create skill functions to support the new fields, ensuring they are correctly stored and updated in the database.
- Incremented app version to 0.8.143 and updated changelog to reflect these changes.
- Added MediaWiki API configuration options to docker-compose.yml for production environment.
- Improved error handling in the SMW client by introducing a response parsing method to manage API errors more effectively.
- Updated methods in the SMW client to ensure robust handling of API responses, preventing potential crashes due to unexpected data structures.
- Enhanced the import functionality to handle last imported timestamps more gracefully.
- Added support for style direction mappings in the backend, allowing for improved categorization of exercises.
- Introduced a new function to normalize property synonyms, enhancing the mapping of exercise properties.
- Updated the exercise catalog assignment logic to include style directions, ensuring proper database entries.
- Enhanced the ExercisesListPage with new filtering options for style directions, improving user experience and search capabilities.
- Added imported_by parameter to _run_import and _upsert_exercise
- Exercises now automatically get created_by set to importing user
- Fixes visibility issue where imported exercises were invisible
Related: Wiki import - exercise visibility fix
Problem 1: Reimport fails completely (all exercises fail)
Root cause: Line 465 existing[0] instead of existing['id']
Fix: Changed to existing['id'] for dict-cursor compatibility
Problem 2: 221 exercises skipped, only 2 in database
Root cause: Orphaned wiki_import_references from failed imports
Fix: New admin endpoint to manually cleanup orphaned references
New endpoints:
- DELETE /api/import/mediawiki/admin/cleanup-orphaned-references
Deletes all references where local_id doesn't exist in target table
Returns count of deleted references per type
- GET /api/import/mediawiki/admin/stats
Shows references vs. actual count + orphaned count
Helps diagnose import state issues
Usage:
1. Call cleanup endpoint to remove orphaned references
2. Run normal import (reimport=false)
3. Should import all previously failed exercises
Problem: 221 exercises skipped, but only 2 actually in database
Root cause: Failed imports created wiki_import_references entries,
but exercises were never saved (due to earlier bugs)
Duplicate check only verified references existed, not actual data.
Result: All subsequent imports skipped 'already imported' items.
Solution:
- Enhanced duplicate check to verify BOTH reference AND actual entity
- If reference exists but entity is missing → delete orphaned reference
- Item will be re-imported on same run (no manual cleanup needed)
Check logic:
1. Find reference in wiki_import_references
2. Check if exercises/skills/methods table has matching local_id
3. If both exist → skip (true duplicate)
4. If reference exists but entity missing → delete reference + re-import
This auto-heals broken state from previous failed imports.
Error: 100 exercises failed with error '0' (TypeError suppressed)
Root cause: More row[0] dict access in catalog assignment functions
Previous fix only covered fetchone()[0] in RETURNING clauses.
Missed 6 more locations in SELECT queries where row is dict-like.
Fixed locations:
- Line 511: focus_area_id in exercise_focus_areas INSERT
- Line 524: style_direction_id in exercise_training_styles INSERT
- Line 535: target_group_id in exercise_target_groups INSERT
- Line 546: skill_id in exercise_skills INSERT
- Line 568: skill_id in exercise_skills INSERT (with levels)
- Line 589: category_id assignment in _upsert_skill
All changed from row[0] to row['id'] for RealDictCursor compatibility.
This should fix the remaining 100 import failures.
Error: 500 Internal Server Error on execute import
Root cause: Using fetchone()[0] with RealDictCursor (dict-like rows)
PostgreSQL RealDictCursor returns dict-like objects, not tuples.
Accessing [0] on a dict throws TypeError.
Fix: Changed all fetchone()[0] to fetchone()['id']
Locations:
- Line 163: log_id after INSERT INTO wiki_import_log
- Line 485: ex_id after INSERT INTO exercises
- Line 599: skill_id after INSERT INTO skills
- Line 616: method_id after INSERT INTO training_methods
This matches the pattern used in other routers (exercises.py, etc.)
- Implemented a new SQL migration for wiki import tracking tables.
- Created an import router for handling MediaWiki imports of exercises, skills, and methods.
- Developed a Semantic MediaWiki API client for direct API interactions.
- Added a mapper to convert SMW properties to local database fields.
- Introduced background tasks for asynchronous import processing.
- Implemented logging and error handling for import operations.
- Added endpoints for previewing imports, checking import status, and managing import references.
