Board
| [SCI/E 논문 게재] A Multi-Stage Framework for Story-Based Question Generation, Evaluation, and Cognitive Domain Assignment for Potential Cognitive Training Applications | |||
| 작성일 | 2026-10-02 | 조회수 | 9 |
|---|---|---|---|
| 첨부파일 | electronics-15-04518.pdf | ||
|
Title: A Multi-Stage Framework for Story-Based Question Generation, Evaluation, and Cognitive Domain Assignment for Potential Cognitive Training Applications Authors: Chanyi Lee, Sang Ho Oh Journal: Electronics Abstract: This study proposes a multi-stage framework for generating, evaluating, refining, and assigning cognitive domains to story-based candidate questions for potential cognitive training applications. Using 20 FairytaleQA stories, 1668 questions were initially generated, of which 1351 remained after quality evaluation, refinement, and Solver–Supervisor review. The framework combines criterion-based LLM evaluation, iterative self-refinement, supervisory review, evidence-grounded answerability assessment, and task-based multi-label cognitive-domain assignment. In the question-quality ablation study, Supervisor + Refinement achieved an 83.9% branching pass rate with 3.59 mean refinement iterations, compared with 68.7% and 4.63 iterations without the Evaluation Supervisor. Relative to Zero-shot generation, coherence, consistency, fluency, and relevance increased by 1.28, 1.21, 0.13, and 1.96 points, respectively. Independent evaluation with Granite and Phi-4-mini consistently showed improvements in consistency and relevance. For domain assignment, the Full Framework achieved inter-seed Jaccard, MASI, Exact Match, and Fleiss’ κ values of 0.917, 0.904, 0.883, and 0.874, respectively. These results support the framework’s utility for improving LLM-assessed question quality and assignment stability, while expert validation remains necessary to establish cognitive validity and training suitability.
|
|||
| 다음 | 다음 게시글이 없습니다. |
|---|---|
| 이전 | [국제 학술대회 발표] 최민서 학생 |