2026· Bulletin Sukhoi State Technical University of Gomel· 0 citations
TL;DR
An algorithm and a prototype software system based on large language models are proposed to automate the creation and verification of unique assignments for laboratory work in information technology disciplines and makes it possible to significantly reduce instructors' time expenditure while preserving pedagogical control over the content of the educational process.
Abstract
An algorithm and a prototype software system based on large language models are proposed to automate the creation and verification of unique assignments for laboratory work in information technology disciplines. The solution is based on a three-agent pipeline that provides the generation of a reference master assignment, parametric variation of unique task instances, complexity control, and automatic error correction. An evaluation of the efficiency and correctness of the proposed solution was carried out. The testing results confirmed the high quality of the generated laboratory assignments and compliance with the requirements for the uniqueness of task variants. It was noted that the proposed solution makes it possible to significantly reduce instructors' time expenditure while preserving pedagogical control over the content of the educational process
Recent developments in Large Language Models (LLMs) have created new opportunities to automate educational assessment and reduce workload for instructors. However, concerns regarding grading consistency, transparency, and pedagogical reliability continue to limit the adoption of fully automated assessment systems. In this study, we propose a Human-in-the-Loop framework for the automated evaluation of engineering laboratory reports based on Retrieval-Augmented Generation (RAG). The proposed framework integrates text extraction, structured content extraction, contextual retrieval from grading rubrics and laboratory resources, rubric-based evaluation, automatic feedback generation, and instructor validation into a unified grading workflow. The RAG module retrieves contextual information so that the language model can generate assessments that align with the course goals and are based on educational information about the specific assignments, thus ensuring consistent evaluation based on the rubric. The system was tested using a set of 56 laboratory reports collected from undergraduate courses in Electrical and Electronics Engineering. The experimental results indicate a strong agreement between the grades provided by the AI and the instructor, with a Pearson Correlation Coefficient of 0.988, a Mean Absolute Error (MAE) of 3.55, and a Root Mean Squared Error (RMSE) of 3.77. Besides, 89.29% of the reports were scored within ±5 points of the instructor scores. The grading time was reduced from 392 minutes to 84 minutes, a workload reduction of 78.57%. The results demonstrate that the integration of Retrieval-Augmented Generation, rubric-based evaluation, and Human-in-the-Loop validation constitutes an effective approach for AI-supported assessment of engineering laboratory reports, maintaining instructor oversight and educational integrity.
Amina Abbi, M. Skouri, M. Raoufi· International Journal of Adv...· 0 citations
A comparative analysis of six LLMs for generating formative feedback on introductory Java programs containing predefined defects under controlled conditions reveals substantial cross-model variation, particularly in multi-defect scenarios.
Melina Najimi, Saba Yazdani, Marzieh Ahmadzadeh· Proceedings of the Canadian...· 0 citations
This paper introduces CodeOwl, an AI-driven tool that automates the generation of tiered Parsons problems automatically, and evaluated CodeOwl with a mixed-method framework comprising complexity analysis, expert ratings, and user studies.
Luca Cisternino, Florian Obermuller, Gordon Fraser· arXiv.org· 0 citations
The time complexity of algorithms is a critical concept in computer science and engineering, and it is recognized as a fundamental topic in the Association for Computing Machinery (ACM) curricular recommendations. In alignment with these guidelines, numerous universities worldwide incorporate this topic into their introductory computer science courses, which typically have large student enrollments annually. Consequently, there is an increasing demand for automation in both instructional and assessment processes. This study explores the potential of Large Language Models (LLMs) to assist teaching staff in generating source code segments with predefined time complexity and determining the time complexity of given code segments, with applications in educational and examination contexts. We proposed a novel methodology for LLM evaluation in the aforementioned context and evaluated three prominent LLMs: ChatGPT, Gemini, and Llama, on their ability to generate and analyze C code segments exhibiting linear, logarithmic, quadratic, and exponential time complexities. A framework was developed to automate the prompt and segment generation and time complexity determination using two mainstream prompt engineering methods: zero-shot and chain-of-thought, and assessed the differences in code generation and time complexity analysis. A total of 960 generated segments were assessed on the correctness of time complexity, structural appropriateness, and suitability for exam use. The results suggest that ChatGPT is the most suitable LLM for generating segments with predefined time complexity (success rate goes up to 61%). All LLMs yielded the best results in generating linear segments, while exponential complexity posed the greatest challenge overall. A subset of generated segments was extracted to evaluate the time complexity determination capabilities. All three LLMs were asked to find the time complexity of each extracted segment. The most accurate LLM is ChatGPT (79.6%). We also assessed how good each LLM is in determining the time complexity of segments generated by itself. Llama outperforms others in that task (83% of successful determinations) when the zero-shot prompt method is used. The findings suggest that current LLMs cannot fully automate question generation and time complexity problem solving. However, they can substantially support the process and reduce the workload for educators.
Đorđe Pešić, Milena Vujošević Janičić, Marko Mišić et al.· PeerJ Computer Science· 0 citations
Aim.
To present a set of requirements helping to visualise AI results for monitoring the compliance of educational materials with regulatory requirements, ensuring their effective integration into the practical activities of a teacher.
Methodology.
The research is based on the analysis of regulatory legal documents, as well as on the theoretical understanding of approaches and principles outlined in works devoted to Explainable artificial intelligence (XAI), the digital educational environment, and pedagogical design. The key method was system analysis, which made it possible to synthesise requirements for the interface of the educational content verification system from various sources.
Results.
A set of requirements for displaying the results of educational materials verification has been formulated and substantiated, including: a multi-window interface for simultaneously displaying the lesson plan, identified inconsistencies, regulatory references, and AI recommendations, high-precision rendering of mathematical expressions, transparent visualization of AI (XAI) logic to enhance teacher trust, integrated symbolic and spatial representation of educational content, cognitive load management through visual analytics, and engagement monitoring features and risk management indicators.
Research implications.
The theoretical significance of the work lies in the systematisation and adaptation of XAI requirements and provisions of national standards for the design of educational products with AI algorithms to the task of constructing user interfaces for educational systems using AI technologies. The practical significance consists in the development of a set of specific parameters (resolution, pixel density, colour space, etc.) that can be used in the creation and selection of hardware and software tools for the implementation of intelligent systems in the educational process, increasing their transparency and , the level of trust on the part of teachers as a consequence.
Conclusions.
The level of compliance of educational materials developed on the basis of artificial intelligence systems and the effectiveness of their application in the educational process are determined not only by the quality of their algorithms, but also by the thoughtfulness of the user interface. The work formulates and substantiates a set of requirements for the visualisation of the results of such systems, integrating the provisions of Explainable AI, national standards for the use of AI in education, and modern principles of pedagogical design. This set can serve as a regulatory basis for the design of interfaces of educational AI systems.
V. Belyaev, Yu. Obydenkov, A. Rulev· Moscow Pedagogical Journal· 0 citations
Test-based learning is effective in fostering knowledge retention, but manually creating assessment questions remains time-consuming and limits personalized student practice. The advent of Large Language Models (LLMs) has introduced new possibilities for Automatic Question Generation (AQG). Motivated by this context, this survey provides a comprehensive overview of AQG using LLMs, focusing on educational applications. Following the PRISMA methodology, we reviewed 132 studies published between 2023 and 2025. Our contributions include a taxonomy of question types by response openness, an analysis of AQG efforts across knowledge fields, educational levels, evaluation strategies, and difficulty control. We also identify recurring challenges and research opportunities.
Roberto Oliveira, A. Hernández, M. Garbin et al.· ACM Computing Surveys· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
