Skip to content
Review

Structured AI Demonstrations and Student LLM Use in Engineering Mechanics: Study Design and Preliminary Results

Jul 2026 · arXiv.org · Vol abs/2607.28710 · 0 citations · 49 references
Computer Science

TL;DR

This manuscript presents a descriptive study design and preliminary findings from an undergraduate engineering mechanics course conducted in Spring 2026, and details a reproducible survey instrument used to capture student AI usage patterns, attitudes, and verification practices, which are subsequently linked to academic performance metrics.

Abstract

The rapid integration of large language models (LLMs) into undergraduate education presents an urgent challenge for engineering instructors. Despite widespread student adoption, there remains a critical lack of domain-specific empirical evidence to guide pedagogical policies and classroom interventions. This manuscript presents a descriptive study design and preliminary findings from an undergraduate engineering mechanics course conducted in Spring 2026. We detail a reproducible survey instrument used to capture student AI usage patterns, attitudes, and verification practices, which are subsequently linked to academic performance metrics. Additionally, we document a deployable sequence of nine structured, instructor-led AI demonstrations designed to model strategic LLM delegation and evaluation. While our preliminary data highlight shifting student behaviors and complex relationships between AI reliance and course outcomes, the primary contribution of this work is the provision of an open-access methodological framework. By making our complete study design, survey tools, and demonstration materials publicly available, we urge other engineering educators to collect and share similar empirical data. Navigating this unprecedented technological shift will require a collaborative, evidence-based approach to fully understand its long-term impacts on student learning.

View source

Similar papers

Review Open access Aug 2026

Generative AI as a Catalyst for Deeper Design Skills in Undergraduate Engineering

Generative AI is transforming how engineers conceptualize and evaluate design ideas, presenting both opportunities and risks for engineering education. This practice-based paper reviews current literature on AI integration into design courses, capturing best practices and proposals for AI scaffolding. It then describes practical experiences integrating conversational AI into undergraduate engineering design courses at the University of Ottawa, from first-year introductory design through fourth-year capstone projects. AI tools were introduced incrementally, supporting ideation, scenario exploration, critique of alternatives, and structured feedback, alongside instruction in prompt design, critical evaluation, and ethical considerations. The paper describes pedagogical framing, specific activities, student guidelines, and instructor observations of engagement patterns across year levels. Faculty observed that early reluctance and superficial use gave way to broader design exploration and more precise design rationale with sustained, guided practice. Challenges include managing expectations, variation in prompting skill, and maintaining student agency.

H. Sadek, Franz Newland · 0 citations
Review Open access Aug 2026

Improving Student Perception and Confidence Using Programming in Manufacturing Engineering

Programming is increasingly integral to engineering education, yet students often struggle with its cognitive demands and unfamiliar logical frameworks. Python, with its intuitive syntax and practical utility, has shown promise in bridging this gap, particularly when applied to real engineering contexts where programming tasks are grounded in familiar disciplinary problems. This paper presents the design and implementation of Python-based Jupyter Notebooks integrated into a second-year manufacturing processes laboratory course, where students have prior but limited exposure to Python from earlier coursework. The Notebooks enable students to perform engineering calculations, visualize experimental results, and work through data processing tasks within the context of existing laboratory activities. They also provide instructors with a means to scaffold learning and guide laboratory activities. Instructional resources were created to take advantage of these opportunities. They were organized and hosted as an open educational resource using the Jupyter Book framework and GitHub Pages to support independent student access throughout the term.  Formative survey feedback indicated that students found the notebooks accessible and practical, and most expressed intent to apply Python in future engineering work. Students also identified areas for improvement, particularly around opportunities to write code themselves and the clarity of inline documentation. These findings highlight a core design tension between scaffolding accessibility and skill development and inform recommendations for educators seeking to embed computational tools within discipline-specific laboratory courses.

Meet Upadhyay, Casey Keulen · 0 citations

Rewriting the Curriculum: Integrating AI Prompt Engineering into EFL Writing Instruction

It is concluded that prompt engineering should be formally integrated into EFL writing instruction to cultivate essential digital-age literacy skills and advocate for curriculum innovation that effectively bridges traditional composition pedagogy with the emerging demands of AI-mediated communication.

Eslam Yacoub · 0 citations
#large language models Review Oct 2026

LLMs in Civil Engineering: Education Usage Patterns, Verification Practices, and Curriculum Implications from a Taxonomy-Aligned Student Survey

Large language models (LLMs) are rapidly entering civil engineering research and practice, yet little is known about their use in educational contexts. This study reports results from an institutional case study based on a taxonomy-aligned survey of 109 respondents (103 undergraduates, four graduate students, and two faculty) in civil engineering–related programs at a large US university. The survey examined adoption patterns, task functions, verification practices, disclosure norms, and training needs. Undergraduates primarily used LLMs for tutoring and concept explanation (83%) and design ideation (67%), with limited adoption in coding (7%) and technical reasoning (41%). Verification practices were robust: 86% recalculated manually, 52% checked against standards, and only 4% reported nonverification, yielding a median of two methods per user. Ethical orientations favored conditional disclosure for major contributions (53%) and placed primary responsibility for errors on the human user (75%). Demand for formal training was high, especially among those with greater adoption, familiarity, and verification breadth. Results reveal a developmental gap between student practices, which emphasize low-risk learning and ideation, and research and faculty practices, which emphasize technically rigorous applications. The study underscores the need for curricular pathways that guide students from exploratory uses toward responsibly verified technical tasks within similar educational contexts. By linking a civil engineering–specific taxonomy of LLM functions with educational survey data, this work offers institutionally grounded empirical evidence on artificial intelligence (AI) literacy in civil engineering education and highlights directions for curriculum and assessment design.

Zhenhua Huang · 0 citations
Open access Aug 2026

Comparative Evaluation of Large Language Models in Computer Programming Education

A comparative analysis of six LLMs for generating formative feedback on introductory Java programs containing predefined defects under controlled conditions reveals substantial cross-model variation, particularly in multi-defect scenarios.

Melina Najimi, Saba Yazdani, Marzieh Ahmadzadeh · 0 citations
Review Aug 2026

Curriculum as Code: An AI-Assisted Architecture for Instructional Design in STEM Education

Contribution: This paper presents a six-phase AI-assisted instructional design architecture based on the Curriculum as Code paradigm, integrating Generative AI with LaTeX and Python to automate the creation of reproducible, visually consistent, and technically precise materials for STEM education. Background: Creating customized instructional materials for active learning imposes a heavy workload on faculty. Standard presentation tools lack robust support for technical content, while current AI applications often hallucinate and fail to formalize the instructional authoring process, limiting their utility for rigorous academic design. Intended Outcomes: The framework aims to reduce preparation time while ensuring mathematical accuracy, adherence to institutional visual identity, and preservation of the instructor's tacit pedagogical knowledge through explicit rules. Application Design: The solution comprises a six-phase pipeline that replaces ad-hoc prompt engineering with a systematic workflow, utilizing text-based interfaces and code-driven generation (LaTeX/Beamer for slides, Python for figures), governed by pedagogical constraints, contextual calibrations, and automated review cycles. Findings: Validated over one year across 8 modules and 28 project contexts in a Project-Based Learning environment, the architecture significantly reduced instructor workload. Generated assets underwent independent peer review and were deployed by six different faculty members, confirming scalability beyond a single author. Based on over 600 voluntary student evaluations, materials achieved high quality ratings from 8.5 to 9.9/10. Results indicate high reproducibility, minimized hallucinations, and sustained pedagogical and visual fidelity, suggesting viability for broad STEM educational applications.

Henrique Mohallem Paiva · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.