Skip to content
Open access

When AI is Not AI

Aug 2026 · Journal of Ethics and Emerging Technologies · Vol 36, pp. 1-16 · 0 citations · 15 references

TL;DR

Structured human-computer interactions with higher education chatbots to explore whether these chatbots were programmed to provide financial counseling to college students found that many systems marked as AI chatbots fell short of adaptive, generative capabilities which are the essence of AI systems.

Abstract

Empirical research into these higher education chatbots has largely focused on how chatbots are used in two domains: 1) helping students through the enrollment management (ex: admissions, financial aid, housing) process and 2) providing instructor support to enhance student learning in a curricular context. This study reports on structured human-computer interactions with higher education chatbots to explore whether these chatbots were programmed to provide financial counseling to college students. During our structured interactions with chatbots, we learned that many systems marked as AI chatbots fell short of adaptive, generative capabilities which are the essence of AI systems (Regulations (EU) 2024/1689, Art. 3(1)). Specifically, we identified five points along the capability spectrum of generating responses: 1) chatbots were often “offline” during business hours, rendering them non-functional; 2) chatbots were fillable forms, thus functioning like emails; 3) chatbots were decision trees and not large language models, thus functioning like search engines and lacking adaptive response generation; 4) chatbots were actually chat applications staffed by human agents without disclosure of their non-automation; and 5) artificial intelligence chatbots were not overly intelligent in its responses to our queries. We address the implications of these findings for both research and communication ethics, sustainability, and institutional efficiency.

Read PDF

Similar papers

AI is Not AI: Interacting With Chatbots in Higher Education and Implications for Ethics, Sustainability, and Institutional Efficiency

Structured human-computer interactions with higher education chatbots to explore whether these chatbots were programmed to provide financial counseling to college students found that many systems marked as AI chatbots fell short of adaptive, generative capabilities which are the essence of AI systems.

Richard Simonds, Z. Taylor, Sara Ray · 0 citations
Open access Aug 2026

Understanding Student Effectiveness with AI Chatbots in Introductory Programming Education: An Interaction Analysis Study

Generative AI chatbots are increasingly used in introductory programming courses, but whether they support learning or encourage over-reliance remains unclear. To examine which interaction features are associated with productive persistence, we analyzed 198 CS1 student–chatbot conversation logs using a five-dimensional coding scheme: query type, response relevance, response type, dialogue outcome, and effectiveness marker. Persistence was measured as the number of student prompts per log. Negative binomial models showed that relevance failures predicted longer interactions, particularly false positives (IRR = 2.49) and false negatives (IRR = 1.65), both p < 0.001. Logistic models showed that false negatives strongly predicted unresolved sessions (OR = 9.94, p = 0.008), suggesting that the inability to answer accelerates abandonment. In contrast, conceptual (Socratic) responses predicted progressive effectiveness markers. Overall, interaction quality, not mere usage, appears to drive productive persistence, highlighting the importance of minimizing false negatives and encouraging Socratic scaffolding in educational chatbot design and deployment.

Rubaina Khan, Joshua Siderius, Kyle James Ross et al. · 0 citations
Open access 2026

From Chatbot to Adaptive Syllabus-aware AI-mechanism: LLM Personalized Teaching Assistant in Higher Education

Michael, a syllabus-aware AI teaching assistant designed to scaffold reasoning through structured, hint-first dialogue aligned with course progression, rather than providing direct solutions, is introduced, suggesting that curriculum-aligned constraints and hint-first scaffolding can support instructional integration without displacing pedagogical goals.

Or Peretz, Roei Zerahia · 0 citations
Review Open access Aug 2026

ChatGPT Use and Abuse: A 3 Year Look at Undergraduate Perceptions and Usage in the Classroom

The release of ChatGPT, a generative artificial intelligence (AI) model, in November of 2022 marked a pivotal transition in how students use technology to learn. The ability of these models to process natural language inquiries and produce cogent, highly tailored answers has sparked serious concern among educators regarding its implications for academic integrity and the future of education. The current research surveyed (n = 845) college students in a cross-sectional design administered over the course of seven semesters and focused on students’ use of ChatGPT in the classroom and their views on the appropriateness of these actions. Results indicated that students’ awareness of ChatGPT rose rapidly following its release (Spring 2023: 58.5%; subsequent semesters: 81.8% to 100%). Cheating behavior (e.g., asking ChatGPT to write a final draft of a paper) increased over the course of data collection (r = 0.24, p < 0.001), with 33.9% of students reporting using the software sometimes to always to help with class assignments and 12.7% reporting using it to write essays/papers or parts of essays/papers due for class. The discussion focuses on key trends regarding student cheating behavior using ChatGPT over time as well as possible explanations for the changes in student usage.

Adelia C. Ehrlich, Christopher L. Groves, Luke J. Tacke et al. · 0 citations
Open access Aug 2026

Artificial Intelligence and education: Opportunity or risk? Opinion from the conversational AI chatbots

ChatGPT is prominent in terms of the resources available for learning and teaching, as well as the benefits that teachers and students find in its application, awareness of the risks and challenges they face in education in order to make appropriate use of large language models (LLMs).

Arminda Suárez-Perdomo, David Abreu · 0 citations
Open access Jul 2026

Harnessing AI and chatbots to develop an interactive learning activity: student perceptions and outcomes

ABSTRACT Artificial intelligence (AI) has transformed various industries, and education is no exception. This paper details how AI can be utilized to support the development of an interactive, choose-your-own-adventure learning activity for pre-health students. With ChatGPT assistance, we identified and implemented the open-source platform Twine to create a branching, scenario-based activity focused on sexually transmitted diseases (STDs), allowing students to engage with clinical decision-making and apply course concepts. Additionally, ChatGPT’s natural language processing capabilities allowed educators to create a script-based learning experience informed by prior student feedback, providing students with immediate, targeted feedback and support. This study describes how AI was used to develop a novel learning activity that encourages student engagement and application of course concepts. Students who piloted this activity in an upper-division course reported the activity to be an effective and engaging learning experience that helped them apply course content to real-life scenarios. Importantly, the use of AI helped streamline the development of this activity, lowering barriers to creating complex educational activities that may otherwise be difficult to implement. Here, we demonstrate how AI can be leveraged to support the development of interactive, scenario-based learning activities and provide practical guidance for instructors.

Usean J. Redic, Deavin F Tendean, Andrea Nicholas · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.