Curated Intent Classification Dataset and Robustness Test Set for a Two-Stage Mental Health Chatbot

Abstract

This record provides the evaluation data accompanying the paper “A Safety-First Two-Stage Mental Health SupportChatbot: Robust Crisis Routing andRetrieval-Grounded Response Generation” (SU26 DAP Group 4, FPT University, Can Tho Campus). The data is designed to stress-test the robustness of a two-stage mental-health support chatbot (intent classification followed by response generation) against adversarial and trap queries typical of psychological counseling contexts, such as: Ambiguous crisis signals, out-of-scope requests disguised as mental-health questions, isk keywords used in benign contexts (lexical traps), indirect crisis expressions, slang and typos, and mixed-intent messages. Files train_curated_v8.csv (218 rows, 4 labels: crisis 66, out_of_scope 60, smalltalk 48, support 44): a hand-curated set of hard examples for the stage-1 safety router. It covers 22 risk-related keyword groups (e.g., kill, die, cut, hang, jump, pills, end, disappear). For each group, it pairs benign uses of the keyword (out-of-scope requests, casual slang, everyday distress) with genuine crisis statements. It also includes indirect crisis expressions that contain no explicit keyword. Columns: text, label, word_group (keyword group, 22 values), type (trap 152, indirect_crisis 43, crisis_contrast 23). robust_test.csv (258 rows, 4 labels: support 110, crisis 56, smalltalk 48, out_of_scope 44): held-out probe messages used for the robustness evaluation reported in Section [IV]. It spans 10 stress categories: short and long support messages, typos/slang, lexical traps, direct and indirect crisis, mixed-intent messages, small talk, and short/long out-of-scope requests. Columns: text, label, category (10 values), source (seed 41 = initial seed set, huno 217 = extended set written by the authors). Construction All samples were authored by the research team. A portion was generated with large language models (Claude, Anthropic) or produced programmatically (e.g., paraphrasing, noise injection, template expansion), then manually reviewed and labeled by the authors. Privacy and intended use The dataset contains no real user conversations or personal data. It is intended for research on chatbot robustness and safety evaluation only. It is not a clinical resource and must not be used to train systems that provide diagnosis or replace professional mental-health care.