TL;DR
A dual-engine, AI-powered resume screening system designed for transparency and reproducibility, with a reproducible SBERT→XGBoost→SHAP classification pipeline, and a practitioner-oriented user interface that operationalizes explainability and auditability is presented.
A reproducible SBERT→XGBoost→SHAP classification pipeline, an LLM comparator with a standardized evaluation template, and a practitioner-oriented user interface that operationalizes explainability and auditability are presented.
LLM-based evaluators of natural language generation (NLG) quality are widely deployed as scoring tools and as automated training signals, yet the internal procedure by which they assign a rating remains poorly understood. We investigate this procedure mechanistically through an eight-attack perturbation taxonomy across the Readability and Adequacy dimensions of NLG quality, a generation pipeline that produces paired clean and corrupt summaries with controlled error intensity and explicit token-level modification maps, and a four-experiment battery of causal tracing, logit-lens vocabulary projection, and attention-head knockout applied to Themis (Llama-3-8B) and Prometheus (Mistral-7B). Both evaluators implement a structured, coherent evaluation pipeline operating in two stages: below layer 15, attention performs local error comparison and routes the result to the final input position; above it, the MLP cascade integrates the signal and writes the rating, with the decision crystallizing in the residual stream at a sharp late layer (L = 26 on Themis, L = 25 on Prometheus). Furthermore, a base-model control at the same scale (Llama-3-8B) reproduces the routing architecture and crystallization but not the stage separation, isolating the two mechanisms that fine-tuning specifically installs, suppression of below-L15 MLP contribution at the last position and a two-layer advance of the crystallization depth, indicating that fine-tuning sculpts an existing substrate rather than building the pipeline from scratch. We release the source code and data at https://github.com/himil-v/judge-mech
Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.
Recruiting teams are increasingly confronted with the volume-versus-quality problem: the number of applications has grown faster than the ability of manual resume screening, and keyword-driven applicant tracking systems (ATS) routinely miss qualified candidates whose resumes don’t surface the exact terms a filter is looking for. In this paper, we propose a comprehensive end-to-end intelligent recruitment system, “Intelli360”, which exploits Natural Language Processing (NLP), supervised Machine Learning (ML) and predictive analytics to automate the process of resume parsing, skill extraction, ATS score prediction and candidate ranking. The system extracts structured information from unstructured resumes, computes a weighted resume-matching score against job descriptions and predicts an ATS suitability score. Recruiter and candidate dashboards, an AI chatbot and automated interview scheduling support the surrounding workflow. Intelli360 was benchmarked against a traditional keyword-based ATS and manual screening on a multi-domain resume set covering Artificial Intelligence, Data Science, Software Engineering, Web Development and Cybersecurity roles. Intelli360 achieved a matching accuracy of 91.8% compared to 82.3% for the keyword-based ATS and 72.5% for manual screening, a precision of 90.6%, recall of 92.9% and F1-score of 91.7%. Average resume screening time reduced from 18.4 seconds to 4.6 seconds per resume Average shortlisting time reduced from 6.5 days to 1.8 days Our results suggest that the combination of semantic, NLP-based skill extraction and ML-based ranking provides a significant improvement overrule-based and keyword-based recruitment pipelines. Further work is needed to statistically validate these gains (crossvalidation, confusion-matrix analysis, and significance testing). Index Terms—recruitment automation, natural language processing, machine learning, resume parsing, ATS score prediction, candidate ranking, predictive hiring analytics.
Most automated essay scoring (AES) systems output a single holistic score without interpretable evidence and rely on closed APIs that introduce data privacy and cost barriers. We present ArguLens, an opensource, locally deployable system that decomposes AES into three decoupled components: a discourse-move classifier (Qwen2.5-7B-Instruct fine-tuned with LoRA on PERSUADE 2.0), a grade-independent LightGBM scorer over 31 linguistic and discourse features, and a label-aware feedback generator served through vLLM with a Qwen2.5-14BInstruct backbone. A Gradio web UI exposes pluggable inference backends and supports single-essay and batch scoring with downloadable per-essay breakdowns. On an essaydisjoint PERSUADE 2.0 test split, the logitprobe classifier achieves 82.6% accuracy and 0.727 macro-F1; under prompt-grouped 5-fold cross-validation the scorer reaches a mean QWK of 0.813 under an oracle discoursefeature protocol, and an ablation shows that adding gold discourse annotations yields an increment of +0.055 QWK over the lexical+syntactic configuration (paired t-test, p = 0.010). This is a component-level diagnostic rather than an end-to-end classifier-to-scorer result. The feedback generator ships with a structured evaluation protocol; its human-rater study is left to future work. The system is released under Apache 2.0 at https://github.com/wwrwbs/AI_AWE.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
