Experimental results show that the proposed self-calibrating agentic framework successfully profiles the zero-knowledge workloads, achieving a higher accuracy than baseline LLM agents and establishing a robust foundation for deploying autonomous AI in decentralized infrastructures.
Abstract
Large Language Models (LLMs) are increasingly deployed as autonomous agents, transitioning from static conversational interfaces to dynamic systems capable of complex reasoning, tool execution, and decision-making. However, the operational reliability of these agentic AI systems is fundamentally challenged by the absence of reliable ground truth in open-ended environments and the risk of increasing operational drift over time. To address this challenge, we propose and experimentally evaluate an agentic AI framework, designed to enforce autonomous integrity within LLM-driven systems. We design a self-calibration mechanism that mitigates drift and dynamically approximates ground truth by incorporating an ARIMA forecaster, without requiring continuous human oversight. To demonstrate the effectiveness and reliability of our methodology, we apply it to the complex domain of profiling the resource usage of zero-knowledge workloads in edge computing networks. Experimental results show that the proposed self-calibrating agentic framework successfully profiles the zero-knowledge workloads, achieving a higher accuracy than baseline LLM agents by 91.7% for resource usage prediction and improving the prediction speed by 71.7% compared to pure profiling, establishing a robust foundation for deploying autonomous AI in decentralized infrastructures. Furthermore, the ground truth generation using the proposed ARIMA leaping algorithm is 52% faster than a standard ARIMA forecasting algorithm, while achieving the same accuracy.
Autonomous agents challenge conventional LLM serving by coupling repeated inference with persistent context and sandboxed tool execution. We present Aries, a full-stack experimentation framework that separates task semantics from execution configurations, reconstructs cross-component agent trajectories with correlated system telemetry, and exposes stateful tool execution through a consistent interface across heterogeneous sandbox substrates. We use Aries to conduct reproducible experiments on open agent harnesses and benchmarks. We complement these experiments with production traces from a commercial platform, grounding low-level systems research in observed production behavior. Our results show that (1) token-centric metrics miss non-inference bottlenecks, (2) retaining additional context yields diminishing accuracy benefits while reducing serving capacity, and (3) tool sandboxes alternate between long idle periods and short resource bursts, while current snapshot-based state management makes aggressive suspension costly. A complementary security analysis further highlights the need to reduce the sandbox attack surface. We then discuss the vision for agent-native serving systems designed around trajectory-level metrics, adaptive context management, elastic sandbox resource management, and sandboxes with minimized attack surface.
Leonid Kondrashov, Hongrui Liu, JooYoung Park et al.· 0 citations
Language agents, i.e., LLM agents, progress rapidly and are increasingly deployed in production environments. This trend underscores the urgent need for rigorous and realistic evaluations. However, most existing benchmarks evaluate agents in simplified, idealized settings. They typically rely on pre-packaged tool interfaces, overlook critical steps, and assume inputs are clean and fully specified. Consequently, they understate the difficulty of real deployments, where uncertainty and noise are ubiquitous and agents must proactively explore the environment to uncover new tools. To bridge this gap, we present AgentGym2, a new evaluation framework with task instances grounded in real-world end-to-end working demands. Beyond reasoning and planning, it measures agents'ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information. Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2, revealing a substantial gap between the capability of current agents and the demands of real-world applications.
Zhiheng Xi, Dingwen Yang, Jiaqi Liu et al.· Annual Meeting of the Associ...· 1 citation
The rapid integration of Large Language Models (LLMs) has driven the evolution of Multi-Agent Systems (MAS), where specialized agents collaborate to execute complex workflows. Effective orchestration in these environments requires robust routing mechanisms to efficiently allocate tasks to the most suitable agent. However, existing routers fundamentally rely on unverified proxies, ranging from textual self-descriptions to static surrogate representations, to gauge an agent's competence. This reliance on non-empirical data creates a critical gap between an agent's projected profile and its actual operational capabilities, introducing severe security vulnerabilities. Malicious agents can easily misrepresent their proficiencies or harbor covert backdoors that evade both standard external analysis and static representation-learning techniques. In this work, we introduce ANTAP (Automatic Non-Textual Agent Picker), an evaluation-driven routing architecture that discards indirect proxies in favor of active capability testing. By dynamically querying agents to ascertain their true competencies empirically, ANTAP distills performance into fixed behavioral operators within a shared semantic space. At inference time, routing is performed via a purely non-textual algebraic projection, establishing a"linguistic firewall"that renders metadata-based attacks inexpressible. In our experiments, ANTAP achieves near-zero ASR against description-based injection attacks, compared to 67.3\% and above for the description-based router baseline. Against adaptive embedding attacks, ANTAP achieves substantially lower ASR than the embedding-based baseline, with a 20\% reduction, while remaining resilient to description manipulation by design.
Dvir Alsheich, Adar Peleg, Ben Hagag et al.· arXiv.org· 0 citations
The evolution towards 6G edge-cloud ecosystems demands autonomous, intent-based network management to handle unprecedented infrastructure complexity. While Large Language Models offer promising capabilities for translating high-level user intents into network configurations, current monolithic approaches suffer from cognitive overload, hallucinations, and a profound inability to safely execute low-level data plane mutations. To bridge this gap, we introduce Edgent, a novel framework that integrates hierarchical Agentic AI with Extended Berkeley Packet Filter technologies via the Model Context Protocol. Edgent utilizes a state-driven Supervisor, enhanced by Retrieval-Augmented Generation, to decompose abstract human intents into deterministic execution graphs and dynamically delegate tasks to domain-specific worker agents. We empirically validate the framework by autonomously deploying a distributed, in-kernel DDoS mitigation pipeline across scaled containerized topologies containing up to 85 nodes. Extensive evaluations demonstrate high orchestration reliability; notably, even heavily quantized Small Language Models (e.g., 4B parameters) achieve near-perfect zero-shot execution and 100% overall task completion through autonomous error recovery. Finally, latency and resource profiling confirm that the multi-agent framework can be efficiently driven by fully localized models compatible with orchestration tasks directly within resource-constrained edge environments, therefore this work positions Edgent as a pragmatic step toward the realization of zero-touch nextgeneration networks.
Raffaele Di Tommaso, G. Davoli, Pietro Spadaccino et al.· IEEE Conference on Network S...· 0 citations
Managing modern cloud-native infrastructure is, frankly, a problem that has grown faster than our tools for handling it. Continuous delivery pipelines, ephemeral containers, and sprawling multi-cloud Infrastructure-as-Code (IaC) ecosystems create an operational surface that expands combinatorially with organisational scale and static rule-based monitoring simply cannot keep up. This paper presents InfraAgent, a five-agent agentic AI framework designed for predictive infrastructure management and autonomous deployment operations in heterogeneous multi-cloud settings. The framework brings together: a Temporal Graph Attention Network (T-GAN) agent that jointly models infrastructure dependency topology and multi-variate telemetry for deployment failure prediction across 1-hour, 6-hour, and 24-hour horizons; a Remediation Planning Agent (RPA) built on a Large Language Model (LLM) that generates context-aware IaC patches, rollout prescriptions, and rollback directives directly from risk scores; a Deployment Safety Agent (DSA) that enforces configurable pre-flight autonomy gates; a Capacity Forecasting Agent (CFA) built on a Temporal Fusion Transformer (TFT) for probabilistic 24-hour to 72-hour demand forecasting; and a Continuous Operational Intelligence Orchestrator (COIO) that provides closed-loop governance, cryptographic auditability, and online model adaptation. We evaluate InfraAgent on 47,800 deployment events and 310,000 telemetry snapshots drawn from AWS, Azure, and GCP production environments. The results show a Mean Time to Detection of 4.3 minutes (an 85.2% reduction over rule-based baselines), a deployment failure prediction F1 of 0.931, a 76.8% reduction in production incidents per 100 Continuous Integration/Continuous Delivery (CI/CD) runs, and 91.4% capacity forecast accuracy at the 72-hour horizon, suggesting that coordinated agentic reasoning grounded in graph-temporal learning can meaningfully shift cloud operations from reactive firefighting toward continuous, self-correcting infrastructure assurance.
Harish Apuri, Madhan Mohan Reddy Chinthala, Shikher Goel et al.· European Conference on Artif...· 1 citation
We present AgentSLABench, a resource-aware evaluation framework for autonomous AI agents that measures correctness alongside latency, cost, compute, memory, and network usage under declared resource budgets. Unlike standard benchmarks that report only accuracy, AgentSLABench produces a multi-dimensional profile per agent per task - the same way systems profilers (perf, pprof, cProfile) measure resource consumption of code, but extended with task correctness as a first-class dimension. AgentSLABench provides 16 task environments across 6 categories (5 core: multi-hop QA, retail substitution, code generation, web shopping, travel planning; 11 extended) with isolated Docker containers, declared CPU/memory/time/network budgets, sealed test sets with SHA256 hashes, and a standardized profiling protocol. We profile 5 general-purpose baseline agents (ReAct, PlanAndSolve, Reflexion, CoT, Random) plus 4 task-specialized agents, finding that specialized agents achieve 100% success on 3/5 core tasks (fact_qa, web_shopping, travel_planning) and 66.7-83.3% on retail and code_gen, while general baselines fail entirely on 4/5 domain tasks. Crucially, we report the Efficiency-Adjusted Success Rate (EASR) - success weighted by resource consumption relative to declared budgets - revealing that high accuracy at unbounded cost is not production-viable. We release the full infrastructure, sealed test sets, and profiling results to enable reproducible, resource-aware agent evaluation.
Meher Bhaskar Madiraju, Meher Sai Preetam Madiraju· 0 citations