FlashRT is presented, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization.
Abstract
Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.
Large Language Model (LLM) applications increasingly rely on multi-agent and retrieval-augmented generation (RAG) architectures to solve complex, knowledge-intensive tasks. However, when deployed as generic platforms serving multiple customers and heterogeneous user groups, existing systems often rely on agent-level routing and prompt hardcoding, leading to poor modularity, limited reuse, and weak controllability. In particular, current approaches lack an explicit abstraction for modeling system capabilities and controlling which capabilities are accessible to different users. We propose an expert-guided multi-agent architecture that separates execution from capability modeling. Agents are responsible for LLM interaction and tool execution, while experts represent indivisible business or task capabilities and guide agent orchestration through dynamic prompt injection. This design enables a generic, multi-instance system in which different customer deployments and user groups share the same agent implementations while exposing different capability sets at the business level. The architecture enforces strong guarantees in terms of answerability, traceability, and controllability. A first working implementation has been developed, and an open-source release is currently in preparation.
Ying Zhang, M. Guillaume, Edouard Leleux et al.· 2026 6th International Confe...· 0 citations
Integrated sensing and communications (ISAC) is moving from proof-of-concept demonstrations to system-level deployment in sixth-generation (6G) networks. Because sensing and communication share hardware, spectrum, and waveform resources, ISAC design now involves many tightly coupled choices, including waveform selection, sensing algorithm setup, resource scheduling, and deployment planning. This design space is already too large to manage well through manual tuning or isolated optimizers. This article introduces the \textit{Agent Compiler}, a large language model (LLM)-enabled compilation layer that translates high-level engineering intent into complete and executable ISAC system configurations. The Agent Compiler works in four stages: intent parsing, task decomposition, policy graph synthesis, and infrastructure mapping. It produces a verifiable intermediate representation called the ISAC Policy Graph (IPG). A runtime engine then deploys the compiled configuration and supports closed-loop adaptation at three levels: fast parameter updates, partial recompilation of affected subgraphs, and full workflow recompilation. The core design principle is strict time-scale separation: the LLM handles slow-loop strategic decisions, while proven algorithms retain real-time control in the fast loop. A UAV-assisted disaster rescue example illustrates the full compilation process. We also discuss open issues, including compilation latency, output reliability, constraint verification, and pipeline security, to guide future research.
Lijie Zheng, Xudong Zhong, Baoquan Ren et al.· 0 citations
GPU kernel agents and GPU programming languages have advanced separately, leaving expert kernels difficult to reproduce. Agents usually treat the compiler as a fixed black box and receive only errors, correctness outcomes, and timing, while existing DSLs either hide critical scheduling decisions or expose them through difficult layout abstractions. We present CAKE, a compiler-agent co-design in which agents author CAKE IR, a typed, hardware-explicit schedule representation. CAKE exposes warp roles, memory movement, synchronization, and pipelines while supporting verification, cost modeling, and localized diagnostics. The harness itself evolves: recurring failures become verifier rules, IR primitives, model calibrations, and reusable optimization tactics. In matched implementation-hidden Flash-KMeans clean starts on B200, the best CAKE IR candidate at an 80-million-token budget runs at 1.144x the tuned FlashML baseline, compared with 0.928x for direct CUDA/PTX. Beyond this benchmark, agent-generated Kimi Delta Attention achieves a 2.05x geometric-mean speedup over official FlashKDA and passes end-to-end serving validation. Dispatcher-backed KNN and KMeans improve performance by 1.42x to 2.12x across more than 400 shapes, and four kernel changes are available as upstream PRs. CAKE targets NVIDIA GPUs from Ampere through Blackwell and separates single-shape evolution from library generalization and dispatch.
Zihao Ye, Yingyi Huang, H. Jin et al.· 0 citations
The rise of large language model code agents—Claude Code, Gemini CLI, Codex CLI—has enabled developers to drive models through multi-step programming operations in real codebases via code agent interfaces. Existing work, however, has primarily applied these tools to functional tasks such as bug fixing and feature implementation, leaving performance engineering—a discipline that demands runtime evidence, domain expertise, and rigorous validation—largely unexplored. Moreover, most AI-assisted performance optimization approaches are tightly coupled to a specific model or interface, making them brittle in the face of rapid code agent evolution. We propose a code agent-agnostic agentic scaffold for automated code performance optimization. The scaffold connects to mainstream model code agents through the Model Context Protocol (MCP) and is equipped with a procedural memory module (agent skill) that encodes the complete optimization control loop. This enables the system to autonomously execute the entire pipeline: project-level runtime analysis, hotspot identification and benchmark extraction, Abstract Syntax Tree (AST)-precise code localization, candidate patch generation, functional verification, performance measurement, and version rollback. When straightforward optimization strategies fail, the system escalates by invoking the Retrieval-Augmented Generation (RAG) tool in the MCP layer to retrieve structurally similar human optimization cases as reference patches. We evaluate the system on the 10 CBench benchmark suite [7, 9], and experimental results show our methods can achieve 1.5 × speedup with \(100\%\) success rate. Furthermore, our methods can significantly reduce the token consumption comparing with baseline methods.
Graphical user interface (GUI) agents are systems powered by large multimodal models (LMMs). They perceive screen state and execute user instructions through GUI actions such as clicking, typing, and scrolling on desktops and mobile devices. However, current agents scale poorly to long-horizon tasks: actions incur costly LMM inferences, and performance degrades as context grows. Humans divide such workloads among collaborators who complete sub-tasks in parallel. Yet parallel coordination among GUI agents has received little attention. To close this gap, we introduce ParaGUIBench, to our knowledge, the first benchmark dedicated to parallel execution and coordination of multiple GUI agents on separate desktop instances. It consists of three components: a multi-device Docker infrastructure with a shared file system; a dataset of 233 tasks spanning six task categories; and an evaluation system with efficiency metrics, including step reduction ratio and token cost. We further introduce ParaGUI, a planner-worker agent that decomposes GUI tasks and dispatches sub-tasks to concurrent workers on separate desktop instances. On ParaGUIBench, ParaGUI reaches a 46.4% success rate, outperforming the strongest serial baseline (Claude Sonnet 4.6) by 12.9 points while using roughly half the steps and less than half the tokens. These results show that parallel execution can improve both success rate and efficiency on decomposable, long-horizon GUI tasks, pointing to a direction worth further study.
Zedong Yu, Qianxing Li, Zhi Gao et al.· 0 citations
Agentic coding tools present new opportunities to transform research workflows. The performance of agent systems built depends on both large language models (LLMs) and the harness around LLMs, which is the orchestration code that determines an agent's behavior. We present ToFu, an agentic harness for researchers that reads your codebase, edits files, runs commands, and integrates with your development tools. ToFu plays a dual role in research. As a research assistant, it supports practical research workflows with superior token efficiency, lower cost, and multilingual capability compared with existing agentic harnesses. Its release under the MIT License further enables local deployment for privacy-sensitive users. As a research object, ToFu provides a white-box agentic harness that allows researchers to inspect, modify, and evaluate its orchestration logic, tool-use behavior, and harness design, while retaining strong benchmark performance and an application-level user experience.