AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
TL;DR
This work investigates the effect of per-prompt tool-call budgeting using a 0.5B policy and informing a budget K in the prompt, and investigates how problems complexity relates to tool calls.
Whether an instruction-tuned model calls a tool can be controlled by a single linear direction in its residual stream, extracted without any training from the model's own tool-use preference signal and turned into an inference-time intervention with no prompt change.
This work forms LLM-driven equation discovery as an iterative search process that unifies Best-of-N, sequential refinement, tree search, and evolution-style methods under a common compute-allocation view and finds that search width is the dominant allocation parameter.
This work empirically compare programmatic tool calling to native JSON tool calling across 14 language models on BFCL v4 and demonstrates that programmatic tool calling is a viable and robust alternative to JSON tool calling, with performance tracking model capability across release generations.
ToolRobustBench provides a deterministic and cascade-aware benchmark for diagnosing robustness beyond clean tool-calling accuracy, where a tool-calling agent is an LLM system that selects a tool, supplies structured arguments, and interprets its returned feedback.
Large language model agents derive much of their capability from tool use. Existing research on tool use has largely focused on selecting the right tool and orchestrating the order of calls. However, correctly filling the parameters of a tool call is equally critical for successful execution and has received far less attention. In domains such as cloud networking, even frontier models correctly complete fewer than half of tool calls. Inspired by recent analyses showing that LLM hidden states encode rich information about model predictions, we discover that while the model generates a parameter value, its hidden state contains a strong correctness signal: a simple linear probe can accurately predict whether the value will be correct. Based on this observation, we propose a unified probe-guided framework with two complementary approaches: probe-filtered bootstrapped training (PBT), which uses the probe to filter reliable self-generated calls for fine-tuning, and probe-guided reranking (PGR), which uses the probe to select better candidates during inference. To support systematic evaluation, we release ParamBench, a benchmark built from real cloud-network APIs that categorizes every instance into five difficulty levels according to parameter nesting depth, cross-parameter dependencies, and the reasoning required to derive values from earlier calls. Extensive experiments across 5 open models on ParamBench and 6 external benchmarks demonstrate that our method substantially improves parameter generation, raising the average exact match from 19.7% to 59.6%.
This work introduces Test-Time Scaling via Error Localization (TTEL), an inference-time algorithm that utilizes fixed or environment feedback to perform token-level error localization and establishes strictly dominating Pareto frontiers across sequential reasoning domains.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.