Authenticated state transitions need proof engines that conserve memory, avoid repeated setup, and preserve request semantics across parallel execution. We present StateSync-GKR, a reusable Rust sumcheck/GKR core with a sparse-Merkle frontend, and connect its design mechanisms to controlled measurements. Sparse layer reduction and derived wiring reduce prover ownership and repeated verification work. Cube/affine fusion and parallel path constraints keep the circuit schedule and proof-size growth compact. Reusable preparation and indexed parallel jobs amortize setup while maintaining independent results. Three-state records, strict original-request decoding, and separate core/model interfaces expose the application and assurance boundaries. With circuit, field, transcript and prover driver held fixed, sparse reduction lowers additional requested allocation 3,242-fold and peak process resident memory 89.46-fold relative to StateSync-GKR's test-only dense reference for a two-layer mixed circuit of width 4096. Direct depth-24 fresh/prepared measurements show 3.46-4.45-fold lower per-request proving-path latency. CPU and full-proof delivery studies distinguish useful parallel work from queueing and return-path costs, retaining all outcomes and latency budgets. The evaluation supports larger circuits within finite RAM, lower repeated-request cost, and explicit deployment choices for this artifact. It preserves the cryptographic claim boundary of the existing construction.
JustFit is presented, an MLX runtime that coordinates their overlapping allocations: KVExec executes and checkpoints four-bit KV with bounded workspace, PhaseSwap loads phase-dependent components, and StateTrans preserves history across execution modes.
TomasuLLM is presented, a runtime that executes agent tool calls out of trajectory order while preserving task-execution correctness, and improves the reported benchmark means and scales with tool latency.
Jiang-Nan Yu, Ce-Yu Xu, Meng-Ming Li et al.· 0 citations
Tool-calling agents are mostly optimized for accuracy; in deployment, however, their growing cost is latency: every tool call is decoded autoregressively. Speculative decoding accelerates this without changing the output, and retrieved past calls make strong drafts under real use, where each user's behavior tends to re...
Zi-He Song, Wen-Ting Zhao, Ye Liu et al.· 0 citations
Large language model (LLM) routers are increasingly deployed to optimize quality, latency, and cost through dynamic model selection and request rewriting. However, a compromised router can cause ineligible selection on the control plane, unendorsed rewriting or unfaithful relay on the data plane, undermining LLM routin...
Meng-Yu Yao, Zi-Qi Zhang, Shao-Fei Li et al.· 0 citations
Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in the current worker round. Schedulers assume that a matched prefix is ready, which holds for full attention but fails for hybrid LLMs. There, a branch (the consumer) may...
Hui-Hao Jing, Wen-Bin Hu, Shao-Jin Chen et al.· 0 citations
Long-context serving runs out of KV memory before the hardware reaches its useful throughput. HiSparse stores history in host memory and fetches only the selected entries, but it uses one cache budget at every load and does not handle draft-model speculative decoding such as DSpark and DFlash2. We present Adaptive DHiS...
Naji Alazhar, Wassel Alazhar· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.