Context-Aware Inlining: Using Call-Stack Profiles for Fast and Smaller Binaries

Effect of Abstractions and Prompting Strategies on LLM-Guided High-Performance Optimizations

It is demonstrated that LLMs provided with specific optimization goals achieve better measured performance and validity rates when generating C code compared to creating computation pipelines and optimization schedules with established frameworks, suggesting that future development should explore alternative approaches for verifiable LLM-guided code optimization.

Leveraging Copy-and-Patch JIT Compilation for Low-Overhead Dynamic Program Analysis

Characterizing Type Feedback in Just-In-Time Compilation

A characterization study of how feedback information is utilized by JIT compilers, providing motivation for future work on reducing recording overhead, and evaluating two oracle-based configurations that establish upper bounds on achievable reduction.

Torchy: A Tracing JIT Compiler for PyTorch (Extended Version)

Torchy is presented, a tracing JIT compiler for PyTorch, one of the mainstream eager-mode frameworks, that achieves similar performance as data-flow frameworks, while providing the same semantics of straight-away execution.

From Runtime Reflection to Compile-Time Specialization: A Template-Based Approach to Runtime Libraries

: In low-level programming, performance is usually the primary concern, the programmer having almost full control over the generated code, and therefore its metrics. This control is achieved through system-level programming languages, and most of them these days employ language design concepts such as generic programming and runtime type reflection, which the programmer can leverage depending on their needs (e.g. choosing between performance and flexibility). In the past, some proof-of-concept work has been done to convert D’s runtime library (DRuntime) from a monolithic, TypeInfo -based architecture to a template-based one, observing initial performance improvements of up to 25%. In this paper, we consolidate and extend those results, detailing the performance impact of each hook and evaluating the trade-offs between performance, compilation time and binary size. Our findings confirm an increase in performance of up to 99% for the template-based implementations, with minimal impact on compilation time and binary size. This work is not limited to the D programming language, but can be extended to any other language that supports the same feature set, such as C++, Rust, and Go.

Multi-level Code Optimization via Mixture of Prompts

Runtime efficiency is a critical factor that impacts both software quality and user satisfaction. There are many approaches proposed for code optimization to improve runtime efficiency. Traditional code optimization methods operate on intermediate representations (IRs) during compilation for static languages. They are effective but struggle to handle dynamic languages that do not require compilation. Recently, large language models (LLMs) have been leveraged to directly optimize source code in dynamic languages. However, these methods fail to identify suitable optimization targets and usually conduct incomprehensive single-level optimization. To address these challenges, we propose Optimo, a multi-level LLM-based code optimization approach built on a novel Mixture-of-Prompts (MoP) architecture. In the MoP architecture, Optimo identifies time-critical code structures as performance bottlenecks via differential profiling. These structures are then routed to some optimization strategies, akin to expert models in MoE, each tailored to optimize specific code patterns. Unlike traditional approaches that focus only on statement-level optimizations, Optimo operates at four levels of abstraction, ranging from coarse-grained algorithmic improvements to fine-grained optimizations in API usage. We evaluate Optimo on two code efficiency benchmarks, COFFE and Effibench. Our results demonstrate that Optimo achieves an up to 57.48% opt%, i.e., the percentage of optimized programs that are correct and at least 10% faster than the original programs, and an up to 3.97x speedup when optimizing human-written code, and it consistently outperforms the best baseline by up to 96.51% in terms of opt%. Furthermore, Optimo achieves an up to 42.42% opt% and an up to 13.51x speedup when optimizing LLM-generated code.