Quantifying the impact of feature-driven shortcuts in machine learning benchmarks for circuit performance prediction

Abstract

Machine learning is increasingly used in Electronic Design Automation (EDA) to predict circuit performance, but the reliability of these models can be compromised by features whose correlation with the target largely reflects the target’s own definition rather than genuine, generalizable structure. This study investigates such a case in a CktGNN-derived benchmark, where a bandwidth feature correlates almost perfectly (r = 0.999619) with the Figure of Merit (FoM) target. Using the original CktGNN data directly, we show this correlation is substantially explained by FoM being a composite metric with bandwidth weighted roughly 6-8x more heavily than its other constituent terms (linear fit fom ~ bw + pm + gain, R² = 0.999992), a property of the metric’s definition rather than an unintended leakage pathway. Whatever its origin, the dependency still creates a trivial shortcut for feature-based (non-graph) machine learning models that include it, which is the practically important consequence this study quantifies. We generated a synthetic dataset of 10,000 samples to measure this shortcut’s impact under controlled conditions, evaluating six model families- Linear Regression, Random Forest, XGBoost, CatBoost, LightGBM, and a Multi-Layer Perceptron, across 10 repeated random splits and four different Figure-of-Merit weighting configurations. Removing the shortcut feature produced a statistically significant R² drop for Linear Regression (up to 37.8% relative, Wilcoxon p = 0.002, Cliff’s delta = 1.00), for the MLP (up to 14.6%, p = 0.002–0.010 across configurations), and for Random Forest (up to 13.2%, p < 0.02 across all configurations), while gradient-boosted tree ensembles (XGBoost, CatBoost, LightGBM) were comparatively robust (drops mostly under 5%, frequently not statistically significant). Global permutation and local SHAP attribution analyses reveal this divergence stems from the model’s internal mechanics, whereas linear and parallel-ensemble models rely almost exclusively on the shortcut, sequential gradient boosters distribute attribution across secondary parameter interactions, providing a buffer against the shortcut’s removal. We further validate the with/without comparison directly on the original CktGNN data, where the same qualitative pattern held despite only two features remaining once bandwidth was excluded. This work provides a real-data-grounded account of how a formula-driven feature dependency can still function as a practical shortcut in feature-based ML benchmarking, and offers a replicable, statistical protocol with repeated splits, effect sizes, and confidence intervals, for auditing similar dependencies in other EDA datasets.