Skip to content
Open access

Budgeted Two-Stage Pre-Validation for Retrieval-Grounded Machine Learning Pipelines: A Group-Held-Out Evaluation

Nov 2026 · Academic Journal of Applied Sciences · 0 citations · 29 references

Abstract

Retrieval-grounded machine-learning pipeline synthesis must reconcile code templates with package versions and resource budgets. We study a two-stage pre-validator adapted from concrete dependency-satisfiability and resource-prediction mechanisms: an inexpensive retrieval-profile margin ranks candidate manifests; a budgeted second stage invokes a gradient-boosted peak-memory predictor and applies bounded repairs. Actual execution of 48 public-dataset learning workflows supplies memory profiles, while eight controlled configuration mutations per workload define explicit local feasibility outcomes. Four group-disjoint test folds provide 384 held-out candidates. At an exact 50% second-stage invocation budget, retrieval-prioritized validation yields 372/384 feasible configurations (96.9%), matching exhaustive checking, versus 281/384 (73.2%) for one matched random schedule. Across 300 random schedules the mean is 74.2%. A 16-group paired bootstrap puts the advantage over that seeded random baseline at 19.8–27.1 percentage points. Uncertainty offsets and semantic novelty are not independently beneficial on this coarse-fault benchmark: point prediction and a simple memory-margin ranking tie the proposed method. Calibration also deteriorates on a shifted fold. These findings support auditable, low-cost screening for synthetic configuration faults, without treating local feasibility as actual cloud deployment success or deterministic template synthesis as LLM generation.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.