Speed vs Structure: Building Reproducible AI Workflows with the UCL Unified AI Research Platform

Abstract

Event Type Walkthrough Abstract In this 30-minute walk-through, we demonstrate how AI model development can be effectively balanced between rapid experimentation and robust, reproducible software engineering practices using Unified AI, UCL’s in-house Kubernetes-based research platform. Through an end-to-end example using open-access datasets to train a foundational model for synthetic image generation (https://github.com/xfetus/fetal-ultrasound-edm2), we present two complementary workflows. First, we demonstrate rapid prototyping using Kubeflow-TrainerCommand with the notebook training-edm2-model-scratch-volume.ipynb, where dependencies are installed and training is executed directly in a scratch volume. This enables fast iteration and exploratory development, allowing researchers to quickly validate ideas and test modelling assumptions with minimal setup. We then move to a structured, production-oriented workflow using training-edm2-model-ghcr.ipynb. Here, containerised environments and a central GitHub image registry manage code and dependencies, with execution still handled via Kubeflow-TrainerCommand. This highlights how the same task can support reproducibility, version control, scalability, and long-term maintainability. We highlight trade-offs between speed and robustness in real-world AI workflows. The session is reproducible for UCL researchers with Unified AI accounts and useful for external participants exploring practical adoption. It is designed to encourage discussion before and after the event, support knowledge exchange, and foster future collaboration. We conclude by reflecting on broader good practice, including how institutional platforms such as Unified AI, currently an in-house UCL platform requiring UCL ID credentials, can reduce reliance on external vendors, improve cost efficiency, and support more consistent and reproducible research across teams. This also creates opportunities for cross-faculty collaboration and discussion of how similar approaches are being adopted elsewhere. Prerequisites This content is aimed at Research Software Engineers, data scientists and data stewards of all levels, from beginners to experts, who are interested in AI model development. For those at beginner to intermediate level, some familiarity with GitHub workflows and Python environments would be useful, particularly package management for libraries such as PyTorch, and the drivers relevant to your setup. For more experienced attendees, prior exposure to Kubernetes will help: running kubectl commands, debugging Docker images, and configuring your GitHub account accordingly. If you'd like to replicate the workflows at your own institution, some Kubernetes dependencies will also be needed. To prototype model development locally, you'll need a laptop with an NVIDIA GPU. This lets you run local images and a few epochs of the model to confirm the workflow runs end to end, before scaling up to an A100 or H100 via Kubernetes. Outcomes For Research Software Engineers (RSEs), the key benefits of this session include: (1) developing, strengthening, and exploring new skills across the end-to-end lifecycle of AI model development; (2) gaining a deeper understanding of real-world requirements, constraints, and domain-specific needs; (3) learning from practical case studies to better support collaborative development, manage technical teams, and address common scaling and adoption challenges in AI systems. Participants will also be invited to join a growing community network through GitHub and Etherpad-based collaboration spaces. These platforms will be used to share resources, continue discussions beyond the session, and contribute to an evolving set of community-driven outputs. This includes opportunities to shape future activities such as hackathons, workshops, training sessions, and other collaborative events focused on advancing sustainable AI engineering practices. In-Person or Online Delivery Hybrid (Both In-Person and Online Presenters) Acknowledgements We acknowledge UCL Unified AI Services (UAIServices@UCL) for providing an in-house platform supporting AI model development for researchers across UCL. We also acknowledge the agile delivery support provided by Dorothy Chung.