This work presents the design and implementation of a governance-aware, multi-tenant AI sandbox for structured experimentation and the generation of reusable evaluation evidence across projects and stakeholder groups.
Abstract
Collaborative AI experimentation across industry and academia requires platforms that enable rapid prototyping while preserving controlled access, tenant separation, and transparent workflows. Despite growing interest in AI sandboxes, there is still limited practical guidance on how to design and implement platforms that integrate experimentation capabilities with governance requirements. This work presents the design and implementation of a governance-aware, multi-tenant AI sandbox for structured experimentation and the generation of reusable evaluation evidence across projects and stakeholder groups. The sandbox was developed within an industry-academia collaboration based on requirements that were iteratively refined with industrial partners. Its reference architecture separates the multi-tenant user interface from the backend control plane and places execution and data-management functions in dedicated layers. The platform supports governed user onboarding, project-centered collaboration, managed access to AI services, approval workflows, audit logging, and traceable experimentation. Experiment configurations, contextual information, and governance decisions are stored as persistent records, allowing evidence and outcomes to be compared and reused across projects. The development process provides practical lessons for deploying and extending governance-aware AI sandbox platforms in collaborative research and industrial environments.
Modern enterprise applications rely on extensive third-party code, automated build systems, cloud-native infrastructure and rapidly changing vulnerability intelligence. Security controls are then spread out across the development, the software supply-chain assurance and production operations making it hard to relate some process deficiency with the subsequent impact seen in operations. This paper proposes an AI-driven DevSecOps approach focusing on governance with connections between Secure Software Development, Software Provenance, Runtime Observability, AI for analysis, Deterministic Policy Enforcement, and Responsible Human Decision Making. The framework is developed in a design-science methodology, and structured as the following layers: mission and regulatory context; AI-augmented DevSecOps pipeline; cloud-native runtime; unified observability and threat intelligence; AI intelligence and decision support; and policy governance with human oversight. It has a core principle that AI can correlate evidence, assess risk, and inform action but cannot override the requirements of policy, or authorize high impact operational decisions. A healthcare software-supply-chain scenario is the basis for an example of - a controlled rollback enabled by software bills of materials, provenance records, runtime telemetry, vulnerability intelligence, AI reasoning, policy checks, and human approval. It provides an end-to-end conceptual architecture, a decision model driven by policy, and a standards-based foundation for implementation. Since the evaluation is scenario based, the framework should be viewed as a design artifact and it is necessary to prototype and empirically validate it across multiple sectors.
Mir Fawad, Mir Jawad Yaqoob, Khawar Muhammad Saad· American Journal of Engineer...· 0 citations
Platform engineering has become, over the past several years, the default way large organizations manage the growing complexity of software delivery. That arrival has coincided with a second, less discussed obligation. Platform teams must now build platforms that host AI workloads, retrieval-augmented generation services, model inference endpoints, autonomous agents, while also embedding AI into the platform's own control plane to manage the infrastructure itself. This article calls the pairing a dual mandate and argues, from eighteen years spent migrating enterprise build and deployment infrastructure through several earlier paradigm shifts, that the two obligations are not separable line items but one reinforcing system. Neither the choice of model nor the surrounding tooling binds them together; governance does: provenance, scoped authority, bounded autonomy, and measurement. Drawing on recent platform-engineering, MLOps, and AI-governance literature alongside the author's own experience navigating prior infrastructure transitions, from sequential build systems to distributed CI/CD, from manual provisioning to Infrastructure as Code, this article defines the dual mandate precisely, examines each half in turn, explains why governance becomes the binding constraint at their intersection, and proposes a reference architecture a platform team can apply directly. The analysis is offered as a synthesis of published literature and the author's cross-paradigm operational experience, not as an empirical study of a specific AI production deployment. Where the discussion extends into AI-workload specifics beyond that direct experience, this is stated as informed extrapolation from platform-engineering fundamentals rather than implied first-hand practice.
Sonu Kumar· International journal of com...· 0 citations
A contract-bounded runtime architecture, a source-preserving data substrate, and a falsifiable measurement protocol are contributed, which proposes a cluster-period randomized crossover experiment with a four-state verdict: supported, falsified, conditional-engineering, or inconclusive.
Ya-Xiao Liu, Peng Liu, Yi-Wen Liu et al.· 0 citations
Through applied case studies in pharmaceutical discovery and financial systems, common design patterns that make agentic systems successful are analyzed, and practical mitigation strategies for failure modes are discussed, such as verification pipelines, fallback mechanisms, and human-in-the-loop supervision.
Grace Hui Yang, P. Venkit, Hooman Sedghamiz et al.· Proceedings of the 32nd ACM...· 0 citations
A practical framework for organizations introducing API performance testing from scratch and a knowledge management framing that treats performance testing outputs as reusable organizational assets are contributed.
Inga Zilinskiene· European Conference on Knowl...· 0 citations
Enterprise automation has outgrown the platforms that built the category. Robotic process automation (RPA) suites such as UiPath, Automation Anywhere, and WorkFusion were designed a decade ago around record-and-replay bots and heavyweight, often Java-based, orchestration consoles. They remain dominant, but engineering organizations increasingly report the same friction: steep learning curves, weak integration between development environments and the orchestration console, and licensing models built for enterprise procurement cycles rather than iterative, code-first delivery.
This paper proposes an architecture for a Python-native orchestration platform that borrows the operational role of a traditional RPA "control tower" — scheduling, recovery, human-in-the-loop task management, and governance — while adopting the engineering practices of modern workflow-orchestration systems such as Temporal and Prefect. The result is a system that a mid-sized engineering team can operate, extend, and audit, without the multi-year integration overhead associated with legacy RPA suites.
The architecture is presented in four layers — presentation, orchestration and execution, intelligence and observability, and data and infrastructure — each mapped to specific, currently available open-source components. Two reference diagrams accompany the paper: a system architecture view and a process-execution flow view. The paper closes with security, compliance, and deployment considerations relevant to enterprise evaluation.
Unknown authors· International Journal For Mu...· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comJul 30, 2026
Computer-use AI agents struggle with multi-step workflows like email and customer support. Echoverse trains agents in realistic environments rather than simply providing more training tasks, helping them improve as the tasks, tests, and environments evolve. The post Echoverse: Deep, evolving environments for computer-use agents appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduJun 30, 2026
Computer scientist Phillip Isola cuts through the hype to explain how AI agents work and what the future might hold for this rapidly advancing technology.
