Luigi Usai Recursive Self Improvement Tool (RSI)

Abstract

# UNIVERSAL RECURSIVE SELF-IMPROVEMENT (RSI) FRAMEWORK### An Agentic, Fault-Tolerant, Quine-Closed Architecture for Autonomous Algorithmic Self-Optimization**Author:** Luigi Usai **Framework Version:** 1.0.0 **Release Date:** 2026-09-11 **Target Platform:** Python >= 3.10 (Linux, macOS, Windows) **Licensing:** AFERO AGPL 3.0 e successive / Open Source Scientific Software ---## 1. ABSTRACT & EXECUTIVE SUMMARYThe **Universal Recursive Self-Improvement (RSI) Framework** is a formal, production-grade Python architecture designed to enable autonomous, safe, and verifiable self-optimization in computational entities. Grounded in rigor meeting NASA fault-tolerance standards and Google high-performance software engineering specifications, the framework formalizes the OODA (Observe-Orient-Decide-Act) operational loop for arbitrary executable targets—spanning algorithmic functions, machine learning pipelines, and abstract syntax tree (AST) source code.Crucially, the architecture resolves the foundational challenge of algorithmic self-modification through:1. **Reflexive Quine Closure (Meta-RSI):** The orchestrator itself implements the `SelfModifiable` formal contract, allowing recursive meta-optimization of its own exploration hyperparameters, cooling rates, and selection operators without code degeneration.2. **Process-Isolated Sandboxing with Zero-Deadlock IPC:** Complete crash and hang containment via Python `spawn` context, native hard-timeout watchdogs, active daemon Out-Of-Memory (OOM) monitoring (`EXIT_CODE_OOM = 42`), and pipe-draining before process synchronization.3. **Two-Phase Commit (2PC) Candidate Staging:** Candidate code modifications undergo static AST security verification and subprocess sandbox staging prior to in-memory commit, eliminating runtime poisoning of the host environment.4. **Multi-Objective Pareto Dominance & Metropolis Annealing:** Rigorous multi-metric evaluation across mean/p95 latency, throughput, and memory footprint, augmented with hypervolume scalarization and Metropolis simulated annealing to escape local Pareto optima.---## 2. THEORETICAL & MATHEMATICAL FOUNDATIONS### 2.1 The OODA Algorithmic CycleThe optimization sequence operates in four atomic, certified stages:* **OBSERVE:** Quantifies the active baseline state through functional invariant regression testing and statistical benchmarking with warmup cycles and jitter filtering.* **ORIENT:** Generates a mutated candidate state via an exploration operator parameterized by an annealing temperature $T \in (0, 1]$.* **DECIDE:** Evaluates the candidate against the baseline using Pareto dominance over a metric vector $\mathbf{M} = (m_{\text{latency}}, m_{\text{p95}}, m_{\text{memory}}, m_{\text{throughput}})$. If the candidate is non-dominant, stochastic acceptance is evaluated via the Metropolis criterion: $$P(\text{accept}) = \min\left(1, \exp\left(\frac{\Delta \mathcal{S}}{T}\right)\right)$$ where $\Delta \mathcal{S}$ represents the scalarized hypervolume difference.* **ACT:** Executes an atomic commit of the candidate state if accepted, or a guaranteed transactional rollback to the certified baseline snapshot if rejected or if invariant violations occur.### 2.2 Formal State Snapshot & Cryptographic VerificationState representations conform to the `StateSnapshot` protocol:$$\text{Snapshot} = \langle \mathcal{P}, \text{gen}, \text{timestamp}, \mathcal{M}, \mathcal{H}_{\text{SHA256}}(\text{canonical}(\mathcal{P})) \rangle$$Every mutation preserves auditability through immutable cryptographic digest verification, preventing stealth corruption across generations.### 2.3 Reflexive Quine ClosureLet $\mathcal{O}$ be the `RSIOrchestrator` and $\mathcal{T}$ be the target software entity. If $\mathcal{T} = \mathcal{O}$, the orchestrator exports its internal search hyperparameter state (temperature, decay rate $\gamma$, minimum temperature $T_{\min}$, and Metropolis flags) into a certified `StateSnapshot`, mutates its own search space, and benchmarks its own efficiency, achieving formal recursive self-improvement without external scaffolding.---## 3. ARCHITECTURAL SUBSYSTEMS & IMPLEMENTATION### 3.1 Sandboxing & Fault Isolation (`rsi.sandbox`)* **`ProcessIsolatedSandbox`:** Spawns isolated operating system worker processes using `multiprocessing.get_context("spawn")`. Guarantees clean interpreter memory space, preventing deadlock on cloned locks (vital for POSIX and Windows NT).* **Anti-Deadlock IPC Protocol:** Consumes `multiprocessing.Queue.get(timeout=timeout)` strictly prior to invoking `process.join()`. This eliminates kernel pipe buffer saturation when candidates output massive stdout/stderr logs (>64KB).* **Active OOM Watchdog:** A dedicated daemon monitor thread samples memory at 100 Hz (10ms intervals). If peak allocation exceeds `max_memory_mb`, it terminates the worker instantly via `os._exit(42)`, preventing host OS thrashing.* **Fail-Safe Exception Encapsulation:** Catches segfaults, memory aborts, fatal Python exceptions, and hard infinite loops, converting them into standardized `SandboxResult` failure descriptors without crashing the parent process.### 3.2 Dynamic AST Engine & Two-Phase Commit (`rsi.adapters.DynamicASTTarget`)* **Static Multi-Level Security:** Enforces whitelist-only imports (`math`, `itertools`, `collections`), prevents reflection exploits (`__globals__`, `__subclasses__`), and blocks malicious calls (`eval`, `exec`, `sys.exit`).* **Phase 1 (Sandbox Staging):** The candidate AST source is compiled and benchmarked exclusively inside an isolated process via `stage_candidate_in_sandbox()`.* **Phase 2 (Atomic Commit):** Upon sandbox validation and invariant satisfaction, the candidate AST is compiled into the host execution namespace; otherwise, the host state remains untouched.### 3.3 Universal Plug-and-Play Integration (`rsi.adapters`)* **`@rsi_optimizable`:** A non-invasive Python decorator that transforms any existing algorithmic function into an autonomous self-optimizing engine with one line of code: ```python @rsi_optimizable( benchmark_inputs=[100, 200, 500], invariants=[lambda runner: runner(100) == 200], tunable_params={"batch_size": 1, "worker_threads": 1}, bounds={"batch_size": (1.0, 64.0), "worker_threads": (1.0, 8.0)} ) def compute_service(n, params=None): ... compute_service.optimize(epochs=20) ```* **`UniversalCodeTarget` & `UniversalFunctionTarget`:** Zero-boilerplate adapters for arbitrary source code snippets and callable objects.### 3.4 Invariant Test Harness & Statistical Benchmarking (`rsi.harness`)* **NASA-Standard Invariant Harness:** Evaluates mathematical assertions before and after mutation to guarantee functional equivalence and semantic correctness.* **Google-Standard Statistical Benchmarking:** Incorporates warmup execution cycles, statistical sample filtering, jitter rejection, and percentile analysis (p50, p90, p95, p99).---## 4. VERIFICATION & QUALITY ASSURANCEThe repository includes a comprehensive, automated test harness:* **`tests/test_sandbox.py`:** Validates timeout enforcement on infinite loops, OOM hard-kill (`EXIT_CODE_OOM`), graceful handling of abnormal terminations (`sys.exit`), and high-volume IPC buffer non-deadlock.* **`tests/test_ast_target.py`:** Validates AST static security policies, reflection attack prevention, safe globals scope preservation, and Two-Phase Commit staging.* **`tests/test_harness.py`:** Validates functional invariant regression tests and statistical metrics.* **`tests/test_pareto.py`:** Validates strict multi-dimensional Pareto dominance and hypervolume scalarization.* **`tests/test_rsi_integration.py`:** Validates end-to-end decorator workflows, Quine meta-optimization, and transactional rollback.**Test Results:** 31/31 unit and integration tests passing with 100% pass rate.---## 5. REPOSITORY STRUCTURE```textRSI/├── rsi/│ ├── __init__.py # Top-level API exports (v1.0.0)│ ├── core/│ │ ├── contracts.py # SelfModifiable, StateSnapshot, MetricVector│ │ ├── metrics.py # ParetoEvaluator & Hypervolume scalarization│ │ └── state.py # ImmutableStateSnapshot with SHA-256│ ├── sandbox/│ │ ├── runner.py # ProcessIsolatedSandbox, anti-deadlock IPC│ │ └── watchdog.py # ExecutionLimits, SandboxResult, OOM (42)│ ├── adapters/│ │ ├── ast_target.py # DynamicASTTarget with Two-Phase Commit (2PC)│ │ └── universal.py # @rsi_optimizable, UniversalCodeTarget│ ├── harness/│ │ ├── regression.py # Invariant regression verification harness│ │ └── statistical.py # Jitter-filtered statistical benchmarking│ ├── mutators/│ │ ├── ast_mutator.py # ASTSourceRewriter, ASTConstantMutator│ │ ├── ast_security.py # SecureASTValidator, ASTSecurityPolicy│ │ ├── param_mutator.py # HyperparameterMutator with Metropolis│ │ └── agentic_mutator.py # LLM-guided AgenticCodeMutator│ └── engine/│ └── rsi_orchestrator.py # RSIOrchestrator, OODA engine, Quine closure├── tests/ # Full automated test suite (31 tests)├── example_usage.py # End-to-end reference demonstration└── zenodo.doc # Complete Zenodo deposit description```---## 6. KEYWORDS & RESEARCH CATEGORIES**Keywords:** Recursive Self-Improvement (RSI), Software Self-Optimization, Autonomous Agentic Coding, Process Sandboxing, Two-Phase Commit, Pareto Dominance, Quine Closure, Fault-Tolerant Systems, AST Mutation, NASA Quality Standards, Metropolis Simulated Annealing. Architettura Neuro-Simbolica a Ciclo Chiuso per il Recursive Self-Improvement (RSI): Transizione da Spazi Vettoriali Continui a Ipergrafi Semantici NN -Dimensionali e Regolarizzazione Topologica Addendum Tecnico / Note di RicercaAutore: Luigi UsaiORCID: 0009-0003-3001-717XData: 20 Settembre 2026Classificazione MSC / ACM: I.2.6 (Artificial Intelligence – Learning); I.2.4 (Knowledge R