Reaction-network simulation software is commonly checked only by aggregate signals such as time course run without error or the plausibility of tracked species. This work shows this is insufficient for a specific construct within SBML (the standard encoding format for reaction-network models), the AlgebraicRule, which fixes a species, compartment, or parameter's value through an exact algebraic constraint. Using the official SBML Test Suite as an external reference, we audit COPASI, a widely used open-source systems-biology simulator, against every algebraic-rule test case that provides a Level 3 Version 1 encoding and a published reference trajectory. A substring search returns 107 cases, but one does not contain an algebraic rule at all, and seven carry the rule inside an SBML hierarchical Model Composition submodel, not the top-level model. Structural verification resolves these issues, yielding a corrected corpus of 106 cases, independently confirmed by an unrelated simulator's load-time behaviour. Against this corpus, COPASI 4.46.300 matches the reference trajectory in only 7 cases (6.6%); none of these seven demonstrates genuine algebraic-constraint resolution, because the matches arise from coincidental unit defaults or kinetic cancellation. In 81 cases (76.4%) the algebraically determined quantity is silently absent from output, with no error or warning in the scripting API's message channel; in 16 cases (15.1%) the quantity is present but numerically wrong, by 22% to 765% in 15 of the 16 and by roughly three orders of magnitude in the sixteenth; and two cases (1.9%) crash the interpreter. As a control, libRoadRunner (and Tellurium) rejects every case with an explicit exception naming algebraic rules as unsupported, with no silent degradations, wrong answers, or crashes. We trace COPASI's dominant failure to its internal representation: a rule-governed quantity has no status for "governed by an algebraic constraint," only fixed, assignment, reaction, ODE, or time. Consequently, a compartment fixed by an algebraic rule silently defaults to 1 regardless of the rule's target; the defect is numerically invisible under first-order kinetics (where the size cancels) but produces large errors under second-order kinetics. A minimal model demonstrates the practical consequence: an algebraically defined species drives a downstream reaction that remains dead throughout the simulation, while an unrelated reaction proceeds correctly, with no visible output indicating that a pathway is dead. We report this audit along with four errors found and corrected in our own methodology, quantifying each correction's effect. All findings are packaged as a reproducible comparison against the SBML Test Suite's reference results, and set out for filing with the COPASI issue tracker. Code and reproducibility: https://github.com/TesfayZ/copasi-algebraic-rule-audit
Reaction-network simulation software is commonly checked only by aggregate signals such as time course run without error or the plausibility of tracked species. This work shows this is insufficient for a specific construct within SBML (the standard encoding format for reaction-network models), the AlgebraicRule, which fixes a species, compartment, or parameter's value through an exact algebraic constraint. Using the official SBML Test Suite as an external reference, we audit COPASI, a widely used open-source systems-biology simulator, against every algebraic-rule test case that provides a Level 3 Version 1 encoding and a published reference trajectory. A substring search returns 107 cases, but one does not contain an algebraic rule at all, and seven carry the rule inside an SBML hierarchical Model Composition submodel, not the top-level model. Structural verification resolves these issues, yielding a corrected corpus of 106 cases, independently confirmed by an unrelated simulator's load-time behaviour. Against this corpus, COPASI 4.46.300 matches the reference trajectory in only 7 cases (6.6%); none of these seven demonstrates genuine algebraic-constraint resolution, because the matches arise from coincidental unit defaults or kinetic cancellation. In 81 cases (76.4%) the algebraically determined quantity is silently absent from output, with no error or warning in the scripting API's message channel; in 16 cases (15.1%) the quantity is present but numerically wrong, by 22% to 765% in 15 of the 16 and by roughly three orders of magnitude in the sixteenth; and two cases (1.9%) crash the interpreter. As a control, libRoadRunner (and Tellurium) rejects every case with an explicit exception naming algebraic rules as unsupported, with no silent degradations, wrong answers, or crashes. We trace COPASI's dominant failure to its internal representation: a rule-governed quantity has no status for "governed by an algebraic constraint," only fixed, assignment, reaction, ODE, or time. Consequently, a compartment fixed by an algebraic rule silently defaults to 1 regardless of the rule's target; the defect is numerically invisible under first-order kinetics (where the size cancels) but produces large errors under second-order kinetics. A minimal model demonstrates the practical consequence: an algebraically defined species drives a downstream reaction that remains dead throughout the simulation, while an unrelated reaction proceeds correctly, with no visible output indicating that a pathway is dead. We report this audit along with four errors found and corrected in our own methodology, quantifying each correction's effect. All findings are packaged as a reproducible comparison against the SBML Test Suite's reference results, and set out for filing with the COPASI issue tracker. Code and reproducibility: https://github.com/TesfayZ/copasi-algebraic-rule-audit
A large literature builds physical structure into learned dynamics on the premise that models respecting the underlying physics predict better. This work tests that premise using exact polynomial invariants recovered from trajectories and canonicalised as reduced Gröbner bases over ℚ. On Acrobot, exactness provides little benefit for prediction: a consistency regulariser reduces algebraic residual while leaving rollout fidelity essentially unchanged, and one-step validation error remains the better predictor of long-horizon accuracy. Likewise, a shaping potential recovered from a system with a 100% mass error accelerates learning as effectively as the correct potential, indicating that shaping depends on potential geometry rather than physical correctness. Exact canonical invariants instead prove valuable for diagnosis. This paper develops two procedures: screening, which identifies the violated physical constraint, and attribution, which recovers the faulty invariant and identifies the responsible physical parameter. Two algorithmic components enable this diagnostic: normal-form deflation, which removes algebraically trivial multiples exactly, and quotient-space recovery, which avoids tolerance-based nullspace dimension estimation. Across fifteen injected faults, screening localises every broken constraint with no observed false alarms, whereas observation-space baselines localise none; attribution recovers the responsible parameter on all seven single-parameter faults. Paired difference tests detect all faults, showing that the advantage is localisation rather than detection. Perturbing reference generators by 10⁻⁴ preserves 14–15/15 localisations, showing that screening does not require exactness, whereas ideal-equality decisions distinguish perturbations of only 10⁻¹², showing that exact canonical representations are required for algebraic comparison. Applied to 350 release pairs across eleven RL environments, the diagnostic finds no evidence of changed simulator dynamics, instead revealing properties of the benchmark implementations themselves. Code and reproducibility: https://github.com/TesfayZ/algebraicRLtest.git
A large literature builds physical structure into learned dynamics on the premise that models respecting the underlying physics predict better. This work tests that premise using exact polynomial invariants recovered from trajectories and canonicalised as reduced Gröbner bases over ℚ. On Acrobot, exactness provides little benefit for prediction: a consistency regulariser reduces algebraic residual while leaving rollout fidelity essentially unchanged, and one-step validation error remains the better predictor of long-horizon accuracy. Likewise, a shaping potential recovered from a system with a 100% mass error accelerates learning as effectively as the correct potential, indicating that shaping depends on potential geometry rather than physical correctness. Exact canonical invariants instead prove valuable for diagnosis. This paper develops two procedures: screening, which identifies the violated physical constraint, and attribution, which recovers the faulty invariant and identifies the responsible physical parameter. Two algorithmic components enable this diagnostic: normal-form deflation, which removes algebraically trivial multiples exactly, and quotient-space recovery, which avoids tolerance-based nullspace dimension estimation. Across fifteen injected faults, screening localises every broken constraint with no observed false alarms, whereas observation-space baselines localise none; attribution recovers the responsible parameter on all seven single-parameter faults. Paired difference tests detect all faults, showing that the advantage is localisation rather than detection. Perturbing reference generators by 10⁻⁴ preserves 14–15/15 localisations, showing that screening does not require exactness, whereas ideal-equality decisions distinguish perturbations of only 10⁻¹², showing that exact canonical representations are required for algebraic comparison. Applied to 350 release pairs across eleven RL environments, the diagnostic finds no evidence of changed simulator dynamics, instead revealing properties of the benchmark implementations themselves. Code and reproducibility: https://github.com/TesfayZ/algebraicRLtest.git
