Structured Episodic Memory (SEM)

Abstract

Structured Episodic Memory (SEM) is an experimental non-neural agent architecture that combines overlapping state abstractions, associative value memories, eligibility-based temporal credit assignment, goal-conditioned contact control, and context-dependent strategy memory. We evaluate SEM in Pong using structured state observations rather than pixels and without an analytical future-trajectory or interception solver. Across 40 sequential training runs comprising 2,200 incoming outcomes, the trained controller achieved a lifetime hit rate of 0.958, while frozen evaluation on six fresh seeds reached a mean hit rate of 0.974. Controlled ablations show that temporal eligibility improves motor learning, goal-conditioned aim memory improves contact control, fast memory accelerates adaptation to hidden opponent changes, and context-specific memory is required for high performance when action values conflict across contexts. Under a hard compound physics shift, the frozen controller retained a mean hit rate above 0.92; coarse overlapping abstractions preserved most of this performance even when exact fine-grained state keys were unavailable. SEM also partially re-learned a hidden actuator inversion from sparse hit/miss feedback and showed faster reacquisition on repeated exposure than on first exposure. These results do not establish superiority over neural or conventional reinforcement-learning methods, nor do they imply learned representations. Instead, they characterize how structured associative memory and sparse temporal credit can support motor control, contextual adaptation, transfer, and partial retention within the tested Pong environment.