tl;dr: Dream-RSI is a framework for recursive self-improvement in AI agents that treats accumulated discovery history as a "replay simulator," letting the system refine exploration policies via cheap off-policy feedback instead of expensive online rollouts. Improved policies are redeployed to drive new discoveries, expanding the simulator in a self-reinforcing loop. The authors report competitive or better results with lower discovery costs across algorithm engineering, math optimization, and GPU kernel engineering tasks.