Training AI to Paint with Code(surya.website)
218 points by Tiberium 17 days ago | 28 comments
tl;dr: Researchers trained Qwen 3.5 35B via reinforcement learning (GRPO) to generate images by writing p5.brush JavaScript code, making the artifact editable rather than locked behind prompts. The main challenges were designing a reward function for subjective aesthetics—solved with pairwise judging against a hand-rated pool of 581 reference paintings—and preventing API hallucinations, which was fixed by replacing a 400-line API reference with a short allowlist evolved via GEPA. The authors concede it's slower than standard image generation but argue it opens up finer-grained creative control.
HN Discussion:
  • SVG would be a more direct code-as-image approach than JavaScript producing pixels
  • Enthusiasm for the approach as a fresh direction in AI-generated art
  • Appreciation for keeping humans as creators rather than spectators of locked-in outputs
  • Connects the work to prior research combining RL with visual generation techniques
  • Curiosity about extending the approach to other code-based drawing languages like Logo