← sources

ClashRoyaleAi: an open-source, deterministic Clash Royale simulator for RL, with recurrent PPO, lookahead search and expert iteration [P]

source: reddit (reddit:MachineLearning)published 2026-09-27 12:30ingested 2026-09-28 02:38

retained text

[The opponent plans by simulation: every second it scores each candidate play by running the match 10 seconds ahead in the engine.](https://i.redd.it/xts780wef2sh1.gif)

Our PPO agent learned to park its Cannon behind its own King. Losing a building in a fight cost reward, and letting it decay cost nothing, so it found the loophole.



It's one of many things we learned building a Clash Royale simulator from scratch so an agent could learn the game. The engine is deterministic C++ with Python bindings, plays a full match in about 10 ms on one laptop core, and can fork any game state in microseconds, so lookahead is cheap.



Best result so far: a simple 1-ply lookahead took the policy from 0.625 to 0.944 win rate against a heuristic bot (160 paired matches). Distilling it back into the network kept only +0.045.



The agent isn't strong yet, and RL isn't my home field, so feedback from people who know it better would mean a lot.



Repo: [https://github.com/itzik123/ClashRoyaleAi](https://github.com/itzik123/ClashRoyaleAi)

Built with my friend Ambash (most of the card roster). I used AI coding tools as a pair programmer.

document fetched 2026-09-28 00:28 · canonical https://reddit.com/r/MachineLearning/comments/1wrj0t3/clashroyaleai_an_opensource_deterministic_clash