Agentic genetic programming needs a world to fail in
· Christoph Heuwieser
Video: AI plays Age of Empires II, Emergent Garden, 15 August 2026
Emergent Garden's video "AI plays Age of Empires II" starts as a model shoot-out: Claude Fable, GPT Soul, Gemini Pro and Kimi K3 each write a bot script for the game. The interesting part starts at 8:57, "Optimizing Strategies", where he hands the job to a loop. A coding agent writes strategies, a round-robin tournament picks the winner, and the agent writes challengers to beat it, overnight. The code is on GitHub as AgentsOfEmpires, his extended fork of AutoDE.
"This is not a test of intelligence. It is an optimization algorithm," he says. The strategy file is the genome, and "unlike a normal genetic program, we're not using random mutations. We're using a language model to make mutations." He calls it "a kind of agentic genetic programming". That's his phrase, not an established field, and his loop is closer to a hill climb than to textbook genetic programming. The shape is what matters: a model proposes the change, a fitness signal decides whether it stays.
The loop is only as good as its scorer
He is candid about where it breaks: "these games are so long, there's a lot of randomness at play, and you can't run multiple games in parallel." The harness drives the real game with the mouse, so a match can take most of an hour. The repository's notes make the noise concrete: four unchanged scripts were re-run in the battle scenario, and only 2 of 6 matchup winners repeated. The overnight run cost about 15 dollars in all; the slow part was the games. The winner beats the default AI: "a tiny improvement, but it is something."
Every published version of the idea got past this the same way, with a cheap, trusted evaluator the model cannot touch:
- ELM (2022) used a code model as the mutation operator for programs that build small virtual walkers, scored in a physics simulator.
- FunSearch (2023) and AlphaEvolve (2025) let a language model propose programs and automated evaluators score them. The model never grades its own work.
- Eureka (2023) has a language model write reward functions for robots and judges each one by training policies in GPU-parallel simulation.
- Karpathy's autoresearch time-boxes every experiment to five minutes and keeps the evaluation code off-limits to the agent.
So the world a loop is scored in has to be:
- Fast, because evolution needs many generations.
- Cheap, because most candidates fail.
- Repeatable, so selection follows the change, not luck.
- Parallel, to score many candidates, and each more than once.
- Close enough to reality that a winner means something outside it.
- Out of the agent's reach, or the loop learns to game the scorer.
- Safe to fail in, because almost every mutation is a bad one.
A game is its own reality. A company isn't.
In a game, the game is the real thing: a lost match costs nothing, and a strategy that wins there has simply won.
Point the same loop at a producing organisation and none of the list holds. You can't run your plant twice from the same morning, a month of trial takes a month, and a bad mutation costs real orders. Before you can evolve anything there, you need a model of it to fail in.
Rehearse your policies in a realistic organisation
That is what World-Sim on a4sx is for: a simulated company that operates. Orders arrive (by email, in the pump-maker template), simulated staff key them into siloed business systems (ERP, MES, WMS, planning, mail and more) with realistic lag and mistakes, and goods are planned, produced with quirks and shipped. Nine templates range from a pump maker to a bakery, a brewery and a moulding shop: realistic generic companies, not a model of yours, covering business systems and material flow, not machine physics.
For a loop, the part that matters is the evaluation oracle. You send it a candidate (a Python run(w, params) function and its params) and a number of simulated days. It forks the live world into two identical copies, same state and same seeded generator, attaches your candidate to one and runs both. Back come the per-KPI delta, a fitness number (a direction-signed sum of the deltas) and an HMAC-signed receipt binding a hash of the candidate to its score. The candidate only ever runs in the forks, so the live world is untouched, and the ground truth stays behind the oracle: the agent sees deltas, never the scorer.
params is a genome you can tune; the source of run is a genome you can rewrite. The spec's own examples, on the pump-maker template (seed 7, 30 simulated days): keying email orders the same day raised on-time shipments by 6.1 points, and a reorder where a buyer emails the vendor and waits for confirmation raised them by 17.5 points and cut open orders from 17 to 8. Your instance's history differs, so your numbers will too.
What you could evolve
Each is a run() behaviour scored by the template's own KPIs. Possible experiments, not results:
- A reorder policy (floor, quantity, vendor, when to ask for confirmation): component stock-out days, on-time shipments, open orders.
- An order-intake runbook for incoming order emails: entry lag, entry error rate, orders lost.
- A dispatch and dock-booking rule: late shipments, freight overpay.
- A maintenance and mould-life policy for the moulding shop: OEE, unplanned downtime, moulds over their shot life, scrap.
- A quarantine-release and reorder policy for the bakery or the roastery: lots in quarantine, expiry write-offs, stock-out days.
The loop is the same with the game swapped out: write or mutate a behaviour, send it to the oracle, keep the child only if the KPIs you chose improve, go again.
Limits, and where the loop has to stop
The sim is not your plant. A policy that wins in a simulated company wins against that model's assumptions; robotics calls this the reality gap. Use the sim to rehearse and pre-screen, not as proof.
Evolution finds loopholes. It optimises exactly what you measure (specification gaming). The default fitness is crude: it adds deltas in mixed units, percentage points next to counts next to euros, so one large-unit KPI can dominate. Choose and weight the KPIs yourself, keep a few the loop never sees, and never let it touch the evaluator.
Noise doesn't disappear. The paired fork removes noise within one call, but calls at different times fork a different live state. Score a promising child more than once before you believe it.
Never let the loop act on a real plant. Evolve in the sim, then promote the winner as reviewed text (code, a runbook, a rule set) through human review and a staged rollout. The signed receipt is evidence of what was measured, not permission to deploy.
A world to fail in
The fix for a slow, noisy loop isn't a smarter model but a world to fail in that is repeatable, paired and out of the agent's reach. For a game, the game is that world. For a producing organisation, you need a simulated one first. Each generation is also an agent session, and that trail of rejected candidates may be worth keeping too.
To try it, point your agent at the World-Sim spec. A simulation costs €99 a month.