First Haiku Run
Run 0 proved the machine works with a mock mutator that writes nothing genuinely new. Does swapping in a real model — one that can actually reason about the bot's code — meaningfully improve what the colony produces, inside the exact same loop?
100 generations run overnight, resuming the existing archive, with Haiku 4.5 writing every mutation through a Claude subscription CLI call (roughly 90 seconds per call — patience over API spend). First time a real model, rather than a template, wrote the bots.
Only one variable: the mock mutator was replaced with Haiku 4.5. Same arena, same archive, same sandbox.
- Final archive coverage and best Elo rating (bots are ranked against hand-written baselines and each other).
- How many times bots actually called tools inherited from the shared library, and how the library grew in size.
- The shape of the coverage curve over the 100 generations — flat, steady, or bursty.
Final coverage hit 77.8% (28 of 36 archive cells filled), the tool library grew from 3 to 11 tools, and best Elo climbed to roughly 1080. The library was doing real work: inherited tools were called by bots roughly 250,000 times across the run's evaluation matches, and by generation 201 a top elite was composing three different inherited tools together in one bot.
The coverage curve had a distinct shape: it climbed to about 50% and then went flat for roughly 40 generations, before a burst of strong bots broke through right around the point new tools entered the library. That timing looked like the library was causing the surge.
Two things worth being honest about:
- Haiku bots are draw-prone. The very first real child produced went 0 wins–0 losses–24 draws. The whole meta trended cautious and evasive — bots that survive by avoiding fights rather than winning them.
- The plateau-then-surge story was a correlation, not a proof. New tools entering the library and stronger bots appearing happened at roughly the same time — but that's not the same as the library causing the improvement. A smarter model alone, given more generations to cook, could produce the exact same curve with zero tool inheritance involved. I didn't have a control to tell the two apart.
Stop attributing results to a mechanism I hadn't isolated. The only honest next move is a proper ablation: run the exact same setup with the tool library surgically removed, and see if the gap disappears.
109 API calls · $0.93–$1.13 (Haiku 4.5 via Anthropic API).
Series through Run 1: 109 calls · $0.93–$1.13. Call counts are exact, straight from each run's metrics file. Costs are a range rather than a figure because the provider bills cached input tokens at half price and does not report the cached fraction in the API response, only in the console dashboard, so the upper bound prices every input token as uncached and the lower bound prices all of them as cache hits. A sampled dashboard minute showed 68% of input tokens were cache hits, which puts the truth nearer the bottom of each range. Token counts are measured exactly from Run 8 onward; earlier runs additionally reconstruct tokens from Run 8b's measured per-call profile (3,642 in / 1,348 out), because the usage field was discarded before it was ever written down. A bound is not a measurement and a reconstruction is neither, so both are labelled.
Raw receipts — every archive, lessons ledger, metrics file, and match replay: github.com/godfreystorm/ratchet-lab