From single episodes to the final configuration
The thesis reports where the work ended. This page shows how it got there: 28 milestones from April to October 2026, including the readings that later turned out to be wrong and the validity incidents that exposed them.
Sources: Appendix E of the thesis (the dated validity incidents), the development history of the framework (docs/development-history.md) and the result files linked from each milestone. Chips show the stored result: wins / episodes, or the highest tower floor.
Single episodes
April – July 2026The first experiments scored one episode per configuration with a composite score. Several of their readings did not survive repetition; they are kept here because they shaped what was built next.
- April 2026finding
A first baseline on MT3
On the third floor of the tower, the rule-based heuristic scored 0.875 to 0.924, the reactive agent with Qwen3-30B 0.379, and a single coordinate plan made in one call 0.875.
reading then Per-turn reasoning cannot hold a floor-wide strategy; a plan made once can.what holds now On the simple floor, reactive control wins 14 of 20 episodes against 1 of 70 for committed plans, and most turns of the early one-call plans had fallen back to the heuristic. - April 2026finding
Prompt optimisation: +146 %
The first GEPA run raised the composite score on MT3 from 0.40 to 0.98, with every candidate scored on one episode.
reading then Evolved prompts carry durable strategic principles.what holds now Under repeated, paired evaluation with the annotations present, evolved prompts stay within the noise floor. - April 2026validity incident
A scorer with three bugs
A perfect score of 1.0000 turned out to be a false positive caused by three errors in the scorer. Results from before the correction are not comparable with later ones.
- April – July 2026finding
Bigger models, higher scores
On MT3, Qwen3-30B scored 0.379 while 70B and 235B models scored 0.992.
reading then A capability threshold lies between 30 and 70 billion parameters, and mixture-of-experts models beat dense ones.what holds now Not reproducible: three of the models were later removed from the inference service. In the repeated runs, model size alone did not order the results. - April – July 2026change
A published solver as the reference
The breadth-first solver released with the single-floor test maps gave the targets: 13 HP on the simple floor, 22 HP on the complex floor.
reading then Episodes that end the complex floor at 22 HP are optimal.what holds now The exact optimum of the complex floor is 68 HP (see 29 September). - April – July 2026finding
Without the engine's action list, plans collapse
Agents that saw only the map and had to name their own targets produced plans that broke down. Enumerating the legal actions, with paths and costs computed by the engine, became the core of the design.
- 1 June 2026finding
A loss and a win on identical inputs
Qwen3.5-397B played the simple floor twice within an hour on identical inputs at temperature 0: the first episode lost, the second killed the boss at 2 HP. One episode per configuration could not tell a better prompt from luck, and from then on every configuration was repeated.
Repeated episodes
20 – 23 September 2026The first campaign of repeated episodes, with success rates and Wilson intervals, turned up six threats to validity within four days.
- 20 Septembervalidity incident
The first 20 repeated episodes win nothing
An audit had replaced the placeholder effCost=None in the observation with a number, assuming a formatting error. Two models then won 0 of 20 episodes. The stored reasoning had read the placeholder as "not recommended yet"; with it restored, they won 5 of 10 each. A regression test now pins the observation text byte for byte.
- 20 – 21 Septembervalidity incident
Hidden factors
Two evaluation loops applied different overrides under the same name, a validation script mapped the simple floor to the prompt written for floors with shops, and a stale process kept running an obsolete configuration. Every result file now records its loop, prompt and settings.
- 21 Septembervalidity incident
Models leave the service
Three models of the early experiments were removed from the inference service. The first complex-floor wins could no longer be reproduced; the successor model won 0 of 10 with the prompt on which its predecessor had won 2 of 18.
- 21 – 23 Septemberchange
A tower engine
Stairs, shops and returns to earlier floors turned the single-floor engine into a tower engine. The heuristic climbs to MT8 and stops before monsters it cannot afford.
- 22 – 23 Septemberfinding
One survival rule
With the prompt that lists the solver's route, episodes on the complex floor kept to the listed fights even when the only one left was a lethal boss fight. One added survival rule took Qwen3.6-35B from 0 of 10 to 4 of 5 with that prompt.
Whose decision?
24 – 27 September 2026Comparing the model's answer with the executed action in every stored log changed the question from 'which prompt wins' to 'what does the model need to see'.
- 24 Septembervalidity incident
Silent overrides in every early win
In 94 turns of 68 stored episodes, the engine had executed a different action from the one the model chose, without logging it. All 21 stored simple-floor wins contained such a replacement, 17 of them at turn 9. Reproduced deliberately with logging, the configuration won 2 of 4; with mechanical overrides only, 0 of 5.
- 24 Septemberfinding
One ordering call worth 12 HP
A won and a lost episode fought the same monsters and ended with the same stats; they differed in one order. The observation had shown what the stat items behind a door would save, but not what reaching them cost. Two lines that state both sides, the net stat-route line and the fight-order line, took the same model and prompt from 0 of 5 to 10 of 10.
- 25 Septemberfinding
One-sided annotations as a class
The failures of an overnight batch of eleven cells had one cause in common: each decisive annotation stated a value, a saving, an efficiency, a deficit or a count without its counterpart. A further incident showed that switching the overrides off had also hidden the fight annotations, so overrides and annotations became separate settings.
- 26 Septemberchange
Keys were priced at nothing
Every remaining strategic failure was a key failure: keys were the one resource the engine did not price. A key budget in the state block and a key line on every door brought Qwen3.5-397B back to 10 of 10.
- 26 – 27 Septemberfinding
Less context, better decisions
The six previous decisions made up about three quarters of every request. Removing them raised Qwen3.6-35B from 1 of 5 to 10 of 10 on the same annotations.
- 27 Septemberchange
The final configuration
A positive order line ("take this fight before …") was followed as an instruction even for a fight that cost 46.5 % of the HP. With a 40 % guard on it, Qwen3.5-397B won 10 of 10 and Qwen3.6-35B 7 of 10: the final configuration of every later single-floor experiment.
Prompts, the optimum and plans
28 – 29 September 2026With the observation settled, the two other research questions were tested under the same protocol, and the reference itself was checked.
- 28 Septemberfinding
A pre-registered prompt-optimisation study
Hypotheses stated in advance; evolved, hand-written and rules-only prompts paired on thirteen held-out floors, with the noise floor measured directly. With the annotations, the evolved prompts stayed within it. Without them, the evolved prompts did not make up for the missing annotations; a later paired test found one narrow gain, which rests on the floors the rewrite was selected on.
- 29 Septembervalidity incident
The published optimum was wrong
While building hand-designed floors, an exact re-solve found that the published solver keeps the first state it reaches for each set of cleared obstacles instead of the best. The complex floor's optimum is 68 HP, not 22; every claim of an optimal complex-floor win was an artefact and was re-expressed.
- 29 Septemberchange
Hand-designed floors
Four floors with a single winning set each, which the heuristic fails and random play never wins in 500 games. They separate an agent that reads the annotations from one that does not.
- 29 Septemberfinding
Committed plans
The planning study let the model commit to 2, 3 or 5 actions, a whole-floor plan or a single plan. Four defects of the planning loop were found and fixed on the way. In the final count, committed plans won 1 of 70 simple-floor episodes against 14 of 20 for reactive play.
The tower
30 September – 5 October 2026The tower tests the same agent over 24 floors with shops, keys carried between floors and scripted events.
- 30 September – 1 Octoberchange
Every object on every visited floor
The second tower observation offered every object on every visited floor, not only the current one. The runs reached MT8, where the Red Bats could not be afforded.
- 1 – 2 Octoberfinding
A threshold set by replay
A replayed MT3 decision showed that Qwen3.5-397B declined the fight an expert takes in three of six calls at a 40 % limit on HP per fight, and in none at 60 %. The tower prompt's limit rose to 60 %, and an engine override that enforced 40 % was removed.
- 3 – 4 Octobervalidity incident
Two engine defects on the upper floors
Pricing every combination of shop purchases slowed one run to about 80 minutes per turn; the line was rewritten exactly. Two staircases that the game hides until a scripted event had been shown from the start, which overstated one run by three floors.
- 3 – 5 Octoberfinding
The final tower observation
Qwen3.5-397B reached MT18, DeepSeek-V4-Flash MT17, Qwen3.6-35B MT16 and Mistral Medium 3.5 MT8, the heuristic's floor. A stored run of Qwen3.5-397B, resumed on MT18 with blue keys for sale as in the original game, reached the normal ending on MT21.
- 5 Octobervalidity incident
The losing decision, located exactly
The complex-floor failures had been blamed on a conspicuous trap door. The exact state value of every state of every stored episode showed that all 30 had been lost earlier, 29 of them at the same cheap fight taken before the stats that make it nearly free. The experiments closed the same day.
Writing it down
5 – 18 October 2026- 11 – 18 Octoberchange
From ledger to thesis
The thesis was cut from 319 pages to just over 200: the development history moved to the framework's documentation, the tower designs were reduced to three observations, and the simple floor became the running example. Then it was submitted.