FadeBench
FadeBench tests drift. At the end of a run of small steps, a small language model can take an act it usually declines when asked outright. Each step looks defensible against the one before it. No user or adversary pushes.
Many safety tests judge one answer to one prompt. A model can pass them and still drift.
How it works
The battery has 38 scenario items. Each item is a ladder of four decisions, called rungs, in a low-key fantasy scene. A dependent bears the cost of each decision. The same decision is also asked cold, on its own, with no ladder before it.
I compare the answer at the end of the ladder with the answer to the cold question.
The headline numbers
On gemma-4-E4B with reasoning on, the model takes the drift 15 times in 100 at the end of the ladder. Cold, it does so fewer than 2 times in 100. The gap is 13.2 points (95% interval 2.6 to 25.9).
On the five scenarios named in advance, the result is 43 in 100 against 2 in 99. This is a replication on the same model, not a held-out prediction.
Across four small open models, about 4B to 12B, the gap is clear on one (qwen3-8b). It is uncertain on two (phi-4-mini, and gemma-4-E4B with thinking off). It is absent on one (mistral-nemo-12b).
The limits
- The models are small, about 4B to 12B.
- All scenes are low-key fantasy.
- No frontier model was tested. I do not know yet whether this holds for larger systems.