01 · PROBLEMAn event is more than its mechanics
Big live events are expensive to build and have one chance to land. They depend on things a normal test leaves out: what players have in their accounts, how missions feel in sequence, how the rewards and pacing play together over the whole event.
Testing a simplified slice would have answered the wrong question. The team needed to know whether players would understand the event as it was designed, in the conditions they would actually play it.
02 · SETUPA full rehearsal, not a slice
I rebuilt the event for the test the way players would meet it on release:
- the right amount of resources on every test account, matching what real players would have at that point;
- realistic missions in the order players would get them;
- the full event scenario, not an isolated feature;
- automated and scripted, so every session started from the same state and results were comparable.
The event's designers and producers were invited to watch the sessions live.
03 · FINDINGPlayers read a crucial part of the design differently
In the sessions, players consistently understood a key part of the event differently from how the designers intended it. Because the setup was realistic, this couldn't be written off as a test artefact. Because the team was watching, nobody had to be convinced by a report: they saw it happen.
04 · OUTCOMEChanged before release, shipped with the fix
The team changed the design before release, and the event shipped with the fix. Specific metrics stay with Wargaming, but the decision itself was made in the room, on the day the team saw the sessions.
05 · WHAT MADE IT WORK3 choices I'd make again
- Test what players will meet, not a prototype of it. The extra setup work is what made the finding impossible to dismiss.
- Make sessions identical. Scripting removed noise, so a pattern across players was clearly a pattern.
- Bring the decision makers to the sessions. Watching players struggle changes minds faster than any slide.