Localization regressions: why it was fine last build is not evidence
A source string gets a small edit for clarity. Nobody flags it for retranslation, because the change looks cosmetic in English. Three languages down the line, the old translation no longer matches the new meaning — and nobody notices until a player does.
This is a localization regression: a previously correct translation becomes wrong not because a translator made a new mistake, but because something around it changed. It is the same failure mode as a code regression, and it deserves the same discipline: you cannot rely on memory, you need a baseline and a diff.
Three ways translations go stale without anyone touching them
Source edits are the most common trigger. A designer rewords a tooltip, a writer tightens a line of dialogue, a programmer renames a placeholder. None of these look like translation work from where they sit, but every one of them can invalidate an approved line in every target language.
Bulk retranslation and re-export are the second trigger. Running a batch of strings through a new translation pass, or re-exporting a localization file from a spreadsheet or CAT tool, can shift line ordering, re-wrap text, or quietly touch rows nobody meant to change alongside the ones that needed updating.
Reimporting an old file version is the third, and the sneakiest: a stale export gets pulled back into the build — from a backup, an old branch, an email attachment — and a bug that was already fixed reappears, looking exactly like a fresh issue.
Why it was fine last build is not evidence
It was fine last build only tells you about the specific file, string set, and source text that existed at that point in time. None of those three things are guaranteed to be stable between builds. A localization file is not a fact you check once — it is state that drifts every time source text, translation, or the export pipeline changes.
This is exactly the reasoning that makes engineers distrust it worked on my machine as a QA argument. The fix in both cases is the same: stop relying on recollection and start comparing actual states.
Baseline each run, then diff
Treat every check as a comparison against the last known-good state of that file, not as an isolated pass/fail. Concretely, that means keeping a record of the issues found in the previous run and classifying every issue in the current run against it:
- New — did not appear in the previous run on this file
- Resolved — appeared before and is now absent
- Recurring — appeared before, was fixed at some point, and is back
What this buys you
A resolved-count near zero after a translation pass is a useful early warning, independent of whether the raw issue count looks acceptable. A recurring issue is a strong signal that a specific fix did not survive an export or reimport step — worth investigating the pipeline, not just the string.
It also changes what a review actually looks at. Instead of re-reading every line in a file with hundreds of strings, a translator or reviewer can focus on the handful that are new or recurring, which is both faster and more likely to catch the thing that actually changed.
Treat localization files like code
The underlying idea is not new — it is regression testing, applied to a domain that rarely gets it. Localization files change on every content update, get exported and reimported by more than one tool, and pass through people who are not always looking at the same diff a programmer would see in version control.
Extending the same discipline that code already gets — a baseline, a diff, a classification of what changed and why — turns it was fine last build from a guess into something you can actually check.