Design QA
Visual regression testing
Visual regression testing is an automated check that renders a page or component, takes a screenshot, and compares it pixel by pixel with an approved baseline screenshot of the same thing.
Any difference above a threshold fails the test, so a change to CSS, a dependency or shared layout cannot silently alter what users see. It catches unintended change, not design mismatches.
At a glance
| Compares | A fresh screenshot of a page or component with an approved baseline screenshot of the same page or component |
| Catches | Unintended change: a CSS edit, a dependency upgrade or a shared layout change that moved, hid or restyled something |
| Misses | Anything the baseline already got wrong, and every gap between the build and the design, because the design file is never an input |
| Threshold | A per-pixel color tolerance or a maximum count of changed pixels. Playwright defaults the pixel threshold to 0.2 (Playwright); BackstopJS defaults its mismatch threshold to 0.1 percent of pixels (BackstopJS) |
| Runs | In CI on each pull request against components or key pages, and on a schedule against a deployed environment |
| Found by | Not as a screenshot diff. Visual Comparison compares the live page with the Figma design instead of an earlier screenshot of itself, and Scheduled Scans re-run a saved scan on a schedule and email only the findings added or fixed since the last run |
Why it matters
Functional tests click a button and assert on the result; they never look at the screen, so a button that still works while it sits under a sticky header passes every one of them. Chromatic's docs make the same point: functional tests do not validate rendered pixels (Chromatic). Visual regression testing is the cheapest way to notice that a shared stylesheet, a token change or a dependency upgrade altered pages nobody touched in the pull request. Its limit matters just as much: it compares the UI with its own past, so a page that shipped 20px of card padding where the design said 16px keeps passing forever, because the baseline carries the mistake. That is the gap design QA fills.
How does visual regression testing work?
- 1. Capture a baseline. The first run has nothing to compare with, so it writes reference screenshots: Playwright keeps capturing until two consecutive shots match and stores them next to the test, and Storybook's visual tests create their baselines on the first build (Playwright, Storybook).
- 2. Re-render and diff. Every later run takes a fresh screenshot and compares it with the reference pixel by pixel. BackstopJS describes itself as comparing test screenshots with reference screenshots over time (BackstopJS).
- 3. Apply a threshold. A tolerance decides what counts as a change. Playwright's threshold is a per-pixel color distance, 0.2 by default, with an optional cap on the number of differing pixels; BackstopJS also requires both images to have the same dimensions unless told otherwise.
- 4. Review and accept. A person looks at the highlighted diff. Intended changes are promoted to the new baseline with a command such as Playwright's update-snapshots flag or BackstopJS's approve; unintended ones go back to the author.
- 5. Hold the environment still. Rendering differs by browser, operating system, headless mode and even the machine's power source, so Playwright names each snapshot per browser and platform, and the comparison has to run where the baseline was made (Playwright).
Visual regression testing vs design QA
| Visual regression testing | Design QA | |
|---|---|---|
| Reference | An approved earlier screenshot of the same UI | The approved design file |
| Question it answers | Did anything change since last time? | Does the build match the design? |
| First run | Finds nothing; it only writes the baseline | Finds every mismatch already live |
| Blind spot | A baseline captured with the bug in it | Regressions on pages nobody re-checked |
Common mistakes
- Trusting the first green run. The first run only writes the baseline, so a bug already on the page becomes the reference and every later run agrees with it.
- Comparing across environments. A snapshot taken on a developer's Mac diffs against a Linux runner because fonts and anti-aliasing differ. Keep baselines per browser and platform, and generate them where CI runs.
- Leaving dynamic content in the frame. Dates, carousels, ads and animations change on every run. Stub them or mask the region, or the suite cries wolf until everyone stops reading it.
- Raising the threshold until it is quiet. A tolerance wide enough to ignore font hinting also ignores the 2px shift you wanted to catch. Tune per screenshot and mask instead.
- Calling it design QA. A diff against last week's screenshot says nothing about whether last week's screenshot matched the design. Run both checks.
Frequently asked questions
Is visual regression testing the same as snapshot testing?
No. Snapshot tests compare serialized markup or component output, while visual regression tests compare rendered pixels. The DOM can be identical while a CSS change moves every element, and only the pixel comparison sees it.
Which tools do visual regression testing?
Playwright's toHaveScreenshot assertion, BackstopJS, Chromatic for Storybook, Percy, Applitools and Lost Pixel all work by comparing new screenshots with stored baselines. They differ in where the screenshots render, how diffs are reviewed and what a run costs.
How do you stop visual regression tests from being flaky?
Fix the environment before the threshold: run on the same browser and operating system that produced the baseline, wait for fonts and network requests to settle, stub dynamic content such as dates and ads, and mask animated regions. Only then adjust the tolerance, per screenshot.
Does OverlayQA do visual regression testing?
Not as a screenshot-against-baseline diff. OverlayQA's Visual Comparison compares the live page with the Figma design, which is the mismatch visual regression testing cannot see, and Scheduled Scans re-run saved scans on a schedule and report the findings that were added or fixed since the previous run.
When should visual regression tests run?
On every pull request that can touch the UI, against components in isolation or a few key pages, plus a scheduled run against staging or production to catch changes that arrive through dependencies and content rather than code.
Sources: Playwright, Visual comparisons, BackstopJS, README, Storybook, Visual tests, Chromatic, Visual tests.
Last updated October 22, 2026