Flaky tests shouldn't block your build. Meet Flake filter: Chromatic now auto-detects unstable tests, so you can focus on real changes without getting blocked by flakes. 🧵
Chromatic Flake filter

ALT Chromatic Flake filter

Jul 14, 2026 · 5:54 PM UTC

3
5
267
Flake Filter renders every test multiple times to check its stability. Unstable ones move into an auto-ignored group. Their diffs stop blocking your build.
Chromatic flake filter ignores flaky tests automatically

ALT Chromatic flake filter ignores flaky tests automatically

1
78
Chromatic re-evaluates every test on every build. When a test stops flaking, it automatically rejoins the test suite.
1
58
Ignoring a flaky test doesn't mean ignoring the problem. Every unstable test gets a Playwright trace attached automatically (network, console, DOM) ready to reivew when you are.
Playwright trace is attached automatically via Flake filter with Chromatic

ALT Playwright trace is attached automatically via Flake filter with Chromatic

1
30
Bonus: you can now ignore any test manually too. New story not ready for review? Weird diff after a rebase? Set it aside, come back later.
Ignore flaky tests automatically and manually

ALT Ignore flaky tests automatically and manually

1
61
Flake Filter is live on every plan, on by default, with no setup required. Read the full announcement » chromatic.com/blog/automatic…

Automatic flaky-test filtering

Chromatic now auto-detects unstable tests, so you can focus on real changes without getting blocked by flakes

chromatic.com
55
Sort replies: Relevant Recent Liked
Replying to @chromaticcom
Keeping traces for ignored tests is important. I would also track an owner and time-to-repair, so automatic filtering does not quietly become lost coverage. Capture browser, OS and dependency changes with each flake: an unstable environment can look like an unstable test. More on that distinction: x.com/AgomaMitchell/status/2…
Mitchell Agoma
@AgomaMitchell
Sep 13
Your test code didn’t change. Your test result did. Before blaming “flaky automation”, check what changed underneath it. Homebrew 7.0.0 is out today. Its release notes cover changes to platform support, CI workflows and built-in vulnerability checks. For QA teams, that raises a bigger question: Are you versioning your test scripts—or the conditions that make their results meaningful? Imagine the same UI/API suite running on two machines. One uses yesterday’s browser and dependencies. The other installs newer versions during setup. One fails. The application may have regressed. The environment may have changed. Without evidence, you are guessing. Here is the environment contract I would add to a test framework: 1. RECORD WHAT ACTUALLY RAN Attach the application commit, OS, CPU architecture, runtime, browser and dependency versions to the test report. Record the container image digest where relevant. Never publish secrets in diagnostics. 2. PROVE A CLEAN START Run a scheduled build on a fresh runner with no inherited developer tools. Install declared dependencies, seed synthetic data and execute a small UI/API smoke pack. A failed setup must be reported as a failed setup—not a successful test run. 3. TEST THE SUPPORTED MATRIX Choose environments from product and team requirements. Check the combinations you actually support, including relevant OS and architecture differences. Identical scripts do not prove identical coverage. 4. SEPARATE COLD AND WARM RUNS Test with an empty cache and with the expected cache restored. Check both execution results and timing. A fast pipeline that depends on an undocumented cache is difficult to reproduce. 5. REVIEW TOOLCHAIN CHANGES LIKE CODE Use locked dependencies and immutable references where supported. Trial upgrades in a reviewable change, compare results and keep a recovery path. Controlled updates—not frozen, unpatched dependencies. Where does AI fit? Use it to compare sanitised environment manifests, explain setup errors and propose migration changes. Review those changes before adoption. Do not accept weakened assertions or disabled security controls merely to make CI green. Homebrew’s new vulnerability checks can contribute evidence. They do not certify your entire test environment as secure. For a manual tester becoming an SDET, this is a valuable shift: learn to explain why a test result is trustworthy, not only how to automate a click. Could someone rebuild your test environment tomorrow and explain every version difference? Source: Homebrew 7.0.0 release notes brew.sh/2026/09/13/homebrew-… #TestAutomation #CICD #DevOps #AI
5