How do you deal with flaky automated tests?

Model answer

A flaky test passes and fails on the same code. It erodes trust, because people start ignoring red builds.

Detect and contain. Track failures per test over time and rerun suspects many times to confirm. Move a confirmed flaky test out of the blocking path (quarantine) with an owner and a deadline, instead of deleting it silently or retrying forever.

Find the root cause. Common causes:

Timing and async waits: fixed sleeps or asserting before the UI is ready. Wait for a specific condition or element, not for time.
Shared or leftover state: tests that depend on order or on data from other tests. Make each test independent with its own setup and cleanup.
Environment: unstable test servers, slow shared resources and network calls to real third parties. Use stubs or containers.
Non-deterministic inputs: time, randomness and concurrency. Control clocks and seeds.
Test data collisions in parallel runs.

A retry can reduce noise but hides the problem, so report retried passes and keep them visible.

Finally, add review rules for new tests and keep the flaky rate as a tracked metric.

Key points

Measure flakiness per test and quarantine confirmed offenders with an owner
Replace fixed sleeps with waits on explicit conditions
Make tests independent: own data, setup and cleanup
Control time, randomness and external dependencies
Retries hide flakiness; keep them visible

Likely follow-ups

How would you find which test pollutes shared state?
When is it right to delete a flaky test?

Common mistakes

Adding automatic retries and moving on
Adding longer sleeps instead of waiting on a condition