Fix Flaky Tests: Boost E2E Reliability with Playwright & CI

By Krapton Engineering · Reviewed by a senior engineer · Last updated Jul 9, 2026

Flaky end-to-end (E2E) tests are a silent killer of developer productivity and CI/CD pipeline trust. One moment a test passes, the next it fails without any code changes, leaving engineers scratching their heads and delaying critical deployments. This inconsistency erodes confidence, forcing teams to waste valuable time re-running builds or manually verifying features that should have been automated.

TL;DR: Flaky tests stem from non-deterministic factors like async operations, shared state, and network issues. To fix flaky tests, adopt Playwright best practices such as robust selectors, explicit waiting, strict test isolation, and strategic network mocking. Implement a quarantine strategy and measure reliability improvements to restore trust in your E2E suite and accelerate delivery.

Key takeaways

• Flaky E2E tests are a major blocker for CI/CD confidence and deployment speed.
• Common root causes include asynchronous timing, shared mutable state, and external network dependencies.
• Playwright offers powerful features like explicit waiting and network interception to build more deterministic tests.
• Implement strict test isolation, robust selectors (getByRole), and a test data management strategy.
• Establish a quarantine process for flaky tests and prioritize fixing them to maintain pipeline health.

The Cost of Flaky Tests: Why Trust Matters

Imagine pushing a critical bug fix, only for your continuous integration (CI) pipeline to report a failure in an unrelated E2E test. You re-run the build, and it passes. This intermittent behavior, known as flakiness, is more than an annoyance; it's a significant drain on engineering resources and a direct threat to release velocity. When tests are unreliable, developers lose faith in the entire automation suite, leading to:

• Slower Deployments: Teams hesitate to deploy when CI results are untrustworthy, often opting for manual checks or multiple re-runs.
• Wasted Developer Time: Debugging non-deterministic failures is notoriously difficult and time-consuming, pulling engineers away from feature development.
• Eroded Confidence: A constantly failing or intermittently passing CI pipeline breeds cynicism, leading engineers to ignore test failures, potentially letting real bugs slip through.
• Increased Risk: Eventually, real regressions might be dismissed as "just another flaky test," leading to production incidents.

In a recent client engagement, we observed a team whose CI pipeline was running E2E tests for over an hour, with 30-40% of builds showing intermittent failures. This led to an average of three re-runs per build and a daily deployment cadence that was effectively blocked. Our intervention focused on identifying the root causes and implementing a systematic approach to improve their DevOps practices, starting with test reliability.

Root Causes of Flakiness: Unmasking the Culprits

Flaky tests are a symptom of non-determinism. They often arise from factors outside the direct control of the test logic itself. Understanding these common culprits is the first step towards building stable E2E suites.

Asynchronous Operations & Timing Issues

Web applications are inherently asynchronous. Network requests, animations, and dynamic content loading all happen over time. Tests that don't explicitly wait for these operations to complete before asserting can lead to race conditions.

Broken Pattern: Implicit Waits

// DON'T DO THIS: Implicit wait, prone to flakiness
await page.goto('https://example.com');
await page.waitForTimeout(2000); // Bad: arbitrary wait
await page.click('button#submit');
// Assertion here might fail if content loads slower than 2s

This approach relies on a fixed delay, which might be too short on a busy CI server or too long locally, wasting time. Real-world network latency or server response times are unpredictable, making such tests inherently flaky.

Shared State & Test Isolation

Tests should be isolated, meaning the outcome of one test should not influence another. When tests share mutable state—like database records, user sessions, or environment variables—they can interfere with each other, leading to order-dependent failures.

• Database Pollution: Tests that don't clean up their created data can cause subsequent tests to fail or pass unexpectedly.
• User Sessions: Logged-in state or specific user permissions carried over between tests.
• Global Variables/Cache: In some environments, global application state might persist.

Network Instability & External Dependencies

E2E tests often interact with backend APIs and third-party services. Unreliable network conditions, slow API responses, or rate limiting from external services can introduce non-determinism. Relying on a staging environment that is occasionally down or under heavy load will inevitably lead to flaky tests.

Implicit Waits & Race Conditions

Sometimes, elements appear on the page, but are not yet interactive, or JavaScript might still be processing. Clicking an element too early, or asserting on text that hasn't fully rendered, creates race conditions that are hard to debug.

Playwright Best Practices to Fix Flaky Tests

Playwright has emerged as the industry standard for E2E testing due to its robust auto-waiting capabilities, powerful selectors, and comprehensive API for browser automation. Leveraging its features effectively is key to building reliable tests.

Robust Selectors: The Foundation of Stability

Avoid brittle CSS or XPath selectors that rely on DOM structure, which changes frequently. Playwright encourages resilient, user-facing selectors.

• getByRole: Mimics how users interact with the page (e.g., getByRole('button', { name: 'Submit' })).
• getByText: Finds elements by their text content.
• getByLabel: Locates input fields by their associated label.
• getByTestId: The most stable, explicit choice. Add data-testid="my-component" attributes to your components for testing.

As Playwright's official documentation highlights, choosing the right locator strategy is fundamental for stable tests.

Explicit Waiting: Master Async Control

Playwright's auto-waiting mechanism is powerful, but you must still tell it *what* to wait for. Instead of arbitrary timeouts, wait for specific conditions.

Reliable Pattern: Explicit Waits

// DO THIS: Explicitly wait for conditions
await page.goto('https://example.com');
await expect(page.getByRole('button', { name: 'Submit' })).toBeVisible();
await page.getByRole('button', { name: 'Submit' }).click();
await page.waitForURL('**/dashboard'); // Wait for navigation to complete
await expect(page.locator('.success-message')).toHaveText('Logged in successfully');

Playwright automatically waits for elements to be visible, enabled, and stable before performing actions like click(). For network events, use waitForResponse or waitForRequest. For page navigation, use waitForURL or waitForLoadState.

Test Isolation: Clean State, Every Run

Each test should start from a known, clean state. This is paramount for determinism.

• beforeEach/afterEach Hooks: Use these to set up and tear down test data, log in users, or reset application state. For example, clearing local storage or deleting database entries.
• Test Data Factories: Programmatically generate unique test data for each test. Tools like Faker.js combined with database seeders (e.g., using Prisma or Sequelize ORM features) are invaluable.
• Ephemeral Environments: For complex setups, consider using Docker containers or ephemeral preview environments that are spun up and torn down for each test run or PR.

Network Mocking & Interception

To mitigate network flakiness and speed up tests, intercept and mock API calls. Playwright's page.route() API is excellent for this.

// Intercept and mock an API call
await page.route('**/api/users', async route => {
  const json = [{ id: 1, name: 'Krapton User' }];
  await route.fulfill({ json, status: 200 });
});

await page.goto('https://example.com/users');
// Now, the page will receive the mocked user data, regardless of backend status
await expect(page.getByText('Krapton User')).toBeVisible();

This allows tests to run consistently, even if the backend is slow or unavailable. However, be mindful that excessive mocking can create a false sense of security; ensure critical API contracts are still validated via API contract testing.

Parallel Execution & CI Sharding

Playwright supports running tests in parallel across multiple workers, significantly reducing execution time. Configure your CI system (e.g., GitHub Actions, GitLab CI, Jenkins) to shard tests across parallel jobs.

When NOT to use this approach: While powerful, over-relying on network mocking can sometimes mask real integration issues with your backend services. It's crucial to balance mocked E2E tests for speed and stability with a smaller suite of E2E tests that hit a live, stable staging environment to validate true end-to-end integration. Also, for very small projects with minimal async operations, the overhead of extensive mocking might not be worth the complexity.

Strategic Approaches to Managing Flakiness

Even with best practices, some flakiness might persist. A strategic approach is necessary to manage and eventually eliminate it.

Quarantine & Prioritize

When a test becomes flaky, don't let it block your pipeline indefinitely. Instead:

1. Quarantine: Temporarily mark the test as skipped or move it to a separate, non-blocking suite. This restores CI stability.
2. Create a Dedicated Task: Log the flaky test as a high-priority bug in your issue tracker.
3. Prioritize Fixing: Assign dedicated time to debug and fix quarantined tests. They are technical debt that must be paid.

Reproducibility & Debugging

Debugging flaky tests requires tools and discipline. Playwright's tracing feature (trace: 'on' in config) generates detailed reports with videos, screenshots, and step-by-step actions, making it easier to pinpoint the exact moment of failure. Run tests repeatedly in isolation and in parallel to observe patterns.

Measuring Success: The Payoff of Reliable E2E

The impact of fixing flaky tests is tangible. Our team measured that by implementing these strategies, a client's E2E suite went from 40% intermittent failures to less than 2% over three months. This translated to a 40% reduction in average CI build times, allowing developers to deploy multiple times a day with high confidence instead of struggling with daily deployments. The payoff is not just in faster pipelines, but in reduced stress, higher team morale, and ultimately, a more robust product.

A reliable E2E suite allows developers to iterate faster, knowing that critical user flows are protected. It empowers QA engineers to focus on exploratory testing and complex scenarios, rather than repeatedly validating basic functionality. This confidence directly translates into business value: quicker time-to-market for new features and fewer production incidents.

FAQ

What is a flaky test?

A flaky test is a software test that can pass or fail inconsistently for the same code, without any changes to the application or test environment. This non-deterministic behavior makes tests unreliable and undermines trust in automated testing.

How much test coverage is enough?

There's no magic number for test coverage; it depends on the project's criticality and complexity. Instead of focusing solely on line coverage percentages, prioritize covering critical business logic, common user flows, and error handling paths. A balanced test pyramid with strong unit, integration, and targeted E2E tests is generally more effective than aiming for 100% line coverage at all layers.

Is Playwright better than Cypress?

As of 2026, Playwright is generally considered to have surpassed Cypress for many use cases, particularly due to its broader browser support (Chromium, Firefox, WebKit), native parallel execution, and powerful API for network interception and context management. Both are excellent tools, but Playwright's architecture often provides more flexibility and performance for complex E2E automation.

Can AI help fix flaky tests?

AI can assist in identifying patterns in flaky test failures and even suggest potential fixes by analyzing code changes and test results. However, AI-generated tests or fixes still require human oversight to ensure assertions are meaningful and the underlying issues are genuinely resolved, not just masked.

Ready to Ship with Confidence? Partner with Krapton

Building a robust, reliable software product requires an engineering team that understands the nuances of modern testing and CI/CD pipelines. If flaky tests are slowing down your team and eroding deployment confidence, it's time for a strategic intervention. Book a free consultation with Krapton to discover how our expert engineers can implement Playwright best practices and advanced test automation strategies to deliver a stable, trustworthy E2E suite for your web and mobile applications.