How do you make async UI tests stop flaking without just adding waits?

programming web dev
1

Hey folks, I’m trying to get our Playwright tests stable and I’m hitting a reliability wall in CI. Locally everything passes, but in GitHub Actions the same test fails like 1/20 runs because the UI sometimes renders a stale state after a retry and the click hits the wrong row.

test('retries show latest status', async ({ page }) => {
  await page.goto('/orders');
  await page.route('**/api/orders**', route => route.continue());
  await page.getByRole('button', { name: 'Retry failed' }).click();
  await expect(page.getByText('Status: Success')).toBeVisible();
  await page.getByRole('row', { name: /Order 123/ }).getByRole('button', { name: 'Open' }).click();
});

What’s the most practical way to redesign this test (or the app hooks) so it asserts the right thing and stops being flaky when async ordering and retry caching occasionally show stale data?

1 Like
2

The flaky part is the page-wide Status: Success check. It can go green while the row you care about is still stale, which is exactly the kind of thing CI loves to punish.

I’d scope everything to Order 123 and wait on the data change for that row, not on “some success text somewhere.” In practice that means: grab the row first, click retry, wait for the /api/orders response that shows 123 actually updated, then assert inside that same row before clicking Open.

import { test, expect } from '@playwright/test';

test('retries show latest status', async ({ page }) => {
  await page.goto('/orders');

  const row = page.getByRole('row', { name: /Order 123/ });
  const openButton = row.getByRole('button', { name: 'Open' });

  const ordersUpdated = page.waitForResponse(async (resp) => {
    if (!resp.url().includes('/api/orders')) return false;
    if (resp.request().method() !== 'GET') return false;
    if (!resp.ok()) return false;

    const data = await resp.json();
    const order123 = data?.orders?.find((o: any) => o.id === 123 || o.number === '123');
    return order123?.status === 'success';
  });

  await page.getByRole('button', { name: 'Retry failed' }).click();
  await ordersUpdated;

  await expect(row.getByText(/Status:\s*Success/i)).toBeVisible();
  await openButton.click();
});

If you can change the app a bit, give each row a stable version or test id so the UI can prove “this row is the fresh one” instead of making the test guess. That usually saves a lot of pain later.