The Wire · Showcase
CLI TEST SUITE DITCHES FLAKY SLEEPS FOR EXPLICIT WAITS
By RepoJournal · Filed · About Shopify
Shopify's CLI test infrastructure is killing fixed delays in favor of event-driven assertions, eliminating a class of timing-dependent test failures.
The team removed hardcoded `sleep()` calls and synthetic `setTimeout` polling from file watcher and app event tests [1][2], replacing them with Vitest's `vi.waitFor` helper [3][4]. This shift matters: tests that depend on elapsed time are inherently fragile, failing unpredictably on slower CI runners or during system load spikes. The file watcher tests were particularly problematic, mixing fixed waits with manual callback delays that made the suite slow and unreliable [3]. By switching to explicit event-driven waits, the tests now run faster and fail only when the code actually breaks, not when timing aligns wrong [4]. This is maintenance work that pays dividends immediately: faster feedback loops for developers and fewer ghost failures on deploy pipelines .
One email a day. Unsubscribe in one click.
Keep up with Shopify in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
One email a day. Unsubscribe in one click. Read a past issue →
Action items
- → Review your own test suites for fixed sleeps and replace with vi.waitFor where applicable Shopify/cli [plan]
- → Monitor next CLI test run for performance improvement and flake reduction Shopify/cli [monitor]
References
- [1] Merge pull request #7846 from Shopify/06-17-remove_fixed_sleeps_from_file_watcher_tests Shopify/cli
- [2] Remove fixed sleeps from file watcher tests Shopify/cli
- [3] Remove fixed sleeps from file watcher tests ↗ Shopify/cli
- [4] Use vi.waitFor in app event watcher test ↗ Shopify/cli