The Wire · Showcase
CLI TEST SUITE DITCHES FLAKY SLEEPS FOR EXPLICIT WAITS
By RepoJournal · Filed · About Shopify
Shopify's CLI test infrastructure is killing fixed delays in favor of event-driven assertions, eliminating a class of timing-dependent test failures.
The team removed hardcoded `sleep()` calls and synthetic `setTimeout` polling from file watcher and app event tests [1][2], replacing them with Vitest's `vi.waitFor` helper [3][4]. This shift matters: tests that depend on elapsed time are inherently fragile, failing unpredictably on slower CI runners or during system load spikes. The file watcher tests were particularly problematic, mixing fixed waits with manual callback delays that made the suite slow and unreliable [3]. By switching to explicit event-driven waits, the tests now run faster and fail only when the code actually breaks, not when timing aligns wrong [4]. This is maintenance work that pays dividends immediately: faster feedback loops for developers and fewer ghost failures on deploy pipelines .
Action items
- → Review your own test suites for fixed sleeps and replace with vi.waitFor where applicable Shopify/cli [plan]
- → Monitor next CLI test run for performance improvement and flake reduction Shopify/cli [monitor]
References
- [1] Merge pull request #7846 from Shopify/06-17-remove_fixed_sleeps_from_file_watcher_tests Shopify/cli
- [2] Remove fixed sleeps from file watcher tests Shopify/cli
- [3] Remove fixed sleeps from file watcher tests ↗ Shopify/cli
- [4] Use vi.waitFor in app event watcher test ↗ Shopify/cli
FAQ
- What changed in Shopify on June 18, 2026?
- Shopify's CLI test infrastructure is killing fixed delays in favor of event-driven assertions, eliminating a class of timing-dependent test failures.
- What should Shopify teams do about it?
- Review your own test suites for fixed sleeps and replace with vi.waitFor where applicable • Monitor next CLI test run for performance improvement and flake reduction
- Which Shopify repositories shipped on June 18, 2026?
- Shopify/cli