Promoting from staging to production frequently requires a comprehensive test suite that runs quickly, stays green, and is easy to add new tests to. Across all of our services, we’re up to more than 60 deploys a week – as the number of weekly has grown, it’s important to make sure that our test suite is able to keep up. Our past suite setup had issues adding tests, running tests in both desktop and mobile mode, and making sure all the tests stay green. This led to us not having as strong a test suite as we needed, sometimes with customer-facing defects escaping from staging into production.
Because of these pains, we recently converted to using TestArmada’s Magellan Test runner, which automatically splits a test suite up and runs it in parallel. Want to run the entire suite from your developer box against a staging environment? Want to split up the suite to run eight concurrent tests and run each individually? Want to set a global retry policy for all your end-to-end tests? Magellan has made all of this easy and helped us greatly improve our test suite’s overall quality.