Scale and multiply tests per device, screen orientation, OS version etc


We have multiple scenarios that need to be covered on a broad range of devices, screen orientations, and OS versions, the goal being to write once and test them all without creating separate test runs for each device, screen orientation, OS version and so on.

ex) testing “log in happy flow”

a) log in: on iphone > on OS11 and OS10 (which will have two more branches each) in portrait and landscape mode.

So one test easily becomes 4 scenarios.
-----Add an ipad with two OS and it becomes 8 scenarios.

Can the tests be scaled in such a way to all fit all in one test run? I’ve seen this a little bit with datatables on scenarios, but the problem would be having to create a new data table on each scenario.

How would you suggest solving for covering all bases and widely scaling scenarios?



We are working on a way to execute the same test run on multiple environments:

The feature itself is pretty advanced, but not available for users.
We need to completely redesign the way we display execution metrics all across the test run pages (folder, tests…) before releasing it.

In a meanwhile, you can indeed mix multiple test runs (to handle the systems your running your tests on) and datatables (for screen orientations) to handle your execution campaign :slight_smile:

Hope it helps!