Sequential and parallel scenario runs


#1

Is it possible to run sequential tests? What about parallel tests? How do we go about setting this up?


#2

Hi rebeccaevans, that seems to be the responsibility of how you execute tests but not hiptest :slight_smile: What test framework do you use and what is your test runner?


#3

@tikolakin, BDD with Gherkin, test runner still in the air. Planning manual testing in meantime.

Wondering how anyone handles running parallel and sequential tests rather than have a tester do each case in what seems to be random order by title in ASC.

Sequential:
How are the tests put in the test runs? ex) are they in order they were selected? in the stacking inheritance order of the folders? are they ASC by title? etc. So that we may choose a logical sequence of tests or order them in the run.

Parallel:
Imagine the back and forth messaging nature of a texting application – where one set of tests is done by the sender and the other the recipient but best performed at the same time. Rather than multiple accounts in a single test case per step, wondering if there is a way to perform multiple test cases by single accounts in parallel fashion. Only way I imagine is by using multiple browsers with different hiptest sessions and different tests. So hoping for test case linking (besides just tags).


#4

hey, @dillonbatenich
it’s quite interesting position when we look at it from the manual execution point of view

  1. when you execute test sequentially you just get what you selected when created a test run. Basically, it’s hierarchical organization of scenarios by folders.
  2. I’m not quite sure I got it right. If your aim is to execute test cases by different users (probably different env?) I would go with creating test run per test execution and assign them to a different person for executions (manual I’d guess)

Does it help?


#5

With Behat framwork, tests will be executed by alphabitic order of folders.
The tip i find is to prefix each folders by an index like this : 01 - My first folder


#6

ok, why do you need to execute them in that strict sequence?


#7

Why do sequential testing? Sometimes one test’s ending is another test’s beginning. In each of your tests you can include all setup preconditions etc. but things will run more smoothly if they are also run in a desired sequence–especially for manual tests.


#8

generally, it doesn’t give you 100% guaranty of test stability
test should not depends on each other, otherwise, if one test fails - execution of rest doesn’t make sense

for manual test execution I would agree partially, but to solve it I would look at sth called user journeys - https://www.martinfowler.com/bliki/UserJourneyTest.html
see also story tests https://www.martinfowler.com/bliki/StoryTest.html
I doubt there are ways in hiptest to orginize sceanrios in journey style, and I would doubt it should be like it when we talk about BDD and what it tries to solve.