Test Plan
A Test Plan is a collection of Test Suites that belong to specific tasks or features or have another reason to be run together. In general, the primary use of Test Plans is running several Test Suites integrated into your Continuous Delivery pipeline, although you may still run it manually in the UI.
The image below shows the Test Plan hierarchy in Loadmill.
Creating and running a Test Plan
Let's create and run your first Test Plan together:
Navigate to the Test Plans tab and click on the NEW TEST PLAN button.
Let's look at the Test Plan editor page.
3. Click on MANAGE SUITES to add Test Suites to the Test Plan. You can pick specific Test Suites or all by clicking ALL SUITES. Once the Test Suites are selected, click CLOSE.
🧠 Make sure the Test Suites you would like to run within the Test Plan have test flows with CI toggle on. Otherwise, the Test Suites will be grayed out and won't run.
4. Now you have two options:
Run the Test Plan by clicking RUN TEST PLAN.
Use the RUN BY LABELS option to select relevant labels and then click RUN TEST PLAN so that only Test Suites whose test flows have specific labels will run.
Test Plan navigation bar
The navigation bar allows:
To set Test Suites to run in parallel (up to 12) so you will be able to significantly save running time.
To select a GitHub repository branch to take Test Suite data from while using the GitHub integration.
To configure Test Plan parameters values so that the Test Plan run will take these values in all the Test Suites using the parameters.
4. To define a Setup Flow that will run before all suites of the Test Plan. The Setup Flow common use cases: creation of an account, user, env cleanup and any other action/s you need to perform before running the Test Plan suites. You can either create the flow from scratch in the Setup Flow tab or use a shared flow by clicking on USE A SHARED FLOW.
5. Set Test Plan E-mail or Slack notifications so that you will get one email/slack notification with all the Test Suites' information.
6. To schedule the Test Plan to run periodically to constantly validate your application status.
Integrating Test Plan into CI/CD
To integrate Test Plans into your Continuous Delivery pipeline, use our npm module. See an example of how to launch a Test Plan below:
Find more examples and supported CLI options here.
Analyzing Test Plan results
After running the Test Plan, you will be redirected to the Test Plan Run page. The Test Plan Run report page shows the list of executed Test Suites as defined in the Test Plan. The Test Suites table shows each suite's description, the number of its flows, duration, and status. Use the table filter to filter the Test Suites by their status.
You can easily re-run Test Plan by clicking RE-RUN or Run only Failed Suites.
Last updated
