Loadmill
Learn moreSchedule a demo
  • Introduction
    • Loadmill - AI - Powered Solution
    • Deviceless mobile testing
      • Capturing traffic with Loadmill MITM Proxy
      • Loadmill desktop recorder
        • Generating test flows
      • Installing certificate on mobile devices
        • iOS certificate installation
        • Android certificate installation
      • Configuring proxy on mobile devices
        • iOS Wi-Fi settings
        • Android Wi-Fi settings
      • Troubleshooting
    • What is an API
      • API - Data Fetching
      • Quick examples of API requests
      • What is an API endpoint?
    • API Server Testing
      • Contract testing
      • Regression Testing
  • Quick Guide
    • Create Account
    • Download Test Composer
    • Register your first API flow
    • Running Your API Test
  • Loadmill Test Composer
    • Quickstart
    • Composer Layout
    • Filter Settings
  • Test Editor
    • Layout
    • Flows
      • Generated Flow Code
      • Test Flow editor
      • Flow Controls
      • Add CSV to Flow
      • Flow Execution
    • Steps
      • Request step
      • Code step
      • Extraction & Assertion step
      • Web Socket step
    • Extractions - Set Parameters
    • Assertions - Verify Response
    • Parameters
      • Parameter Execution Order
      • Test Suite Parameters
      • Parameters Sets
    • ⨍(⨯) FUNCTIONS
    • Postscript
      • Running Postscript
      • Accessing w/ Postscript
      • Validating Postscript
    • Login/Authentication Flow
    • Before & After Hooks
  • Load Testing
    • Load Test Editor
    • Load Testing Guide
    • Analyzing Load Test Results
    • Parameterized Load Test
    • Domain Verification
    • Configuration Files
    • Load Testing FAQs
    • Load Testing Troubleshooting
  • User Behavior Testing
    • Overview
    • Setup
    • Recording troubleshooting
    • Additional recording methods
    • Recording settings
    • How to work with Recordings
  • Auth
    • Okta SSO integration
    • API Tokens
    • Testing with CORS
    • REST API
  • Integrations
    • Loadmill Agent
    • CI integration
    • GitHub
      • CI integration
      • Data sync
    • GitLab
    • Bitbucket
    • Jira
    • New Relic
    • Slack integration
    • TestRail integration
    • Database Testing
    • Kafka Testing
    • Datadog Integration
    • ✉️Email Testing
    • Webhook Testing
    • Integrations FAQs
    • XRay
    • TestRail
    • gRPC Support
  • Collaboration
    • Collaboration
    • Teams
    • Groups & Reports
    • Test Suite Collaboration
    • Reviews
    • Shared Flows
    • Labels
  • Reporting
    • API Catalog & Coverage
      • API Catalog
        • Unique Entity ID's Mapping
        • Domain Mapping and grouping
        • Endpoints grouping
        • OpenAPI upload
      • Test Coverage
        • Generating API test coverage report
  • General
    • Billing
      • Usage report
    • Settings
      • 📈Analytics
        • Flow Run History
      • 🧳Import & Export
    • General FAQs
    • General troubleshooting
    • Comparisons
      • Loadmill vs. SoapUI
      • Loadmill vs. JMeter
      • Loadmill vs. Blazemeter
      • Loadmill vs. WebdriverIO
      • Loadmill vs. Potato
    • Miscellaneous
      • Running a Test Suite
      • Test Plan
      • API Testing troubleshooting
      • API Testing FAQs
      • Test Editor
        • API Tests - Data from CSV files
Powered by GitBook
On this page
  • Creating and running a Test Plan
  • Test Plan navigation bar
  • Integrating Test Plan into CI/CD
  • Analyzing Test Plan results
  1. General
  2. Miscellaneous

Test Plan

PreviousRunning a Test SuiteNextAPI Testing troubleshooting

Last updated 3 years ago

A Test Plan is a collection of Test Suites that belong to specific tasks or features or have another reason to be run together. In general, the primary use of Test Plans is running several Test Suites integrated into your Continuous Delivery pipeline, although you may still run it manually in the UI.

The image below shows the Test Plan hierarchy in Loadmill.

Creating and running a Test Plan

Let's create and run your first Test Plan together:

  1. Navigate to the Test Plans tab and click on the NEW TEST PLAN button.

  2. Let's look at the Test Plan editor page.

3. Click on MANAGE SUITES to add Test Suites to the Test Plan. You can pick specific Test Suites or all by clicking ALL SUITES. Once the Test Suites are selected, click CLOSE.

4. Now you have two options:

  • Run the Test Plan by clicking RUN TEST PLAN.

  • Use the RUN BY LABELS option to select relevant labels and then click RUN TEST PLAN so that only Test Suites whose test flows have specific labels will run.

Test Plan navigation bar

The navigation bar allows:

  1. To set Test Suites to run in parallel (up to 12) so you will be able to significantly save running time.

  2. To configure Test Plan parameters values so that the Test Plan run will take these values in all the Test Suites using the parameters.

5. Set Test Plan E-mail or Slack notifications so that you will get one email/slack notification with all the Test Suites' information.

6. To schedule the Test Plan to run periodically to constantly validate your application status.

Integrating Test Plan into CI/CD

To integrate Test Plans into your Continuous Delivery pipeline, use our npm module. See an example of how to launch a Test Plan below:

loadmill --test-plan <test-plan-id> -w -v -t <token> --report --colors

Analyzing Test Plan results

After running the Test Plan, you will be redirected to the Test Plan Run page. The Test Plan Run report page shows the list of executed Test Suites as defined in the Test Plan. The Test Suites table shows each suite's description, the number of its flows, duration, and status. Use the table filter to filter the Test Suites by their status.

You can easily re-run Test Plan by clicking RE-RUN or Run only Failed Suites.

Make sure the Test Suites you would like to run within the Test Plan have test flows with CI toggle on. Otherwise, the Test Suites will be grayed out and won't run.

To select a GitHub repository branch to take Test Suite data from while using .

4. To define a Setup Flow that will run before all suites of the Test Plan. The Setup Flow common use cases: creation of an account, user, env cleanup and any other action/s you need to perform before running the Test Plan suites. You can either create the flow from scratch in the Setup Flow tab or use a by clicking on USE A SHARED FLOW.

Find more examples and supported CLI options .

🧠
the GitHub integration
shared flow
here