# Loadmill - AI - Powered Solution Loadmill automates testing across multiple devices and platforms by replaying and analyzing real user behavior, helping engineers to easily create the tests required for continuous delivery. Loadmill creates and replays thousands of tests based on user flows within minutes.
Loadmill's AI-powered test case generator can help you create comprehensive, reliable test cases for your APIs in minutes. By composing API requests and correlating them with Loadmill's algorithms, you can generate dynamic, automated test cases that are ready to use. With Loadmill, you can quickly and easily create comprehensive test suites for your APIs that will ensure their reliability and performance. Imagine a scenario where you need to add a user to your company’s database by submitting a form and then verifying that a user exists by validating its id provided when it's added to the database. This is a common process where the form will make a `POST` request to the server. The `POST` request connects to the server location through the server's URL path and passes in the user information inside the body of the request. If the user has been successfully added, the server sends back the `status 200` of the request notifying the request has been successful and the id of the user that has been added to the database. ![](/files/of5U54HiB8P0l11TqQ4X) You'll then make a `GET` request to the server by passing in that response `id` from the previous `POST` request (either inside of the body or in your URL). If everything went right and you're successful, you'll get a response back from your server saying whether or not the user's `id` expired. ![](/files/PUTvILAJ2khtPJSWicTw) # Deviceless mobile testing ## Deviceless Mobile Testing with Loadmill Deviceless testing is a revolutionary approach to testing that focuses on capturing user behavior from mobile devices. By creating end-to-end API-driven flows, you can execute tests quickly and efficiently without the need for a physical device. Loadmill utilizes AI algorithms and the Codex engine to understand user behavior and build dynamic flows that can be executed repeatedly, taking into account dynamic values such as IDs and keys. ### Why Deviceless Testing? Traditional testing methods require physical devices or emulators to run tests, which can be time-consuming and resource-intensive. Deviceless testing offers several advantages: 1. **Speed**: Deviceless testing allows for faster test execution since it bypasses the need for a device or emulator. 2. **Scalability**: You can execute multiple tests concurrently, enabling you to scale your testing efforts effortlessly. 3. **Cost-effective**: Reduces the need for purchasing and maintaining a wide range of devices for testing purposes. 4. **Reliability**: API-driven tests are less prone to errors caused by device-specific issues or UI changes. ### How Loadmill's Deviceless Testing Works Loadmill's deviceless testing solution leverages AI algorithms to create dynamic flows based on user behavior. The process consists of the following steps: 1. **Capturing User Behavior**: Loadmill captures user interactions from a mobile device with your application at the API level. This includes requests, responses, and any relevant metadata. 2. **Analyzing Data**: The captured data is analyzed by loadmill's AI algorithms to identify patterns, correlations, and dependencies between different API calls. 3. **Generating Dynamic Flows**: Based on the analysis, Loadmill generates end-to-end API-driven flows that accurately represent user behavior. These flows are designed to be dynamic, taking into account variable values that may change between test executions. 4. **Executing Tests**: The generated flows can be executed as test scenarios within Loadmill. You can run these tests concurrently, allowing you to scale your testing efforts and obtain results quickly. 5. **Monitoring and Reporting**: Loadmill provides real-time monitoring and reporting features, enabling you to track test progress, identify issues, and measure performance metrics.\\

A conceptual diagram of Loadmill fed through MITM proxy to capture mobile behavior

### Getting Started with Deviceless Testing in Loadmill To start using deviceless testing with Loadmill, follow these steps: 1. **Get access to Loadmill:** If you haven't already, [contact us](https://www.loadmill.com/request-access/) to get an account. 2. **Download and install Loadmill Desktop App**: To begin with deviceless testing, download the Loadmill Desktop App at . (This application includes a built-in MITM proxy that will allow you to capture traffic from your mobile device.) 3. **Configure your device and app to use Loadmill's MITM proxy**: In order to capture the API traffic, you need to configure your mobile device and app to communicate through the Loadmill MITM proxy. This can be done by modifying the Wi-Fi settings on your device and updating the app's network configuration, if necessary. Ensure that your mobile device is connected to the same network as your computer running the Loadmill Desktop App.

Loadmill desktop application with MITM proxy shows traffic of a mobile app

# Capturing traffic with Loadmill MITM Proxy Loadmill's Desktop App utilizes a MITM (Man-in-the-Middle) proxy to capture API traffic from your mobile device. This enables you to record user interactions at the API level, providing valuable data for generating dynamic flows and executing deviceless tests. {% hint style="info" %} It is also possible to use 3rd party MITM proxies as well (ie. Charles proxy and proxyman) and export the captured traffic into loadmill. {% endhint %} The MITM proxy intercepts the communication between your mobile device and the backend servers, allowing Loadmill to analyze and record the API calls and responses. This approach ensures that you capture the most accurate and complete data for your testing scenarios. To use the MITM proxy within Loadmill's Desktop App, follow these steps: 1. **Start the Loadmill Desktop App:** Switch to the proxy section\ ![](/files/sjRv7b9iIyz4qN5sd5Oq)\ at this stage the proxy will be paused without content. 2. **Install a Certificate**: Installing the Loadmill certificate on your device establishes a secure trust relationship between your device and the proxy, enabling it to decrypt the traffic for analysis and testing purposes.\ In loadmill proxy, look for the following button ![](/files/EW7v8kc5q6sG4GMg6IcZ) to download the certificate. 3. **Set up your mobile device** to use the proxy server provided by the Loadmill Desktop App. This typically involves configuring your device's Wi-Fi settings to use the proxy server's IP address and port number. 4. **Start the proxy:** By clicking the button 5. **Interact with your application:** you will see new requests accumulating in loadmill.\ ![](/files/62iYsv3idDLhWpmVgc2i) # Installing Loadmill desktop app The Loadmill Desktop App is our official application, available for Windows and macOS. It bundles almost everything that Loadmill has to offer, the testing platform UI, the MITM proxy for capturing application traffic, a dedicated private Loadmill Agent for running tests locally, Hybrid test support, and more. To get started with Loadmill Desktop App, follow the steps below. ## Downloading the Loadmill Desktop App 1. Visit the Latest [Official Loadmill Release](https://github.com/loadmill/desktop-app/releases/latest). 2. Choose the appropriate version for your operating system (Windows or macOS). * For Windows, the installer is named `Loadmill-desktop-app--Setup.exe`. * For macOS, the installer is named `Loadmill--x64.dmg`. 3. Click it to download the installer to your computer. ## Installing the Loadmill Desktop App On Windows To install the Loadmill Desktop App on Windows, follow the instructions in the [Install Windows desktop app](/introduction/deviceless-mobile-testing/installing-loadmill-desktop-app/install-windows-desktop-app) guide. # Install Windows desktop app This document outlines the required prerequisites and setup steps before installing the Loadmill Desktop App on a Windows system. Ensure these requirements are met to avoid installation or runtime issues. *** ## ✅ System Requirements * **Operating System:** Windows 10 or Windows 11 (64-bit) * **Processor:** Intel i3 or higher / AMD equivalent * **Memory:** Minimum 4 GB RAM (8 GB recommended) * **Disk Space:** At least 500 MB free space for installation plus additional space for captured HAR data and logs * **Network:** Stable internet connection required to communicate with Loadmill cloud services *** ## ⚙️ Software & Access Requirements * Administrative privileges to install software and security certificates * Optional: Corporate proxy credentials if behind enterprise firewall (the app supports proxy configuration) *** ## 🖥 Installation on Windows 1. Download the latest version of Loadmill desktop app for Windows from [here](https://github.com/loadmill/desktop-app/releases/download/v3.7.8/Loadmill-desktop-app-3.7.8-Setup.exe) 2. Double-click to run the setup 3. App will get installed in the following folder:\ `C:\Users\\AppData\Local\desktop_app` ![Loadmill Desktop App](/files/hSv9eW0xTJVMB65CMhwA) *** ## 🔐 Certificate Installation (for HTTPS Traffic Capture) * When capturing HTTPS traffic (from browsers or mobile devices), Loadmill installs a local root certificate to decrypt HTTPS traffic * During installation, accept the certificate prompt to allow HTTPS capture * If using for mobile capture: export and install the certificate on the mobile device via Wi-Fi proxy configuration * Ensure both desktop and mobile device are on the same Wi-Fi network for proper capture ![Proxy Tab](/files/bCrYm34FqFO6dM9hpd8m) *** ## 🌐 Network Configuration * Allow outbound HTTPS (TCP port 443) to Loadmill’s cloud endpoints * Ensure that any corporate VPNs or firewalls do not block the Loadmill app executable or proxy ports * For capturing mobile traffic, verify the device proxy is configured to the desktop app’s IP and port (shown within the app) *** ## 📁 Log and Configuration Paths * **Log files:**\ `C:\Users\\AppData\Roaming\Loadmill\logs` * **Configuration and captured data:**\ Stored under the app’s user data directory in `AppData\Roaming\Loadmill` ![Download Logs](/files/V6bDOm50Er5ElsMFUR1z) *** ## 📞 Support For installation or connectivity issues, contact Loadmill Support via or refer to the official documentation at # Loadmill desktop recorder Loadmill Desktop Recorder is a powerful tool designed to simplify and enhance your deviceless testing experience. By capturing user interactions with your application at the API level, the Desktop Recorder allows you to quickly and efficiently create end-to-end API-driven test flows. There are three main components that work together to provide a powerful and efficient testing solution: 1. **Loadmill UI**: Same as the web interface for managing your testing process. With the Loadmill UI, you can edit and execute test flows, and access reporting and run results.\ ![](/files/Cn1qpfwHO9AbwUkyLqq2) 2. **MITM Proxy**: The integrated Man-in-the-Middle (MITM) proxy allows you to capture API traffic from your mobile device by intercepting the communication between the device and the backend servers.\ ![](/files/nSfTYV5mazUSiFSh17sk) 3. **Loadmill Agent**: The Loadmill Agent enables local execution of test scenarios, allowing you to run tests directly from your computer within your network. This feature simplifies the testing process and makes it more accessible, allowing you to quickly and easily debug tests and obtain results. {% hint style="info" %} The agent is controlled by the play/stop button integrated to the top right corner of the application. {% endhint %} ## **Proxy area (Recorder section)** **General controls:**
At the top, you can find controls for: * starting/pausing the proxy * Importing har file, to continue the work * Clearing the list * Listening IP+Port information * Certificate download * Filtering area * Destination suite for generated tests **Proxy entries area:**
Once started and configured, the proxied requests will be collected in the list.\ The controls in the list allow:\ \- Viewing a request details by clicking on it\ \- Deleting requests by selecting them\ \- Exporting har file **Test creation controls:**\ ![](/files/n2hEdl76550HMcT2XO9w)\\ Bottom right area * Analyze button - will mark out irrelevant requests, the user can revert this decision by clicking the button at the top of the entries area. * Create test button - will process the requests and generate parameterized test in loadmill.\ (The test will be found according to the suite selector by switching to loadmill area) # Generating test flows Loadmill allows streamlining tests generation by capturing user behavior on a mobile device. To generate a test follow the following steps: 1. Ensure your device is up and configured correctly with [mitm certificate](/introduction/deviceless-mobile-testing/installing-certificate-on-mobile-devices) and [proxy settings](/introduction/deviceless-mobile-testing/configuring-proxy-on-mobile-devices). 2. Start your application and position it's state to the beginning of a scenario 3. Start loadmill proxy by clicking,\ Tip: make sure you start from an empty list, you can clear the residues by clicking . 4. Drive the application on your device by the scenario you want to capture, make sure requests are captured in the proxy 5. If needed, adjust the filterto focus or relevant domain(s) to reduce potential noise from irrelevant requests. 6. Once finished driving the scenario, click on button.\ loadmill will filter out requests that are highly suspected as irrelevant and strikes them out. You can review the filtered requests and if needed restore them by selecting and clicking. 7. Adjust the destination suite in the suite field: 8. To generate the test, click on 9. To review the test navigate to Loadmill section go to Test suite\ \ And select your test suite,\ The recently created flow in the list will be marked with the recent timestamp. # Installing certificate on mobile devices Depend on the platform and your application, there are various steps that might be required to install new certificate.\ In the next sections you will find the most common methods to deploy new certificate. # iOS certificate installation To deploy a certificate into an iOS device, follow these steps: 1. **Download the certificate**: In loadmill desktop app, go to the proxy section, download the certificate by clicking on![](/files/JdRy4guzbTPxQ4UgUv44)button. Transfer the certificate file (usually in `.cer` or `.crt` format) to your device (ie. via AirDrop or Google drive). 2. **Install the certificate**: Once the certificate file is downloaded, a prompt will appear asking to review and install a profile.\ ![](/files/rzNPU3kRAAerEmLsQaUs) 3. Go to device settings and select "Profile Downloaded"\ ![](/files/vJUXJofitQ3oLAupITYM) 4. Click on install,\ You will be prompted for your device password, proceed to completion.\ ![](/files/mhNNLukN3QxdTzPLovio)\\ 5. **Trust the certificate**: After the certificate is installed, you need to manually trust it. To do this, go to `Settings` > `General` > `About` > `Certificate Trust Settings`. Under the "Enable full trust for root certificates" section, find the Loadmill certificate you just installed and toggle the switch to enable trust for the certificate.\ ![](/files/qHd78Wm1T7UI5b7YvAsq) By following these steps, you can successfully deploy a certificate into your iOS device and configure it to work with the Loadmill MITM proxy for deviceless testing. This will enable you to capture and analyze encrypted API traffic from your iOS device, allowing for more comprehensive and accurate test scenarios. # Android certificate installation More information about certificate installation on android can be found [in our blog here](https://www.loadmill.com/blog/behavior-driven-mobile-testing) # Configuring proxy on mobile devices # iOS Wi-Fi settings {% hint style="info" %} Before following the next steps, make sure you are connected to the same network as the machine hosting loadmill desktop app proxy. {% endhint %} To set the proxy on your wifi connection: 1. Go to the main `Settings` menu and select `Wi-Fi`.\ ![](/files/lEmiuFYoRZKzJ6shhBtP) 2. Tap the "i" icon next to your connected Wi-Fi network.\ ![](/files/WAebG0p1bvNKIJ1Wn7yC) 3. Scroll down to the `HTTP Proxy` section and select "Manual."\ ![](/files/ZG41jGXX9fSkM0DTiKMv) 4. Enter the IP address and port number provided by the Loadmill Desktop App for the MITM proxy server.\ ![](/files/TAs6zKqmQLFe1n8EU6k7)\\ *The IP and the port number can be found at the upper area of the loadmill desktop app in the proxy section.*
5. Save the settings and ensure that your device is still connected to the Wi-Fi network. # Android Wi-Fi settings Setting proxy in wifi setting on android can be found in [our blog here](https://www.loadmill.com/blog/behavior-driven-mobile-testing). # Troubleshooting Q: My application utilize ssl-pinning, as a result I don't see encrypted traffic.\ A: In that case you must build a development version of your application with one of the options:\ (a) disabling ssl-pinning\ (b) whitelisting loadmill-mitm-proxy certificate as part of ssl-pinning\ \ Q: After configuring everything, my application is not responding anymore, neither I see traffic.\ A: Make sure the following:\ (a) The proxy is started\ (b) The proxy port is whitelisted in machine's firewall\ (c) Your device and proxy host machine are connected to the same network and the port is allowed by the network.\ (d) The application does not utilize ssl-pinning or loadmill-ssl-certificate is whitelisted. # What is an API Application Programming Interfaces (APIs) have completely overtaken how we use mobile and web applications by allowing software to interact with other software. Today, we use APIs as a key entry point for any data, service, and integration access. An API is an interface that allows two pieces of software to communicate with each other. It provides a way for one program to request data from another program, and for the other program to respond to that request. An API can be thought of as a set of rules that govern how two pieces of software interact with each other. When one piece of software wants to request data from another piece of software, it does so by following the API's rules. The API then tells the second piece of software how to respond to the request. ![](/files/qktFJl4A4tUyyDdHKEUH) # API - Data Fetching An API request allows you to access specific data or functionality from an internal or external source. For example, you could use an API to retrieve data about the latest user signups on your platform or to submit a new user to your database. ![](/files/YEPoGYwS0ucRdxvTFP0r) # Quick examples of API requests If you're looking to make some API requests, here are a few examples to help you get started. Remember, you'll need to set up your server to make these requests - this is just for illustration purposes. To get started, let's look at a simple example. Suppose we want to get information about the latest users on our site. We can do this by making a GET request to the /users endpoint: In this example, we're using REST API to demonstrate how API calls are being accessed. ## This will return a JSON object containing information about the latest users. `GET` `/` #### Request Body | Name | Type | Description | | ------------------------------------ | ---- | ----------- | | id\* | 456 | | {% tabs %} {% tab title="200: OK " %} ```javascript { // Response } ``` {% endtab %} {% endtabs %} **method** **URL**\ GET /users This will return a JSON object containing information about the latest users. ``` [{ "name": { "title": "mr", "first": "ross", "last": "geller" }, "id": 123, "email": "rossgeller@testmail.com", "phone": "011-962-7516", "gender": "male", "age": "34", "occupation": "scientist", ... }, { "name": { "title": "mrs", "first": "jennifer", "last": "gibson" }, "id": 456, "email": "jennifergibson@testmail.com", "phone": "212-934-5277", "gender": "female", "age": "29", "occupation": "developer", ... }, ... ] ``` Now let's say we want to get information about a specific user. We can do this by making a GET request to the /users/{id} endpoint, where {id} is the id of the user we want to retrieve: **method** **URL**\ GET /users/123 ``` { "name": { "title": "mr", "first": "ross", "last": "geller" }, "id": 123, "email": "rossgeller@testmail.com", "phone": "011-962-7516", "gender": "male", "age": "34", "occupation": "scientist", ... } ``` This will return a JSON object containing information about the user with id 123. We can also create new users by making a POST request to the /users endpoint. For example, to create a new user with the name "John Doe" and has filled all the required parameters for the server to accept your request, we would make the following request: **method** **URL**\ POST /users ``` //Request Body { "name": { "title": "mr", "first": "john", "last": "doe" }, "email": "johndoe@testmail.com", } ``` The server would then accept your request and send back a response with a status code 200 (means the request has been successful) and in this case would send back a generated id of the user-created by this request as shown below: ``` //Response Body { "id":789 } ``` # What is an API endpoint? An API endpoint is a specific URL that your application can access to retrieve data or perform an action. Each endpoint is designed to perform a specific task, such as retrieving data from a database or adding new data to a database. API endpoints are typically defined by the functionality they provide, such as retrieving all users from a database or creating a new user. However, endpoints can also be defined by their formats, such as JSON, XML, or HTML. When you make a request to an API endpoint, the server will respond with the data you requested or the results of the action you performed. The format of the response will be determined by the format of the endpoint (JSON, XML, or HTML). ![](/files/zjD2X2Z3T0fFFhCCK4W0) # API Server Testing #### The importance of testing your server API testing is a critical part of any software development process. A well-tested API ensures that your server is able to handle the traffic and data it needs to in order to function properly. Without proper testing, your server could crash or malfunction, causing major problems for your business. API testing can be time-consuming and difficult, but it's important to make sure that your API is up to the task of handling your data. By testing early and often, you can avoid potential problems down the road. #### How do developers create their API tests today? API testing can be performed at the unit level, but it is often done at the integration or system level. API testing is a critical part of any software development process because it allows developers to ensure that the different components of their applications work together as expected. There are many different ways to approach API testing, but one common approach is to use a tool like Postman to make requests to the API and then assert that the responses are as expected. Another common approach is to use a scripting language like Python or Ruby to write automated tests that make requests to the API and check the responses. No matter what approach you take, some key things to keep in mind when doing API testing. First, you need to clearly understand what the API is supposed to do. Second, you need to have a good understanding of the data that will be returned by the API. And third, you need to be able to test the API in a variety of different scenarios, including positive and negative tests. # Contract testing #### Intro Contract testing is a testing approach used in software development that involves testing the interactions between two different services or systems by defining and verifying the contracts or agreements between them. In a typical scenario, when two services communicate with each other through an API, the contract testing approach involves defining the expected input and output data for each API endpoint, and then writing tests to verify that the API endpoint behaves as expected. Contract testing helps in detecting and preventing issues that may arise due to changes made to the API, without having to run end-to-end tests. This allows for faster and more efficient testing of software systems, reducing the time and effort required for testing. Overall, contract testing helps improve the reliability, stability, and resilience of software systems by ensuring that the different components of the system work together as intended, and are compatible with each other. #### Contract testing with Loadmill Contract testing with Loadmill is easy. To create contract validation of an API call, use [JSON schema](https://docs.loadmill.com/introduction/api-server-testing/pages/-LHDbcfL_yc8unWDJfhd#json-schema-validate-that-a-parameter-comply-with-given-json-schema.) or [JSON contains](/test-editor/assertions#json-contains-validates-that-a-json-contains-a-subset-json-in-such-way-dynamic-fields-can-be-omitted) as following: * **Enforcing JSON Schema of an API:**\ With 'JSON Schema' you can explicitly enforce multiple of aspects of your json data.\ In the request editor assertions insert new entry, use '\_\_responseBody' built-in parameter and ensure the values are of the type "JSON Schema", then insert a valid json-schema.\\

An example of schema validation request in loadmill

In the following table you can see an example of the schema and a typical response that comply with the schema.
SchemaResponse
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "integer"
    },
    "title": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "price": {
      "type": "integer"
    },
    "discountPercentage": {
      "type": "number"
    },
    "rating": {
      "type": "number"
    },
    "stock": {
      "type": "integer"
    },
    "brand": {
      "type": "string"
    },
    "category": {
      "type": "string"
    },
    "thumbnail": {
      "type": "string"
    },
    "images": {
      "type": "array",
      "items": [
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        }
      ]
    }
  },
  "required": [
    "id",
    "title",
    "description",
    "price",
    "discountPercentage",
    "rating",
    "stock",
    "brand",
    "category",
    "thumbnail",
    "images"
  ]
}

{
   "id":1,
   "title":"iPhone 9",
   "description":"An apple mobile which is nothing like apple",
   "price":549,
   "discountPercentage":12.96,
   "rating":4.69,
   "stock":94,
   "brand":"Apple",
   "category":"smartphones",
   "thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg",
   "images":[
      "https://i.dummyjson.com/data/products/1/1.jpg",
      "https://i.dummyjson.com/data/products/1/2.jpg",
      "https://i.dummyjson.com/data/products/1/3.jpg",
      "https://i.dummyjson.com/data/products/1/4.jpg",
      "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
   ]
}
* **Enforcing JSON structure using JSON-Contains:**\ \&#xNAN;**'**Json-contains' is a lightweight capability in Loadmill to enforce JSON data.\ In the request editor assertions insert new entry, use '\_\_responseBody' built-in parameter and ensure the values are of the type "JSON Contains", then insert an expected json or a subset that can contain less fields.\ Note, to ensure existence of a field ignoring it's value use `"*"` notation.

Json-Contains example in loadmill - Enforcing json content

In the following table you can see an example of the expected JSON-Contains object and a typical response that comply with it.
JSON-Contains objectResponse
{
    "id": 1,
    "title": "iPhone 9",
    "description": "An apple mobile which is nothing like apple",
    "price": 549,
    "discountPercentage": "*",
    "stock": 94,
    "brand": "Apple",
    "category": "smartphones",
    "thumbnail": "https://i.dummyjson.com/data/products/1/thumbnail.jpg",
    "images": [
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
    ]
}

{
    "id": 1,
    "title": "iPhone 9",
    "description": "An apple mobile which is nothing like apple",
    "price": 549,
    "discountPercentage": 12.96,
    "rating": 4.69,
    "stock": 94,
    "brand": "Apple",
    "category": "smartphones",
    "thumbnail": "https://i.dummyjson.com/data/products/1/thumbnail.jpg",
    "images": [
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
    ]
}
``` ``` **Automatic assertions**\ Loadmill can generate 'JSON schema' or 'JSON contains' assertions automatically during test composition. Such assertions can serve are a quick baseline that can be adjusted later on.\ To turn automatic JSON validations. 1. head to settings icon -> recordings:\ 2. Locate the 'JSON Validation section in the list', make sure the toggle turned on and adjust the type according to the needs.
if 'JSON Validation' section doesn't exist, try adding it by clicking on ![](/files/9PsKEWmU0JucFmCh6H6p) at the top right corner and selecting it from the list.
4. (optional) Click on the '+' to add ignored keys.\ ![](/files/Gog7Z1ZSqUoQwBnjpCmr) 5. From now on, every generated test with loadmill composers will cary json validation assertions. # Regression Testing #### How do developers create their regression tests? Regression testing is a process of testing changes to software to make sure that the new code doesn't break existing functionality. It's an important part of the software development process, but it can be time-consuming and expensive. There are a few different ways that developers can create regression test suites for APIs. One common way is to use a tool like SoapUI or Postman. These tools allow developers to create test cases and run them against an API to ensure that the API is functioning correctly. Another way to create a regression test suite is to manually write code that makes calls to the API and verifies the results. This approach requires more work up front, but it can be more flexible and allow for more comprehensive testing. #### The issue many companies have in testing their servers Today, many organizations are struggling with regression testing. They're finding that it's taking longer and longer to test their software, and they're not able to keep up with the pace of change. As a result, they're missing bugs and releasing software with serious defects. There are several reasons for this problem. One is that software is becoming increasingly complex. Another is that the pace of change is accelerating, making it harder to keep track of all the changes that need to be tested. Finally, many organizations are under pressure to release new features and products quickly, which leaves less time for regression testing. As a result, they're forced to cut corners and take shortcuts that can lead to serious defects in their software. # Overview Droid CUA is Loadmill's AI-powered desktop app for mobile and web testing. It helps you create, run, and manage tests for Android devices and emulators, iOS simulators on macOS, cloud mobile devices, and browser flows. Instead of writing traditional automation code with selectors and locators, you describe the user flow in natural language and let the agent operate the app from the screen, the same way a person would. With Droid CUA you can: * Connect to a real Android device, Android emulator, iOS simulator, cloud mobile device, or browser. * Create mobile and web tests from plain-English instructions. * Watch the agent execute each action with live logs. * Save reusable `.dcua` test scripts. * Run saved tests again from the desktop app or the CLI. * Use run history and reports to review what happened. *** ## How it helps Droid CUA is useful when you want to validate real mobile flows without spending time building and maintaining fragile selector-based scripts. For example, you can ask the agent to sign in, search for an item, add it to the cart, submit a form, or verify that a confirmation screen is visible. Droid CUA captures the device screen, sends the screen and your instructions to the AI model, receives an action such as tap, scroll, type, or wait, and executes that action on the device. The result is a test workflow that is easier to draft, easier to understand, and closer to how a real user interacts with your app. *** ## Platforms ### Android Droid CUA supports physical Android devices and Android emulators. ### iOS Droid CUA supports iOS simulators on macOS. ### Web Droid CUA supports browser testing with installed Chrome or Edge. ### Cloud devices Droid CUA supports cloud mobile runs from the CLI, including Loadmill Cloud and LambdaTest. Cloud runs use real Android or iOS devices and require the provider credentials plus an app build. ### Desktop support * macOS: Android, iOS simulator, and web workflows * Windows: Android and web workflows *** ## Learn more * [Getting started](/droid-cua/getting-started) * [CLI](/droid-cua/cli) * [Agent Skill](/droid-cua/skill) # Getting Started This guide shows the basic path for running your first Droid CUA test. *** ## Before you start You will need: * The Droid CUA desktop app installed. * A Loadmill account. * A target device, simulator, cloud device, or browser. * Android Debug Bridge (ADB) for Android testing. * Xcode, Appium, and the XCUITest driver for iOS simulator testing on macOS. * Chrome or Edge installed for web testing. For a fuller checklist, see [Setup](/droid-cua/setup). *** ## Step 1: Download the desktop app Download the latest desktop app release: * [Mac](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA.dmg) * [Mac Intel](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-intel.dmg) * [Windows](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-Setup.exe) After the download finishes, install and launch Droid CUA. *** ## Step 2: Sign in When the app opens, sign in with your Loadmill account. If the setup wizard appears, follow the checks it shows. It helps confirm that the required tools for Android or iOS testing are installed. *** ## Step 3: Choose a target Choose the platform you want to test. For Android, connect a physical device with USB debugging enabled, or choose an available emulator. For iOS, choose an installed iOS simulator. iOS simulator testing is available on macOS only. For web testing, choose the web platform and use an installed Chrome or Edge browser. For cloud mobile testing, use the CLI with LambdaTest credentials and a mobile app build. ![Droid CUA device selection screen](/files/CdchvuXyOCgBB7i8vG7b) *** ## Step 4: Create a project Create or open a Droid CUA project and choose where to store your test files. Droid CUA tests are saved as `.dcua` files. Keeping them in your project repository makes it easy to review, edit, and run them later in CI. You can also choose a results folder. This is where Droid CUA stores run reports and execution history. *** ## Step 5: Write a simple first test Start with a short flow that is easy to verify. For example: ``` Open the app. Sign in with the standard test account. Verify that the Home screen is visible. ``` Good Droid CUA instructions describe the user's goal and the visible result. Use the exact button names, screen names, and messages that appear in your app when possible. You can write a test directly, or use Design Mode to describe what you want to test and let the agent create a first draft. ![Droid CUA Design Mode screen](/files/SizinEnBsPcoCZ9xirgq) Saved tests are plain `.dcua` files. They are easy to read, review, and run again later. ![Example Droid CUA test file](/files/pmOtypXv1YJqIPW4Wlfe) *** ## Step 6: Run and review Run the test from the desktop app and watch the live execution log. If the test fails, look for the first place where the agent did something different from what you expected. Then update the instruction, add clearer app context, or split the flow into a smaller test. For the first few tests, keep each flow small. Once the device connection, credentials, and app context are working, you can add broader scenarios. If the problem happens before the test starts, see [Setup troubleshooting](/droid-cua/setup/setup-troubleshooting). # Setup This page covers the basic setup required before running Droid CUA tests on mobile devices, cloud devices, and browsers. *** ## Desktop app Download and install the Droid CUA desktop app: * [Mac](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA.dmg) * [Mac Intel](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-intel.dmg) * [Windows](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-Setup.exe) After installation, launch Droid CUA and sign in with your Loadmill account. When you open Droid CUA for the first time, the setup wizard helps you choose a platform and run the required checks. ![Droid CUA setup wizard welcome screen](/files/WEbhr9fwsjC4B53NLtG8) *** ## Android setup To test Android apps, Droid CUA needs access to a physical Android device or an Android emulator. Install and configure: * Android Debug Bridge (ADB) * Android Emulator CLI, if you want Droid CUA to launch emulators * USB debugging on physical Android devices To confirm ADB is available, run: ```sh adb version ``` To confirm a device or emulator is visible, run: ```sh adb devices ``` If the device appears in the list, Droid CUA should be able to connect to it from the desktop app. Use the **Devices** page to choose the device source, platform, and target device. ![Droid CUA devices page with a connected Android emulator](/files/CdchvuXyOCgBB7i8vG7b) *** ## iOS simulator setup iOS simulator testing is available on macOS only. Install and configure: * Xcode * iOS Simulator * Appium * XCUITest driver After setup, open Droid CUA and choose an installed simulator from the device picker. Use the platform selector on the **Devices** page to switch between Android, iOS, and web targets. *** ## Web setup To test web flows, install a supported browser: * Google Chrome * Microsoft Edge Droid CUA uses Playwright with the installed browser for web execution. In the CLI, use `--target web` with `--instructions`. ```sh droid-cua --target web --browser chrome --instructions tests/search.dcua ``` Use `--browser edge` to run against Microsoft Edge. *** ## Cloud device setup Droid CUA supports cloud mobile runs from the CLI. ### Loadmill Cloud Loadmill Cloud is a paid feature. Contact [Loadmill Support](mailto:support@loadmill.com) to discuss access and have it enabled for your account before using it. After access is enabled, you need: * A Loadmill API token. * A target platform: `android` or `ios`. * The cloud device name and OS version. * An app build: `.apk` for Android or `.ipa` for iOS. Set your API token in your shell or CI environment: ```sh export LOADMILL_API_TOKEN=your-loadmill-api-token ``` Use `--device-source loadmill-cloud` with the platform, device, app, and saved test when you run the CLI. See [CLI](/droid-cua/cli) for an example. ### LambdaTest LambdaTest is another cloud-device option available from the CLI. You will need: * `LAMBDATEST_USERNAME` * `LAMBDATEST_ACCESS_KEY` * A target platform, such as `android` or `ios` * A cloud device name and OS version * An app build: `.apk` for Android or `.ipa` for iOS Set the credentials in your shell or CI environment: ```sh export LAMBDATEST_USERNAME=your-username export LAMBDATEST_ACCESS_KEY=your-access-key ``` Then run Droid CUA with `--device-source lambdatest`. *** ## Project setup Create a Droid CUA project and choose: * A tests folder for saved `.dcua` files. * A results folder for run history, reports, and logs. Keeping `.dcua` files in your app repository makes it easier to review them, commit them, and run the same tests later with the CLI. If your test needs a mobile app build, add it from the **Apps** page. Droid CUA can manage `.apk`, `.ipa`, and `.app` builds. *** ## App context For real app testing, create a short `context.md` file next to your tests. Use it to explain details the agent cannot infer from the screen alone, such as: * What the app does. * Which test accounts to use. * Important screen names and navigation paths. * Common success messages. * Any confusing or similar-looking buttons. Good context makes tests more reliable because the agent has the same product knowledge a teammate would need before testing the app. *** ## Setup troubleshooting If setup does not work as expected, see [Setup troubleshooting](/droid-cua/setup/setup-troubleshooting). # Setup Troubleshooting Use this page when Droid CUA cannot connect to a device, cannot find a required tool, or fails before the test starts. *** ## The app cannot find ADB Make sure Android Debug Bridge is installed and available in your system PATH. Run: ```sh adb version ``` If the command is not found, install Android Platform Tools and restart Droid CUA after updating your PATH. *** ## Android device does not appear Run: ```sh adb devices ``` If the device is not listed: * Confirm the device is connected over USB. * Enable USB debugging on the device. * Approve the debugging prompt on the device screen. * Try reconnecting the cable or restarting ADB. If the device is listed as `unauthorized`, unlock the device and approve the USB debugging prompt. *** ## Android emulator does not start Confirm that Android Emulator CLI is installed and that the emulator can be started outside Droid CUA. If the emulator starts manually but not from Droid CUA, restart the desktop app and select the emulator again from the device picker. *** ## iOS simulator does not appear iOS simulator support requires macOS. Confirm that: * Xcode is installed. * The simulator exists in Xcode. * Appium is installed. * The XCUITest driver is installed. After installing missing tools, restart Droid CUA and open the device picker again. *** ## Web browser is not found For web tests, install Google Chrome or Microsoft Edge and restart Droid CUA. For CLI runs, confirm the selected browser is supported: ```sh droid-cua --target web --browser chrome --instructions tests/search.dcua ``` Use `--browser edge` if you want to run against Microsoft Edge. *** ## LambdaTest cloud device run fails before connecting Check that the required LambdaTest inputs are set: * `LAMBDATEST_USERNAME` * `LAMBDATEST_ACCESS_KEY` * `--device-source lambdatest` * `--platform android` or `--platform ios` * `--device-name` * `--os-version` * `--app` * `--instructions` Android cloud runs require an `.apk` file. iOS cloud runs require an `.ipa` file. *** ## The test starts but the agent gets confused This is usually not a setup problem. Improve the test input instead: * Use the exact labels shown in the app. * Keep the first test short. * Add or improve `context.md`. * Add an assertion that describes something visible on screen. * Split long flows into smaller tests. *** ## CLI run fails before execution Check the most common inputs: * `--instructions` points to an existing `.dcua` file or folder. * `--avd` matches the selected Android device, emulator, or iOS simulator. * `--platform ios` is used for iOS simulator runs. * `--target web` is used for browser runs. * `--config` points to valid JSON. Run with `--debug` to create detailed artifacts for troubleshooting: ```sh droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` # CLI The Droid CUA desktop app is the main place to create and debug tests. The CLI lets you run saved `.dcua` tests from a terminal, CI pipeline, or other automation workflow across mobile, cloud device, and web targets. *** ## Install the CLI Install the package globally: ```sh npm install -g @loadmill/droid-cua ``` Then run: ```sh droid-cua ``` *** ## Choose an LLM provider and CUA model ### Use Loadmill (default) The CLI uses Loadmill by default, with `loadmill-smart` as the recommended, most robust model. You do not need to pass `--llm-provider` or `--cua-model` for a standard Loadmill run. ```sh droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua ``` ### Use Loadmill Pulse for faster execution For a faster, lower-cost Loadmill run, use `loadmill-pulse`. Loadmill is already the default provider, so you only need to set the model: ```sh droid-cua \ --cua-model loadmill-pulse \ --avd adb:emulator-5554 \ --instructions tests/login.dcua ``` `loadmill-beacon` is also available as an experimental Loadmill model. ### Use your own OpenAI API key To use OpenAI instead of Loadmill, set `OPENAI_API_KEY` and explicitly pass `--llm-provider openai`. Passing an OpenAI model to `--cua-model` alone does not change the provider. Choose `gpt-5.6-terra`, `gpt-5.6-luna`, or `gpt-5.4`. ```sh OPENAI_API_KEY=your-openai-api-key \ droid-cua \ --llm-provider openai \ --cua-model gpt-5.4 \ --avd adb:emulator-5554 \ --instructions tests/login.dcua ``` For CI or a config file, use `"llmProviderMode": "openai"` to select OpenAI. You can also set `DROID_CUA_LLM_PROVIDER=openai` as the default provider for a shell environment. *** ## Run a saved Android test Use `--instructions` to point at a `.dcua` file and `--avd` to select the target device or emulator. ```sh droid-cua --avd adb:emulator-5554 --instructions tests/login.dcua ``` You can also point `--instructions` at a folder. Droid CUA will run the `.dcua` files in that folder. ```sh droid-cua --avd adb:emulator-5554 --instructions tests ``` *** ## Run a saved iOS simulator test On macOS, use `--platform ios` and pass the simulator name: ```sh droid-cua --platform ios --avd "iPhone 16" --instructions tests/login.dcua ``` *** ## Run a saved web test Use `--target web` to run a `.dcua` test against an installed browser. ```sh droid-cua --target web --browser chrome --instructions tests/search.dcua ``` Supported browser values are `chrome` and `edge`. *** ## Run on a LambdaTest cloud device Set your LambdaTest credentials: ```sh export LAMBDATEST_USERNAME=your-username export LAMBDATEST_ACCESS_KEY=your-access-key ``` Then run with `--device-source lambdatest` and pass the target device details. ```sh droid-cua \ --device-source lambdatest \ --platform android \ --device-name "Galaxy S24" \ --os-version "14" \ --app ./app-debug.apk \ --instructions tests/login.dcua ``` Android cloud runs require an `.apk` file. iOS cloud runs require an `.ipa` file. *** ## Run on Loadmill Cloud Loadmill Cloud lets you run saved Droid CUA tests on cloud-hosted Android or iOS devices. It is a paid feature; contact [Loadmill Support](mailto:support@loadmill.com) to enable it for your account before setting it up. After Loadmill Cloud is enabled, create a Loadmill API token and make it available to your local shell or CI environment: ```sh export LOADMILL_API_TOKEN=your-loadmill-api-token ``` Start a run with `--device-source loadmill-cloud`, the target platform, device name, OS version, app build, and a saved test: ```sh droid-cua \ --device-source loadmill-cloud \ --platform android \ --device-name "Galaxy S24" \ --os-version "14" \ --app ./app-debug.apk \ --instructions tests/login.dcua ``` Use an `.apk` app build for Android or an `.ipa` app build for iOS. *** ## Use a config file For CI, it is recommended to keep a small JSON config file in the repository and run tests with `--config`. ```sh droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` Example config: ```json { "llmProviderMode": "loadmill", "cuaModel": "loadmill-smart", "promptCustomizations": { "basePromptInstructions": "", "designModeInstructions": "", "executionModeInstructions": "" }, "appContextEnabled": true, "appContextPath": "../tests/context.md", "contextOptimizationEnabled": true, "contextOptimizationThreshold": 30000 } ``` The config file keeps prompt settings and app context consistent between local runs and CI runs. *** ## Common CLI options | Option | Description | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--target` | Target kind, such as `mobile` or `web`. | | `--instructions` | Path to a `.dcua` test file or a folder of `.dcua` files. | | `--avd` | Android device, Android emulator, or iOS simulator name. | | `--platform` | Target platform, such as `android` or `ios`. | | `--browser` | Browser for web runs, such as `chrome` or `edge`. | | `--device-source` | Mobile device source, such as `local`, `lambdatest`, or `loadmill-cloud`. | | `--device-name` | Cloud device name for LambdaTest or Loadmill Cloud. | | `--os-version` | Cloud device OS version for LambdaTest or Loadmill Cloud. | | `--app` | App build path for cloud runs. Use `.apk` for Android or `.ipa` for iOS. | | `--config` | Path to a Droid CUA headless config file. | | `--llm-provider` | AI provider for the run: `loadmill` (default) or `openai`. Set this to `openai` when using your own OpenAI API key. | | `--cua-model` | Model to use for the run. With Loadmill, use `loadmill-smart` (recommended and most robust), `loadmill-pulse` (faster and lower cost), or experimental `loadmill-beacon`. With OpenAI, use `gpt-5.6-terra`, `gpt-5.6-luna`, or `gpt-5.4`. | | `--context` | Path to an app context file. | | `--no-context` | Disable app context for the run. | | `--record` | Save screenshots from the run. | | `--debug` | Write detailed debug artifacts for troubleshooting. | *** ## CI basics In CI, install the CLI, provide the required credentials as secrets, and run the saved test file with a config file. ```yaml jobs: droid-cua: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 - run: npm install -g @loadmill/droid-cua - run: | droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` A successful run exits with code `0`. A failed run exits with code `1`, so CI can fail the build when the mobile test fails. # Agent Skill The bundled Droid CUA Agent Skill gives supported coding agents the instructions they need to work safely with Droid CUA projects. It helps an agent inspect a `.dcua` project, author or update tests and context, check the selected target, and run or diagnose tests with the Droid CUA CLI. Use the skill when you want an agent to work on a Droid CUA test project. It is intentionally limited to Droid CUA work; it is not a general browser automation or mobile-development skill. *** ## Install the skill Install the latest Droid CUA package first: ```sh npm install -g @loadmill/droid-cua ``` You can install the skill for Cursor, Claude Code, or Codex. ```sh droid-cua skill install cursor droid-cua skill install claude-code droid-cua skill install codex ``` By default, the skill is installed globally for your user account. To install it only in the current project, add `--local`: ```sh droid-cua skill install codex --local ``` In the Droid CUA desktop app, you can also choose **Help → Install Droid CUA Skill**, then select your coding agent. The desktop app installs the skill globally. *** ## Use the skill After installation, ask your coding agent to work with Droid CUA or `@loadmill/droid-cua`. The skill directs the agent to: * Inspect the active project without reading or exposing secret values. * Work with existing `.dcua` tests, `context.md`, test data, and CLI configuration. * Confirm the intended test and target before starting a run. * Use Loadmill by default, unless you explicitly ask to use your own OpenAI API key. * Run the narrowest relevant test and report its outcome, screenshots, and report artifacts. For the CLI commands and supported test targets, see [CLI](/droid-cua/cli). *** ## Manage an installation Check whether a skill is installed: ```sh droid-cua skill status codex ``` Remove an installed skill: ```sh droid-cua skill uninstall codex ``` After upgrading `@loadmill/droid-cua`, explicitly update the installed skill: ```sh droid-cua skill install codex --force ``` The installer does not overwrite or remove locally modified skill files unless you supply `--force`. # Create Account Go to [loadmill.com](https://loadmill.com) and sign up for a new account. You can sign up through your Google or Github account. We also recommend signing up for a [POC trial](https://www.loadmill.com/poc) with us to get a dedicated team guiding you to all of the features. # Download Test Composer Once your account is created, download the [**Loadmill Test Composer Chrome Extension**](https://chrome.google.com/webstore/detail/loadmill-recorder/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en)**.** This extension allows you to compose your test scenario by registering your API requests while you navigate the site. It will register any type of API requests being sent and received from the server. You can also apply specific filters if you want to register requests relevant to your test case flow. As well as adding domains in case you want to register our-of-domain APIs such as third-party APIs that is necessary for completing your flow. See the [**Test Composer Setup** ](https://github.com/loadmill/loadmill-docs/tree/master/quick-guide/broken-reference/README.md)for more info on the composer features. # Register your first API flow Navigate to your app and start registering any CRUD (Create/Read/Update/Delete) operation. **To do that:** 1. Navigate to the app/platform of your choice. 2. Go to the Loadmill Test Composer tab inside your dev tools. 3. Create any CRUD operation flow (create and delete any type of operation). 4. Click on Analyze Requests to analyze and optimize your flow from the APIs registered for your test case. 5. Click on Create Test to upload your flow to the Loadmill platform. **Voila! You just registered your first API flow!** Clicking on “Create Test” sends the registered API flow to the Loadmill AI that, through its complex algorithm generates a fully dynamic ready-to-use automated test. The next steps will walk you through the Loadmill platform understanding what and how the Loadmill AI generated the automated test. # Running Your API Test Congrats! You managed to upload your test scenario to the Loadmill platform. Before running your test, let's walk through the key sections of your automated test. #### **Naming Your Flow** Naming your flow is (or should always be) your first step in creating and managing your automated test cases. It allows you to know what your flow is about and what should it do so never forget to name every flow/scenario you compose. When uploading your flow/test scenario to Loadmill, you’ll see the flow added to your test suite on the left side of the app. By default, the name is the date and time you uploaded your test but make sure to rename it to, most commonly, the name of the operation you composed to a test. {% embed url="" %} #### **Generated Test Code** The generated test code from the scenario you registered is the most interesting part of Loadmill. After uploading your flow/scenario to Loadmill, its sophisticated AI generates a complete automated dynamic test build that you can easily view, edit and integrate. The generated code is located at the top of the editor and is written and saved in `.yaml` {% embed url="" %} #### API Components From the generated automated test to your API components, Loadmill makes it easy for you to edit each API component related to your test case. Every component is expandable with various features such as extracting and validating dynamic values from the server response, running custom code to be used in each API call, and managing conditions for every API used for your test case. {% embed url="" %} #### **Running Your Test** Loadmill offers multiple ways of running your test(s) like running your tests manually through the Loadmill app or running them through terminal but the most common one is the CI connection where, once connected to Loadmill, all you need to do is push your code to your CI and Loadmill will run the chosen tests automatically which gives you the freedom to concentrate more on your work rather than going through the hassle of managing multiple tabs and flows manually. # Quick Start Guide ## Quick Start — Demo AUT (Record & Run Your First Test) This guide will help you record and run your **first end-to-end test in Loadmill** using the Maker–Checker demo app.\ Time needed: **10–15 minutes** *** ### What You’ll Need * A Loadmill trial account (logged in) * Google Chrome (latest version) * [Loadmill Test Composer Chrome Extension](https://chromewebstore.google.com/detail/gdkmnfehipofdefhpegbgkkocinlaofd) installed and visible in DevTools * 10–15 minutes of focus *** ### Step 1 — Install the Composer 1. Install the **Loadmill Test Composer** from the Chrome Web Store. 2. If your Chrome is managed, ask IT to approve the extension.
*** ### Step 2 — Verify the Recorder 1. Start by navigating to 2. Open Chrome DevTools (**⌘⌥J** on macOS / **Ctrl+Shift+J** on Windows/Linux). 3. Select the **Loadmill Composer** tab. 4. Make sure the **Record** button is available. 5. Set **Filter Settings** to the demo AUT domain. 👉 Only after this step should you continue to the demo AUT.
*** ### About the Demo AUT — Maker–Checker Bank The demo application is a simple **bank transfer system** showing the classic *Maker–Checker approval workflow* (the “four-eyes principle”). It gives you a realistic but lightweight environment to test Loadmill. * **Maker** logs in, creates a new funds transfer (amount, recipient, description). The transfer is saved with status *Pending*. * **Checker** logs in separately, reviews pending transfers, and either approves or rejects them. * **Maker** can then log back in to see the updated status. **Demo URL & Credentials** * URL: * Maker: `maker / maker1234!` * Checker: `checker / checker1234!` *** ### Step 3 — Record the Maker & Checker Flow Together 1. In the Composer, **create a new Test Suite** by typing the name (e.g., *Maker–Checker Demo Flow*) into the suite selector and pressing **Enter**. 2. Click **Record**. 3. Log in as **Maker** and create a new transfer. 4. Confirm the transfer shows *Pending*. 5. **Click Pause** on the recorder (this ensures both roles are captured cleanly). 6. Log out (or open a private window). 7. Log in as **Checker**. 8. Approve the transfer created earlier. 9. Confirm the status updates to *Approved* (or *Rejected*). 10. Stop the recording only after both Maker and Checker steps are complete. 👉 This demonstrates a **real end-to-end business process** across two roles.
*** ### Step 4 — Analyze Requests 1. Click **Analyze Requests** in the Composer. 2. (Optional) Check the **“Show filtered steps”** box to see which calls were removed. You can override AI filtering decisions here. 👉 This produces a **leaner, faster, and more stable test**.
*** ### Step 5 — Create the Test 1. Once analysis is complete, click **Create Test**. 2. You’ll be redirected to the Loadmill platform. 3. Review the test steps generated from your recording.
*** ### Step 6 — Finalize and Enhance 1. **Review & Rename** your test for clarity. 2. **Check Parameters & Correlations:** Notice how inputs like *amount* and *recipient* were auto-parameterized, and dynamic values like *transaction ID* are automatically reused in later steps. * Rename parameters to readable keys if needed. * 👉 This reduces maintenance and makes tests reusable. 3. **Try an AI Refactor:** Click the **AI Refactor** button and type: ``` change recipient name to test-name ``` * 👉 This shows how Loadmill AI can refactor many steps at once, cutting down manual edits. *** ### Step 7 — Debug and Run the Test 1. Toggle **Debug Mode** (gear icon). 2. Add a **breakpoint** on one of the steps. 3. Run the test and watch it stop at the breakpoint. 👉 Debugging lets you inspect intermediate results and quickly validate logic. *** ### Step 8 — Run the Suite 1. Click **Run Suite** to execute the full flow. 2. Verify that the Maker–Checker flow passes end-to-end.
*** ### Step 9 — Run as Load Test 1. Click the **small arrow** next to **Run Suite** and select **Run as Load Test**. 2. Set the test duration to **2 minutes**. 3. Run with **5 users** (trial accounts are limited to small load tests). 👉 This shows how the **same flow** doubles as both a functional and performance test.
*** ### Step 10 — (Optional) Explore AI Configuration Want to make Loadmill’s AI match your team’s language? 1. Go to **Settings → Algorithm → AI Prompts**. 2. Customize how Loadmill: * Describes steps (tone, style, emojis, business terms) * Groups related steps using AI * Explains failures (e.g., known API errors) * Suggests next steps in a flow 👉 Tailoring AI output makes results clearer, saves triage time, and fits better into your workflow. **Explore here:** [AI Prompts Settings](https://app.loadmill.com/app/user/settings/algorithm?tab=ai-prompts) *** ### Troubleshooting * **Can’t find the recorder?** It may be hidden in Chrome’s toolbar. Pin it. * **No requests?** Check that recording is on and filters are correct. * **Managed Chrome?** Ask IT to install the extension. * **Security warning on AUT?** Expected, safe to proceed (browser flags it as a banking app). # Agent Testing Quick Guide This quick guide shows how to get access to Loadmill Agent Testing, create a simple agent-based test, and run it against a real web UI from the desktop app. *** ## Before you start You will need: * A Loadmill account * A plan that supports AI Agent capabilities (if you have a promo code, you can use it during sign-up to activate access) * The Loadmill desktop app installed *** ## Step 1: Sign up and activate access Create your account here: [Sign up to Loadmill](https://app.loadmill.com/app/signup) *If you already have a Loadmill account, you can skip the sign-up part of this step.* If you received a promo code by email, open [Billing](https://app.loadmill.com/app/user/settings/billing) after signing in and apply the code there. This enables access to the Agent step for eligible accounts. Open the Billing page from your account settings. ![Billing page showing the subscription plan section](/files/XQCdncwI2iycL4f1E0k3) Enter the promo code and activate it. ![Billing page showing the subscription code activation field](/files/yCdBuRvqihuUMMzdjHmj) *** ## Step 2: Download the desktop app Download the desktop app here: [Loadmill desktop app](https://app.loadmill.com/app/user/settings/desktop-app) **Important:** The desktop app is required to run Agent steps. *If you already have the desktop app installed, you can skip this step.* *** ## Step 3: Create your first agentic test 1. Open the Loadmill desktop app and create a new test suite. 2. Add a new step and choose **UI Agent**. ![Add a new UI Agent step](/files/UaJRv2m3bmXFEwZgsYZf) 3. Enter a short prompt that describes the user flow you want the agent to perform on your site and what it should validate. ![Example of a UI Agent step configured in a test](/files/fu9vyEw4uWnpi8QYc6Ak) For more details on configuring the UI Agent step, including the Navigate to URL, Agent Instructions, and AI-Driven UI Validation fields, see the [UI Agent step documentation](/test-editor/steps/ui-agent-step). 4. Run the test you created. ![Example of a UI Agent step run result](/files/RKQKWzm72dRkrPqcL04p) This is the fastest way to get started: give the agent a short, real UI task and validate that it can complete it reliably. {% hint style="info" %} Keep the first run simple. Start with one short path on your site so you can watch how the agent navigates the UI and validate the result. {% endhint %} *** ## Learn more about creating tests in Loadmill Agent Testing is a fast way to validate real user flows, but it works best as part of Loadmill's broader API-first testing approach. If you are new to Loadmill, you can also learn how to create tests with the Loadmill Composer and generate API tests from real user activity. See the [Quick Start Guide](/quick-guide/quick-start-guide) for a step-by-step walkthrough. *** ## Need help? If you have any questions or need help, contact us at . If you would like to book a demo, click [here](https://www.loadmill.com/book-a-demo). # Test Composer Setup Loadmill has several ways to compose full API scenarios - the most common is [Loadmill Composer Chrome Extension](https://chrome.google.com/webstore/detail/loadmill-test-composer/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en). This extension allows you to compose your API test scenario while browsing your application. It can register any type of incoming and ongoing API calls from the server. {% hint style="warning" %} Make sure to create an account if you want to use it. {% endhint %} Once installed, you can locate the Loadmill Composer as part of the dev tools tab. Click Command+Option+J (Mac)/ Ctr+Shift+J (Windows) to open your dev tools and select the Loadmill Composer tab. {% embed url="" %} {% hint style="info" %} **You can drag the Loadmill Composer tab for it to be more visible for you every time you open your dev tools** {% endhint %} {% embed url="" %} [![image-text](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LHDbUNdi3wPd9vSolzU%2Fuploads%2FNRrejIpGspdMBzdTlYHH%2FDownload%20Chrome%20Extention.png?alt=media\&token=8f06b3fe-6ad1-4a1b-95e3-ce0bdcc78853)](https://chrome.google.com/webstore/detail/loadmill-test-composer/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en) # Test Composer Layout Here's a quick walkthrough of the Loadmill Composer: 1. "Start" to start composing your test. 2. API registered in your flow. Click to expand the API to see request/response data. 3. "Analyze Request" for the AI to analyze and correlate your flow. 4. "Create Test" registers the flow in your Loadmill suite. 5. "Filter Settings" set specific URL filters. 6. Input for creating a new Loadmill suite. 7. Downloads scenario as a `.har` file. ![](/files/ZzJLs2HmM5hdoT3hDVeV) The composer detects all API calls sent by your application as you navigate the platform. Each API is registered with its method, URL, request body, and response. You can click on any API to view the contents of the request registered. {% embed url="" %} #### Analyzing Requests What makes the Loadmill Composer unique is the “Analyze Requests” feature. It allows the AI to process and analyze how each request will affect your API test and what kind of parameters you need in order to make it dynamic, allowing the test to be ready for any environment. To start correlating your flow, click “Analyze Requests”. {% hint style="info" %} **After creating a flow, hit “Pause”, so it does not register any incoming API calls. This will keep your API resources from being used unnecessarily.** {% endhint %} {% embed url="" %} #### Creating Tests Hit the “Create Test” button to upload a scenario. Loadmill generates the code for that uploaded scenario, parameterizing the correlated values in the corresponding flow. {% embed url="" %} # Filter Settings You can adjust your filter settings to allow you to personalize the API test. #### **Include Domains** "Include Domains" lets you register API calls from outside of the current domain. You'll find it useful when registering API flows that require external domains for authentication, purchase flows**,** etc. {% embed url="" %} #### Exclude Domains Exclude Domains registers every domain but the inputted domain you type in. {% embed url="" %} #### Filter Containing Strings The filter will remove strings containing any input given (such as ".js", “.img”,”css”, etc..) so that the URLs for these strings don't affect your API flow. This is an essential process for ensuring that your test is composed reliably. {% embed url="" %} #### Downloading Flow The `.har` registered scenario/flow file can be downloaded by clicking on the bottom-left icon. {% embed url="" %} # Algorithm Configuration Guide When creating a test in Loadmill, you can configure a rich set of rules that determine how your test scenario is analyzed, interpreted, and converted into an executable flow. These settings control assertions, filtering, data cleanup, parameter extraction, uniqueness detection, and various advanced behaviors of the test-generation algorithm. This guide explains every configuration option, what it does, when to use it, and how it affects the resulting test. To apply or modify these configurations, go to the Algorithm page in Loadmill and make changes through the UI or click `Edit Algorithm Settings` to edit the JSON directly. ![](/files/HGZfdcZzMhPpcr7BIYmi) Then, edit or add the relevant rules in the appropriate JSON properties. ## Table of Contents #### 1. [Assertions](#assertions) * [Automatic Assertions](#automatic-assertions) * [Automatic Assertions by URL](#automatic-assertions-by-url) * [Automatic Polling by URL](#automatic-polling-by-url) * [Strict Response Validation](#strict-response-validation) #### 2. [Filtering & Control](#filtering-and-control) * [Filter Irrelevant Requests](#filter-irrelevant-requests) * [Filter Request by Body](#filter-request-by-body) * [Headers Filters](#headers-filters) * [Ignored Keys](#ignored-keys) * [Ignored Values](#ignored-values) * [Irrelevant Aura Actions](#irrelevant-aura-actions) * [Irrelevant GraphQL Operations](#irrelevant-graphql-operations) * [Irrelevant SOAP Actions](#irrelevant-soap-actions) * [Keep All MIME Types](#keep-all-mime-types) * [Relevant URLs](#relevant-urls) * [Session Cut URLs](#session-cut-urls) * [URL Blacklist](#url-blacklist) #### 3. [Data-cleanup](#data-cleanup) * [Don't Extract From Response by URL Regex](#dont-extract-from-response-by-url-regex) * [Keep Original Values](#keep-original-values) * [Max Entities of Same Type](#max-entities-of-same-type) * [Reduce Global Parameters](#reduce-global-parameters) #### 4. [Structure & Semantics](#structure-and-semantics) * [Fields Hierarchy](#fields-hierarchy) * [JQuery Ignored Elements](#jquery-ignored-elements) * [JSONPath Ignored Keys](#jsonpath-ignored-keys) #### 5. [Parameter Detection](#parameter-detection) * [Custom Identifiers](#custom-identifiers) * [Custom Uniqueness](#custom-uniqueness) * [Default Values](#default-values) * [Aggregate Request Body Parameters](#aggregate-request-body-parameters) * [Include Failed Requests](#include-failed-requests) * [Keys Value as Key](#keys-value-as-key) * [Max Response Content Byte Size](#max-response-content-byte-size) * [Nested Value Decoding](#nested-value-decoding) * [Non Secret Keys](#non-secret-keys) * [ResponseRegexExtractors](#response-regex-extractors) * [Search Unique Values](#search-unique-values) * [Special Keys](#special-keys) * [Synonyms](#synonyms) * [Value Holder Key Map](#value-holder-key-map) #### 6. [Advanced & Serialization Settings](#advanced-and-serialization-settings) * [Allow Dots](#allow-dots) * [Array Format](#array-format) * [Decode URL-Encoded Body](#decode-url-encoded-body) * [Decode URL-Encoded Query](#decode-url-encoded-query) * [Prettify Request Body](#prettify-request-body) * [Split Origin](#split-origin) * [Xml Decode](#xml-decode) #### 7. [Uniqueness & Extraction Controls](#uniqueness-and-extraction-controls) * [ID Prioritization](#id-prioritization) * [Max Uniqueness Length](#max-uniqueness-length) #### 8. [Post Processing Editing](#post-processing-editing) * [Yaml Regex Replacers](#yaml-regex-replacers) * [Regex Replacers](#regex-replacers) *** ## Assertions ### Automatic Assertions Defines **automatic assertions** that Loadmill applies to extracted parameters. **When to use:** * When your API contains stable response fields that should always be validated. * When you want to detect silent failures (`200 OK` but the operation logically failed). **Structure:** **`assertions`** *object* An object that defines how to generate assertions for extracted parameters. The object includes the following properties: * **`isDefaultAssertion`** *boolean, Defaults to `true`* Defines basic automatic assertions of *Exists* applied to all extracted parameters. * **`specificAssertions`** *array, Defaults to \[ ]* Define specific assertion applied to parameters with defined keys. Each item in the array contains: * **`key`** *string* The key that, when encountered, should create the automatic assertion. * **`assertion`** *{ \[assertionType: string]: string }* Defines the assertion to apply. The object must contain a single assertion operator (e.g., equals, matches, contains) and its expected value. Example: ```json "assertions": { "isDefaultAssertion": true, "specificAssertions": [ { "key": "success", "assertion": { "equals": "yes" } } ] } ``` *** ### Automatic Assertions by URL Extracts a value from responses whose URL matches a pattern, then asserts it. **When to use:** * When a specific endpoint always returns a value you want to verify. **Structure:** **`autoAssertionsByUrl`** *array* Defines automatic assertions for specific endpoints. Each item specifies a URL pattern, a JSONPath to extract the value, and the assertion to apply. Each item in the array contains: * **`url`** *string* Regex pattern to match the request URL. * **`jsonpath`** *string* JSONPath expression to extract the value from the response. * **`assertion`** *{ \[assertionType: string]: string }, Optional* Assertion operator and expected value (e.g., `{ "equals": "processed" }`). * **`loop`** *object, Optional* Adds a polling loop for the matched URL. The loop includes: * **`iterations`** *number* Number of polling attempts. * **`assert`** *{ \[assertionType: string]: string }* Assertion to apply on each polling response. * **`wait`** *number* Wait time (ms) between iterations. Example: ```json "autoAssertionsByUrl": [ { "url": "/api/orders/[0-9]{8}$", "jsonpath": "$.data.status", "assertion": { "equals": "processed" } }, { "url": "/api/orders/[0-9]{8}/status$", "jsonpath": "$.data.status", "loop": { "iterations": 5, "assert": { "equals": "completed" }, "wait": 1000 } } ] ``` *** ### Automatic Polling by URL Defines **automatic polling loops** for endpoints matching specific URL patterns. **When to use:** * When your system performs asynchronous operations requiring repeated polling. * For status endpoints that eventually reach a final state. * When testing processes that aren't instantaneous (e.g., onboarding, file processing). **Structure:** **`automaticLoops`** *array* An array where each item defines the conditions under which an automatic loop should be added. Each item in the array contains: * **`url`** *string* Regex pattern to match the request URL. * **`loop`** *object* The loop includes the following properties: * **`iterations`** *number* Number of polling attempts. * **`assert`** *{ \[assertionType: string]: string }* Assertion to apply on each polling response. * **`wait`** *number* Wait time (ms) between iterations. * **`expectedStatus`** *string, Optional* Expected HTTP status code (e.g., `"ANY"`). Example: ```json "automaticLoops": [ { "url": "https://example.com/orders/[0-9]{9}/status\\?subId=[0-9a-z]{40}", "loop": { "iterations": 20, "assert": { "check": "__status", "equals": "200" }, "wait": 2000 }, "expectedStatus": "ANY" } ] ``` *** ### Strict Response Validation Defines **strict validation** rules for API responses using JSON Schema or "JSON contains" logic. **When to use:** * When your API response structure must be validated (schema-based testing). * When responses contain dynamic values that shouldn't break validation (use `ignoreKeys`). **Structure:** **`strictResponseValidation`** *object* Controls whether Loadmill should automatically generate strict validation rules (JSON Schema or JSON Contains) for each API response. The object includes the following properties: * **`jsonSchema`** *boolean* Enables strict validation of the response against an auto-generated JSON Schema. Ensures the structure, types, and required fields match the expected schema. * **`jsonContains`** *boolean* Enables validation that the response contains specific fields/values. * **`ignoreKeys`** *array* A list of keys to exclude from validation. Useful for dynamic fields (e.g., id, timestamp, uuid) that change between runs. * **`shouldValidateArrayItems`** *boolean* If true, validates each item in arrays individually. Examples: ```json "strictResponseValidation": { "jsonSchema": true, "ignoreKeys": ["id"] } ``` ```json "strictResponseValidation": { "jsonContains": true, "ignoreKeys": ["teamLabelsToIgnore", "name"], "shouldValidateArrayItems": true } ``` *** ## Filtering & Control ### Filter Irrelevant Requests Controls whether to automatically remove requests that don’t impact the outcome of the test, such as telemetry, health checks, and other noise. **When to use:** * When HAR contains a lot of browser noise (Mixpanel, GA, LinkedIn, etc.). * When you want cleaner tests with fewer steps. * When focusing only on business logic requests. **Structure:** **`filterIrrelevantRequests`** *boolean, Defaults to true* When enabled, the generated test is cleaner, more stable, and easier to debug. Example: ```json "filterIrrelevantRequests": true ``` *** ### Filter Request by Body Filters requests according to JSONPath rules. **When to use:** * When URL-based filtering is insufficient. * When the request body is JSON. * When the only reliable way to identify irrelevant requests is by inspecting the request body, for example, in GraphQL requests. **Structure:** **`filterRequestByBody`** *array* Each item in the array contains: * **`query`** *string* JSONPath expression to match a value in the request body. * **`equals`** *string* Value to compare against the result of the JSONPath query. Example: ```json "filterRequestByBody": [ { "query": "$.type", "equals": "telemetry" } ] ``` *** ### Headers Filters Removes specific headers during processing. **When to use:** * To avoid extracting irrelevant or unstable headers. * When headers contain high-frequency noise (Cloudflare, Chromium, WebSocket). * When cleaning request data for readability. **Structure:** **`headersFilters`** *string \[ ]* An array of header names or prefixes to exclude from extraction. Example: ```json "headersFilters": [ "x-", "cf-", "accept", "Sec-WebSocket-Key", "cookie" ] ``` *** ### Ignored Keys Skips extraction for specific keys during parameter extraction. **When to use:** * When certain keys are known to be irrelevant or noisy. **Structure:** **`ignoredKeys`** *string \[ ]* Array of key names to ignore during extraction. Example: ```json "ignoredKeys": ["irrelevantKey", "unusedField"] ``` *** ### Ignored Values Skips extraction of trivial or unhelpful values. **When to use:** * To suppress useless values that appear frequently and create irrelevant parameters. **Structure:** **`ignoredValues`** *array, Defaults to `["undefined", null, "null", true, "true", false, "false"]`* List of values to ignore during parameter extraction. Example: ```json "ignoredValues": ["undefined", null, "null", true, "true", false, "false"] ``` *** ### Irrelevant Aura Actions Filters out Aura actions that are not relevant to the generated test. An Aura action is a request in the Salesforce Lightning framework that performs a specific operation. **When to use:** * When your HAR contains Aura requests with actions that are not relevant to your test scenario. * To remove noise and focus on business-critical Aura actions. **Structure:** **`irrelevantAuraActions`** *string \[ ]* Array of regex patterns. Any Aura action that matches a pattern in this array and has no extracted values will be excluded from the generated test. ```json "irrelevantAuraActions": ["^get"] ``` *** ### Irrelevant GraphQL Operations Filters out irrelevant GraphQL operations that should not be included in the generated test. **When to use:** * When your HAR contains GraphQL requests with operations that are not relevant to your test scenario. * To remove noise and focus on business-critical GraphQL operations. **Structure:** **`irrelevantGraphQLOperations`** *string \[ ]* Array of GraphQL operation names. Any operation that matches a name in this array and has no extracted values will be excluded from the generated test. Example: ```json "irrelevantGraphQLOperations": [ "IntrospectionQuery", "TelemetryEvent" ] ``` *** ### Irrelevant SOAP Actions Filters out SOAP actions that should not be included in the generated test. **When to use:** * When your HAR contains SOAP requests with actions that are not relevant to your test scenario. * To remove noise and focus on business-critical SOAP actions. **Structure:** **`irrelevantSoapActions`** *string \[ ]* Array of SOAP action names. Any action that matches a name in this array and has no extracted values will be excluded from the generated test. Example: ```json "irrelevantSoapActions": [ "GetServerTime", "Ping" ] ``` *** ### Keep All MIME Types Controls whether all MIME types are preserved during extraction. **When to use:** * When you need to extract parameters from non-standard or binary MIME types. * To ensure no data is filtered out based on MIME type. **Structure:** **`keepAllMimeTypes`** *boolean* If true, all MIME types are kept during extraction, including binary and custom types. Example: ```json "keepAllMimeTypes": true ``` *** ### Relevant URLs Forces these URLs not to be filtered. **When to use** * When your flow depends on URLs that look noisy but are actually important. * To override aggressive filtering and ensure specific endpoints are always included. **Structure** **`relevantUrls`** *string \[ ]* An array of URL substrings or patterns. Any request URL containing one of these will be included in the generated test, even if it would otherwise be filtered out. Example: ```json "relevantUrls": [ "banking/sessions/create", "v1/cities" ] ``` *** ### Session Cut URLs Defines URL patterns where a new session should be started during test generation. **When to use:** * When using a backend recordings. * To split flows into distinct sessions for better isolation and analysis. **Structure:** **`sessionCutUrls`** *string \[ ]* Array of URL substrings or regex patterns. When a request matches any of these, a new session is started. Example: ```json "sessionCutUrls": [ "/api/login", "/logout" ] ``` *** ### URL Blacklist Removes requests containing any of the specified substrings. **When to use** * To filter out entire domains. * To eliminate 3rd-party scripts (CDN, tracking, ads). **Structure** **`urlBlackList`** *string \[ ]* an array of substrings. Any request URL containing one of these substrings will be excluded from the generated test. Example: ```json { "urlBlackList": ["your-domain.com", "another-domain"] } ``` *** ## Data-Cleanup ### Don't Extract From Response by URL Regex Skips extraction for responses whose request URL matches a pattern. **When to use:** * When some endpoints return noisy or irrelevant payloads. **Structure:** **`dontExtractFromResponseByURLRegex`** *string \[ ]* Array of regex patterns. Extraction is skipped for responses matching these patterns. Example: ```json "dontExtractFromResponseByURLRegex": ["pattern1", "pattern2"] ``` *** ### Keep Original Values Prevents extraction for matching URLs. **When to use:** * When there are fixed values that you don't want to replace with parameters. **Structure:** **`keepOriginalValues`** *string \[ ]* Array of URL patterns for which values should not be replaced with parameters. Example: ```json "keepOriginalValues": ["/internal/status"] ``` *** ### Max Entities of Same Type Limits how many identical keys are extracted from a single request. **When to use** * When responses include many repetitive objects. **Structure:** **`maxEntitiesOfSameType`** *number, Defaults to 15* Maximum number of identical keys to extract per request. Example: ```json "maxEntitiesOfSameType": 15 ``` *** ### Reduce Global Parameters Controls extraction of global parameters to prevent excessive or irrelevant parameterization. **When to use:** * When too many global parameters are being extracted, leading to noisy or unstable tests. * To focus extraction only on meaningful parameters. **Structure:** **`reduceGlobalParameters`** *boolean, Defaults to false* When enabled, limits extraction of global parameters to only those that are relevant. Example: ```json "reduceGlobalParameters": true ``` *** ## Structure & Semantics ### Fields Hierarchy Defines how extracted JSONPaths use attribute-based selectors instead of index-based selectors for improved stability. **When to use:** * When the entity type can be reliably identified by a specific key. * To avoid unstable extraction paths that depend on array indices. **Structure:** **`fieldsHierarchy`** *string \[ ]* Array of field names. When present, extraction paths will use attribute-based selectors for these fields. Example: ```json "fieldsHierarchy": ["name", "title"] ``` *** ### jQuery Ignored Elements Skips extraction for specific HTML elements when generating jQuery selectors. **When to use:** * When invalid HTML causes Loadmill to fix the markup and generate selectors on the corrected structure. * When you want to avoid extracting or interacting with certain elements, even after HTML correction. **Structure:** **`jqueryIgnoredElements`** *string \[ ]* Array of HTML element names to ignore during jQuery selector generation. Example: ```json "jqueryIgnoredElements": ["tbody"] ``` *** ### JSONPath Ignored Keys Defines patterns for dynamic key segments (such as UUIDs or timestamps) that should be replaced with recursive descent (`..`) in generated JSONPath expressions. **When to use:** * When dynamic map keys (such as UUIDs or timestamps) make JSONPath expressions unstable. * To simplify extraction paths and avoid brittle selectors. **Structure:** **`jsonpathIgnoredKeys`** *string \[ ]* Array of regex patterns. Any key matching a pattern will be replaced with recursive descent in JSONPath. Example: ```json "jsonpathIgnoredKeys": [ "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$" ] ``` *** ## Parameter Detection ### Custom Identifiers Adds custom identifier field names (case-insensitive) for parameter extraction. **When to use:** * When your system uses nonstandard naming for ID fields. * To ensure custom identifier fields are prioritized during extraction and correlation. **Structure:** **`customIdentifiers`** *string \[ ]* Array of field names (case-insensitive) to treat as identifiers during extraction. Example: ```json "customIdentifiers": ["testSuiteCard", "orderNumber"] ``` *** ### Custom Uniqueness Detects unique values and their reoccurrences by specifying keys and regex patterns. Without any additional conditions. **When to use:** * When a key contains a unique and important value that should be replaced everywhere it appears. **Structure:** **`customUniqueness`** *array* An array where each item contains rules that define unique values. Each item in the array contains: * **`key`** *string, Optional* The key whose value should be detected as unique. * **`regex`** *string* Regex pattern to match the unique value. Example: ```json "customUniqueness": [ { "key": "patientNumber", "regex": "^[a-z0-9]{25}$" } ] ``` *** ### Default Values Assigns fallback values to specific fields during test generation. **When to use:** * When you need to extract a specific value from the request body and create correlations for it. * When fixed values must be replaced with dynamic ones (e.g., timestamps, dates, random IDs). **Structure:** **`defaultValues`** *array* an array where each item contains rules that define how to extract parameters and assign them a default value. Each item in the array contains: * **`selector`** *string* The key to identify the field. * **`value`** *string* The fallback value to assign. * **`name`** *string* The parameter name to use for correlation. Example: ```json "defaultValues": [ { "selector": "userId", "value": "123", "name": "userId" } ] ``` *** ### Aggregate Request Body Parameters Aggregates matched request-body objects into a single parameter for correlation instead of extracting only primitive values. **When to use:** * When you want to correlate request-body objects (not just primitive values). * To reduce the number of generated parameters by treating an object as one correlated value. **Structure:** **`aggregateRequestBodyParameters`** *boolean, Defaults to false* When enabled, if a match is found in the request body and the value is an object, the algorithm extracts the object as a single parameter and correlates it across requests, which reduces the number of generated parameters. Example: ```json "aggregateRequestBodyParameters": true ``` *** ### Include Failed Requests Includes failed HTTP requests in the generated test. **When to use:** * For debugging flows where failures are expected or important. * When you want to analyze and reproduce error scenarios. **Structure:** **`includeFailedRequests`** *boolean, Defaults to false* When enabled, failed HTTP requests (non-2xx responses) are included in the generated test. Example: ```json "includeFailedRequests": true ``` *** ### Keys Value as Key Treats the value of a key as a parameter. **When to use:** * When you need to extract a value from a key and use it as a parameter for correlation or substitution. **Structure:** **`keysValueAsKey`** *string \[ ]* Array of key names whose values should be treated as parameters. Example: ```json "keysValueAsKey": ["customTokenField"] ``` *** ### Max Response Content Byte Size Controls the maximum response size (in bytes) that the system will process for parameter extraction. **When to use:** * Increase the default value when you need to extract parameters from very large responses. **Structure:** **`maxResponseContentByteSize`** *number, Defaults of 3000000* Maximum response size (in bytes) that will be processed for extraction. Example: ```json "maxResponseContentByteSize": 3000000 ``` *** ### Nested Value Decoding Parses and extracts parameters from fields containing stringified JSON objects. **When to use:** * When a field contains a stringified JSON object that needs to be parsed for parameter extraction and correlation. **Structure:** **`nestedValueDecoding`** *array* an array where each item contains rules specifying which keys in the response body should undergo additional parsing. Each item in the array contains: * **`key`** *string* The key whose value is a stringified JSON object. * **`type`** *string* The format of the nested value (e.g., `"JSON"`). Example: ```json "nestedValueDecoding": [ { "key": "value", "type": "JSON" } ] ``` *** ### Non-Secret Keys Defines keys whose values are considered non-sensitive and will be extracted and recorded even when operating in secure mode. **When to use:** * When using a backend recordings. * When you want to ensure certain values are always extracted, even in secure recording. * When specific keys do not contain secrets and should be included for correlation or debugging. **Structure:** **`nonSecretKeys`** *string \[ ]* Array of key names whose values are safe to extract and record. Example: ```json "nonSecretKeys": ["sessionId", "userId"] ``` *** ### Response Regex Extractors Extracts values from response bodies using regex capturing groups. **When to use:** * When you need to extract and correlate a value from the response body that cannot be accessed using standard extraction methods (e.g., not available via JSONPath). **Structure:** **`responseRegexExtractors`** *array* an array where each item contains rules for extracting parameters from the response body using regex. Each item in the array contains: * **`regex`** *string* Regex pattern with capturing groups to extract the desired value from the response body. * **`name`** *string* The parameter name to assign to the extracted value. * **`forceExtract`** *boolean, Optional, Defaults to false* If true, always extract the value even if it only appears once and has no correlations. Example: ```json "responseRegexExtractors": [ { "name": "refresh_token_id", "forceExtract": false } ] ``` *** ### Search Unique Values Detects unique values and their reoccurrences based on their values, not just their keys or context. **When to use:** * When you want to identify unique values according to their actual content, regardless of their field names or context. **Structure:** **`searchUniqValues`** *boolean, Defaults to false* When enabled, the algorithm will detect and correlate unique values wherever they appear. Example: ```json "searchUniqValues": true ``` *** ### Special Keys Always extracts these keys from the request body, regardless of other extraction settings. **When to use:** * When `reduceGlobalParameters` is off and you still want to extract certain keys from the request body. **Structure:** **`specialKeys`** *string \[ ]* Array of key names to always extract from the request body. Example: ```json "specialKeys": [ "totalAmount", "delivery", "customerQuoteTime", "subtotalAmount" ] ``` *** ### Synonyms Maps different key names to unify correlation and extraction. **When to use:** * When different endpoints use different naming conventions for the same logical identifier. * To unify extraction and correlation of values across APIs with inconsistent field names. **Structure:** **`synonyms`** *array* Each item is an array of key names that should be treated as synonyms for correlation and extraction. Example: ```json "synonyms": [ ["sku", "product"], ["amountPaid", "totalAmountPaid", "totalAmount"], ["customerQuoteTime", "delivery"] ] ``` *** ### Value Holder Key Map Extracts nested values when a key contains an object instead of a primitive. **When to use:** * When a field stores a nested object instead of a primitive. **Structure:** **`valueHolderKeyMap`** *array* an array where each item contains rules for extracting parameters whose value is located inside one of their nested fields. Each item in the array contains: * **`containerKey`** *string* The key whose value is an object containing the actual value. * **`valueKey`** *string* The key inside the object that holds the value to extract. Example: ```json "valueHolderKeyMap": [ { "containerKey": "password", "valueKey": "text" }, { "containerKey": "id", "valueKey": "value" } ] ``` *** ## Advanced & Serialization Settings ### Allow Dots Controls whether dots (`.`) are allowed in parameter names. **When to use:** * When parameter names may safely contain dots (e.g., `user.name`). * To support APIs or systems that use dotted notation for keys. **Structure:** **`allowDots`** *boolean, Defaults to false* If true, allows dots in parameter names during extraction and correlation. Example: ```json "allowDots": false ``` *** ### Array Format Controls how arrays are serialized in URL-encoded form. **When to use:** * When URL-encoded arrays must follow a specific backend format. * To ensure compatibility with server-side array parsing. **Structure:** **`arrayFormat`** *string, Defaults to "indices"* **Allowed values:** "indices" | "brackets" | "repeat" Specifies the serialization format for arrays in URL-encoded bodies. Example: ```json "arrayFormat": "indices" ``` *** ### Decode URL-Encoded Body Enables decoding of request bodies with `application/x-www-form-urlencoded` content type. **When to use:** * For POST requests with `application/x-www-form-urlencoded`. * When you need to extract parameters from URL-encoded request bodies. **Structure:** **`decodeUrlEncodedBody`** *boolean, Defaults to false* If true, decodes URL-encoded request bodies for parameter extraction. Example: ```json "decodeUrlEncodedBody": true ``` *** ### Decode URL-Encoded Query Enables decoding of URL query strings to improve readability in the generated test. **When to use:** * When your server does not require query parameters to remain encoded. * When you want URL query parameters to appear in a human-readable format for easier review and debugging. **Structure:** **`decodeUrlEncodedQuery`** *boolean, Defaults to true* If true, decodes URL-encoded query strings for better readability in the output. Example: ```json "decodeUrlEncodedQuery": true ``` *** ### Prettify Request Body Controls formatting of request bodies in the generated YAML output. **When to use:** * When you want readable, formatted JSON bodies in your test output. * When performance snapshots require compact bodies (disable prettification). **Structure:** **`prettifyRequestBody`** *boolean, Defaults to true* If true, pretty-prints large JSON bodies in output YAML for readability. If false, leaves request bodies compact (not formatted). Example: ```json "prettifyRequestBody": true ``` *** ### Split Origin Separates origins into host and protocol components. **When to use:** * When combining HAR files from backend recordings. * To ensure requests are grouped and processed by their actual origin. **Structure:** **`splitOrigin`** *boolean, Defaults to false* If enabled, splits the origin into host and protocol for more granular processing. Example: ```json "splitOrigin": false ``` *** ### XML Decode Enables decoding of special formats from systems like Priority ERP (not standard XML). **When to use:** * When working with Priority ERP or systems returning non-standard XML encodings. **Structure:** **`xmlDecode`** *boolean, Defaults to false* If true, enables decoding and extraction from non-standard XML formats. Example: ```json "xmlDecode": true ``` *** ## Uniqueness & Extraction Controls ### ID Prioritization Prioritizes extraction of parameters whose keys match known or custom identifier fields (such as `id`, `uuid`, or those defined in `customIdentifiers`) when multiple candidates exist in an object or array. **When to use:** * When your data contains multiple possible identifiers and you want to ensure the most meaningful one is extracted. * To improve correlation and stability by always preferring standard or custom ID fields. **Structure:** **`idPrioritization`** *boolean, Defaults to false* If true, extraction will prioritize keys matching identifier fields (e.g., `id`, `uuid`, or those in `customIdentifiers`) over other candidates. Example: ```json "idPrioritization": false ``` *** ### Max Uniqueness Length Limits the maximum length of a value considered for uniqueness detection. **When to use:** * To avoid extracting long strings (such as logs, SQL queries, or HTML) as parameters. * When you want to prevent noisy or irrelevant values from being treated as unique identifiers. **Structure:** **`maxUniquenessLength`** *number, Defaults to 40* Maximum length (in characters) of a value to be considered for uniqueness detection. Values longer than this will be ignored. Example: ```json "maxUniquenessLength": 40 ``` *** ## Post Processing Editing ### Regex Replacers Extracts values from requests using regex capturing groups. **When to use:** * When values do not appear in JSON at all (e.g., tokens inside headers or raw bodies). * When there is a repeated value that was not extracted into a parameter. **Structure:** **`regexReplacers`** *array* an array where each item contains a regex that, if matched, should create a parameter for it. Each item in the array contains: * **`name`** *string* The parameter name to assign to the extracted value. * **`regex`** *string* Regex pattern with capturing groups to extract the desired value from the request. * **`reuse`** *boolean, Optional, Defaults to false* If true, reuses an existing parameter even when the values are different. Example: ```json "regexReplacers": [ { "name": "auth_token", "regex": "eyJhbGciOi.*", "reuse": true } ] ``` *** ### YAML Regex Replacers post-processing regex replacements on the generated YAML output. **When to use:** * When you need to mask, transform, or enhance YAML after generation. **Structure:** **`yamlRegexReplacers`** *array* an array where each item contains rules specifying which changes should be applied to the generated YAML. Each item in the array contains: * **`regex`** *string* Regex pattern to match in the YAML output. * **`replace`** *string* Replacement string to substitute for matches. * **`flags`** *string, Defaults to ""* Regex flags (e.g., `"g"` for global, `"i"` for case-insensitive). Example: ```json "yamlRegexReplacers": [ { "regex": "password: \".*\"", "replace": "password: \"***\"", "flags": "g" } ] ``` *** # Layout Welcome to the Test Editor Layout. Here you’ll tour all the editor components to understand better and customize your tests. Loadmill’s Test Editor brings ease of access and customization to API testing. Upload a test scenario and generate dynamic code with multiple, expandable API components including extracting and validating dynamic values from the server response, running custom code for every API call, and managing conditional results for every API used in your test case. You can run your tests locally, or in your CI/CD pipeline with a single click. Here’s a tour of the Test Editor to guide you to all its functionalities:
* In the Flows section on the left column, you can rename or delete your flows. * The API editor in the center are all the expandable API components. * On the Flow Runs (or Trial Runs) in the right column, you can see a single test run per flow that includes all its activity logs. # Flows A flow is a composed set of correlated API requests registered from an operation made by the user. A flow can represent any type of operation like adding and deleting a user to a group chat or simply going through a checkout process. ![](/files/FrfJl6XSmeEma8iUnhI5) You can register flows through the [Test Composer Chrome Extension](https://github.com/loadmill/loadmill-docs/tree/master/test-editor/flows/broken-reference/README.md) or adding the Loadmill User Behavior Module on your server. When uploading your flow/test scenario to Loadmill, you’ll see the flow added to your test suite on the left side of the platform. By default, the name is the date and time you uploaded your test but make sure to rename it to, most commonly, the name of the operation you composed to a test. {% embed url="" %} # Generated Flow Code After uploading your flow/scenario to Loadmill, its sophisticated AI generates a complete automated dynamic test build that you can easily view, edit and integrate. The generated code is located at the top of the editor and is written and saved in `.yaml` You can edit your test code through the Loadmill platform or through your IDE by setting up Data Sync to your repository. See here how to enable Data Sync to your Test Suite. {% embed url="" %} # Test Flow editor API Test flow is a series of API calls (HTTP requests) that represent **a single "user" flow you would like to test**. These requests will be executed sequentially until completion or until the first failure. ![](/files/QEJ1mktw7JmWv4sCuv4K) {% hint style="info" %} 🧠 Control visibility of the flows' list area by clicking on Flows in the Test Suite navigation panel. This way you can better focus on the flow you are currently working on. {% endhint %} ## The Test Flow Toolbar
| Option | Description | | ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | AutoSave | Auto saves the test flow unless its configuration is invalid. | | Login toggle | Toggle this switch to enable the flow to use [the Global Login](https://docs.loadmill.com/api-testing/test-suite-editor/global-login-flow) settings. | | CSV toggle | Easily re-run your test flows by using different parameters values by uploading a [CSV file](https://docs.loadmill.com/api-testing/test-suite-editor/api-tests-data-from-csv-files) containing a list of the values. | | Flow status (default: ![](/files/5RT0s8svV9roWXSotBZj)) | Set the [status of the flow](https://docs.loadmill.com/general/api-testing1/analyzing-an-api-test-results#flow-status). | | Run Flow button (![](/files/HuEf7uBdq9peO008MFWj)) | Run the edited flow as a "Trial run" to debug and validate it. | | Run with overrides (![](/files/FbaGJSD5cy1MWYg1f6hz)) | Run the edited flow as usual, but [override some parameters](https://docs.loadmill.com/test-editor/flows/flow-execution#runnning-with-overrides). | | Debug mode toggle (![](/files/XxZdBGk4sN0C280F8p9B)) | Turn the [Debug Mode](https://docs.loadmill.com/test-editor/flows/flow-controls#debug-mode) on/off. | | Flow scope parameters (![](/files/YltJ7ZZc2XI1unP8U6EK)) | Set parameters which can be used in a specific flow. | | Edit flow code (![](/files/3Ntn9WC9gO1xnfVgVkea)) | Edit the raw YAML code of the saved flow. | | Edit using AI (![](/files/kHVVcDjIR0ih10D7eEOd)) | Describe the desired changes and get a suggestion made by AI. | | Restore deleted requests (![](/files/zX27kkp7f3nG0BD2Gb7k)) | Allows to restore previously deleted requests (up to 10 requests), and the restoration stack clears upon refreshing the page. | | Add request | Add en empty request to the flow | | Paste requests | Paste a request - accepts CURL, and copied requests from the flow. |
| Option | Description | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Export Flow | Save the flow to a YAML file. You can then import the flow into another test suite or as a load test. | | Duplicate Flow | Create a copy of the original flow within the Test Suite. | | Convert to Load Test | Converts the flow to a [Load Test](#converting-api-flow-to-load-test). | | Replace Parameter | Replace the usage of a parameter in this flow with another parameter. (Find more information about parameters [here](https://docs.loadmill.com/api-testing/test-suite-editor/parameters)). | | Extract Parameter | Extract a value to a parameter from all across the test requests. | | Copy Flow to Suite | Copy the flow into another Test Suite. | | Generate Descriptions | Generate a suitable description to all requests in a flow using AI. | | Configure service Definition | For GRPC, allows selecting the service that will be used in the flow. | | Get Last Trial Run | Open the last run's results window. | ### **Converting API flow to Load Test** Easily convert an API flow to Load test by clicking on the Flow options and select "Convert to Load Test". Once you've selected ![](/files/Yfj5y4GZ5kPqkJlB6uJY), a new Load test will open containing all the requests and properties taken from the original API test. # Flow Controls Flows provides additional features and functions that can be found in the top right corner of the editor. {% embed url="" %} ### Debug Mode Working on a flow often requires checking or verifying certain things during execution. For example, if the flow involves adding then removing an item from an online shopping cart, we might want to verify that the item was indeed added to the cart during the execution. Loadmill provides the functionality to execute tests in debug mode. When the debug toggle (![](/files/XxZdBGk4sN0C280F8p9B)) is turned on (![](/files/LAxh1OeljDgD29BXCS6h)), a small checkbox will appear next to each request in the flow, allowing you to set a request as the breakpoint for debugging. Once the flow is executed, the test will be paused right before the checked request is made. The results window will already show the statuses of completed requests, with a 'pending' status indicating the upcoming request, which serves as the breakpoint. {% embed url="" %} #### Export Flow Exports the current flow you're viewing (in `.yaml`). #### Duplicate Flow Duplicates the current flow on your Test Suite. #### Convert to Load Test Coverts the current flow into a Load Test. See here how to setup Load testing with Loadmill. #### Replace Parameters Replaces the given parameter generated by Loadmill or by the user to a new parameter name. **ex:** Loadmill uses `${origin_`*`1}` to define the base url of the test.* By applying Replace Parameters you can replace `${origin1}` to `${base_url}`. {% hint style="info" %} This only changes the name of the parameter without affecting its value. {% endhint %} {% embed url="" %} #### Extract Parameter Extracts a given value from the Test Flow into a parameter. **ex:** Say you have a unique value that you want to parameterize like a visitor id, a specific name or even a phone number. You can extract those values and parameterize them by applying the Extract Parameter tool. {% embed url="" %} {% hint style="info" %} All parameterized values are located in the PARAMETERS tab of the suite. {% endhint %} #### Copy Flow To Suite Copies the current flow to the chosen suite. #### Suggest Failure Once a flow is executed and fails, this button will appear ![](/files/37DZJuF21YyN8fqr5N5V) which, using AI, provides you with a possible reason and solution for the test, according to the tests agenda. For example:

In this example, it seems like one of the request used the wrong method

#### Fix It For Me As AI suggests whats wrong with the test, and gives a possible way to solve it, why wouldn't it just solve it by itself?\ The 'FIX IT FOR ME' button, leads to the edit-using-AI panel prompt prefilled with the suggestion AI provided before.
Then it acts like the regular 'Edit Using AI' feature explained [here](https://docs.loadmill.com/test-editor/flows/test-flow-editor#the-test-flow-toolbar). # Groups A Group lets you bundle multiple steps together under a shared label. This keeps your flow organized and makes it easier to understand at a glance what a set of steps is doing. Groups also unlock features like **Repeat Group**, which lets you run the same set of steps multiple times automatically. ![Steps group in the flow editor](/files/HQR3yHQISoshcHI7Fb71) ## Creating a Group To create a group: 1. In the flow editor, select the steps you want to group together using the checkboxes next to each step. 2. Click the **Group** action that appears at the top. 3. Give your group a descriptive name (e.g., "Create user and add to team"). The grouped steps will now appear together under a collapsible section with your description. ![Group steps in the flow editor](/files/FbofSzEONRR0kMMiRGJA) ## Repeat Group Repeat Group lets you run the same group of steps once for each item in an array. ### How It Works When you configure a Repeat Group, you provide: 1. **An 'array parameter'** — an array to iterate over (e.g., a list of user IDs). 2. **A 'set value to' parameter** — the name of the parameter that will hold the current item on each iteration (e.g., `user_id`) On each iteration, Loadmill takes the next item from the array and assigns it to your chosen parameter. All steps in the group execute with that value before moving on to the next item. You can also set an optional **delay** (in milliseconds) between iterations if needed. ![Repeat group configuration in the flow editor](/files/KZ42NIZ8oL1UeFU10gdO) ### Example: Adding Users to a Team Use Repeat Group when you need to run the same steps (e.g., create user, add to team) once for each item in a list returned by an earlier step. * You have a parameter `userIds` with the value: ```json ["user_1", "user_2", "user_3"] ``` * In the Repeat Group settings: * **Array parameter:** `userIds` * **Set value to:** `user_id` * **Delay:** `300` ms **What happens when the flow runs:** | Iteration | `${user_id}` value | | --------- | ------------------ | | 1 | `user_1` | | 2 | `user_2` | | 3 | `user_3` | Each iteration, the group's steps execute using the current `${user_id}` value. For example, if one of the steps sends a POST request to add a user to a team, the request body might look like: ```json { "userId": "${user_id}", "teamId": "${team_id}" } ``` On the first iteration, `${user_id}` will be `user_1`. On the second, `user_2`. And so on until all items in the list have been processed. ![Repeat group report](/files/FdsezsWHqp8eZ4wImkHJ) # Add CSV to Flow When you're ready to take your automation testing to the next level, it's time to start thinking about testing your flow with multiple variations. This means creating different versions of your test cases and running them against different versions of your software. Doing this can help you find bugs that only show up in specific combinations of your software. This is important because it helps to ensure that your tests are comprehensive and that they cover all possible scenarios. Creating multiple variations of your tests also allows you to test different parts of the API in isolation, which can be helpful in identifying issues. Loadmill provides a quick, easy method in testing different iterations of your flow by simply uploading a CSV file containing the different iteration values. Here's an example for a user management application. This app allows you to add and remove users from your dashboard. {% embed url="" %} To start with, let's try 5 different email addresses, as well as 5 first names and 5 last names. We'll divide the task into 4 easy steps. ### Create a CSV file. The following table shows the variations we want to test. {% hint style="info" %} The **expected\_outcome** will not be uploaded to the test. This is simply to show you the expected outcome for each variation tested. {% endhint %}
emailfirst_namelast_nameexpected_outcome
variation1@loadmill.comJohnDoepass
variation2@random.comDoe2pass
variation3VariationThreefail > wrong email
variation4@John4Doe_4fail > wrong email
variation45@companyJohn_$fail > wrong email
### Update parameter names. The *"Create a user"* API contains hard-coded values that need to be substituted with the parameter names from the CSV headers (***email**, **first\_name** and **last\_name***). To do that, replace the string value to ${...parameter name...}. The ${} wraps the values you set on your CSV file by pointing the parameter name to the column header name. Request Body (before) ```json { "email": "${__escape_quotes(email_in_request_body_1)}", "firstName": "John", "lastName": "Doe", "customId": "johndoe@loadmill.com", "environmentId": "da7167ae9563468a9bcb6eb157011745", "tenants": [ "e7f403f4c30d4bc3840456a7953bedb4" ] } ``` Request Body (After) ```json { "email": "${email}", "firstName": "${first_name}", "lastName": "${last_name}", "customId": "johndoe@loadmill.com", "environmentId": "da7167ae9563468a9bcb6eb157011745", "tenants": [ "e7f403f4c30d4bc3840456a7953bedb4" ] } ``` ### Upload CSV file. Now that you're done, the last step is to upload your CSV file by clicking on the CSV button. {% embed url="" %} ### Run the suite. By running the suite, we see the 5 iterations of the flow we created, each sending the server the values from the uploaded CSV. {% embed url="" %} We notice that the last two test iterations have passed when the expected outcome would have been to fail. That is a crucial part in testing your server as it allows you to understand how your API behaves in those scenarios. Overall, creating multiple variations of your API tests is important because it helps to ensure that the API is functioning correctly and that it meets all expectations. By doing so, you can avoid potential issues and ensure that your API is ready for production. # Flow Execution ### Running with Overrides When running a flow within the flow editor for debugging purposes, it can be very convenient to replace certain parameters for a single run. Instead of changing the parameter value in the global parameters window, you can simply click the 'run with overrides' button (![](/files/2vTqpUSzrw8k5slIAEEM)), and then set values for a specific execution. This won't affect the global parameters, providing a quick and simple way to debug your flows. i.e.: This is the global parameter: ![](/files/09CDVwsKSIZn0nO3ZVdA), and we want to check something specific and change the value of `email` for only one run. We'll use the 'run with overrides' button to execute it and reset its value. For example:

Good to know: the override parameters in this window are saved for subsequent 'runs with overrides' as well.

### Assigning Labels Flow Labeling is an efficient way to save time and enhance productivity during application testing. By executing only relevant flows, you can avoid redundant testing and concentrate on critical aspects. To utilize Flow Labeling, simply label your flows with specific keywords for identification. Then, assign these labels to relevant sections of your application's CI. This ensures that only relevant flows are executed when changes are made. Testing and validation are crucial in product and software development, but maintaining this process amidst diverse requirements can be challenging. Flow labeling is essential for developers to effectively test every aspect of their product. You may need to test multiple operations or ensure your application loads correctly. However, each update or bug fix undergoes a categorical test scenario that may not encompass all test cases from your Test Suite/Plan. Creating and assigning labels to flows is a straightforward process with Loadmill. Just select the flow and click on the label icon ![](/files/pXxY57tcjmOX37W3SxPY) to assign a label of your choice. You can either assign a pre-made label or create a new one yourself. The following Test Suite represents a user management application with 6 different flows. Each flow is labeled ***"operation"*** or ***"sanity"*** or both. {% embed url="" %} {% hint style="info" %} Make sure your flow is attached to the CI in order to run it based on labels. {% endhint %} ### Installing Loadmill NPM Package Before you start, install the [Loadmill NPM Package](https://www.npmjs.com/package/loadmill). ### Running Labeled Flows via CLI Now that we've labeled our flows correctly, we'll run them according to label we pass in to the command line. We want to run all flows within a test suite that are labeled ***"sanity"***. Let's quickly tour what values you need before running our tests.
parameterwhere to find
<test-suite-id>Navigate to your test suite page and the suite id is located inside the url
<token>The token is generated from the access token page
Open your CLI and run the following command: ``` loadmill --test-suite -w -t --labels --report "label1,label2" ``` {% embed url="" %} To conclude, flow labels allow us to focus our tests based on our needs regardless of the amount of test cases we have on our test suites or test plans. We could schedule daily test runs on ***"sanity"*** labeled flows and run "operation" flows for every time we push an update to our code. # Steps Steps are the core components of flows, made up of one or more steps of varying types. They generate, process, and share data. In the code editor, steps can be copied, created, edited, and deleted. #### Steps views There are two views in the flow editor. 1. Summary View: Displays a scrollable area containing all the steps, with each step collapsed to show summarized details.\\

Summary view - Collapsed steps example

2. Expanded view: Displays single steps at a time expanded with all the available functionality.\ note, some function will require additional expansion.

Expanded step example

{% hint style="info" %} Navigating between steps in Expended view can be done by keyboard up/down keys and by clicking on the step name on the left index list (when available on wide views). {% endhint %} **Switching between the views** can be dome by clicking on the title area of a step in non active areas as illustrated below
#### Adding a new step In Summary view, Click on the dotted menu (![](/files/OAb5T8H7O1jmvLk4K1bR)) located in the separation line between any two steps and in the beginning and the end of the flow, depends on where you'd like to introduce the new step.
Select the `+ option`Then click on the desired step type ![](/files/45puyv8cZRUJQTyFkJmP) A new empty step will be added. #### Copying step(s) * To copy a single step click on the copy button (![](/files/nQG9xvLFtqGK41jCEC3n)) located on the top right corner of every Step
To paste the step in the desired place, locate the divider between any two requests or at the beginning or the end of the flow and click on the dotted menu (![](/files/q3HezUu0X2kpXzuYKvSR)) then on the paste button ![](/files/StMeuVajk8aCs4fVXQ8a) * To copy multiple steps, use the checkboxes located left to the description of each step then click on the copy button located at the top.
to paste the copied steps, locate the divider between any two requests or at the beginning or the end of the flow and click on the dotted menu (![](/files/q3HezUu0X2kpXzuYKvSR)) then on the paste button ![](/files/StMeuVajk8aCs4fVXQ8a) same as individual flow. {% hint style="info" %} Tip1: To copy all the requests click on the checkbox at the top then use copy/paste as already demonstrated.\ ![](/files/bDoHgXGLbjEg6szhOP0c) {% endhint %} {% hint style="info" %} On empty flows, this top menu and the dividers will be missing, but you still paste steps by clicking the "paste" label at the top right corner of the screen![](/files/oUj6MIJ8AuJHPWui1811) {% endhint %} # Request step Request is the main building block of tests. It represents a single API call or a user action. ![](/files/-Ml4iL1e0ky9rMeHBILr) ## Request sections When we expand the request editor by clicking at the top-right of the request, we can see that a test request includes two main sections - **Request and Response Handling sections**. ![](/files/-Ml4lQnkGb7kR4glnph6) ![](/files/-Ml4kWzOqsz0DHlxPB9w) ### The Request Handling section ![The request handling section](/files/-Ml4mPc7Zo_SNpLVh1RM) * The **Method** and **URL** fields are the most basic requirements for a valid request. * Black input fields are fields in which you can use [parameters](https://docs.loadmill.com/api-testing/test-suite-editor/parameters) and [functions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions). * It is recommended to set the Request description in a way that describes its action. This will make it easier for you to debug your tests later if this request fails. * Some requests require a **body**. Selecting the right **content-type** for your request will help us highlight the syntax of your request body (i.e. JSON or XML). * The **Headers** section allows to see and set the request headers. You can add header to all requests or as a Global Header so that it will be added to all of the test flow requests and appear in [the Global Login tab](https://docs.loadmill.com/api-testing/test-suite-editor/global-login-flow). #### GraphQL requests A standard GraphQL POST request should use the `application/json` content type, and include a JSON-encoded body of the following form: ```json { "query": "...", "operationName": "...", "variables": { "var1": "value", } } ``` `operationName` and `variables` are optional fields. `operationName` is only required if multiple operations are present in the query. The response should return the queried data in a JSON. i.e., ```json { "data": { ... }, "errors": [ ... ] } ``` Based on ([graphql.org](https://graphql.org/learn/serving-over-http/)). ### The Response Handling section ![](/files/-Ml4nYb5LN9t8YgmOMgP) ![The response handling section](/files/-Ml4naxy0fC4hflfFBkN) The response handling section includes 3 sections: [**Extractions - Set Parameters**](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions), [**Postscript - run code**](https://docs.loadmill.com/api-testing/test-suite-editor/postscript-run-code) (disabled by default), [**Assertions - Verify Response**](https://docs.loadmill.com/api-testing/test-suite-editor/assertions) and Flow control. #### The **Extractions - Set Parameters** section allows you: * To extract values from the response body into an existing or a new parameter using JSONPath, Closure, jQuery, RegExp. * To extract an HTTP header value into a parameter using the Header extractor. * To assign a static value or [**apply a function on another parameter**](https://docs.loadmill.com/api-testing/test-suite-editor/functions) using the "Assign" option. * To add an extraction from [the Suggestions repository](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions#suggestions). Team admins configure the repository within Settings - Suggestions. ![](/files/-Ml4p1yW4r0domoc1bB8) The **Assertions - Verify Response** section allows you to validate the request response. ![](/files/-Ml4pH1lRzMW7AKhSt9L) The **Flow Control** section allows to repeat the request until the parameter's value meets the requirement or number of iteration reached: ![](/files/-Ml4r93brFyxkz-8L5cR) You can also set Skip and Stop conditions between requests within this section: ![](/files/-Ml4rTipI-4BcR58NI9i) # Code step The code steps allows running JavaScript code snippets at any point without spinning an entire request. Among it's usages are introducing initialization logic at first step of a flow, processing specific data right after a generic shared flow, and providing more visibility and clarity in flows and it's special logic steps.
The code step consists of two areas: 1. Code editor - a place to write your JavaScript snippets 2. Flow control - Allows skipping to another step, stopping the flow and add wait time after step execution. # Extraction & Assertion step The extraction & assertion step allows extracting, assigning parameters and add validations without spinning an entire request. Among it's usages are assigning or allocating values at the beginning of a flow of after calling a generic shared flow and we need to extract/re-assign/validate additional flow specific data.
The Extract & Assert step consists of three main areas: 1. Extraction section - a place to assign and extract the parameters.\ Note that the extractions will refer to the last response that was introduced, ie. the previous response.\\
Each extraction consists of three parts: 1. Parameter name - the name that the parameter will be referenced when used, this filed can accept letters, numbers and undescores (\`\_\`). 2. Assignment method/type - A selection of methods that can be used to acquire the value for the parameter. The available methods are: Assign, JSONPath, Clojure, xPath, RegExp, Header and WebSocket. For detailed explanation about each one of the methods please refer to the [Extractions documentation](/test-editor/set-parameters-extractions). 2. Assertions area - A place for data validations (Assertions)
The assertions section consists of three areas: 1. The parameter that carries the value for the validation. 2. The comparison method (the operator), also referred as the value, the available operators are\ Exists, Doesn't exists, Equals, Doesn't equals, Contains, Doesn't contains, Matches, Greater than, Less than, JSON Schema, JSON Contains, XML Contains.\ For detailed explanation about each one of the comparison methods please refer to the [Assertions documentation](/test-editor/assertions). 3. The expression - when applicable, this value will carry the compared value or expression for the validation. 3. Flow control area - Allows skipping to another step, stopping the flow and add wait time after step execution. # Web Socket step The WebSocket step enables real-time, bidirectional communication testing by allowing you to connect to WebSocket endpoints, send messages, and validate responses. This step is particularly useful for testing applications that require persistent connections and real-time data exchange. ## WebSocket Step Operations A WebSocket step can perform three main operations: 1. **Connect** - Establishes a WebSocket connection to the specified endpoint 2. **Send** - Transmits messages through an established WebSocket connection 3. **Receive** - Listens for and validates incoming WebSocket messages ## Step Configuration ### Connection ![](/files/meRaImRnhlwSh8DkPG3n) When establishing a connection: * The URL format follows: `wss://` * Headers can be added to the WebSocket handshake request for authentication or custom protocols * A timeout value (in seconds) can be set for the connection attempt ### Sending Messages ![](/files/m936MgTnxjDlf6EUGMDQ) For sending messages: * Supports two data types: * Text (JSON, plain text) * Binary data * Messages can include dynamic parameters * Headers can be included with the message * Used for both sending data and subscribing to channels * A timeout can be specified for the send operation Note: A Send step without a preceding Connect step will automatically create a new WebSocket connection. ### Receiving Messages ![](/files/AKW8CaCWh1d9nlJWWzYo) When receiving messages: * Messages are fetched based on the **WebSocket URL** * Use URL query parameters (e.g., `wss://example.com/ws?userId=1`) to send and filter messages by specific properties * **Message retrieval options**: * Last - Returns the most recent single message * Last 10 - Returns an array of the last 10 messages * Last 50 - Returns an array of the last 50 messages * **Extractions** support all standard Loadmill extraction methods * Example JSONPath for array filtering: `$[?(@.id == "${my_id}")]` * [Learn more about extractions](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) * **Wait for extractions to be resolved** - Since WebSocket connections are asynchronous, this setting defines how long to wait for messages that match your extraction criteria. Default is 5 seconds, but you may need to adjust this based on your application's behavior. * **Assertions** - Use the same powerful assertion capabilities as HTTP requests to validate your WebSocket messages, including JSON/XML validation, value comparisons, and regex matching. [Learn more about assertions](https://docs.loadmill.com/api-testing/test-suite-editor/assertions) * The **Flow Control** section allows to repeat the request until the parameter's value meets the requirement or number of iteration reached. ## Example Run Let's look at a simple WebSocket test flow in action: ![](/files/pOLXeMJ7eegswZ6qGY9w) This example demonstrates a complete WebSocket interaction: #### 1. Connect to WebSocket: * Establishes connection to a WebSocket server #### 2. Send Some Data: * Sends a message on the connection #### 3. Assert message received: * Waits for all extractions to be resolved * Once resolved, validates that the extracted `${message}` exists * Received messages are displayed for easy debugging # Playwright step The **Playwright step** allows you to run UI automation scripts using [Playwright](https://playwright.dev/) as part of your Loadmill flow. This is especially useful when validating UI behavior alongside your API steps — for example, logging in through the UI, verifying elements on the page, or simulating full end-user workflows. ![Playwright step editor](/files/hPsnaKZMCKzK7i8GIpxv) ## Usage The Playwright step contains a single **code editor** where you write your Playwright script. > 🧠 Only write the contents of a `test()` block — do **not** include the test declaration itself. ✅ **Correct:** ```ts await page.goto('https://www.loadmill.com'); await expect(page).toHaveTitle(/Loadmill/); await page.getByRole('textbox', { name: 'Email' }).fill(email); ``` ❌ **Incorrect:** ```ts test('Login test', async ({ page }) => { await page.goto('https://www.loadmill.com'); await expect(page).toHaveTitle(/Loadmill/); await page.getByRole('textbox', { name: 'Email' }).fill(email); }); ``` ## Parameters and context In the Playwright step, **all suite and flow parameters** are available directly as variables — no need to use `${param}` syntax like in other inputs. ✅ **Correct:** ```ts await page.getByRole('textbox', { name: 'Email' }).fill(email); ``` ❌ **Incorrect:** ```ts await page.getByRole('textbox', { name: 'Email' }).fill(${email}); ``` ## Cookies Session cookies are passed automatically into the Playwright browser context, so you can stay authenticated between API and UI steps. To manually set cookies, you can use Playwright’s built-in [`context.addCookies`](https://playwright.dev/docs/api/class-browsercontext#browser-context-add-cookies) method: ```ts await context.addCookies([ { name: 'my_cookie', value: '12345', domain: 'your-app.com', path: '/', httpOnly: true, secure: true, sameSite: 'Lax', }, ]); ``` ## Attaching screenshots to the report You can attach screenshots directly to your test report using Playwright’s `testInfo.attach` method. ✅ Example: ```ts await page.goto('https://loadmill.com'); const screenshot = await page.screenshot(); testInfo.attach('Loadmill Page', { body: screenshot, contentType: 'image/png', }); ``` ## Changing timeout You can change the default timeout for Playwright actions using the [`setDefaultTimeout`](https://playwright.dev/docs/api/class-page#page-set-default-timeout) method: ```ts page.setDefaultTimeout(10000); // 10 seconds ``` ## Changing view resolution To set the browser viewport size, use the [`setViewportSize`](https://playwright.dev/docs/api/class-page#page-set-viewport-size) method: ```ts await page.setViewportSize({ width: 1280, height: 720 }); ``` ## Requirements To use Playwright steps, a [private agent](https://docs.loadmill.com/integrations/testing-localhost-application) with UI tests enabled is required. You can enable UI testing using one of the following options: * **Desktop App** - UI testing is supported out of the box with no additional configuration required. * **Docker** - Set the following environment variable when running the container: ```bash docker run -it --rm -e LOADMILL_AGENT_TOKEN= -e UI_TESTS_ENABLED=true loadmill/agent ``` * **NPM Package** - Run the agent with the `--ui-tests` flag: ```bash loadmill-agent start -t INSERT_TOKEN_HERE --ui-tests ``` > ⚠️ This option requires additional dependencies and lacks the isolation benefits of Docker or the Desktop app. # UI Agent step The **UI Agent step** allows you to automate UI interactions using natural language prompts. Instead of writing code, you describe what you want the agent to do, and it autonomously navigates your application, performs actions, and validates results. This step is especially useful for: * **Hybrid testing** — If most of your test logic is in API steps but you need to validate something in the UI, you can add a UI Agent step to open the app and verify a specific element or state without building a full end-to-end UI test. * **Hard-to-automate actions** — When certain actions are difficult to replicate via API (e.g., complex UI interactions, OAuth flows, or third-party widgets), you can use the UI Agent step to perform those actions and then continue with your API-based test approach. * **End-to-end UI testing** — Quickly create full UI tests by describing user flows in natural language. ![UI Agent step editor](/files/fu9vyEw4uWnpi8QYc6Ak) ## Step structure The UI Agent step consists of three main parts: ### 1. Navigate to URL The starting point for the agent. This is the URL the browser navigates to when the step begins. **Example:** ``` https://example.com/login ``` ### 2. Agent Instructions (optional) Instructions describing the actions the agent should perform before reaching the UI state you want to validate. This field is optional — if you only need to validate something on the page without performing any actions, you can leave it empty. Each action should be written on a separate line. Use this to describe interactions such as: * Clicking buttons or links * Filling out forms * Navigating through menus * Scrolling or hovering **Example:** ``` Enter "user@example.com" in the email field. Enter "password123" in the password field. Click the Login button. ``` ### 3. AI-Driven UI Validation Describe what you want to verify on the UI state after the actions are complete. This is where you specify the expected outcome of the test. Common validations include: * Verifying an element is visible * Checking that a button is clickable * Confirming an item was deleted or removed * Validating that specific text or content is displayed * Ensuring error messages appear (or don't appear) **Example:** ``` Verify that the dashboard page is displayed and the welcome message shows "Hello, User". ``` ## Advanced settings ### Timeout Configure the maximum time (in seconds) the agent is allowed to complete the step. If the step takes longer than the specified timeout, it will fail. Set a reasonable timeout based on the complexity of your flow and the expected response time of your application. ## Parameters You can use parameters in any of the UI Agent step fields. Parameters work the same way as in other parts of Loadmill — use the `${parameterName}` syntax, and the value will be resolved at runtime. **Example:** ``` Enter "${userEmail}" in the email field. Enter "${userPassword}" in the password field. ``` This allows you to: * Reuse the same test with different data sets * Reference values extracted from previous steps * Use suite-level or flow-level parameters ## Viewing results After running a test with UI Agent steps, you can view: * **Validated screenshot** — A screenshot of the UI state that was validated * **Step-by-step actions** the agent performed * **Validation results** showing what passed or failed If the test failed, a **session recording** is also provided. This allows you to watch exactly what the agent did during execution and identify where it went wrong, making it easy to debug and tweak your prompts to improve stability. ![UI Agent step run result](/files/RKQKWzm72dRkrPqcL04p) ## Best practices * **Be specific in your instructions** — The more precise your prompts, the more reliable the test. Instead of "click the button", say "click the Submit button" or "click the blue button labeled Continue". * **One action per line** — Write each instruction on a separate line for clarity and consistency. * **Keep validations focused** — Use a single, clear validation statement. Avoid combining multiple unrelated checks in one validation. * **Start simple** — Begin with short, focused flows and expand as you gain confidence in the agent's behavior. * **Use parameters for variable data** — Avoid hardcoding values that may change; use parameters instead to make tests reusable. ## Requirements To use UI Agent steps, you need: * A Loadmill account with AI Agent capabilities enabled * The [Loadmill desktop app](https://app.loadmill.com/app/user/settings/desktop-app) installed and running {% hint style="info" %} UI Agent steps require the desktop app because the agent needs to control a real browser to interact with your application's UI. {% endhint %} ## Related documentation * [Agent Testing Quick Guide](/quick-guide/agent-testing-quick-guide) - Get started with agent testing * [Playwright step](/test-editor/steps/playwright-step) - For code-based UI automation # Extractions - Set Parameters Use the **Extractions - Set Parameters** section of the request to extract values from the responses. These extracted values can be use in any of the following requests or to assert and verify your success criteria. ![The request Extractions - Set Parameters section](/files/-Ml9SIw6apzelQsYPuvq) ## Extraction Types ### Assign Use this to assign any combination of existing parameter or response values to another parameter ![](/files/-Ml4wxHOLpnEkj6LomQZ) ### JSONPath Use this extractor to extract values from JSON responses. JSONPath is a query language for JSON objects, much like XPath for XMLs. ![](/files/-Ml4xTxazsrtLhgsH-Pj) You can easily find a relevant JSON Path expression by using our JSON Path Finder feature. To use it, just run your test flow, go to the Response body section, click on JSONPATH and copy the JSON Path expression you need. ![](/files/-MZHL8kEPdzyenDocHew) ![Copying a JSON Path expression from the finder](/files/-MZHLLZDiwG--47lse6Q) Also find helpful JSON Path expressions [here](https://goessner.net/articles/JsonPath/index.html#e2). ### Clojure Use this extractor to extract values from Clojure (EDN content type) responses. You need to enter JSONPath Query, Clojure queried as JSON. ![](/files/-Ml4xqLtYAdC5jlqkRlN) ### jQuery Use this extractor to extract values from HTML or XML responses. jQuery extraction syntax is very similar to XPath and may be very useful to extract values from HTML pages (such as [security tokens](https://portswigger.net/web-security/csrf/tokens)). ![](/files/-Ml4y8IEMq0TepNcmdjG) ### RegExp Use this extractor to extract values from any type of response using RegExp. The first value captured by the regex (in parentheses) will be set as the parameter's value. You can test your regex expressions using this useful online tool - [link](https://regex101.com). ![](/files/-Ml4yOojCAMr1gVPkRMa) ### Header Use this extractor to extract values from any of the response headers. ![](/files/-Ml4ygdP5i_ge2e3i0Z0) ## Suggestions In many cases Loadmill users use the same or similar extractions and [assertions](https://docs.loadmill.com/api-testing/test-suite-editor/assertions). We've implemented the Suggestions feature that allows team admins to configure a repository of extractions and assertions within **Settings - Suggestions**. Then, each user can add the extractions from the repository by clicking **+ SUGGESTIONS**. ![](/files/-Ml4ysn3rQNF_J71IsXR) By default, there are a few common extraction and assertion examples in the repository. Team admins can also navigate to the Suggestions Settings directly from within the suggestion dialog window. ![](/files/-M_G7232wBDcWdKiNqM3) ## Autocomplete {% hint style="info" %} :man\_mage: Use the autocomplete option to see a list of existing parameters. To see it, press Shift + Cmd (Ctrl for Windows) + Space. See how it works below. {% endhint %} ![](/files/1vy5dZarpZ7rgqGoKVlR) # Assertions - Verify Response When testing an API, it is usually not enough to verify that all our HTTP requests completed successfully. Often, it is necessary to make sure that we got the correct response from the server by examining the **response status, body or headers**. With [Loadmill](https://www.loadmill.com), this is made easy by using **Assertions**. Assertions are used in conjunction with [parameters](/test-editor/parameters) to do just that: examine the server's response and assert its correctness. You may have an arbitrary number of assertions executed after each successful request. If an assertion fails, the next request will **not be executed** and the test scenario will be marked as **failed**, but the subsequent assertions for the current request **will be executed** nonetheless. ## Assertions editor Assertions editor can be found in loadmill under each request cubical on it's expanded form.\\

Assertion section in request editor

Furthermore, there is an option to disable an assertion by clicking the three dots (![](/files/aHyW1ye2vxBOGm5ffYPd)) in the assertion line. A disabled assertion will still be evaluated, but it won't cause the test to fail.
## Assertions Creation from Response It is also possible to create assertions directly from the response panel from the 'JSONPATH' window (only when the response is a JSON). After executing the test, click on one of the responses to expand it, which will open the response panel, as follows:
Click on ![](/files/TicC5ocbItUyBaxmBben), then navigate to the desired parameter. For example, let's find the 'content-length':
Now, simply click on the plus sign located in the top right corner, set the extraction, and add the desired assertion right from there!
You can then see both the extraction and the assertion in the request editor panel:
## Assertion Types The target of an assertion is always a parameter value. You may use built-in parameters, default parameters or any parameter extracted from the current or previous requests in the **current scenario** as the target. ![The request Assertions - Verify Response section](/files/AHGrkGD6YwCjp89VsVTC) When using assertions, it will usually require to use [extractions](/test-editor/set-parameters-extractions) first to set the parameter with proper data and only then to attempt to validate it's form. There are several types of assertions: #### **Exists** - Validates that a parameter exists and not null. * Example 1:\ Assigning empty term to `param1` will result in assertion failure\ ![Click to enlarge the image](/files/AytStooaHpofsBv1vRDz)![](/files/wYEOq5U1tMZl25O0tgUq) * Example 2:\ Assigning any value to `param1` will result in passed assertion\ ![Click to enlarge the image](/files/mR8N4TSZAZpqGRaeDa2F)![](/files/lHTdef4Vu1Hr3hqqfh6y) #### **Doesn't exist** - Validates that a parameter doesn't exists or null. * Example 1:\ Assigning any value to `param1` will result in assertion failure\ ![Click to enlarge the image](/files/y4JEJpe77IMRnNRHTisi)![](/files/VUszTrxUNkFNb3Qkj6fV) * Example 2:\ Not setting `param1` entirely will result in passed assetion\ ![Click to enlarge the image](/files/NWv3A3PxdYSgIg4PDZ8u)![](/files/VYgSeJeRIOSXjKN0zs3Y) #### **Equals** - Validates that a parameter is equal to the given expression. The equality check is **case sensitive.** * Example 1:\ The assigned `param1` doesn't equals to loadmill hence will fail in assertion\ ![Click to enlarge the image](/files/VXXerdgTmy4jbzsvD0Q7)![](/files/Od2RTLZfZr9uPOVrVMf7) * Example 2:\ Extracting `$.args.app` json path field to `param1`(from the result that is not displayed) should be equal to "Demo" and will result the assertion to pass\ Assigning numeric value to `param2` of 11.0 in postscript should be equal to "11", and the assertion will pass.\ ![Click to enlarge the image](/files/eSamPaQZOjzFmACiIema)![](/files/mLCFop4OSQNvIzkJXr6t)\ \&#xNAN;***Note** that all values are implicitly compared as strings.* #### **Doesn't equal** - Validates that a parameter is NOT equal to the given expression. The equality check is **case sensitive**. * Example 1:\ `param1` shouldn't be equals to 'demo' but it is, hence the assertion will fail\ ![Click to enlarge the image](/files/gQjaj2p6DSAddWszqx0y)![](/files/MXZo4ca7TRYHkcFecZhe) * Example 2:\ `param1` doesn't equals to 'loadmill' hence the assertion will pass\ ![Click to enlarge the image](/files/xJm99VgVFBlUR08vW3LV)![](/files/eFB66jd3pBqa4h9nq5pN) #### **Contains** - Validates that a parameter contains a sub-string. The containment check is **case sensitive**. * Example 1:\ When `param1` set with the value 'loadmill-demo' and doesn't contains the string 'example' the assertion will fail.\ ![Click to enlarge the image](/files/C1UKMWTkBo9pe8TFVSBM)![](/files/vJzqSp8xVSsU1NOWvdO3) * Example 2:\ When `param1` set with the value 'loadmill-demo' and contains the string 'demo', the assertion will pass.\ ![Click to enlarge the image](/files/bRldyTBf7vrJugDqv6G7)![](/files/k6bDSDgB4vbRJpsS05z3) #### **Doesn't contain** - Validates that a parameter DOESN'T contain the given expression. The containment check is **case sensitive**. * Example 1:\ When `param1` set with the value 'loadmill-demo' and contains the string 'demo', the assertion will fail.\ ![Click to enlarge the image](/files/IWfl6e2CEDhlROTjbn8b)![](/files/vpLRcE70Jhn6pDeRN7zQ) * Example 2:\ When `param1` set with the value 'loadmill-demo' and doesn't contain the string 'example', the assertion will pass.\ ![Click to enlarge the image](/files/TCWWpeHsuGIz3128J8CJ)![](/files/QaQyDlJkKprkt3GSTWEw) #### **Matches** - Validates that a parameter matches the given regular expression. * Example 1:\ The assigned epoch representation of the current time to `param1` will contain 13 digits would not match the regex (`^\d{10}$`), hence the assertion will fail\ ![Click to enlarge the image](/files/04ZAf9VIaMni1RiTvQKk)![](/files/GfjGr5BZyPcSagiEHkbI) * Example 2:\ The assigned epoch representation of the current time to `param1` will contain 13 digits and would match the regex (`^\d{13}$`), hence the assertion will pass\ ![Click to enlarge the image](/files/YH6l7XF6Anj5ZWir6pza)![](/files/K4A1dORgUyycbjNhwHb1) #### **Greater than** - Validates that a parameter is greater than the given expression. * Example 1:\ `param1` is assigned with the value 31 that is not greater than 40 hence the assertion will fail.\ ![Click to enlarge the image](/files/fN2IRr0hn1ay4D1v8VjU)![](/files/EPf0fd9miUcSahkw1pea) * Example 2:\ `param1` is assigned with the value 31 which is greater than 30 will make the assertion to pass\ ![Click to enlarge the image](/files/zuma6YdYSQaLJFRvyh5m)![](/files/RPEMmLXOahPQPhoRnCMx) #### **Less than** - Validates that a parameter is less than the given expression. * Example 1:\ `param1` is assigned with the value 30 that is not less than 31 hence the assertion will fail.\ ![Click to enlarge the image](/files/gytaFPlctV14xBOqlSc2)![](/files/O2CdLyfFIv74MtQNR3Bw) * Example 2:\ `param1` is assigned with the value 31 which is less than 40 will make the assertion to pass\ ![Click to enlarge the image](/files/i0zphUifu15KJzOhPn9w)![](/files/edFNEzz2zYmxAPP3HIp9) #### **JSON Schema** - Validate that a parameter comply with given [JSON Schema](https://json-schema.org/). * Example 1:\ Given `param1` was set with a json as below, opposed to the expected schema the field "name" is not numeric hence the assertion will fail.\ ![Click to enlarge the image](/files/OtSxrxdbvPorpJi65QoM)![](/files/KkzF3VANzORSXyvgCFOw)\ Note, the assertion will suggests to exclude the failing field from the validation schema. * Example 2:\ Given `param1` was set with a json as below and the expected schema describes it correctly, the assertion will pass.\ ![Click to enlarge the image](/files/uDoa481cKTgOJ2YTHwwp)![](/files/kIC3iWMentq6ntFR6CL8) {% hint style="info" %} To prepare initial JSON schema out of a JSON you can search for 'from json to schema' in google or use a [tool like this](https://transform.tools/json-to-json-schema). {% endhint %} #### **JSON Contains -** Validates that a json contains a subset json, in such way dynamic fields can be omitted to avoid false negatives. Accepting any value for individual field will accept "\*" notation\ Note, the subset json object should preserve the same hierarchal structure as the containing object while fields can be omitted. * Example 1:\ Given `param1` was set with a json as below, any mismatch\ in a key or value on same hierarchy will result in assertion error:\ ![Click to enlarge the image](/files/1IonoVYmgE0zPN9Dk1LK)![](/files/um46p7eZl1lzxYOFaNjZ)\ Note to the "Exclude ..." notations, clicking on them will adjust the value of the assertion to exclude the failing field. * Example 2:\ Given `param1` was set with a json as below, any partial subset in json-contains validation value with same structure will result pass in assertion:\ ![Click to enlarge the image](/files/wILT9qro8LGJjXKLJYHd)![](/files/nH3rBs7Dq2RyJM4hPGM2)\ \- Using star notation to accept any value for `latitude` and `country` fields.\ ![Click to enlarge the image](/files/IBWkKpjtJOWo05isJdP1)\ Note, the "\*" notation works only on simple values (strings/numerics/booleans) #### **XML Contains -** Validates that an xml contains a subset xml, in such way dynamic fields can be omitted to avoid false negatives. Accepting any value for individual field will accept "\*" notation\ Note, the subset xml object should preserve the same hierarchal structure as the containing object while fields can be omitted. * Example 1:\ Given `param1` was set with an xml as below, any mismatch\ in a key or value on same hierarchy will result in assertion error: ![](/files/yZAfXDcuaKmkUttn6a28) \\ * Example 2:\ Given `param1` was set with an xml as below, any partial subset in xml-contains validation value with same structure will result pass in assertion:\ ![](/files/xmGsM8u871zQTIn0UeoP)![](/files/QKbY6RWWIeyHVLiuMBnD)\ \ \- Using star notation to accept any value for `latitude` and `country` fields.\ ![](/files/c5NMw4ybXSfSzlXKyUsE)\ Note, the "\*" notation works only on simple values (strings/numerics/booleans) You may embed parameters in any assertion expression. These parameters will be evaluated right before the assertion is executed. See **an example of the assertion** that validates the generated ID is in [the format of UUID](https://docs.loadmill.com/api-testing/test-suite-editor/functions#__is_uuid-target) below: ![](/files/R7ncXB2e3h333VvLC9mR) {% hint style="info" %} :man\_mage: When creating tests via our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder), Loadmill will create **two automatic assertion types** for you: 1. Default assertions - for any extraction we find in the recorded test, we add a default assertion to it (example: "id" exists). 2. Specific assertions - for any parameter with given user key (for example, "success") we extract it and assign to it an assertion (example: "success" equals "true"). Optional automatic JSON Schema assertion: you can enable [this assertion](https://docs.loadmill.com/working-with-the-recorder/recorder-settings#strict-response-validation) within Settings - Recordings. {% endhint %} ## Suggestions In many cases Loadmill users use the same or similar extractions and assertions. We've implemented the Suggestions feature that allows team admins to configure a repository of [extractions](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) and assertions within **Settings - Suggestions**. Then, each user can add the assertions from the repository by clicking on **+ SUGGESTIONS**. ![](/files/-Ml5-FKwVzDSE-IyLRTR) By default, there are a few common extraction and assertion examples in the repository. Team admins can also navigate to the Suggestions Settings directly from within the suggestion dialog window. ![](/files/-M_G8F1z_TeFwD_IfSS2) ## Caveats Keep in mind that all parameter values are **textual**, i.e. a parameter has **no type** such as `Number` or `Array` that we know from common programming languages. This is important in order to avoid confusion when using parameter extractors such as **JSONPath**. For example, consider the following scenario: 1. Extract the value for `books` via the JSONPath query `student.books` on ```javascript { "student": { "books": [] } } ``` 2. Assert `books` is **Not Empty**. You may expect this assertion to fail but, in fact, it will succeed. This is because the parameter `books` is evaluated to `[]` and therefore is considered as a non-empty string. One possible way to correct this is to use a **RegExp** assertion on the `books` parameter instead: `\[[^\s]+\]`. # Parameters Working with [Loadmill](https://www.loadmill.com) makes it very easy to convert recorded browser/network sessions (via HAR files) into test scenarios. But once a scenario has been created, it often needs to be **parameterized** in order to denote the dynamic parts of your API. This is best explained by example, so let's have a look at the following test scenario: 1. User adds a blog post: `curl https://www.myblog.com/posts --data "Hello World!"` 2. User likes his own post: `curl -X PUT https://www.myblog.com/posts/123/like` The `123` part is the identifier of the blog post created by the user - it cannot be known in advance because it is generated by the server at runtime. This is a very common use case for **parameters** and is handled by two simple steps: 1. [Extract](#parameter-extraction) the ID from the first response into a parameter, e.g. `postId` 2. Embed the parameter in the second request URL: `https://www.myblog.com/posts/${postId}/like` You may embed parameters with the `${}` syntax in the request URL as well as the request body, request headers, [extractions](#parameter-extraction), [assertions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/assertions) and more. Note that any parameterized expression such as `posts/${postId}` will remain as-is if no such parameter is defined or extracted before the expression is evaluated. ## Default Parameters Another common use case for parametrization is when you want to **reuse test scenarios** on different environments or with small adjustments. Let's extend our first example by requiring that the user provide credentials via **basic authentication**. This means our new URLs could look like this: 1. `https://testUser:testPassword@www.myblog.com/posts` 2. `https://testUser:testPassword@www.myblog.com/posts/${postId}/like` If you would like to use different credentials for every test run, you may replace the username and password with parameters and set their values in the [Test Suite Parameters](https://docs.loadmill.com/api-testing/test-suite-editor/test-suite-parameters) tab whenever you reuse the test. So now the URLs will look like: 1. `https://${user}:${pass}@www.myblog.com/posts` 2. `https://${user}:${pass}@www.myblog.com/posts/${postId}/like` Using parameter defaults is especially useful for automated testing and CI where you may be testing a different server every time you run a test. Please refer to the [Loadmill CLI and npm module](https://www.npmjs.com/package/loadmill#parameters) for more information about how to inject parameters dynamically in such scenarios. ## Parameter Extraction Parameters can be defined and populated with values dynamically after each request in your test scenario. ![](/files/-Ml4uy9yMuihq1xt7j9C) There are several **extraction query types** that may be used: * **JSONPath** - used for extracting values from a JSON response. For example, the query `$.post.id` will extract the value `123` from this JSON response: ```javascript { "post": { "id": 123, "text": "Hello World!" } } ``` ![](/files/-Ml4vF6b33EbcWRkRTRE) * **JQuery (Cheerio)** - used for extracting values from XML/HTML responses. We use a subset of the JQuery selector syntax called [Cheerio](https://cheerio.js.org). You may add an optional (but very useful) **attribute** input to your query that selects an attribute value from the first element found by the jQuery. If you do not provide an attribute to select, the query will simply output the inner content of said element. * **JS RegExp** - used for extracting arbitrary values from any kind of textual response via regular expressions with capture groups. For example, we can extract the `id` field from the same JSON response we've seen above using a regular expression: `.*"id":\s*([0-9]*)`. * **Header** - used for extracting response header values via header names. * **Assignment** - used for assigning an explicit value to a parameter or [a function](https://docs.loadmill.com/api-testing/test-suite-editor/functions) to an existing parameter. Previously defined or built-in parameters may be embedded within the string, e.g. `https://${host}/path/to/glory` or `The time is ${__now}`. * **Clojure** - used for extracting values from Clojure (EDN content type) responses. Querying the data is done using JSONPath. For example, the query `$[":user"][":id"]` will extract the value `56` from this EDN response: ``` {:user {:role :viewer, :name "Rivi", :teams nil, :id "56"}} ``` Previously defined or built-in parameters may be embedded within **any kind of extraction query**. These parameters will be evaluated right before the query itself is evaluated. ## Built-in Parameters There are several **built-in** parameters that you can use in your test scenario. They are: * `__status` The status code of the last HTTP response. * `__statusText` The status text of the last HTTP response. * `__responseTime` The total response time (in milliseconds) of the last HTTP response. * `__testRunId` - The test run id: (Suite / Flow / Load) * `__testStartTime` - The test run start time (UTC in milliseconds) * `__launchedBy` - The name of the user running the test. **Note:** some previous built-in parameters are now defined as no-argument [parameter functions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions) and can still be used in the same way. ## Advanced Usage So far, we've only seen how to inject a simple parameter value into an arbitrary expression, e.g. `Hello ${name}`. However, it is also possible to inject a ***computed value*** using [Parameter Operators](#operators) or [Parameter Functions](https://docs.loadmill.com/api-testing/test-suite-editor/functions), e.g. `The total price is ${price + __mult(price,tax)}`. You may use operators or functions anywhere parameters may be used. Computed values can be extremely useful when you need to introduce conditional behavior to your test. Say you want to skip **Purchase Request** if the preceding **Get Price Request** response returns a price above the current budget. This could be accomplished by extracting the price to a parameter and setting the **Purchase Request** skip condition to `${budget <= price}`. ![](/files/-MlFgzMp-Gg8mHMv6fD3) You may also use ***literal values*** within expressions, e.g. `${__if_then_else(is_good,'Success!',':_(')}` - but be aware there are some [syntactic limitations](#notes-and-limitations). Function calls without arguments can be used with or without parentheses, e.g. `__random_uuid` can be used instead of `__random_uuid()`. See below the full list of supported [operators](#operators) and [functions](https://docs.loadmill.com/api-testing/test-suite-editor/functions). ### Notes And Limitations Current syntax has some limitations. Syntax errors are easy to spot in the GUI - a malformed expression will simply not be highlighted. If an expression is invalid due to its syntax it will simply remain as-is and will not be computed nor replaced at run-time. If, however, an operator or a function receives invalid input (e.g. division by zero or any of the parameter arguments not having a value) the test will fail with an error. Note that predefined parameter values (AKA [Parameter Defaults](#default-parameters)) are computed whenever a test configuration is validated - thus computation related errors will render the test configuration invalid. Current syntax limitations are: * Operators ***must*** be separated from their arguments by spaces, e.g. `${x + y}` is fine but `${x+y}` will not be computed. * Spaces are ***not allowed*** anywhere else within an expression, e.g. both `${__add(x, y)}` and `${fullName == 'John Doe'}` will not be computed. * **Literal values** may not contain whitespace characters, commas (`,`) or single quotes (`'`). These cannot be escaped - simply define a previous parameter with the desired value when the need arises. Note that numeric literal values need to be quoted the same as any other value, e.g. `${x > '0'}` is OK but `${x > 0}` will not be computed. * You may chain multiple operations together, e.g. `${x * y + z}` but you may ***not*** use parentheses to set precedence, e.g. `${(x * y) + z}` will not be computed. This can usually be worked around using functions though, e.g. `${__mult(x,y) + z}` * All operators have the ***same precedence*** - computations always conform to right-associativity, i.e. `${x * y + z - j + k}` will be computed as `x * (y + (z - (j + k)))`. * Computations may ***not be nested***, i.e. you may not pass a computed value as an argument to function, e.g. `${__mult(x,y) + z}` is OK but neither `${__mult(x + y,z)}` nor `${__mult(__add(x,y),z)}` will be computed. * Unary operators, e.g. `${-x}` are ***not*** supported. This can be overcome using functions such as [\_\_neg](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions#__neg-p-1) or [\_\_not](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions#__not-p-1). ### Operators The currently supported operators are: #### Textual Operators * `=` Strict equals operator. Aliases: `==` and `===`. * `!=` Strict not-equals operator. Alias: `!==`. May be applied to any two parameters which have values. #### Boolean Operators * `|` Logical OR operator. Alias: `||`. * `&` Logical AND operator. Alias: `&&`. May be applied to any two parameters which have values. #### **True Semantics** A parameter translates to boolean `true` if and only if * It has a value ***and*** * The value is not an empty string ***and*** * The value is not equal to `false`, `FALSE`, `FaLsE` or any other combination of upper-case and lower-case letters that forms the word `false`. The computed value of a valid boolean operation is either exactly `true` or exactly `false`. #### Numeric Operators * `+` Addition operator. * `-` Subtraction operator. * `*` Multiplication operator. * `/` Division operator. * `<` Less-then operator. * `<=` Less-then-or-equals operator. * `>` Greater-then operator. * `>=` Greater-then-or-equals operator. May be applied to any two parameters which have values that translate to ***finite numbers***. Computed values are ***not*** rounded to integers. ### # Parameter Execution Order When using parameters in Loadmill tests, it is important to understand their hierarchy. To figure it out, let's look at a basic example below: The user has a test flow with a GET request to his application home page. ![](/files/-Ml4vmiAEkyqOfJ6k6zb) The "base\_url" parameter is used to store URLs and choose them depending on which environment the user would like to run tests against. This parameter can be defined by several ways, let's review them: ![](/files/-MZIBxk_UWTCbBGu90Ho) 1. **Test Suite Parameters tab** - this is a place where the user defines a default value for the parameters he is going to use in the Test Suite. In our case - "base\_url". ![](/files/NYDf5U9Kmk3ztHcs24TW) 2\. **Test Plan parameter value** - once the user has a few Test Suites, he groups them into a higher level called [Test Plan](https://docs.loadmill.com/api-testing/test-plan) per a specific task or feature they belong to. Then, the user can configure the "base\_url" parameter value that will be sent to all Test Suites during the Test Plan run. This value will override the value from the Test Suite Parameters tab (if there is any). ![](/files/UpZyMSdkVgfdwyqwOdVE) 3\. **npm module parameter value** - when running tests in CI/CD, the user initiates their execution by using [our npm module](https://www.npmjs.com/package/loadmill) and he can add a relevant "base\_url" parameter value to the command and **this value will** be taken for the run and **override the parameter value set in #2 and #1 above** (if there is any)**.** ``` loadmill --test-plan -w -v -t --report base_url=your_current_url ``` 4\. **CSV file parameter value** - in some cases the user may need to run the same request multiple times using different parameter values, so he can use a [CSV file for that](https://docs.loadmill.com/api-testing/test-suite-editor/api-tests-data-from-csv-files). ![](/files/-MZILUTBTbuUsQthSWio) **The value from the CSV file will** be taken during the test run and **override the parameter value from sections above** (if there is any). 5\. **Extraction parameter value** - when the user extracts value into a parameter within the request's [Extractions - Set parameters](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) section, **this value will override the parameter value from all sections above during the test run**. # Test Suite Parameters The Test Suite Parameters tab allows to configure default parameters which are relevant to the execution context. e.g. target host, login credentials, and user input. Default parameters can be used in requests using the template strings notation: `${parameter_name}`. ![](/files/X8Iyz5ZAMbfftmMXBkLl) ![](/files/aWgjfGzgszdm4ZSeQXhf) ### Parameters scope - *DEPRECATED* ⚠️ Note: This feature is deprecated. Use [Suite Execution Mode](/test-editor/parameters/suite-execution-mode) instead. By default, Test Suite parameters have the **Entire Test Suite Run** execution scope. The alternative option is to select the **Single Flow Run** scope. ![](/files/XWQPRbAL9Test7MYQyeF) Look at the example below to better understand what execution scopes mean. Let's say, I have a test flow where I test creating and editing a blog post. I've extracted ID of the post that has been created and now I want to use it in the following requests of the same flow. This is a classic example of a self-contained test and It will work regardless of the execution scope selected. But in case I would like to use the post ID parameter in another flow of the Test Suite I should use the "Entire Test Suite Run" scope, otherwise, Loadmill won't recognize that parameter. ### Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # Suite Execution Mode Loadmill lets you control how flows inside a test suite interact with each other and how failures are handled. The *Execution Mode* determines whether flows share state, how retries behave, and whether the suite stops on the first failed flow. *** ## **Where to configure Execution Mode** You can configure Execution Mode from the suite’s **Run Settings**: 1. Open your test suite. 2. Navigate to **Run Settings** in the top navigation bar. 3. Scroll to the **Execution Mode** section. 4. Select either **Dependent** or **Independent**. ## ![](/files/j2PbbmOGhibpAPUUKSe3) ## **Modes** ### **Dependent Mode** Flows run sequentially and share execution state. **Characteristics:** * **Shared cookies & parameters** — any extracted values from one flow are available in the next. * **Stops on first failure** — if one flow fails, the entire suite ends immediately. * **Retry reruns the entire suite** — to rebuild all shared context correctly. **Best for:** * Multi-step flows where later tests rely on earlier outputs. * Ensuring consistent global state. *** ### **Independent Mode** Each flow runs in complete isolation (except `Before All`). **Characteristics:** * **No shared state** — every flow receives a clean environment. * **Suite continues even if a flow fails** — all flows execute regardless of failures. * **Retry reruns only failed flows** — much faster iteration when debugging. **Best for:** * Collections of unrelated flows. * Wide coverage testing where independence improves stability. * Re-tring specific failures without rerunning everything. *** ## **How Execution Mode Works** * **Before All** always runs before the suite starts. * In **Dependent mode**, all flows share one continuous execution context (cookies and parameters). * In **Independent mode**, each flow gets its own isolated environment (cookies, parameters, and context are not reused). * Retries follow the selected mode to ensure deterministic behavior. *** # Parameters Sets Inside the Parameters page, there is an option to insert a parameters set. This adds each parameter defined in the desired set to the current parameters list. ### Creating a Parameters Set The page is located under TEST DESIGN -> Parameters Sets.
Creation of a new set: {% embed url="" %} A set that hasn't been published even once will be saved as a 'draft':
Changed but unpublished sets will be saved as 'modified' (). ### Importing/Exporting Parameters Sets A Parameters Set can also be represented in a YAML file with the following structure: ```yaml meta: description: Example Parameters Set parameters: - name1: "Foo" - name2: "Bar" ``` An example use case is to duplicate a set and modify it (![](/files/GEnayTfHK6NiLx9kreqY), ![](/files/QWKTrkAn0EqTB71PZVy9)). ### Exporting as a key-value file From inside the parameters sets editor, it is possible to download only the parameters as a key-value .txt file, e.g.: ``` name1=Foo name2=Bar ``` It is good for Loadmill's CI integration, where inline parameters can be overridden or added by providing a key-value file instead (See "Parameters" in [Loadmill's NPM README](https://www.npmjs.com/package/loadmill?activeTab=readme)).
# ⨍(⨯) FUNCTIONS Loadmill has builtin functions available to use while creating a test. These functions will be evaluated in run time as part of the test. ## Numeric Functions ### `__abs(p1)` Returns the absolute value of `p1`. * `${__abs('-42')} // returns 42` * `${__abs(p1)} // returns 42, assuming parameter p1 equals 42` ### `__add(p1,[p2,[...]])` Same as the `+` operator, applied to any number of arguments. * `${__add('42','3')} // returns 45` * `${__add(p1,p2)} // returns 10, assuming p1 is 7 and p2 is 3` * `${__add(p1,p2,p3)} // returns 15, assuming p1 is 7, p2 is 3 and p3 is 5` ### `__sub(p1,p2)` Same as the `-` operator. * `${__sub('42','3')} // returns 39` * `${__sub(p1,p2)} // returns 4, assuming p1 is 7 and p2 is 3` * `${__sub(p1,p2,p3)} // returns 3, assuming p1 is 7, p2 is 3 and p3 is 1` ### `__neg(p1)` Unary minus, equivalent to `__sub('0',p1)`. * `${__neg('4')} // returns -4` * `${__neg('-4')} // returns 4` * `${__neg(p1)} // returns -4, assuming p1 is 4` ### `__mult(p1,[p2,[...]])` Same as the `*` operator, applied to any number of arguments. * `${__mult('2','3')} // returns 6` * `${__mult(p1,p2,p3)} // returns 24 assuming p1 is 2, p2 is 3 and p3 is 4` ### `__div(p1,p2)` Same as the `/` operator. * `${__div('6','3')} // returns 2` * `${__div(p1,p2)} // returns 4 assuming p1 is 24 and p2 is 6` ## Conditional Functions ### `__true()` Always returns `true`. ### `__false()` Always returns `false`. ### `__and(p1,[p2,[...]])` * `${__and('true','true')} // returns true` * `${__and('true','false')} // returns false` * `${__and('true','')} // returns false` * `${__and(p1,p2)} // returns true, assuming p1 and p2 aren't false nor ''` * `${__and()} // returns true` * `${__and('-42')} // returns true` Logical AND (same as the `&` operator), applied to any number of arguments. ### `__or(p1,[p2,[...]])` Logical OR (same as the `|` operator), applied to any number of arguments. * `${__or('true','true')} // returns true` * `${__or('true','false')} // returns true` * `${__or('false','false')} // returns false` * `${__or('','false')} // returns false` * `${__or(p1,p2)} // returns true, assuming p1 or p2 aren't false nor ''` * `${__or()} // returns true` * `${__or('-42')} // returns true` ### `__not(p1)` Logical NOT. See also [True Semantics](/test-editor/parameters#true-semantics). * `${__not('false')} // returns true` * `${__not('true')} // returns false` * `${__not('')} // returns true` * `${__not(p1)} // returns true, assuming p1 is't false nor ''` * `${__not('-42')} // returns false` ### `__eq(p1,p2)` Same as the `==` operator. * `${__eq('42','3')} //returns false` * `${__eq('false','false')} //returns true` * `${__eq('FaLsE','false')} //returns false` * `${__eq('','')} //returns true` * `${__eq(p1,p2)} //returns true if both p1 and p2 have the same value` ### `__neq(p1,p2)` Same as the `!=` operator. * `${__neq('42','3')} //returns true` * `${__neq('false','false')} //returns false` * `${__neq('FaLsE','false')} //returns true` * `${__neq('','')} //returns false` * `${__neq(p1,p2)} //returns true if both p1 and p2 have a different value` ### `__eqi(p1,p2)` Same as `__eq` but case-insensitive. * `${__eqi('42','3')} //returns false` * `${__eqi('false','false')} //returns true` * `${__eqi('FaLsE','false')} //returns true` * `${__eqi('','')} //returns true` * `${__eqi(p1,p2)} //returns true if both p1 and p2 have the same value` ### `__neqi(p1,p2)` Same as `__neq` but case-insensitive. * `${__neqi('42','3')} //returns true` * `${__neqi('false','false')} //returns false` * `${__neqi('FaLsE','false')} //returns false` * `${__neqi('','')} //returns false` * `${__neqi(p1,p2)} //returns true if both p1 and p2 have a different value` ### `__lt(p1,p2)` Same as the `<` operator. * `${__lt('42','3')} //returns false` * `${__lt('3','42')} //returns true` * `${__lt('42','42')} //returns false` * `${__lt(p1,p2)} //returns true, assuming p1 is 3 and p2 is 42` ### `__lte(p1,p2)` Same as the `<=` operator. * `${__lte('42','3')} //returns false` * `${__lte('3','42')} //returns true` * `${__lte('42','42')} //returns true` * `${__lte(p1,p2)} //returns true, assuming p1 is 3 and p2 is 42` ### `__gt(p1,p2)` Same as the `>` operator. * `${__gt('42','3')} //returns true` * `${__gt('3','42')} //returns false` * `${__gt('42','42')} //returns false` * `${__gt(p1,p2)} //returns true, assuming p1 is 42 and p2 is 3` ### `__gte(p1,p2)` Same as the `>=` operator. * `${__gte('42','3')} //returns true` * `${__gte('3','42')} //returns false` * `${__gte('42','42')} //returns true` * `${__gte(p1,p2)} //returns true, assuming p1 is 42 and p2 is 3` ### `__matches(target,regex)` Returns `true` if and only if the `target` matches the `regex`. * `${__matches(target,'.*search.*')} // returns true, assuming target is 'A text to search in'` ### `__contains(target,search)` Returns `true` if and only if the `target` contains the string `search`. * `${__contains(target,'search')} // returns true, assuming target is 'A text to search in'` * `${__contains(target,'SEARCH')} // returns false, assuming target is 'A text to search in'` * `${__contains(target,'.*search.*')} // returns false, assuming target is 'A text to search in'` ### `__containsi(target,search)` Same as `__contains` but case-insensitive. * `${__containsi(target,'search')} // returns true, assuming target is 'A text to search in'` * `${__containsi(target,'SEARCH')} // returns true, assuming target is 'A text to search in'` * `${__containsi(target,'.*search.*')} // returns false, assuming target is 'A text to search in'` ### `__if_then_else(condition,then,else)` Returns `then` if `condition` is [semantically true](/test-editor/parameters#true-semantics), otherwise returns `else`. * `${__if_then_else(p1,'good','bad')} // returns 'good', assuming p1 is true or has a value` * `${__if_then_else(p1,'good','bad')} // returns 'bad', assuming p1 is false or ''` ### `__is_number(target)` Returns true if target is not empty and not a NAN. * `${__is_number('42')} // returns true` * `${__is_number('')} // returns false` * `${__is_number('Hi')} // returns false` * `${__is_number(p1)} // returns true assuming p1 is a number i.e. 42` ### `__is_uuid(target)` Returns true if target is in the format of UUID. * `${__is_uuid('123e4567-e89b-12d3-a456-426614174000')} // returns true` * `${__is_uuid('42')} // returns false` ### `__switch(target,case1,value1,[case2,value2,[...],[default]])` Returns `value1` if `target` equals `case1`, otherwise returns `value2` if `target` equals `case2` and so on. If no match is made, the returned value will be an empty string - a default value may be given as the last argument. * `${__switch(p1,'3','Hi','42','towel','none')} // returns 'Hi' if p1 is 3, 'towel' if p1 is 42 and 'none' if p1 is neither 3 nor 42` ### `__switchi(target,case1,value1,[case2,value2,[...],[default]])` Same as `__switch` but case-insensitive. * `${__switchi(p1,'this','Hi','that','towel','none')} // returns 'Hi' if p1 is 'this' or 'THIS', 'towel' if p1 is 'that' or 'THAT' and 'none' if p1 is neither of those` ### `__pick(selection,p1,[p2,[...]])` Returns one of `p1` or `p2` or `p3`, etc. according to the `selection` - either a zero-based index (e.g. `2` picks `p3`) or the word `random` in order to pick a random value. * `${__pick('0','A','B','C')} // returns 'A'` * `${__pick('1','A','B','C')} // returns 'B'` * `${__pick(p1,'A','B','C')} // returns 'C', assuming p1 is 2` * `${__pick('random','A','B','C')} // returns one of 'A', 'B' and 'C'` ### `__pick_random(p1,[p2,[...]])` Same as `__pick('random',p1,[p2,[...]])`. ### `__split_pick(target,delim,[selection=0])` Splits the value of `target` into multiple strings separated by `delim` and returns one of them according to the `selection` as defined by the `__pick` function. If a `selection` argument is not provided, the first value will be returned. * `${__split_pick(p1,'API')} // returns 'loadmill' , assuming p1 value is 'loadmill API testing'` * `${__split_pick(p1,'API','0')} // returns 'loadmill' , assuming p1 value is 'loadmill API testing'` * `${__split_pick(p1,'API','1')} // returns 'testing' , assuming p1 value is 'loadmill API testing'` ## Textual Functions ### `__usd()` Returns the `$` character. ### `__length(target)` Counts the number of characters in `target`. * `${__length('loadmill')} // returns 8` * `${__length('')} // returns 0` ### `__escape_regexp(target)` Returns the value of `target` after escaping special RegExp characters. * `${__escape_regexp('.*search.*')} // returns \.\*search\.\*` ### `__encode_url(target)` Returns the value of `target` after [URL encoding](https://en.wikipedia.org/wiki/Percent-encoding) special characters. * `${__encode_url(p1)} // returns 'this%20is%20url' assuming p1 is 'this is url'` ### `__decode_url(target)` Returns the value of `target` after [URL decoding](https://en.wikipedia.org/wiki/Percent-encoding). * `${__decode_url(p1)} // returns 'this is url' assuming p1 is 'this%20is%20url'` ### `__encode_base64(target)` Returns the value of `target` after [Base64](https://en.wikipedia.org/wiki/Base64) encoding. * `${__encode_base64(p1)} // returns 'ZXhhbXBsZQ==' assuming p1 is 'example'` ### `__decode_base64(target)` Returns the value of `target` after [Base64](https://en.wikipedia.org/wiki/Base64) decoding. * `${__decode_base64(p1)} // returns 'example' assuming p1 is 'ZXhhbXBsZQ=='` ### `__hex_to_base64(target)` Accepts a string representation of a hexadecimal number and returns its value in its Base64 representation. * `${__hex_to_base64(p1)} // returns 'q83vmHZUMg==' assuming p1 is 'abcdef987654321'` ### `__escape_quotes(target)` Returns the value of `target` after escaping special characters. This function is like escape quotes in JavaScript and will escape characters like \n \r \t and \\". * `${__escape_quotes('"escapeMe"')} // returns \"escapeMe\"` * `${__escape_quotes(P1)} // returns \"escapeMe\", assuming p is "escapeMe"` ### `__lower(target)` Returns the value of `target` after converting all characters to lower case. * `${__lower('LoadMill')} // returns loadmill` ### `__upper(target)` Returns the value of `target` after converting all characters to upper case. * `${__upper('LoadMill')} // returns LOADMILL` ### `__slice(target,[begin=0,[end]])` Returns a sub-string of `target` which starts at `begin` index (inclusive) and ends at `end` index (exclusive). Both indexes are zero-based. * `${__slice('Loadmill')} // returns 'Loadmill'` * `${__slice('Loadmill','0')} // returns 'Loadmill'` * `${__slice('Loadmill','2')} // returns 'admill'` * `${__slice('Loadmill','2','4')} // returns 'ad'` * `${__slice('Loadmill',p1,p2)} // returns 'mill', assuming p1 is 4 and p2 is 8` * `${__slice(p1,'0','2')} // returns 'Lo', assuming p1 is Loadmill` ### `__replace(target,[begin=0,[end]])` Returns a new string with one, some, or all matches of a `searchValue` replaced by a `replaceValue`. `searchValue` can be either a string or a regex. If it's a regex, then the `options` argument can be used as the regex options. {% hint style="info" %} Regex tokens must be escaped in `searchValue`. (`searchValue='+'` is bad, `searchValue='\+'` is good) {% endhint %} * `${__replace(p1,'a','x')} // returns 'xaa' assuming p1 is 'aaa'` * `${__replace('AAA','a','x','i')} // returns 'xAA' (here we used the i = case insensitive option)` ### `__replace_all(target, searchValue, replaceValue)` Returns a new string with all matches of a `searchValue` replaced by a `replaceValue` (Same as `__replace`, but the g \[= global] option is always on). * `${__replace_all(p1,'a','x')} // returns 'xxx' assuming p1 is 'aaa'` ## Array functions ### `__array_length(target)` Counts the number of elements in `target` array. * `${__array_length('[]')} // returns 0` * `${__array_length(p1)} // returns 4, assuming p1 is [1,"str",true,null]` ### `__array_matches(target,regex)` Returns a boolean string of whether all the elements in the target array match the given regex. * `${__array_matches(array,'.*a.*')} // returns true, assuming array is ["apple","banana","orange"]` * `${__array_matches(array,'.*Z.*')} // returns false, assuming array is ["apple","banana","orange"]` ### `__array_in_range(target,[min=0,max=2^32])` Returns a boolean string of whether all the elements in the target array match the given range. * `${__array_in_range(array,'0','100')} // returns true, assuming array is ["100","99","1"]` * `${__array_in_range(array,'0','100')} // returns true, assuming array is [100,99,1]` ### **`__array_includes(src,target)`** Returns a boolean string of whether each element in the target array is also in the src array. * `${__array_includes(srcArray,targetArray)} // returns true, assuming srcArray is ["apple","banana","orange"] and targetArray is ["apple","banana"]` * `${__array_includes(srcArray,targetArray)} // returns false, assuming srcArray is ["apple","banana","orange"] and targetArray is ["apple","banana","lemon"]` ### `__array_sum(target)` Returns the sum of all the elements in the target array. * `${__array_sum(targetArray)} // returns 401, assuming targetArray is ["400","0","1"]` * `${__array_sum(targetArray)} // returns 188, assuming targetArray is [100,80,8]` * `${__array_sum(targetArray)} // returns NaN, assuming targetArray is [100,"some-word",8]` ### `__array_sort(target)` Returns the sorted array in the ascending order. * `${__array_sort(targetArray) // returns ["apple","banana","orange"], assuming targetArray is ["banana","apple","orange"]` ### `__array_sort_numbers(target)` Returns the sorted array of numbers in the ascending order. * `${__array_sort_numbers(targetArray) // returns [2,19,34,55,90,109,136,156,188,190], assuming targetArray is [109, 136, 156, 188, 19, 190, 2, 34, 55, 90]` ### `__array_is_unique(target)` Returns a boolean string of whether all elements in the array are unique. * `${__array_is_unique(targetArray) // returns true, assuming targetArray is ["cat","dog","bird"]` * `${__array_is_unique(targetArray) // returns false, assuming targetArray is ["cat","cat","dog"]` ### `__array_pluck(target,'key1','key2','key3:newKeyName',['key4',[...]])` Returns an array of objects with specified JSON keys with an option to rename JSON keys. Returns an empty array in case the target array is undefined. * `${__array_pluck(array,'author','book')} // returns [{"author":"Bulgakov","book":"The Master and Margarita"},{"author":"Bulgakov","book":"The Fatal Eggs"}], assuming array is [{"author":"Bulgakov","book":"The Master and Margarita","year":"1967"},{"author":"Bulgakov","book":"The Fatal Eggs","year":"1925"}]` * `${__array_pluck(array,'author','book','year:publishment_year')} // returns [{"author":"Bulgakov","book":"The Master and Margarita","publishment_year":"1967"},{"author":"Bulgakov","book":"The Fatal Eggs","publishment_year":"1925"}], assuming array is [{"author":"Bulgakov","book":"The Master and Margarita","year":"1967"},{"author":"Bulgakov","book":"The Fatal Eggs","year":"1925"}]` ### `__sequence_array(p1)` Returns an array containing a sequence of numbers from 0 to `p1`. * `${__sequence_array('5')} // returns [0,1,2,3,4,5]` * `${__sequence_array(p1)} // returns [0,1,2,3,4,5,6,7,8,9,10], assuming p1 is '10'` ### `__array_missing_element(sub_array,full_array)` Returns the next suggested number from `full_array` which is not in `sub_array.` * `${__array_missing_element(sub_array,full_array) // returns 2 assuming sub_array is [0,1,3,4] and full_array is [0,1,2,3,4,5,6,7,8,9,10]` * `${__array_missing_element(sub_array,full_array) // returns "banana", assuming sub_array is ["apple","mellon"] and full_array is ["apple","banana","mellon","strawberry"]` * `${__array_missing_element(sub_array,full_array} // returns 3 assuming sub_array is [0,1,2,4,5,6,7] and full_array is [1,2,3,4]` ## **Extraction Functions** See also [Parameter Extractions](/test-editor/parameters#parameter-extraction). ### `__regexp(target,regexp,[default])` Extracts a value from `target` using `regexp` as a JS RegExp. If there is no match, an empty string will be returned or, if present, the given `default` value. * `${__regexp(p1,'.*(search).*')} // returns 'search' assuming p1 is 'A text to search in'` * `${__regexp(p1,'(.*search.*)')} // returns 'A text to search in' assuming p1 is 'A text to search in'` * `${__regexp(p1,'.*(bad).*','none')} // returns 'none' assuming p1 is 'A text to search in'` ### `__json_keys(target,[default])` Returns the keys of target object in an array. Works similar to [Object.keys](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys). If `target` is empty, an empty array will be returned or, if present, the given `default` value. * `${__json_keys('{"key1":"val1"}')}// returns ["key1"]` * `${__json_keys(target)} // returns ["key1","key2"] assuming target is {"key1":"val1","key2":"val2"}` ### `__jsonpath(target,jsonpath,[default])` Extracts a value from `target` using `jsonpath` as a JSONPath query. If there is no match, an empty string will be returned or, if present, the given `default` value. * `${__jsonpath('{"key":"val"}','$.key')} // returns 'val'` * `${__jsonpath(target,jsonpath)} // returns 'val' assuming target is '{"key":"val"}' and jsonpath is '$.key'` * `${__jsonpath('{"key":"val"}','$.notHere','none')} // returns 'none'` ### `__jsonpath_all(target,jsonpath)` Similar to `__jsonpath` but returns **all results from the JSONPath** and not only one as happens when using `__jsonpath.` ### `__xpath(target,xpath)` Extracts a value from `target` using `xpath` as an XPath query. If there is no match, an empty string will be returned. * `${__xpath('val','//key/text()')} // returns 'val'` * `${__xpath(target,xpath)} // returns 'val' assuming target is 'val' and xpath is '//key/text()'` * `${__xpath('val','//missing/text()')} // returns ''` ### `__jsonpath_keys(target,jsonpath)` Returns the keys of the extracted value from `target` queried by `jsonpath` in an array. Works similar to [Object.keys](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys). * `${__jsonpath_keys('{"key1":"val1"}','$')}// returns ["key1"]` * `${__jsonpath_keys(target,jsonpath)} // returns ["key1","key2"] assuming target is {"data": {"key1":"val1","key2":"val2"}} and jsonpath is '$.data'` ### `__jsonpath_apply(target,jsonpath,new_value)` Returns an object or an array of objects with the updated key value as set in `new_value` on specific key as set in `jsonpath`. * `${__jsonpath_apply(target,jsonpath,'USD')} // returns '[{"country":"Japan","currency":"USD"},{"country":"China","currency":"Yuan"}]' assuming target is '[{"country":"Japan","currency":"Yen"},{"country":"China","currency":"Yuan"}]', jsonpath is '$[0].currency' and new_value is 'USD'.` ### `__jquery(target,jquery,[selection=0,[attribute,[default]]])` Extracts a value from `target` using `jquery` as a jQuery selector. If multiple elements are matched, `selection` is applied as in `__pick` with the first element being selected by default. If a non-empty `attribute` is given, the returned value will be the attribute value of the selected element, otherwise, the inner content of the element will be returned. If there is no match, an empty string will be returned or, if present, the given `default` value. * `${__jquery('

Hello world

','h2.title')} //returns Hello world` * `${__jquery(p1,jquery)} //returns Hello world, assuming p1 is '

Hello world

' and jquery is h2.title` ## Randomization Functions ### `__random_uuid()` Returns a random v4 UUID string. ### `__random_boolean([probability=50])` Returns a random boolean value. You may pass an integer between 0 and 100 as the `probability` to get `true` - defaults to 50%. * `${__random_boolean()} // 50% true / false` * `${__random_boolean('70')} // returns true 70% of the times` * `${__random_boolean(p1)} // returns true 20% of the times, assuming p1 is 20` ### `__random_number([max],[min=0,max=2^32])` Returns a random integer between 0 and 2^32. By passing a positive integer you can set a lower maximum, e.g. `__random_number('30')` will resolve to a number between 0 and 30, inclusive. You can also set the minimum, e.g. `__random_number('10','30')` will be between 10 and 30, inclusive. * `${__random_number('30')} // returns a number between 0 and 30` * `${__random_number('10','30')} // returns a number between 10 and 30` * `${__random_number(p1)} // returns a number between 0 and the value of p1` ### `__random_chars([length=10])` Returns a random string of `length` alpha-numeric characters. * `${__random_chars('5')} // returns g2Niu` * `${__random_chars()} // returns YIeT6JXgbJ` * `${__random_chars(p1)} // returns YXgbT, assuming p1 is 5` ### `__random_digits([length=10])` Returns a random string of `length` decimal digits. * `${__random_digits('5')} // returns 83547` * `${__random_digits()} // returns 8354718624` * `${__random_digits(p1)} // returns 83524, assuming p1 is 5` ### `__random_letters([length=10])` Returns a random string of `length` english letters. * `${__random_letters()} // returns YMfvLoYhjh` * `${__random_letters('5')} // returns diMdf` * `${__random_letters(p1)} // returns diMdf, assuming p1 is 5` ### `__random_uppers([length=10])` Returns a random string of `length` upper-case english letters. * `${__random_uppers()} // returns STNOMFSLZC` * `${__random_uppers('5')} // returns WPVRA` * `${__random_uppers(p1)} // returns WPVRA, assuming p1 is 5` ### `__random_lowers([length=10])` Returns a random string of `length` lower-case english letters. * `${__random_lowers()} // returns osmcjtudhe` * `${__random_lowers('5')} // returns alrmv` * `${__random_lowers(p1)} // returns alrmv, assuming p1 is5` ### `__random_hex([length=10])` Returns a random string of `length` hexadecimal digits. * `${__random_hex()} // returns d8c3f7ce22` * `${__random_hex('5')} // returns e5a4b` * `${__random_hex(p1)} // returns e5a4b, assuming p1 is 5` ### `__random_from(chars,[length=10])` Returns a random string of `length` characters present in `chars`. For example, you may generate a random **uppercase** hexadecimal string using `__random_from('0123456789ABCDEF')`. * `__random_from('0123456789ABCDEF') // returns 313249C4FA` * `__random_from('Loadmill') // returns lilodaaLod` * `__random_from(p1) // returns lilodaaLod, assuming p1 is 'Loadmill'` * `__random_from('Loadmill','3') // returns ido` ### `__random_seeded_number([seed='',length=10])` Returns a random string of up to `length` (but no more than 10) decimal digits based on a given seed. Same seed will return the same number. * `__random_seeded_number() // returns 994027583` * `__random_seeded_number('0123456789ABCDEF') // returns 2082602645` * `__random_seeded_number('0123456789ABCDEF','5') // returns 20826` ## Hash Functions ### `__sha1(secret,[secretFormat='utf-8'])` Creating hash digests of data using an algorithm of `sha1`. The function returns a hash in hexadecimal digits. Supported secretFormats are `utf-8` and `hex`. * `${__sha1('mySecret')} alias ${__sha1('mySecret','utf-8')}` * `${__sha1('mySecret','hex')}` * `${__sha1(p1,p2) assuming p1 contains your secret and p2 - your secretFormat.` ### `__sha256(secret,[secretFormat='utf-8'])` Creating hash digests of data using an algorithm of `sha256`. The function returns a hash in a hexadecimal digits. Supported secretFormats are `utf-8` and `hex`. * `${__sha256('mySecret')} alias ${__sha256('mySecret','utf-8')}` * `${__sha256('mySecret','hex')}` * `${__sha256(p1,p2) assuming p1 contains your secret and p2 - your secretFormat.` ## XML/JSON Conversion Functions ### `__xml_to_json(xml, [normalize])` Converts an XML string to its JSON representation. By default, normalization is enabled unless `normalize` is explicitly set to `'false'`. *normalize*: Trims whitespace at the beginning and end of text nodes, and also inside text nodes. * `${__xml_to_json('value')} // returns {"root":{"item":"value"}}` * `${__xml_to_json(xml,'false')} // disables normalization` ### `__json_to_xml(json)` Converts a JSON string to its XML representation. The output is headless (no XML declaration). * `${__json_to_xml('{"root":{"item":"value"}}')} // returns value` ## Time Functions ### `__now([length=13])` Returns the current time (of evaluation) given as UTC milliseconds of `length` digits. Alias: `_now_ms`. * `${__now()} // returns 1604422320167` * `${__now('5')} // returns 16044` * `${'Yo'} ${__now()} // returns 'Yo 1604422320167'` ### `__now_iso([addedMinutes=0])` The same as `__now` but given in ISO-8601 format while adding `addedMinutes` . For example, you may generate the current time + 15 minutes using `__now_iso('15')`. * `${__now_iso()} // returns 2020-11-03T16:54:24.526Z` * `${__now_iso('15')} // returns 2020-11-03T17:09:24.526Z` * `${__now_iso('-15')} // returns 2020-11-03T16:39:24.526Z` ### **`__date_iso()`** The same as `__now_iso` but given in a "date only" format - i.e. `2020-03-03` * `${__date_iso()} // returns 2020-11-03` * `${__date_iso('15')} // returns 2020-11-18` * `${__date_iso('-15')} // returns 2020-10-19` ### **`__formatted_date([format='',addedMinutes=0])`** Formatted date according to the supplied format while adding `addedMinutes`. * `${__formatted_date()} // returns 2024-03-12T13:41:29+00:00` * `${__formatted_date('DD/MM/YYYY')} // returns 12/03/2024` * `${__formatted_date('DD/MM/YY','60000')} // returns 13/3/24` ### **`__format_unit_date(unixTimestamp, [format])`** Formats a given UNIX timestamp according to the supplied format. * `${__format_unix_date('1741534304753','DD/MM/YYYY')} // returns 09/03/2025` * `${__format_unix_date('1741534962788')} // returns 2025-03-09T15:42:42+00:00` For more formatting options, have a look [here](https://day.js.org/docs/en/display/format) # Postscript Create and run various assertions by adding custom codes. Developers are always looking for ways to make their lives easier and writing code is no different. In this section, we'll take a look at how post-request scripts (or postscripts) can help you streamline your HTTP requests. Postscripts are small pieces of code that you can use to automate parts of the request-response cycle. ### Writing Postscripts With Loadmill, writing Postscript code is as easy as writing plain javascript. The server response is stored into an identifier shown as a dollar sign (`$`). To access specific parameter values we would use the same method as we would when accessing an object: ```javascript const apple = { "genus": "Malus", "name": "Apple", "id": 6, "family": "Rosaceae", "order": "Rosales", ... } const appleFamily = apple.family; // returns "Rosaceae" ``` Similarly, when receiving a server response, we would view it the same: ```javascript const response = $; //returns the complete server response const responseProperty = response.propertyName; // or response["propertyNameavascript"] // returns the accessed property value inside the response ``` ### Accessing Postscript Extractions The Postscript feature is very useful when you want to use coded variables on different API requests. With this feature, you can access then throughout your automated flow. This is helpful when you need to store and use information from a previous API request in subsequent requests. In this test case we have an API returning a list of fruits as a `JSON` object.\ From that list, we'll be extracting the name and the calories from the "Apple" object and use it on our next requests: ```javascript const apple = $.find((fruit)=>fruit.name == "Apple"); // the $ represend the whole json object // we then use an array method to find the "Apple" object ``` We can now access this object and it's attributes such that: ```javascript const appleName = apple.name; // returns "Apple" const appleCalories = apple.nutritions.calories; //returns 52 ``` To test this out we can use the console.log method to print out appleName and appleCategories {% embed url="" %} You can now access those variables within the rest of your flow. {% embed url="" %} ### Freedom of Coding Postscript is a powerful tool that gives you complete control over the API. With just a few lines of code, you can create conditions, loops and functions for your API tests. Developers find it useful because they can test their APIs the same way they would in a standard programming language. In our previous example, we tested Fruits API to better understand the Postscript execution. We will now explore the freedom of building code from our API. Here, we'll create a new array containing all the fruits family. ```javascript const fruits = $; //returns all fruits in JSON const fruitsFamily = []; for(var i = 0; ifruit.name == "Orange") // returns true ``` {% endcode %} Similarly, when an API returns an array of fruits and their properties, you can validate the existence of a specific fruit with the exact same code. {% hint style="info" %} **Note: The $ (dollar sign) returns the entire API response in object type.** {% endhint %} {% embed url="" %} ```javascript const fruits = $; // returns the entire json response in object type. const orangeOnList = fruits.some((fruit)=>fruit.name == "Orange") // returns true ``` {% embed url="" %} # Accessing w/ Postscript The Postscript feature is very useful when you want to use coded variables on different API requests. With this feature, you can access then throughout your automated flow. This is helpful when you need to store and use information from a previous API request in subsequent requests. In this test case we have an API returning a list of fruits as a `JSON` object.\ From that list, we'll be extracting the name and the calories from the "Apple" object and use it on our next requests. ### Extracting w/ Postscript ```javascript const apple = $.find((fruit)=>fruit.name == "Apple"); // the $ represend the whole json response object // we then use an array method to find the "Apple" object ``` We can now access this object and it's attributes such that: ```javascript const appleName = apple.name; // returns "Apple" const appleCalories = apple.nutritions.calories; //returns 52 ``` To test this out we can use the `console.log` method to print out **appleName** and **appleCategories** {% embed url="" %} You can now access those variables within the rest of your flow. {% embed url="" %} ### Using Postscript with Pre-Extracted Values You can use Postscript print out values from the Extractions section used across the flow. Here are the steps: 1. Create any extraction from the Extractions section. 2. Open your Postscript section. 3. Assign the extracted parameter to postscript.\ ie: `const name = extractedParameterName;` {% embed url="" %} # Validating Postscript Validating postscript values Most scenarios with postscript require you to validate specific values to ensure a successful API test flow. Loadmill provides 2 methods in validating postscript values: Using Mocha or Assertions. #### Postscript & Mocha Mocha is a JavaScript test framework making asynchronous testing very simple. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases. In the previous article we showed how to verify if a value exists in an array or API response. In our example we tested with a fruits API that returns a list of fruits with their properties and verified if a specific fruit was part of that list. ```javascript const fruits = $; // returns the entire json response in object type. const orangeOnList = fruits.some((fruit)=>fruit.name == "Orange") // returns true ``` In order to validate the API based on the value of orangeOnList we would add the following line of code: ```javascript //assert.equal(parameter, value) assert.equal(orangeOnList, true) ``` {% embed url="" %} Furthermore, a validation error would result the test to fail providing a comprehensive error message: {% embed url="" %} You can reference yourself with the Mocha documentation for more useful functions to better test your API flows at #### Postscript & Assertion Another way to validate a value from postscript is via the Assertion section of the API container. Once your value is saved in a variable, you can use that variable for validation. {% hint style="warning" %} Make sure to convert any variable to a string type while using the Assertion section. {% endhint %} In this case our code should look like the following: ```javascript const fruits = $; // returns the entire json response in object type. const orangeOnList = fruits.some((fruit)=>fruit.name == "Orange").toString() // returns "true" ``` This now converts the boolean value `true` to a string type. To validate your API based on orangeOnList: 1. Add an Assertion. 2. Select/type isOrangeOnList on the parameters list. 3. Select “Equals” from the Values dropdown. 4. Type `true` on the field. 5. Run your API test. {% embed url="" %} # Using Built-in Functions in Postscript Using built-in functions in Postscript You can use [Loadmill's built-in functions](/test-editor/functions) in your Postscript code to perform common operations. You can invoke these functions like you would call it in Loadmill's extractions or assertions. For example, to add two numbers: ```javascript const result = __add(1, 2); console.log(result); // 3 ``` Obviously, the `__add` example is a bit simplistic and you can use the JavaScript `+` operator directly, but it illustrates how to use built-in functions. To really leverage the power of built-in functions, you can use them for more complex operations which are not easily achievable with basic JavaScript. ## More Interesting Examples Some of Loadmill's built-in functions are very useful and powerful and can help you save time and effort in your Postscript code. Here are a few examples: ```javascript // Generate a random UUID const id = __random_uuid(); // Hash a string using SHA-256 const hash = __sha256('mySecret', 'hex'); // Get the next suggested element from full_array which is not in sub_array. const nextMissingElement = __array_missing_element([0,1,3,4], [0,1,2,3,4,5,6,7,8,9,10]) // returns 2 ``` And many more complex operations are available. You can find the full list of built-in functions in the [built-in functions documentation](/test-editor/functions). ### ⚠️ String Inputs and Outputs in Built-in Functions When working with Loadmill's built-in functions in Postscript, **remember that all inputs and outputs must be strings**. This is a common source of confusion, especially when dealing with complex types like arrays or objects. For example, this will **not** work as expected: ```javascript // ❌ Bad: passing arrays directly const nextMissingElement = __array_missing_element([0,1,3,4], [0,1,2,3,4,5,6,7,8,9,10]); ``` That’s because the function expects string representations of the arrays. To fix this, convert them using `JSON.stringify`: ```javascript // ✅ Good: Correct usage with JSON.stringify const array1 = JSON.stringify([0, 1, 3, 4]); const array2 = JSON.stringify([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]); const nextMissingElement = __array_missing_element(array1, array2); const result = JSON.parse(nextMissingElement); console.log(result); // output: 2 ``` Another example: ```javascript const arrAsString = JSON.stringify([109, 136, 156, 188, 19, 190, 2, 34, 55, 90]) const sortedArr = __array_sort_numbers(arrAsString) console.log(sortedArr); // output: [2, 19, 34, 55, 90, 109, 136, 156, 188, 190] ``` 🔁 Tip: Use `JSON.stringify` for inputs and `JSON.parse` for outputs when working with arrays, objects, numbers or booleans. Never assume the return value is a native type, it will always be a string. # Login/Authentication Flow Within the Login tab of each Test Suite, you can define a short flow that will run **before your test flows**. This setup flow is most commonly used to execute requests that will authenticate the test flow itself via cookies or authorization tokens. This flow can also be used to retrieve and extract data necessary for the execution of your test flows. Cookies received and parameters extracted by the login flow requests will be accessible by the requests of the actual test flows of that suite. Global authentication headers configured below can use parameters extracted by the login flow requests (In a `${param}` syntax), and will override any matching headers in the test flow requests of that suite. ![](/files/JdzfGyu90ABEuASsbCn8) ![](/files/5QY2rE6pNgCwt2n2QB94) You can create your login flow from scratch within the Login tab OR use a better way - create it as a [shared flow](https://docs.loadmill.com/collaboration/shared-flows) so you can re-use it in other test suites. # Before & After Hooks Configure tests to run before and after each/all Test Suite flows. Often while writing tests, you have some setup work that needs to happen before tests run, and you have some teardown work that needs to happen after tests run. From now on, you can do it and even more by using our before & after hooks. There are **4 optional hook types** that can be configured: * Before all test flows * Before each test flow * After each test flow * After all test flows ## How it works Let's take a simple yet comprehensive example on how to use the hooks. For example, I have a few tests aka test flows in Loadmill that have a shared setup and teardown logic. First, I need to create a new user and login. Then, my test flow/s should run but it is important to get user data before each test flow. Eventually, I need to delete the user that has been created. Let's configure this logic together. Navigate to/create a Test Suite containing relevant tests -> BEFORE/AFTER tab. On this page, you can create before & after hooks from scratch or use pre-defined [shared flows](https://docs.loadmill.com/collaboration/shared-flows). ![](/files/qh1J4pXnUDyQtUM1UEwE) {% hint style="info" %} :brain: Make sure all test flows residing in the Test Suite are aligned with the Before & After logic. You may configure all hooks or just one depending on your use case. {% endhint %} In our case, we will create the "Before All" flow that will include requests to create a new user. Then, we will use our well-known [Login tab](https://docs.loadmill.com/api-testing/test-suite-editor/global-login-flow) to create a login flow. The "Before Each" flow will include a request to get the user data. After that, our test flow/s run and eventually, we delete the created user within the "After All" flow. Let's run the test flow and see results. ![](/files/-MbemKq0mLiGJtI8GYiC) If we enter the test flow itself, we will see our Login and Before Each flows as well. ![](/files/-MbempbJIUlLqNh0i_jm) # Load Test Editor ![](/files/FYnehjmXBkrkmwk80NBM) Let's take a look at the Load Test Editor. ![](/files/QjBxZLZQhS11Ek7ouMIP) | Option | Description | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 The requests' area | The main area of your load test. Double-click a request to manage its headers, parameters and assertions. [Learn More](https://docs.loadmill.com/api-testing/test-suite-editor/request-editor) | | 2. Tested Domains | Manage domains/hosts tested by the load test. See your domain in red? Check how to verify it [here](https://docs.loadmill.com/load-testing/setup/domain-verification). | | 3. DRY RUN | Run your load test as [a single iteration](https://docs.loadmill.com/load-testing/getting-started#running-a-trial-test). | | RUN TEST | Run your load test [at scale](https://docs.loadmill.com/getting-started/getting-started-1#running-a-load-test-at-scale). | | 4. Advanced Settings | Configure Iterations, E-mail notifications, enable Cookies, [Import CSV](https://docs.loadmill.com/load-testing/working-with-the-test-editor/data-from-csv-files) and more. | | 5. Replace Parameter | Easily replace the parameter you wish so all of the flow's requests will be affected. | | Extract Parameter | [Extract a value to a parameter](https://docs.loadmill.com/load-testing/working-with-the-test-editor/quick-parameter-editing) from all across the test requests. | # Load Testing Guide Running your first load test. The Loadmill's Load Testing module allows users **to replay their API tests at scale** and use real traffic with the ability to target specific countries or go global. ![](/files/ZL4CLj98Vg2C2ksEXe1c) In this quick guide to Loadmill we will learn how to: 1. Create and run a very simple load test in just a few seconds. 2. Create an API test by using our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder) and run it as a load test in Loadmill. 3. Run load tests as DRY RUN or at scale. ## Let's create and run your first Load test together. There are two ways to do that: * Creating and running a load test from scratch in Loadmill. * Recording an API test by using our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder) and running it as a load test in Loadmill. ### Creating and running a load test from scratch in Loadmill: {% embed url="" %} The simplest load test you can create is an HTTP GET request to your web server. {% hint style="warning" %} :brain: Load test may include **100 requests maximum**. {% endhint %} All you need to do is: 1. Click the **NEW TEST** button on the top navigation menu. This will get you to our test editor. 2. Give your new test a short description `My First Test` 3. Copy and paste the website URL into the **URL** text box. In our case, it's going to be `https://loadmill-test-blog.herokuapp.com`. This is a ghost blogging server we are using for testing and demos, so you can use it for tests. 4. Click the **DRY RUN** button as we recommend running your load test for the first time as a [trial test](https://docs.loadmill.com/getting-started/getting-started-1#running-a-trial-test). ![](/files/5FdWjobbC8AlwsfMMvSm) That's it! You have just created and run your first load test in under a minute 🎉 See how to run it at scale [here](https://docs.loadmill.com/getting-started/getting-started-1#running-a-load-test-at-scale). ### Recording an API test by using [our Chrome recorder extension](https://chrome.google.com/webstore/detail/loadmill-recorder/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en) and running it as a load test in Loadmill: 1. Create an API test as explained [here](https://docs.loadmill.com/api-testing/getting-started#creating-and-running-an-api-test-by-using-our-chrome-recorder-extension). 2. Go to Loadmill and open the recorded test in your Test Suite. 3. Click on three dots next to the run icon - Click **Convert To Load Test**. 4. To run the load test for the first time, we recommend running it as a trial test. See how to do that below. ### Running a trial test Before we run a massive load test, we usually want to run just a single iteration of the scenario, to make sure everything works as expected. This is what we call a **DRY RUN**. It's also a very easy way to test your API and can be exported to a **Functional Test** which you can later [use in your CI](https://docs.loadmill.com/integrations/npm-modal). When we are creating a new test or extending/maintaining an old one, we like to iterate several times - modifying the test and trying it out - until we get the result we want. To start a trial run, click the **DRY RUN** button at the bottom of the editor. ![](/files/-MNOl2TQuaHSF8rKCw_Z) ### Running a load test at scale Now that we know that our test is configured correctly, we can run it at scale. To do that: 1. Click the **RUN TEST** button at the bottom of the test editor. 2. Set Test Duration (minutes) and **Maximum Concurrent Sessions OR Maximum Requests Per Second** you would like this test flow to run. 3. Click on **Advanced** to set additional parameters. For example, you can set **Ramp-up** **Duration** (1 minute by default) hence a number of concurrent sessions will increase linearly during the ramp-up. 4. Click **RUN**. ![](/files/VufxX0zFCHVBLFemvOtZ) {% hint style="info" %} high-volume:man\_mage: **Note**: At some point you may notice a message stating that your domain is not *verified* - this only means that you may not run high-volume tests before proving ownership of your domain/hostname. You can learn how to do that [**here**](https://docs.loadmill.com/load-testing/setup/domain-verification) (it's actually pretty easy). {% endhint %} As the test progresses we can see the number of concurrent sessions **ramping up** linearly and see how **response time metrics** change over time. ![](/files/-MNOnaYkgPfUE5jbBYw_) In addition, you can **adjust concurrency** **during a test run**. This is a very powerful feature that allows to see how the test performs with dynamic concurrency changes in real time. ![](/files/-MNSNWvW-a2253t7erkD) Loadmill keeps track of the request **error rate** and fails the test if it passes a certain threshold (%50 by default). We can see the response time for each request and are able to drill down and see the error statistics for each one. {% hint style="info" %} :brain: You can easily run your API Test Suite as load tests by clicking on the drop-down icon within the **RUN SUITE** button - **Run as load test**. {% endhint %} ![](/files/7nNWpas6b4MBa1B2tF0Q) Keep exploring the guides to learn how to analyze load test results, set thresholds and more. ### Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # Analyzing Load Test Results Load tests results display several types of data series over the duration of the test. The data is divided into two graphs: ![](/files/-Lwm27tkNucbDzpEGh-u) ### Performance Over Time In the graph the main x-axis is the running time of the test, the y-axis on the left is response time measured in milliseconds, and the right y-axis is the number of active users. On this graph you can track how the number of active users correlates with your application performance using the following metrics: * **Concurrency** (black) - the number of active users in the test. * **Avg. Response Time** (green) - the average response time for all successful requests in a given moment in time. * **50% Response Time** **(purple)** - the 50th percentile for response time, also known as Median. This is the response time value, half of the requests measured below it, and half of the request measured above it. * **95% Response Time** **(orange)** - the 95th percentile for response time. 5% of the requests measured slower response time than this value. * **Error rate (red)** - The percentage of failed requests out of the total number of attempted requests for the given moment in time. In case the load test reaches a certain threshold (%50 by default), it will fail. The difference between average, 50th percentile and 95th percentile, also known as Standard Deviation, can be a useful tool to identify bottlenecks in your applications performance. ### Throughput This graph represents the number of successful requests per second. As the number of active sessions increases, the number of attempted requests will go higher. When the number of active sessions is constant for some time, changes in throughput will indicate changes in your applications ability to respond fast to requests. # Parameterized Load Test Creating a load test by using parameters and extracting response values In our [first simple test](https://docs.loadmill.com/load-testing/getting-started#creating-and-running-a-load-test-from-scratch-in-loadmill) we have simulated users loading the homepage of our blog, our second test is going to simulate several users publishing new entries to the blog. * First create a new empty test. Only logged in users can post to our blog, so let's start by executing a login request and store the authentication token returned by it into a parameter for later use. * Give your new test a short description `Login & Create a blog post` * Update the request description to `Blog Login` * Set the Request #1 method to `POST` and its URL to - `https://loadmill-test-blog.herokuapp.com/ghost/api/v0.1/authentication/token/` * Click the "ADVANCED" button to expand the advanced section of the request. * Set the Content-Type to `application/x-www-form-urlencoded` (last in the list). * And paste this as the request body - `grant_type=password&username=a@b.com&password=Test1234&client_id=ghost-admin&client_secret=b91601629baf` * Expand the “Extractions - Set Parameters” card and set a value to a parameter named `access_token` using a `JSONPath` extractor querying for `access_token`. This will create a value extractor that will execute a [JSONPath query](http://goessner.net/articles/JsonPath/) against the JSON response of the request and extract the authentication token into a parameter named `access_token`. * Verify that a value was set to the parameter `access_token` by the `JSONPath` extractor by expanding the ‘Verify Response’ card of the request and creating an assertion. Add an assertion that states that the `access_token` parameter `Is True` (Use the "Exists" assertion type). ![](/files/-MNSi4P0GO61k5y_XlvU) * Click **DRY RUN** and execute a trial run to make sure it is working. * Now that we have the authentication token stored in `access_token` we can use it to publish on our blog * Create a new request by clicking the "+ ADD REQUEST" button below the first request. * Set its description to `Publish Blog Post` Set it as a `POST` request to `https://loadmill-test-blog.herokuapp.com/ghost/api/v0.1/posts/` * Click on the "ADVANCED" button and set the Content-Type to `application/json` and the body to this JSON- ```javascript { "posts": [ { "title": "Title ${__random_chars}", "slug": "${__random_chars}", "markdown": "Text ${__random_chars}", "status": "published" } ] } ``` * This JSON body defines the blog post that we are going to publish. Notice that we have used one of [Loadmill's functions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions) to make it a little more interesting - `${__random_chars}`. The `__random_chars` parameter will change to 10 random characters during test/trial execution. * Now, let's use the `access_token` parameter value we extracted from the login response to authenticate this request. Expand the header card of publish request and add a header with the name `Authorization` and the value `Bearer ${access_token}` * Run a trial of the whole scenario and go to `https://loadmill-test-blog.herokuapp.com/` to see that our blog posts are getting published. Now that we know that our full scenario is working we can run it as a load test. Click the "RUN TEST" button at the bottom of the test and run the test 🎉 # Domain Verification **Domain verification** is the process of verifying that your domain or hostname actually belongs to you, so we can load-test it. For example, if you have an app hosted on `myapp.com` you would like test requests from [Loadmill](https://www.loadmill.com) to be sent to it during a load test. However, if we allow ***anyone*** to send requests to `myapp.com` it could be abused by a malicious user to attack your servers by flooding them with requests. The way this problem is solved is by using **domain verification**, a simple process in which you are asked to: 1. Go to **Settings > Domains**. 2. Enter the hostname or sub-domain you would like to test in the text area at the top of the page and click **Verify**. 3. You will be prompted and asked to host a small text file, containing a **Verification Token**, on your server in order to prove you own your domain, e.g. `myapp.com/loadmill-challenge/aSnd5K8L86Pggg1rGLPgLlf6guK.txt`. 4. Once the file is hosted on your server, click **Verify** to complete the process. If you have access to your site's **DNS configuration**, you can [prove ownership without changing code on your server](#dns-verification). {% hint style="warning" %} :brain: **Note:** There is no need to prove ownership of a sub-domain once its parent is verified. For example, if you have already completed the verification process for `myapp.com` you can load-test `www.myapp.com` or `blog.myapp.com` as well. {% endhint %} ## Express Middleware If you are using [node.js](https://nodejs.org) and [express](https://expressjs.com), you can easily serve the verification file using our npm module: [express-loadmill](https://www.npmjs.com/package/express-loadmill). It may also be used to enable [crowdsourced load testing](/auth/testing-with-cors) for high volume load tests. ## DNS Verification If you do not want or unable to serve a file on your server, you can prove ownership of your domain via DNS. According to the documentation of your hosting service or DNS provider, add a DNS record of the following format: * **Type:** `TXT` * **Name:** The sub-domain you would like to test, e.g. `www` or `myapp.com` * **Value:** `loadmill-challenge=` where `` stands for the **Verification Token** you were asked to serve in a text file. E.g. `loadmill-challenge=aSnd5K8L86Pggg1rGLPgLlf6guK` You can make sure that the text record was properly added by looking it up using DNS lookup tools like this one - [MxToolbox](https://mxtoolbox.com/TXTLookup.aspx). Once the DNS record is set, click **Verify** to complete the process. {% hint style="info" %} :man\_mage: **Note:** DNS records may take a while to be propagated, so if the verification initially fails, try again after a few minutes. {% endhint %} # Configuration Files Learn how to work with test configuration files Loadmill test configuration files are an extension of the [YAML file format](https://en.wikipedia.org/wiki/YAML). The test configuration example below was converted into JSON to be used in [our REST API](https://docs.loadmill.com/integrations/rest-api#create-load-test) and demonstrates some of the key elements such as - [extractions](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions), [assertions](/test-editor/assertions) and parameter defaults. {% code title="example.json" %} ```javascript { "meta": { "description": "New Flow" }, "requests": [ { "description": "Basic flow", "method": "POST", "url": "httpbin.org/anything", "postData": { "text": "{\"parameter\":\"value\"}", "mimeType": "application/json" }, "extract": [ { "parameter_value": { "jsonPath": "$.json.parameter" } } ], "assert": [ { "check": "parameter_value" } ] } ], "parameters": [], "skipLoginFlow": true, "useCookies": true, "authenticationHeaders": [], "notifications": [], "concurrency": 50, "duration": 120000, "rampUp": 60000, "iterationDelay": 1000, "maxIterations": "5", "targetedCountries": [] } ``` {% endcode %} # Load Testing FAQs ### **Is there a way to run a load test by throughput instead of setting the maximum number of concurrent sessions?** Sure, click the arrow next to **Max Concurrent Sessions**, select **Max Requests Per Second** and set the number you wish. ![](/files/-MXWJu-dK2FYG_BRosx8) ### **I've got an error message below\... what should I do?** ![Unable to run load test](/files/-MMGEWvKJqxrsswqBkWg) The message you've got means that you've reached your plan's maximum credit. Please go to **Settings -> Billing -> Change Plan** and upgrade. If you don't have access to the Billing tab, contact your team admin who will be able to buy more credit. ### **Can I add multiple conditions to a flow control step?** Yes, you can combine multiple conditions by using an **assignment extraction** in the **Extractions - Set Parameters** section. Use the following syntax when assigning a value to a parameter: ``` ${condition1 && condition2} ``` This ensures that the combined condition is evaluated before being used in the flow control step. Do not place this expression directly in the loop input parameter itself. # Load Testing Troubleshooting Debugging some common issues with load tests ### Load test does not start If your test is stuck at "Pending" state, it is possible that some of its requirements are not being met. For example, if your test is using crowdsourced traffic from a specific country, but there not enough simulation devices are available from this location, your tests will wait in "Pending" state until those will become available. ### **I’m getting an error ‘Unverified domains are already under test’ but our domain has been verified. What do I miss here?** Please make sure your tests are not using other domains like facebook, salesforce etc. Contact support at if you need [an additional domain to get verified](https://docs.loadmill.com/load-testing/setup/domain-verification) or any other assistance is required. ### **Error message: domain name \[domain] is not on white list** As mentioned in the [domain verification ](/load-testing/domain-verification)page, you are only allowed to test websites that you own. Before your test starts, the list of participating domains (white list) is calculated based on all of the test requests. In case one of your request uses a dynamic parameter to generate one of the test request's URL, we might miss this domain, and not add it to the test's white list. ![HTTP request domain name \[random-website-name.com\] is not on white list](/files/-LuZRsOmQRUGtcVYnK9A) A simple workaround would be to add at least one request for the dynamic domain in one of the other requests. (If you want you can even use a skip condition to disable this request from actually running). # Overview Loadmill users can easily record real user sessions as Loadmill test flows, then integrate the flows into their Continuous Delivery pipeline and run them per each code-change. Loadmill provides two main options to record real user sessions: **Loadmill Chrome recorder extension** and **Loadmill node recorder**. ## Loadmill Chrome recorder extension 1. Download the extension [here](https://chrome.google.com/webstore/detail/loadmill-recorder/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en). 2. Go to [Test Suites](https://app.loadmill.com/app/api-tests/test-suites) in Loadmill and create a Test Suite. 3. Open the Chrome Developer Tools => switch to the **Loadmill Rec** tab and select name of the created Test Suite in the extension. ![](/files/tHa0G74W5OAH6UpumwBn) {% hint style="warning" %} :brain: If you are not logged in to Loadmill, you will be prompted to the login page. Please note that sometimes other [installed extensions](https://docs.loadmill.com/working-with-the-recorder/recording-troubleshooting) like Adblock may cause issues, thus we recommend switching them off while recording. {% endhint %} 4\. Click on the "Clear" button within the extension to clear the captured requests list and navigate to the tested website. 5\. Start recording your test. By default, the recorder captures only requests from the main domain. In case you have more domains to record, click on FILTER SETTINGS and add them, also you can filter irrelevant strings out in these settings. ![](/files/3fb4T9EEKSryzZodY4w8) Note, you can also configure URL Filters within [the Recordings Settings](https://docs.loadmill.com/working-with-the-recorder/recorder-settings#url-filters) so that the entire team will have the same filters. 6\. To make sure your recorded session includes only relevant requests, you can click on the expanding arrow within requests to see their request and response body. Delete irrelevant requests by clicking on the "Delete" icon. 7\. When you are done recording your session, click the "Upload" button at the bottom. 8\. Click on the small box next to the Test Suite name to open the recorded test in your Test Suite. {% hint style="info" %} :mage: For each test flow Loadmill automatically creates [**extractions**](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) and [**assertions**](https://docs.loadmill.com/api-testing/test-suite-editor/assertions): 1. Default assertions - for any extraction we find in the recorded test, we add a default assertion to it (example: "id" exists). 2. Specific assertions - for any parameter with given user key (for example, "success") we extract it and assign to it an assertion (example: "success" equals "true"). {% endhint %} 9\. Run the Test Suite by pressing the "RUN SUITE" button and see [your results](https://docs.loadmill.com/api-testing/analyzing-an-api-test-results). 🥳 ## Loadmill node recorder We've developed **a smart recorder** that allows users to capture traffic in the QA or Production environment and replay it as API tests. ![The Recordings tabs](/files/qPi2LuIjsUkZei6zHd82) ![](/files/-LpTJchKQFqAbttAcw6m) 1. Create a new recording from the [Recordings tab](https://app.loadmill.com/app/recordings/my-recordings) in Loadmill. 2. Play your test scenarios in the recorded app. By default, the traffic is split to different sessions by its source (different IPs) and intermissions between user actions. (If you wish to manually split a recorded session in your test environment, use the [Loadmill session splitter](https://chrome.google.com/webstore/detail/loadmill-session-splitter/beknfelcpakgnojjfcdpjddhnckekhni), else you can use our [advanced Recordings settings](https://docs.loadmill.com/working-with-the-recorder/recorder-settings)) 3. When you're ready, stop the recording and [generate a Test Suite from it](https://docs.loadmill.com/working-with-the-recorder/working-with-the-recorder). See how to set the recorder up in [the next guide](https://docs.loadmill.com/working-with-the-recorder/setting-up-the-recorder). ## Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # Setup It only takes a few minutes to install the Loadmill recorder on your website. Once installed, you’ll be able to record user behavior on your website and generate a replay-able Test Suite. ## Create an Application Before you'll start recording users flows, you'll need to set up an application to record from: 1. Go to the **Recordings** tab , located in the navigation area at the left, and click on the **App Setup** tab. 2. In the "Setup" tab, click on + "ADD APPLICATION" button. 3. Name the application in the description and fill in its URL. ![](/files/-MNZ-bsiZ3pO4u34O5Kl) 4\. Click on "GET TRACKING ID" button. ![](/files/-MNZ1Jj_1iYmtyTlYBDy) 5\. There are three options how to install the recorder: * Option 1: [Server side](https://www.npmjs.com/package/@loadmill/node-recorder) aka Node * Option 2: Client side aka HTML * Option 3: Testing in Developer Tools aka DEVTOOLS Choose a relevant option for you and right after you've installed the recorder - the application will appear in the applications table. ![](/files/-MNZ3_1g-As3hih-LADb) ## # Recording troubleshooting ### I'm trying to record my tests by using the Chrome recorder extension but getting error that I need to login to Loadmill however I'm logged in... Any idea on how to fix it? There are a few ones 🙂: * It might be related to the 3rd party blocking policy, see how to disable it [here](https://support.cloudhq.net/how-to-enable-3rd-party-cookies-in-google-chrome-browser/). * If the above option didn't help, you might have the "don't track" option enabled in Chrome and you will need to whitelist app.loadmill.com, see where to do this whitelisting [here](https://support.google.com/chrome/answer/2790761?hl=en\&co=GENIE.Platform%3DDesktop). * Sometimes other installed extensions like Adblock may cause this, thus we recommend switching them off while recording. # Additional recording methods Up to now, we've reviewed the following recording methods: * The [Loadmill Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder#loadmill-chrome-recorder-extension) * The [Loadmill Recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder) In case you're already using a recording solution and would like Loadmill to generate an API Test Suite from it, the following recording methods can be imported to Loadmill : 1. [GoReplay](https://goreplay.org/): An open-source network monitoring tool which can record your live traffic, and use it for shadowing, load testing, or detailed analysis and monitoring. You can import Gor files in the recording page by clicking IMPORT RECORDING. 2. [HAR file](https://en.wikipedia.org/wiki/HAR_%28file_format%29): Use the browser's Dev Tools to record a HAR file. You can import HAR files in the API and Load Test Editor. 3. [Charles proxy](https://www.charlesproxy.com/): An HTTP proxy that allows a developer to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet. Export sessions to HAR files and import them as Loadmill Test Flows. # Recording settings Use the advanced Recording Settings to create meaningful replay-able tests that are easy to configure. Team Admins can control these settings [on this page](https://app.loadmill.com/app/user/settings/recordings). ![](/files/odZwNU5R4CnnCGSO9w28) {% hint style="info" %} **Reduce Global Parameters** is the only setting enabled by default that makes sure only relevant parameters will be created during the recording import. :point\_down: These are recording rules you can add & configure for our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder#loadmill-chrome-recorder-extension) & [recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder). {% endhint %} ### Split sessions by URLs (for the [Recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder) use only) In general we want the test cases created based on recorded data, to be short and focused as possible. By default, Loadmill will split a recorded user session into two separate Loadmill test flow, if the user hasn't been active for 5 minutes. In many cases, you will want to control when a session is split into separate test cases. Using this setting, you can add API URLs that will define the beginning of a new test case each time they are called. {% hint style="warning" %} :brain: **Note:** You can also use raw JavaScript to fire an event from your code to split a recorded session:`navigator.serviceWorker.controller.postMessage({ type: "reset-session" });` {% endhint %} ### URL Filters Use this setting to filter API calls that are not relevant for your tests i.e. marketing, monitoring, analytics and other external integrations. ### Special Keys Use this setting to manage important Special Keys located in your POST requests body. This way, we can make sure they won’t be missed during the recording processing phase. ### Entities Synonyms Entities Synonyms help us detect same entities with different names like a 'Jet' and an 'Airplane'. Here you can add and manage entities synonyms. This way, we can make sure they won’t be missed during the recording processing phase. Synonyms must be entered in a list and separated by commas. i.e. Plane, Aircraft, Jet. ### Ignored Keys Ignored keys are a list of keys that will be ignored while generating tests from recordings.\ Keys in this list will never be extracted to a parameter in the created tests. ### Default values Default values are a list of keys that once detected will be assigned with a default value. This value may be overridden on runtime. ### Non-Secret Keys (for the [Recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder) use only) Non-secret keys will be used to identify and record values even when recording in secure mode.\ Keys and paths to identify non-secret values are given in a JSONPath-like syntax. {% hint style="warning" %} :brain: **Note** : Do not record private user data. It is recommended to test these keys locally before recording real users. {% endhint %} ### Regex replacers Regex replacers will be used to identify any strings that match the regex and extract them as a parameter. ### Strict Response Validation When enabled, this option will add JSON Schema assertion to every recorded JSON response. ### Automatic extractions and assertions Configure which JSON keys should be extracted into parameters and which assertions should be created for them. For example, I've recorded a test flow that includes the "total\_score" JSON key and it should be equal to 100 at the beginning of the test flow, so I can add an automatic extraction and assertion for it: ![](/files/-MZwhSYlzE06okrNmE2s) Thus, when uploading a recording, Loadmill will automatically create the extraction and assertion in the test flow in case there is such a JSON key. ![The automatic extraction in the flow](/files/-Ml915ChuNjkVueXkHw9) ![The automatic assertion in the flow](/files/-Ml91DvEwKgkywqFRCy7) # How to work with Recordings ## Start Recording After you're all set with your application, it's time to start recording your users session: 1. Go to the "Recordings" tab and click on + NEW RECORDING. 2. In the dialog, select the wanted application from the list. 3. Click START RECORDING. 4. Once the recording starts, you'll be able to see its progress in the recordings table. **Note:** you can have 1 active recording per app. 5. Click on the running recording in order to review its progress. 6. In order to stop a recording, click on the STOP button. ## Awesome! I've recorded my application, now what? ![](/files/H0mywt9i4q3biE5ZCPb5) When you stop the recording process, you can review its list of tracked API's (method, path, Avg response time & Throughput). The next step will be Generating a repayable Test Suite containing your test flows: Click "GENERATE TEST SUITE". Check out more cool things you can do with recordings in the [Recordings Settings](https://docs.loadmill.com/working-with-the-recorder/recorder-settings) guide. # Overview Loadmill now offers Playwright as a first-class testing experience, fully integrated for true end-to-end and hybrid testing. You can run UI/browser tests, API tests, or combine both in a single flow — all on Loadmill’s automated infrastructure. No need to manage Playwright environments or runners: browser provisioning, scaling, and orchestration are handled for you. ## Why it matters: * Unified UI and API testing under one platform * Built-in infrastructure management, reporting, and AI-driven maintenance * Fast, stable, and scalable tests with reduced operational complexity ![Hybrid Testing Overview](/files/uGFqL5gWk2t0DqJIRK0T) # Playwright Integration Capabilities ## Record Playwright Tests Easily create Playwright tests by recording your actions in the browser. This feature makes test creation accessible to everyone, even those without coding experience, and speeds up the process for QA and product teams. **Note:** To record Playwright tests, you must have the Loadmill Desktop App installed. ### How to use: 1. Open a flow in the Test Editor. 2. Click the "Record" button in the Playwright step (Desktop App required). ![Record Playwright Button](/files/EmBLBeZfoA5jeZ4ZOqcC) 3. Interact with your application in the browser window that opens. 4. When finished, close the browser. Loadmill will generate Playwright code based on your actions. **Tip:** You can edit the generated script for advanced scenarios. ## Cross-Browser Testing Run your Playwright tests across multiple browsers and devices directly in Loadmill. This ensures your application works as expected for all users, regardless of their browser or device. By automating cross-browser validation, you save time and catch compatibility issues early. ### How to use: 1. In the Test Editor, add a Playwright step to your flow. ![Add Playwright Step](/files/PcNQrtRZslvBHcca8mDo) 2. In the Playwright step settings, select the devices you want to test (e.g., Desktop Chrome, Firefox, Mobile Safari) ![Select Browsers](/files/kGvK13RGLqVLhpRZU5Oe) 3. Save and run your test suite. Loadmill will automatically provision and execute tests across the selected browsers. ![Select Browsers](/files/6dwGmnu6qa3aCKQzVure) ## Debugging Run your Playwright tests in debug mode and step through the Playwright script inside the flow context in Loadmill. This lets you view exactly what's happening in the test, analyze issues, and recreate bugs more effectively. Debug mode provides a step-by-step view, making it easier to write, troubleshoot, and refine your Playwright scripts. **Note:** To debug Playwright steps, you must have the Loadmill Desktop App installed. ### How to use: 1. Open your flow in the Test Editor. 2. Run the test in debug mode. [Learn more about Debug Mode](https://docs.loadmill.com/test-editor/flows/flow-controls#debug-mode) 3. Step through the Playwright script, viewing each action and its effect in real time. ![View Playwright Step Logs](/files/JpncK6HBbFKEplY9RmRQ) 4. Use this interactive view to analyze issues, recreate bugs, and improve your test scripts. ## Automated Orchestration & Infrastructure Loadmill automatically provisions browsers, executes tests in parallel, and scales your test runs. You don’t need to manage infrastructure or runners, making your testing pipeline faster, more reliable, and maintenance-free. ### How to use: Simply run your test suite with Playwright steps. Loadmill automatically manages browser provisioning, parallel execution, and scaling. No additional setup or runners required. **Note:** For large test suites, Loadmill will automatically balance load and optimize execution speed. # AI-Native E2E Testing ## Why Loadmill MCP? Loadmill MCP brings the power of AI-native workflows to E2E and behavioral testing. Inspired by the latest advances in agentic workflows, Loadmill MCP lets you: * **Generate new E2E tests in seconds from natural language.** * **Run tests as you develop, with instant feedback and AI-powered insights.** * **Automate repetitive QA tasks and empower your team to ship with confidence.** Test generation is also supported for hybrid testing (combining API and UI steps). [Learn more in the Hybrid Testing section.](/hybrid-testing/overview) *** ## Core Features * **AI-Native Test Authoring:** Describe what you want to test in plain text. Instantly get a ready-to-run E2E suite—no boilerplate, no manual scripting. * **Continuous Feedback Loop:** Run tests directly from your IDE or any AI tool, see results and logs in real time, and get actionable suggestions for failures. * **Self-Verifying Workflows:** Loadmill MCP enables agents to generate, run, and validate tests in a closed loop—ensuring your E2E flows work as intended after every change. * **Collaboration for All:** Anyone on your team, regardless of experience, can contribute meaningful tests and improve coverage. No prior testing knowledge required. *** ## Real-World Workflow: Main Use Cases ### 1. Run E2E Tests & Analyze Results While Developing With Loadmill MCP, you can instantly validate your changes as you code—without leaving your IDE. This approach keeps you focused, reduces context switching, and ensures that every update is automatically checked against your real-world E2E scenarios. The result: faster feedback, fewer bugs, and greater confidence in every release. This example shows how you can use Loadmill MCP to quickly validate your changes and build with confidence. The workflow below is shown with VS Code and GitHub Copilot, but you can achieve the same results with any compatible IDE or AI tool: **Step 1: Make changes and run tests** After updating your code (for example, changing a function or fixing a bug), trigger all relevant E2E tests. Test results appear instantly—if there are failures, you’ll see them right away. ![Run tests and see failures](/files/7ZpbpTllXaxtyJXqEX04) **Step 2: Analyze failures and fix the issue** Use Loadmill MCP to get and analyze the failures, leveraging its insights and suggestions to help you find a solution. Update your code to address the issues. ![Analyze failures and fix code](/files/Hr2gt3DLIlHvQJjIjFfS) **Step 3: Re-run tests and confirm success** Re-run the tests. This time, all tests pass, giving you confidence to continue development. ![All tests pass](/files/2PJ1pkIFz0KI3Io1DAv4) *** ### 2. Generate New E2E or Hybrid Tests in Seconds Using Free Text With Loadmill MCP, creating new tests is as simple as describing what you want to check. Just type your scenario in plain text and let the AI instantly generate a ready-to-run E2E test flow or suite for you. ![Generating a test from free text](/files/Jtvcg5bzZPdY31I7jBIQ) This workflow is designed for speed and flexibility: * **Instant test generation:** Go from a free-text idea to a fully structured test in seconds. No manual scripting, no previous testing experience needed. * **Flexible integration:** Once generated, you can either: * **Import the test file directly into Loadmill** and run it as part of your existing suites, or * **Sync the test via git**—push it to GitHub (or any version control system) and have it run automatically as part of your CI/CD pipeline. * **Always up-to-date and spec-compliant:** Every test generated by Loadmill MCP is built using the latest Loadmill test-suite schema, so you can be sure your tests are always aligned with current Loadmill standards—no outdated formats, no compatibility worries. *** ## Getting Started ### Installation ```bash npm install -g @loadmill/mcp ``` ### Connecting Your IDE **First, generate a Loadmill API token from your** [**account settings**](https://app.loadmill.com/app/user/settings/security)**.** #### VS Code Add a server configuration to your IDE: ```json { "servers": { "loadmill": { "type": "stdio", "command": "npx", "args": ["@loadmill/mcp"], "env": { "LOADMILL_API_TOKEN": "${input:loadmill-api-token}" } } }, "inputs": [ { "id": "loadmill-api-token", "type": "promptString", "description": "Loadmill API token", "password": true } ] } ``` #### Cursor Setup Steps: 1. Open the Cursor application. 2. Navigate to **Cursor Settings → MCP & Integrations → MCP Tools → New MCP Server**. 3. This opens `MCP.json` where you paste the following configuration: ```json { "mcpServers": { "loadmill": { "command": "npx", "args": ["@loadmill/mcp"], "env": { "LOADMILL_API_TOKEN": "REPLACE_WITH_YOUR_LOADMILL_API_TOKEN" } } } } ``` Set your `LOADMILL_API_TOKEN` environment variable before launching Cursor. *** ## Resources * [Loadmill MCP official npm page](https://www.npmjs.com/package/@loadmill/mcp) # Loadmill Test Authoring Workflow ### 🚀 Overview This guide explains how to create and update Loadmill test suites directly in your IDE (Cursor, VSCode, or any other editor) using the Loadmill MCP. With the MCP workflow you can: * Create and update Loadmill tests directly from your IDE * Validate changes instantly * Run tests to ensure they work as expected * Integrate new tests into your pipeline > ⚠️ Before you get started, make sure you meet the [Prerequisites](#prerequisites) below for this workflow. ### 🎥 Video Guide Watch this demo to see how to create a new test suite using the Loadmill MCP: {% file src="/files/SdHjGBp0Y0ZVwQ4uwpAO" %} ### 📦 Where the Work Happens All Loadmill suites live in a Git repository, and this repository is the **entry point** for creating and updating tests. By working directly in this repo: * The actual changes happen alongside the suite files, just like any other code change. * You are able to leverage GitHub MCP to pull context from anywhere in your system—PRs, API files, spec documents—but the resulting test updates remain local to the suite repo. * The familiar Git workflow (branches → commits → PRs → review) applies naturally. ### 🧩 The `AGENTS.md` File Each repository that contains Loadmill suites should include an `AGENTS.md` file that guides the IDE agent. This file defines: * Where suites and flows live * How the agent should structure and modify them * When validation or execution should occur * When to consult Loadmill Docs or GitHub MCP `AGENTS.md` acts as a **baseline behavior model**, and you may extend it with project-specific conventions—for example: * Naming standards * Which Loadmill shared flows to use * Common authentication patterns * Internal testing best practices With this file in place, the agent handles the workflow automatically. You only need to describe *what* to build or change. *** ## 🔧 Main Workflows There are two primary workflows when developing Loadmill tests in your IDE. ### Updating an Existing Test Suite Use this process when adding flows or modifying existing test logic. #### High-Level Steps 1. **Identify the suite to update**: Locate the relevant YAML file under `loadmill-suites/`. 2. **Apply the required changes**: Describe the new flow or modifications, the agent updates the YAML. 3. **Ensure the suite is valid**: The agent validates the suite and resolves any issues. 4. **Commit the changes to a branch**: A new Git branch is created, changes are committed, and the branch is pushed. 5. **Run the suite via MCP**: The suite is executed using that branch, and a link to the run is provided. 6. **Adjust if needed**: Update the suite and re-run until everything passes. 7. **Open a pull request**: Review and merge your changes. 8. *(Optional)* **Sync the suite in Loadmill UI**: Use the GitHub Sync tab to check out the correct commit to view or edit it in the Loadmill UI. **Important:** This last action will overwrite any unsynced changes in this suite in Loadmill, so be sure to sync first if needed. ### Creating a New Test Suite Use this workflow when introducing a new suite for a feature or area. #### High-Level Steps 1. **Create an empty suite**: The agent uses Loadmill MCP to generate a new suite in Loadmill. 2. **Create the actual test**: A new YAML file is created under `loadmill-suites/` with the generated content. 3. **Ensure the test is valid**: Validation is performed and issues are fixed. 4. **Commit to a new branch**: The suite is committed and pushed. 5. **Run the suite**: The agent executes it and provides a link to the run results. 6. **Adjust and re-run if necessary**: Continue until the suite passes. 7. **Open a pull request and merge**: Complete the standard Git workflow. 8. *(Optional)* **Sync the suite in the Loadmill UI**: Checkout the commit using the GitHub Sync tab to view or edit it in the Loadmill UI. ### 🧭 Your Role as the Developer The agent handles the lifecycle—validation, execution, branching, committing.\ You are responsible for: * Describing the tests or updates you want * Providing relevant context if needed (API changes, feature behavior, etc.) * Reviewing the generated test logic * Merging the resulting pull requests The intent comes from you; the workflow is handled automatically. ### ✅ Summary * Loadmill MCP enables you to build and update tests directly inside your IDE. * Test logic lives in the repository and follows your normal development workflow. * Loadmill MCP validates and executes tests; GitHub MCP manages branches and commits; Loadmill Docs MCP provides reference material. * The `AGENTS.md` file defines your automation model and can be customized. * Two main workflows are supported at the moment: updating an existing suite, or creating a new one. **Note:** This functionality is currently in beta. Please provide feedback to help us improve it! *** ### 📘 Prerequisites #### 1. Your Loadmill Suites Must Be Synced to Git Your Loadmill test suites must be connected and synced to a Git repository. This enables version control, collaboration, and integration with the MCP workflow. For setup instructions, see: [Syncing Loadmill with GitHub](/integrations/github-integration/data-sync-connection-to-github) #### 2. MCP Configuration To use this workflow, you need to configure your MCP setup to include Loadmill and GitHub servers. Example Cursor config: ```json { "mcpServers": { "loadmill": { "command": "npx", "args": ["@loadmill/mcp"], "env": { "LOADMILL_API_TOKEN": "REPLACE_WITH_YOUR_LOADMILL_API_TOKEN" } }, "loadmill-docs": { "url": "https://docs.loadmill.com/~gitbook/mcp", }, "github": { "url": "https://api.githubcopilot.com/mcp/", "headers": { "Authorization": "Bearer REPLACE_WITH_YOUR_GITHUB_TOKEN" } } } } ``` #### 3. `AGENTS.md` File Your Loadmill tests repository should include an `AGENTS.md` file at the root level to guide the IDE agent. This file defines conventions, structure, and automation rules for your Loadmill suites and can be customized to fit your project's needs. Download the file and add it to your repository to get started: {% file src="/files/L5KbBN9Ja3kG5ah5vwFA" %} {% file src="/files/D4aHTekWaijC16OCvfUX" %} # Loadmill Agent Skill ## Overview Agent Skills are a new open standard for giving AI coding agents specialized capabilities. Tools like **Cursor**, **Claude Code**, **OpenAI Codex**, and **Goose** all support skills—portable instruction packages that agents discover and load automatically when a relevant task comes up. The **Loadmill Agent Skill** gives your IDE agent everything it needs to create, maintain, and run Loadmill E2E tests as a natural part of your development workflow. Instead of context-switching to a separate testing tool, you describe what you need and the agent handles the rest—authoring YAML test files, validating them, running them via Loadmill MCP, and managing the Git workflow around them. > **Prerequisite:** The Loadmill Agent Skill requires the **Loadmill MCP** server to be installed and configured in your IDE. See the [Installation guide](https://github.com/loadmill/loadmill-docs/tree/master/loadmill-mcp/loadmill-mcp/README.md#installation) for setup instructions. *** ## Use Cases ### 1. Creating New Tests Describe the scenario you want to test in plain text and the agent takes it from there. It creates a new Loadmill test suite, writes the flows, validates the YAML against the Loadmill schema, commits to a branch, runs the suite, and hands you a link to the results—all without leaving your IDE. You can also point the agent at a **pull request for a new feature** and have it generate tests that cover the new behavior end-to-end. Or, direct it to a **buggy or uncovered section of your codebase** and let it create tests that fill the gap—turning blind spots into well-tested flows. No prior Loadmill experience is needed; the skill encodes the structure, conventions, and best practices for you. ### 2. Test Maintenance When your application code changes, tests need to change too. The Loadmill Agent Skill helps you keep your test suites in sync with your codebase. Point the agent at a PR, a diff, or simply describe what changed, and it will: * Identify which test suites and flows are affected. * Update assertions, request payloads, or flow logic to match the new behavior. * Validate and re-run the updated tests to confirm everything passes. This turns what used to be a manual, error-prone maintenance task into a quick conversation with your agent. ### 3. Running Regression Tests During Development While building a new feature, you want continuous confidence that existing E2E flows still work—without waiting for CI. With the skill installed, you can ask your agent to run the relevant test suites or a full test plan against your **localhost** as you develop. The agent will: * Execute the tests via Loadmill MCP against your local environment. * Analyze failures and surface actionable insights. * Help you fix any broken tests caused by intentional changes. * Re-run until all regressions are resolved. This creates a tight feedback loop—write code, run tests against your local server, fix issues—all inside your IDE, long before you push. *** ## Getting Started ### 1. Install Loadmill MCP Follow the [Installation guide](https://docs.loadmill.com/loadmill-mcp/loadmill-mcp#installation) to configure the Loadmill MCP server in your IDE. ### 2. Download the Skill Download the skill package below and extract it into your project's skills directory: | IDE / Agent | Directory | | ------------ | ------------------------- | | Cursor | `.cursor/skills/loadmill` | | Claude Code | `.claude/skills/loadmill` | | OpenAI Codex | `.agents/skills/loadmill` | | Goose | `.goose/skills/loadmill` | ### {% file src="/files/kGvhlSHp8gFSXkol0Ztg" %} ### 3. Start Using It Once the skill is in place, your agent will automatically pick it up. Try prompts like: * *"Create an E2E test suite for the user registration flow"* * *"Update the checkout tests to reflect the new payment API"* * *"Run all regression tests for the billing module"* The agent will follow the workflows defined in the skill—validating, committing, running, and reporting results—while keeping you in control at every step. # Okta SSO integration Okta is an identity and access management service platform that allows to secure every identity, from customers to the workforce with Single Sign-On (SSO), Multi-factor Authentication and more. The Okta SSO integration with Loadmill enables Okta users to authenticate through Okta and then log into Loadmill without authenticating again. ### Configuration **Requirements** * Install the Loadmill application in your Okta instance * Complete the steps below to set everything up **Configuration steps** After installing the application, you need to obtain some information that you will have to send to Loadmill #### Gather information from Okta: In the Okta admin page, click on the Loadmill application and then navigate to the General tab. Copy the following values: * Client Id * Client secret (click the eye button to toggle the visibility) * Okta domain **Send the information to Loadmill** Once you have all the information (summarized below), email it to **:** * Client Id * Client Name (how you wish this team to be called - i.e. 'The Rebel Alliance') * Client secret (click the eye button to toggle the visibility) * Okta domain * Company email suffix (i.e. my-comany.com) Loadmill support will handle your request and get back to you once the integration is configured. #### Note Loadmill - Okta SSO integration is limited to one email suffix per integration. Meaning that if, for example, your email is and the email suffix you supplied to us was [snow.com](mailto:jon@snow.com) then this organization integration instance will support only email with that suffix. ### Authentication Once you've configured the integration with Okta, follow steps below to log into Loadmill: Navigate to the [Login](https://app.loadmill.com/app/login) page => Click **SSO Log In**. ![](/files/x1yNEL2gKO3RZSpix6Qx) Enter your email and click on the "SSO LOG IN" ![](/files/Bdd4W20jVDSceYjRQm2t) Choose the relevant account to login to ![](/files/ud58ztwOC55gDOq3tyat) Enter your Okta credentials in the page you were redirected too. ![](/files/auLuqmLfoWZJL2Cz3pP6) In case, you are using Multi-Factor authentication, put the code you've got and click **Verify**. ![](/files/WbAqR73nphcc8ufH80SW) That's it, you're in! 🎉 ### Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # API Tokens In order to use the [Loadmill npm module](https://www.npmjs.com/package/loadmill) or the [Loadmill REST API](/auth/rest-api), you will need to generate an **API Token**. ## Generating a Token 1. Navigate to **Settings** > [**Security**](https://app.loadmill.com/app/user/settings/security)**.** ![](/files/xlvGvHJ4FFCv07NDpL28) 2\. Enter description for the token's intended usage and click **GENERATE**. You will be given an arbitrary string of characters that will be shown to you ONLY ONCE - this is your API token, make sure to copy and store it in a safe place. ## Using a Token Tokens may be used in one of three ways: 1. Passed in the options object to the [Loadmill npm module](https://www.npmjs.com/package/loadmill), e.g. `require('loadmill')({token: "ewszIIzEDIeORLwTSrcEvrKKEWQeR9yzQqyCCKlK"})` 2. Given as a **basic authentication** username with a blank password, e.g. `https://ewszIIzEDIeORLwTSrcEvrKKEWQeR9yzQqyCCKlK@app.loadmill.com/api/tests` 3. Given in an **Authorization** HTTP header as a bearer token, e.g. `Authorization: Bearer ewszIIzEDIeORLwTSrcEvrKKEWQeR9yzQqyCCKlK` ## Protecting Tokens An API token enables its bearer to create and run load tests within your account. You can do almost anything you can do in our web interface using tokens, except changing your password, making payments and generating more tokens. Therefore, you should keep your token as safe as you can from untrusted third parties. If you suspect it may have been compromised, you may revoke the token via **Settings** > [**Security**](https://app.loadmill.com/app/user/settings/security) and generate a new one. # Testing with CORS ## Testing with CORS For high volume load tests, [Loadmill](https://www.loadmill.com) uses real browser sessions to test your application/web server. This requires that you allow [CORS](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing) in your server code (just for us - no third parties need to be allowed). [Enabling CORS](https://enable-cors.org/server.html) mainly involves reading request headers and writing response headers accordingly on your server. The following example code shows exactly how to enable CORS for Loadmill tests in an [express.js](http://expressjs.com) application. Express users **don't need to copy this example**, simply use our npm module: [express-loadmill](https://www.npmjs.com/package/express-loadmill) ```javascript var app = require("express")(); // Handle CORS before processing requests: app.use(function (req, res, next) { // This request header contains the host of the CORS client: var origin = req.header("Origin"); // This request header contains the HTTP request method: var requestMethod = req.header("Access-Control-Request-Method"); if (origin === "http://app.loadmill.com" || origin === "https://app.loadmill.com") { // This response header allows CORS from loadmill.com: res.header("Access-Control-Allow-Origin", origin); // This response header is required only if you use cookies in your tests: res.header("Access-Control-Allow-Credentials", 'true'); if (req.method === 'OPTIONS' && origin && requestMethod) { // It's a pre-flight request: // This request header contains the request headers of the pre-flighted request: var requestHeaders = req.header("Access-Control-Request-Headers"); setPreFlightHeaders(res, requestMethod || "", requestHeaders || ""); return res.sendStatus(204); } else { // If your test scenario involves reading response headers, // we automatically include them in this request header: var exposedHeaders = req.header("Loadmill-Request-Expose-Headers") || ""; // This response header allows the test client to read the desired headers from the response: res.header("Access-Control-Expose-Headers", exposedHeaders); } } return next(); }); function setPreFlightHeaders(res, allowedMethod, allowedHeaders) { res.header({ // This response header asks the browser not to pre-flight // the same request URL again for the next 24 hours: "Access-Control-Max-Age": "86400", // These response headers approve the request method and headers specified: "Access-Control-Allow-Methods": allowedMethod, "Access-Control-Allow-Headers": allowedHeaders }); } ``` If you are not sure how to enable CORS in your application or would like to see examples for different languages/frameworks, please contact us at . ## Express Middleware If you *are* using [node.js](https://nodejs.org) and [express](https://expressjs.com) you can simply use our npm module: [express-loadmill](https://www.npmjs.com/package/express-loadmill). It may also be used for quick and easy [domain verification](/load-testing/domain-verification). # REST API Loadmill's API is an interface for interacting with Loadmill’s servers. Use it to create and launch your tests. Before getting started you’ll need to generate an API token, take a look how to generate it [here](https://docs.loadmill.com/integrations/api-tokens). ## Run Test Suite `POST` `https://app.loadmill.com/api/v1/test-suites/:id/run` Run a predefined test suite #### Path Parameters | Name | Type | Description | | ---- | ------ | ------------------------------ | | id | string | UUID of the Test Suite to run. | #### Query Parameters | Name | Type | Description | | ------------- | ------- | -------------------------------------------------------------------------------------------------------- | | forceAllFlows | boolean | Default = false - running only flows marked for execution with CI toggle. If true - executing all flows. | #### Headers | Name | Type | Description | | ----------------------------------------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization\* | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | #### Request Body | Name | Type | Description | | --------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | additionalDescription | string | Add an additional description at the end of the current suite's description | | overrideParameters | object |

name: value

pairs to override the default parameters values of this specific run. i.e.

{{"paramName1":"paramVal1"}, ...}

| {% tabs %} {% tab title="200 Test suite has launched successfully." %} ``` {testSuiteRunId: "running-uuid"} ``` {% endtab %} {% endtabs %} ## Get Test Suite Run (Test Suite results) `GET` `https://app.loadmill.com/api/v1/test-suites-runs/:id` Get a launched Test Suite results #### Path Parameters | Name | Type | Description | | ---- | ------ | ----------------------------------------------------------------------------- | | id | string | The running uuid. You get this ID in the response when launching a Test Suite | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | {% tabs %} {% tab title="200 " %} ``` { "id": "test suite run uuid, "description": "test suite run description", "startTime": timestamp, "endTime": timestamp | null, "conf": { "useCookies": true }, "status": "PASSED" | "FAILED" | "RUNNING", "displayName": "Display name of the executing user", "testSuiteId": "origin test suite uuid" | null (if deleted), "testSuiteFlowRuns": [ { "id": "test suite flow run uuid", "startTime": timestamp, "endTime": timestamp | null, "conf": {}, // executed flow conf "description": "Flow description", "status": "PASSED" | "FAILED" | "RUNNING", "numOfRequests": number of requests in this flow conf, "avgResTime": 199, // in ms "duration": "422 ms" } ], "progress": "1/1", // flows progress "successRate": "100%", // flows success rate "avgResTime": "422 ms" } ``` {% endtab %} {% endtabs %} ## Get Test Suite Flow Run `GET` `https://app.loadmill.com/api/v1/test-suites-runs/flows/:id` Get Test Suite Flow results #### Path Parameters | Name | Type | Description | | ---- | ------ | ------------------------------------------------------------------------------ | | id | string | The flow running uuid. You get this ID when fetching the Test Suite Run entity | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | {% tabs %} {% tab title="200 " %} ``` { "id": "test suite flow run uuid", "description": "Flow description", "conf": {...}, // the executed flow conf "status": "PASSED" | "FAILED" | "RUNNING", "startTime": timestamp, "testSuiteFlowId": "origin test suite flow uuid" | null (if deleted), "testSuiteId": "origin test suite uuid" | null (if deleted) } ``` {% endtab %} {% endtabs %} ## Stop Test Suite Run `PUT` `https://app.loadmill.com/api/v1/test-suites-runs/:id` Stop a launched Test Suite #### Path Parameters | Name | Type | Description | | ---- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | string |

The Test Suite Run uuid. You can get this ID in the response when getting the Test Suite Run ().

\

| #### Query Parameters | Name | Type | Description | | -------------- | ------ | ------------------------------------------------------------------------------------------ | | forceFail=true | string | Using this filter option the user can stop the Test Suite Run and put its status to FAILED | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | #### Request Body | Name | Type | Description | | --------------------- | ------ | ---------------------------------------------------------- | | additionalDescription | string | Will be added at the end of the Test Suite Run description | {% tabs %} {% tab title="200 Test Suite Run has been successfully stopped." %} ``` ``` {% endtab %} {% endtabs %} ## Create Load Test `POST` `https://app.loadmill.com/api/v1/tests` Create a load test from load test configuration #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | #### Request Body | Name | Type | Description | | ------ | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | config | object | A load test configuration. The JSON test configuration may be exported from the Loadmill test editor or from an old test run. See a config example here - | {% tabs %} {% tab title="200 " %} ``` { testId: "load test id" } ``` {% endtab %} {% endtabs %} ## Create Load from Test Suite Flow `POST` `https://app.loadmill.com/api/v1/test-suites/:suiteId/flows/:flowId/loads` Create a draft load test from a test suite flow #### Path Parameters | Name | Type | Description | | ------- | ------ | ------------------------------------------------------ | | suiteId | string | The suite UUID in which the flow resides | | flowId | string | The flow UUID. The load will be created from this flow | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | #### Request Body | Name | Type | Description | | --------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------- | | loadTestOptions | object | Optional load test configuration options. If not provided, default values will be used. See the table below for available options. | #### Load Test Option | Name | Type | Description | | ----------------- | ------- | ------------------------------------------------------------------ | | targetedCountries | array | Array of country codes for targeted geographic distribution. | | monitoredEnv | string | Environment identifier for monitoring purposes | | rampUp | number | Ramp-up time in seconds. Default value is used if not specified | | alwaysDistributed | boolean | Whether to always use distributed load testing. Default: `false` | | useStaticIp | boolean | Whether to use static IP addresses. Default: `false` | | sensitiveData | boolean | Whether the test contains sensitive data. Default: `false` | | duration | number | Test duration in seconds. Default value is used if not specified | | concurrency | number | Number of concurrent users. Default value is used if not specified | | rps | number | Requests per second. Default value is used if not specified | {% tabs %} {% tab title="200 " %} ``` { testId: "load test id" } ``` {% endtab %} {% endtabs %} ## Set Flow `PATCH` `https://app.loadmill.com/api/v1/test-suites/:suiteId/flows/:flowId/pools` Create a parameter pool and attach it to a flow #### Path Parameters | Name | Type | Description | | ------- | ------ | --------------------------------------------------------------- | | suiteId | string | The suite UUID in which the flow resides | | flowId | string | The flow UUID. The parameter pool will be attached to this flow | #### Query Parameters | Name | Type | Description | | ---- | ------ | ------------------------------- | | name | string | name of the parameter pool data | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | content-type | string | must be text/csv | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | #### Request Body | Name | Type | Description | | ---- | ------ | --------------------------- | | body | string | The CSV data in CSV format. | {% tabs %} {% tab title="200 " %} ``` ``` {% endtab %} {% endtabs %} ## Run Load Test `PUT` `https://app.loadmill.com/api/v1/tests/:id/load` Run an existing load test. Be aware that a given load test can be run only once. In order to rerun it you will have to recreate it. #### Path Parameters | Name | Type | Description | | ---- | ------ | ------------ | | id | string | Load test ID | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | {% tabs %} {% tab title="200 " %} ``` ``` {% endtab %} {% endtabs %} ## Get Load Test `GET` `https://app.loadmill.com/api/v1/tests/:id` Returns the Load Test data. If the Load Test has ended it would contain its result. #### Path Parameters | Name | Type | Description | | ---- | ------ | ------------ | | id | string | Load test ID | #### Headers | Name | Type | Description | | ------------- | ------ | -------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security" | {% tabs %} {% tab title="200 " %} ``` { "id": "load test id", "startTime": timestamp, "endTime": timestamp, "usedSeconds": "total seconds used by all sessions", "description": "load description", "conf": {}, // load test configuraion "result": "aborted" | "done" | "failed", "failedIterations": number, "successfulIterations": number } ``` {% endtab %} {% endtabs %} ## Labels `GET` `https://app.loadmill.com/api/v1/labels` Returns all user's team's labels #### Query Parameters | Name | Type | Description | | -------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------- | | filter=active\&filter=evaluating | string | Using this filter option the user can get only the labels who are attached to flows with a specific status. | #### Headers | Name | Type | Description | | ------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization | string | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | {% tabs %} {% tab title="200 " %} ``` { "teamLabels": [ { "id": "123e4567-e89b-12d3-a456-426614174001", "description": "my label", }, { "id": "123e4567-e89b-12d3-a456-426614174002", "description": "another label", } ] } ``` {% endtab %} {% endtabs %} ## Run Test Plan `POST` `https://app.loadmill.com/api/v1/test-plans/:test-plan-id/run` Run a predefined test plan #### Path Parameters | Name | Type | Description | | ---------------------------------------------- | ------ | ---------------------------- | | test-plan-id\* | String | UUID of the Test Plan to run | #### Headers | Name | Type | Description | | ----------------------------------------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization\* | String | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | #### Request Body | Name | Type | Description | | --------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | additionalDescription | String | Add an additional description at the end of the current plan's description | | labels | Array |

An array of strings representing labels to filter the test plan execution. i.e. \["label1", "label2"]

Only test flows with matching labels will be included in the execution.

| | labelsExpression | String | String representing of labels expression to filter the test plan execution. i.e. `(label1 \| label2) & !label3` An expression may contain the characters `( ) & \| ! (` | | overrideParameters | Object | `name:value` pairs to override the default parameters values of this specific run. i.e. `{{"paramName1":"paramVal1"}, ...}` | | pool | String | Execute tests from a dedicated agent's pool (when using private agent) | | parallel | String | Set the concurrency of a running test suites in a test plan. Max concurrency is 10 | | maxFlakyFlowRetries | String | The maximum number of retries for flaky test flows. If a test flow fails due to flakiness, it will be retried up to the specified number of times | {% tabs %} {% tab title="200: OK return the an object with testPlanRunId property in it" %} {% endtab %} {% endtabs %} ## Get Test Plan Run `GET` `https://app.loadmill.com/api/v1/test-plans-runs/:plan-run-id` Get the Test Plan Run result #### Path Parameters | Name | Type | Description | | -------------------------------------------------- | ------ | ----------------------------------------------------- | | test-plan-run-id\* | String | test plan run id - given when launching the test plan | #### Headers | Name | Type | Description | | ------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization | String | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | {% tabs %} {% tab title="200: OK " %} {% endtab %} {% endtabs %} ## Re-run Test Plan `POST` `https://app.loadmill.com/api/v1/test-plans-runs/:test-plan-run-id/re-run` Re-run a test plan run #### Path Parameters | Name | Type | Description | | -------------------------------------------------- | ------ | ----------------------------------- | | test-run-plan-id\* | String | UUID of the Test Plan Run to re-run | #### Query Parameters | Name | Type | Description | | ---------- | ------- | --------------------------------------------------------------------------- | | onlyFailed | boolean | Default = false. When set to true only failed or stopped suites will re-run | #### Headers | Name | Type | Description | | ----------------------------------------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization\* | String | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | {% tabs %} {% tab title="200: OK return an object with testPlanRunId property in it" %} {% endtab %} {% endtabs %} ## Get test suites `GET` `https://app.loadmill.com/api/v1/test-suites` Returns a list of test suites #### Query Parameters | Name | Type | Description | | ----------- | ------ | -------------------------------------------------------------------------- | | search | String | filter test suites by the given search phrase | | rowsPerPage | String | How many suites to return, Values might be `10, 25, 50, 100` default to 10 | #### Headers | Name | Type | Description | | ----------------------------------------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization\* | String | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | {% tabs %} {% tab title="200: OK Returns an object with an array property of testSuites" %} {% endtab %} {% endtabs %} ## Get Test Plans `GET` `https://app.loadmill.com/api/v1/test-plans` Returns a list of test plans #### Query Parameters | Name | Type | Description | | ----------- | ------ | -------------------------------------------------------------------------- | | search | String | filter test suites by the given search phrase | | rowsPerPage | String | How many suites to return, Values might be `10, 25, 50, 100` default to 10 | #### Headers | Name | Type | Description | | ----------------------------------------------- | ------ | --------------------------------------------------------------------------------------- | | Authorization\* | String | Authentication token - you can generate it in the "User menu"> "Settings" > "Security". | # Loadmill Agent Running API and Load tests locally or on your servers. Running API and load tests locally on your own servers can offer several benefits, including faster test execution, greater control over the testing environment, improved isolation of issues, and enhanced security. By running tests locally, you can avoid the latency and potential security risks associated with sending requests over the internet, and have more control over the testing environment to simulate different conditions and more effectively identify and fix issues with your API or application. The Loadmill Agent is a tool that enables you to run load and performance tests on your servers or local environment. It acts as a proxy, intercepting requests made to your API or application and forwarding them to the Loadmill testing platform. The Loadmill Agent can be installed on your servers or locally on your development machine and configured to intercept requests made to specific endpoints or domains. You can download the Loadmill agent via npm package or use our newly release desktop app for macOS. ### MacOS Desktop App Loadmill has recently released a new desktop app for macOS that allows users to perform regular and local testing using a private agent. The app is designed to be easy to use and provides a convenient way to access Loadmill's powerful testing capabilities from a Mac computer. One of the key features of the app is the private agent, which allows users to conduct testing on their own devices without having to rely on external servers. This is particularly useful for local testing, as it allows users to test their applications in a controlled environment without having to worry about network connectivity or other external factors. To use the private agent, users will need to generate a Loadmill [security token](https://app.loadmill.com/app/user/settings/security). Once the token has been generated, it can be used to authenticate the private agent and begin testing. {% tabs %} {% tab title="Agent Installation" %} {% embed url="" %} {% endtab %} {% tab title="Agent Run" %} {% embed url="" %} {% endtab %} {% endtabs %} ### NPM Package/Docker Installation Run the following command if you have Docker installed on your machine: `docker run -it --rm -e LOADMILL_AGENT_TOKEN= loadmill/agent` For more information, please [click here](https://hub.docker.com/r/loadmill/agent) #### NPM Package **Prerequisites** 1. Node version 14 or higher installed. 2. [Generated Token](https://docs.loadmill.com/integrations/api-tokens) **Installation**
using npmusing yarn

npm i @loadmill/agent -g

npm i @loadmill/agent // in case you can't install it globally.

yarn add @loadmill/agent -g
#### Running the Agent | installed globally (-g) | installed locally | | --------------------------------- | -------------------------------------------------------------------- | | `loadmill-agent start -t ` | `./node_modules/@loadmill/agent/bin/loadmill-agent start -t ` | {% embed url="" %} #### Commands The start command accepts the following options: * `--token` - (REQUIRED) can be retrieved [through the UI](https://docs.loadmill.com/integrations/api-tokens). * `--loads-capacity` - (default = 50). Optionally you can supply the number of users this agent can simulate. It can be 0 in order not to run load tests. * `--no-api` - can be supplied in order to run [Load tests](https://docs.loadmill.com/load-testing/getting-started) only, [API tests](https://docs.loadmill.com/api-testing/getting-started) will run on Loadmill servers. * `--pool` - can be supplied in order to set the agent pool (the limit is 256 characters). When using multiple Loadmill Agents, this parameter helps you distinguish between running agents. Then, use [our npm module](https://www.npmjs.com/package/loadmill) with supplying the `--pool ` parameter to assign test runs to a specific agent. * `--config` - alternatively, you can supply a path to a yaml file that will contain all the options above. ### Insecure Certificates You can set the env var `NODE_TLS_REJECT_UNAUTHORIZED=0` at the beginning of the start command when you're testing a system that uses https but has a self-signed or invalid SSL certificate. Just type this: `NODE_TLS_REJECT_UNAUTHORIZED=0 loadmill-agent start -t ` IMPORTANT: The Loadmill agent won't be able to verify that it's talking to the right website in this case. \\ # CI integration The easiest way to integrate your API tests into your CI/CD pipeline is by using [**our npm module**](https://www.npmjs.com/package/loadmill). ### How it works 1. Install the npm module as described [here](https://www.npmjs.com/package/loadmill). Make sure you have node.js version 14 or higher installed on your machine. 2. Create an API token that will be used for the integration. To generate it, navigate to Settings - [Security](https://app.loadmill.com/app/user/settings/security). 3. Create a Test Plan and include into it all Test Suites you would like to run in the CI. Read more about Test Plans [here](https://docs.loadmill.com/api-testing/test-plan). 4. Initiate execution of the Test Plan by running command: ``` loadmill --test-plan -w -v -t --report --colors // Take the parameter value from its URL going after /api-tests/test-plans/. ``` Other supported options can be found [here](https://www.npmjs.com/package/loadmill). ### Docker Image You can also use Dockerfile attached to run the Test Plan you wish: {% file src="/files/0NcNCBIP7K8rKHs9lwcL" %} To build it use this command : `docker build -t loadmill-runner .` To run it use this command: `docker run -it --env TP_ID=3085de2f-bc42-4727-9c4b-ab649ca4a285 --env TOKEN= --env BUILD_NUMBER= loadmill-runner` In case you like short cuts, you can go ahead and use the npm package [docker wrapper](https://hub.docker.com/r/loadmill/runner) instead :) # GitHub ![](/files/-MUxNeyyDf9dcYPDh8_P) **GitHub** is a code hosting platform for version control and collaboration. It lets users work together on projects from anywhere. With our integration with GitHub, you can easily integrate your Loadmill tests with your GitHub repository and [they will run per each successful release of your app](https://docs.loadmill.com/integrations/github-integration/ci-integration). In addition, the integration enables you [to create a backup of your Test Suites in GitHub](https://docs.loadmill.com/integrations/github-integration/data-sync-connection-to-github) and easily track changes made to them. # CI integration Loadmill enables users to connect tests to a GitHub repository so that the tests will run per each successful release of the application associated with the repository. Configure the integration by following the steps below: 1. Navigate to **Settings** - **Integrations** - CONNECT TO GITHUB. Note: only team admins can perform this action. ![](/files/KlKNtHrGElm9PpDlyXTy) 2\. You will promptly be redirected to GitHub and then back to Loadmill, select a relevant repository and click **CONFIGURE**. 3\. Choose a relevant Test Plan and click **SAVE CONFIGURATION**. Find more information about Test Plans [here](https://docs.loadmill.com/api-testing/test-plan). ![Selecting Test Plan](/files/-MUi1U-lt3qH6Q0-C-rl) From now on, the chosen Test Plan will run upon a successful release to the application associated with the repository with the application url as the target url. Watch your tests update in real time on commits and the pull request screen in GitHub as shown below. Click on **Details** to navigate to the Test Plan run. ![Pull request screen](/files/-MUi6jxhUHGbnUptqCXD) Each Test Plan run will include the commit message within the Test Suite names: ![Test Plan run screen](/files/-MUiAVcoQVsenY1vgSTu) Thus, it will be easy to identify which commit the Test Plan run is referring to. To reset the integration, click on **Configured** next to the repository name and click **CLEAR CONFIGURATION**. # Data sync Loadmill provides the ability to store the Test Suite data in your GitHub repository so you always have a backup when needed. In addition, the data sync connection to GitHub enables you to always track changes made to Test Suites and take the data from the repo for test runs. ### Setup 1. Navigate to **Settings** - **Integrations** - **CONNECT TO GITHUB**. Note: only team admins can perform this action. 2. You will promptly be redirected to GitHub and then back to Loadmill, scroll down to the **Setup data sync connection to GitHub** section. 3. Choose a repository to enable data sync to. ![The GitHub data sync integration setup page](/files/-MUwhG-SO6bjVAHk00MG) 4\. Click on **CONNECT TO REPO**. Now it's time to fill it in with your Test Suite data. :partying\_face: ### **Usage** To sync your Test Suite data with your GitHub repository, follow the steps below: 1. Navigate to a relevant Test Suite and click on the **GITHUB SYNC** tab. ![](/files/aOGf9TD3MQ1VGohU7QtK) 2\. Choose or create a branch where you would like to sync the Test Suite data to. Then, enter an initial commit message and click COMMIT to sync the Test Suite data. ![](/files/-MdWAqWS-NENvkIhJpm3) **What happens next?** Loadmill creates the **loadmill-suites** folder in the branch you have selected/created. In addition, a new file with the Test Suite data is created in the loadmill-suites folder. Its name matches the Test Suite description and ID. Easily navigate to the commit by clicking on **GO TO COMMIT** at the bottom left corner of the screen. ![](/files/-MUrs6SDlufkPhs5wCQ0) The commit message in GitHub includes your Loadmill username and the commit message you entered in Loadmill. ![](/files/-Ml3zxNZIp__Il2HGjrN) **Another great thing about this feature** is that, over time, you can always track changes made to Test Suites. Just remember to enter commit messages to sync the Test Suite changes to GitHub. ![](/files/-Ml406scP-J4lmetXIJi) In addition, you can also see the difference aka DIFF between the last committed version and the "working copy" version with the changes that weren't committed to GitHub yet. ![](/files/puQ19Zhxj8HAgSis8Cu4) Moreover, **you can always revert the Test Suite data to the state of any commit you wish**. Just select it within the Branch history section and click **CHECKOUT**. ![](/files/-MdWE-AooPX7Sry5XP2n) ### Running Test Plan with the data committed to GitHub Once you have Test Suite data committed to your GitHub repository, you can select it when running your Test Plan. Navigate to the **Test Plan** - **Run Settings** - **Versioned Test Configuration (Recommended)** and select the branch where you committed test suite changes to. ![](/files/2IzKCDX5CF3GqIkUFHJA) Selecting the branch means that the latest version of it with committed changes will be taken as the Test Plan configuration and any other uncommitted test suite changes won't be taken into account. To reset the integration, click on **DISCONNECT** within Settings - Integrations - Setup data sync connection to Github. # GitLab You can easily set up Loadmill to automatically run your tests as part of your continuous integration (CI) workflow. ### Gitlab Integration Setup 1. Go to **Settings** > **Integrations**. 2. Click on the connect to gitlab button. 3. Authorize Loadmill to use your account. 4. Select the project to enable data sync to and click on connect to project. {% embed url="" %} ### Syncing your Test Suite Now that your GitLab integration is all set up head over and sync your Test Suite to your repository. #### To sync your Test Suite to your GitLab repository: 1. Select the Test Suite you wish to connect your repository to. 2. Select the "GitLab Sync" tab. 3. Select or create a new branch from the Sync Actions section. 4. Enter a commit message (example: my first commit). 5. Click on commit. {% embed url="" %} ### Merging Branches Loadmill makes it easy to merge your commits directly through its interface. Simply select the commits you want to merge, and Loadmill will handle the rest. {% embed url="" %} ### Running Test Plans with GitLab You can configure your Test Plan to run with the latest syncs of your test suites repositories. That means your Test Plan will run with the latest Test Suite synced commits. **To configure this option:** 1. Head over to any Test Plan you wish to connect your repository to. 2. Select the "Run Settings" tab of the Test Plan. 3. Select the branch containing the synced Test Suite. Your Test Plan will now run on the latest committed version of the Test Suites. {% hint style="warning" %} Changing your Test Suites locally will not affect the Test Plan run until you commit your changes to GitLab. {% endhint %} {% embed url="" %} ### Syncing from your GitLab Repository 1. Edit your Test Suite file from your GitLab repository. 2. Commit the changes made. 3. Go back to your Test Suite > GitLab Sync on the Loadmill app. 4. Notice the latest commit under the "Branch History" section. 5. Click on "Checkout". 6. See the changes on your Test Suite. {% embed url="" %} # Bitbucket Bitbucket integration is a great way to speed up your development process. By integrating Bitbucket with Loadmill, you can automatically test your code as you make changes. This makes it easy to find and fix bugs quickly and ensures that your code is always up to date. {% hint style="warning" %} Make sure to have an Atlassian account to connect to Bitbucket. {% endhint %} ### Bitbucket Integration Setup 1. Go to **Settings** > **Integrations**. 2. Click on the connect to bitbucket button. 3. Grant Loadmill access to your Bitbucket account. 4. Select your repository. {% hint style="info" %} You might be redirected to the Atlassian login page when connecting Loadmill to Bitbucket. Login using your Atlassian credentials and set up a Bitbucket username to complete the registration. {% endhint %} {% embed url="" %} ### Syncing your Test Suite Now that your Bitbucket integration is all set up head over and sync your Test Suite to your repository. #### To sync your Test Suite to your Bitbucket repository: 1. Select the Test Suite you wish to connect your repository to. 2. Select the "Bitbucket Sync" tab. 3. Select or create a new branch from the Sync Actions section. 4. Enter a commit message (example: my first commit). 5. Click on commit. {% embed url="" %} ### Running Test Plans with Bitbucket You can configure your Test Plan to run with the latest syncs of your test suites repositories. That means your Test Plan will run with the latest Test Suite synced commits. **To configure this option:** 1. Head over to any Test Plan you wish to connect your repository to. 2. Select the "Run Settings" tab of the Test Plan. 3. Select the branch containing the synced Test Suite. Your Test Plan will now run on the latest committed version of the Test Suites. {% hint style="warning" %} Changing your Test Suites locally will not affect the Test Plan run until you commit your changes to Bitbucket. {% endhint %} {% embed url="" %} ### Syncing from your Bitbucket Repository Loadmill can similarly track syncs made directly through Bitbucket. Simply commit a new change from your repository and you'll be able to see the commit change under the Bitbucket Sync tab of your Test Suite. To do that: 1. Edit your Test Suite file from your Bitbucket repository. 2. Commit the changes made. 3. Enter your commit message and create a pull request from this change. 4. Click on "Merge" from your pull request. 5. Go back to your Test Suite > Bitbucket Sync on the Loadmill app. 6. Notice the latest commit under the "Branch History" section. 7. Click on "Checkout". 8. See the changes on your Test Suite. {% embed url="" %} ### Creating/Updating Branch To create a new branch (or update an existing one) from the Loadmill app: 1. Head over to your Test Suite. 2. Select the "Bitbucket Sync" Tab. 3. Under Sync **Actions** > **Commits**. 4. Type in the new branch name (or select an existing one). 5. Commit the changes done on your test suite. 6. Click on "Create Pull Request" Clicking "Create Pull Request" enables you to view the changes in the Test Suite and create a pull request for your synced Bitbucket branch. There, you'll be able to manage and merge when ready. {% embed url="" %} # Jira Loadmill's Jira integration is a great way to automatically create tickets when tests fail or the expected outcome is not what was planned. You can set your own automatic ticket information directly from the Loadmill platform, making it easy to keep track of your testing progress. This integration is a great way to improve your workflow and ensure that your testing is always on track. ### Prerequisites Loadmill supports Jira version 5 and later versions. ### Setting up your Jira integration #### Step 1 - Getting Jira Credentials **Project Key** 1. Login to your Jira account. 2. Navigate to Projects > View all projects You can see the project key on the second column after the project name. {% hint style="info" %} You can also create a new project and gather the project key by following the same steps above. {% endhint %} {% embed url="" %} **Jira Server URL** The Jira server url is the origin url of your Jira account: ****** **API Token** 1. Navigate to Settings > Atlassian account settings. 2. Select Security on the left menu. 3. Click on "Create and manage API tokens". {% embed url="" %} **Email** The email is the account email used for your Jira account. **Setting up Issue Types** Many projects have their own default issue types. Make sure the issue type of your choice is created in the project. You can verify this by going through the following steps: 1. Navigate to Projects > View all projects. 2. Select the "Project Settings" by clicking on the 3 dots \[•••] of the project. 3. Select "Issue Types" from the left menu. Make sure the issue type is present. You can create a new issue type by clicking on the\ "+ Add issue type" button at the end of the issue types menu. {% embed url="" %} #### Step 2 - Configuring Jira on Loadmill 1. Login to your Loadmill Account 2. Navigate to Settings > Integration. 3. Select the CONNECT TO JIRA integration. A window will pop up. 4. Paste all the credentials copied from Step 1. 5. Click connect {% embed url="" %} #### Setting up Fields Loadmill automatically sets the test description on the default fields of the issue type. You can add fields to the ticket layout by dragging and dropping the field type from the fields section to your right. You can also set the type of description you wish to have aside from the default ones assigned and custom your ticket message using the Loadmill dynamic variables. The Loadmill dynamic variables are dynamic variables taken from the test run to help creating a ticket more comfortably without the copy/paste hassle. Those variables are: 1. Test Environment: `${environment}` 2. Test Description: `${test_description}` 3. Test URL: `${test_url}` 4. Run URL: `${run_url}` 5. Failed Assertion: `${failed_assertions}` 6. HTTP Status: `${http_status}` 7. Error Message: `${error_message}` **Quick Example** 1. Login to your Jira account and into your issue type layout. 2. Drag and drop the "Paragraph" component to the ticket layout. 3. Give a title to the "Paragraph" component and click on "Save". *ex: Test Update* 4. Login to your Loadmill account > Go to Settings > Integrations > Add Jira Field. 5. Select the component name your created from step 2 and 3. 6. Give the value a description using the dynamic variables.\ ex: Test Update on `${environment}`: `${test_description}` Below is a more detailed example of using dynamic variables
Test Update env: `${environment}` test url: `${test_url}` test run: `${run_url}` http status: `${http_status}` description: `${test_description}` error message: `${error_message}`
{% embed url="" %} #### Creating the ticket Once your integration is complete you can now run your test cases (or use the current test runs) and open a Jira ticket based on fail criteria. 1. Select and run your test case/flow. 2. Select the failed API requests of your failed test. 3. Click on the report to jira button. This will open a new tab with the ticket information with the dynamic values already set on the ticket description. You can add more information if and when you see fit and when ready click on the "Create" button at the bottom of the Jira ticket to create your ticket issue. {% embed url="" %} # New Relic Easily stream events and view all your data in one centralized dashboard within your New Relic account ### Events Once integrating with your New Relic account Loadmill will publish the following events: * LoadmillTestPlan: at the end of each test plan run * LoadmillTestSuite: at the end of each test suite run * LoadmillTestSuiteFlow: at the end of each test suite flow run * LoadmillFlowStatusChanged: at each flows status change (e.g. from draft to evaluating) ### Integration Setup 1. Go to Loadmill > Settings > Integrations > New Relic 2. Insert the required credentials. 3. Test the connection 4. Click Connect From that point on, Loadmill will start streaming events into your New Relic account. Now you can create you custom alerts and dashboards or use our [predefined dashboard](https://newrelic.com/instant-observability/loadmill)

Loadmill's predefined dashboard

# Slack integration Configure real-time notifications to reach any Slack channel/s you wish in just a few clicks. Slack has more than 12 million daily active users and 156,000 organizations subscribe to the app so it's become essential to get notified about important events via Slack. We introduce our **Loadmill app on Slack** that allows to send notifications per API Test, Load Test, Test Plan run to the Slack channel you wish. Follow the steps below to configure the integration. 1. Navigate to Settings => Integrations. Click CONNECT TO SLACK. ![](/files/VAcoVVvipPR0lbNWt9Aj) 2\. You will be redirected to the page where you need to click **Allow** in order to give relevant permissions to the Loadmill app. 3\. You are almost set, the only thing is left to do is to choose what notifications you would like to get and in which channel. To do that, go to Notifications (the tab above Integrations) => ADD NOTIFICATION => Notification type: Slack => choose channel => select relevant test type and status. 🧠 Tip: the most popular notifications are about Failed Test Plan Runs :) ![](/files/ceZZqxCHk0XoeYZSn2k5) That's it, you did it! 🎉 From now on, you won't miss any important Loadmill event. ![Loadmill notification in Slack](/files/-MeZIRZbvDKhV5CxKnHr) ### Retrieving a Slack Channel ID Occasionally, due to limitations with the Slack API, Loadmill may be unable to automatically retrieve the Slack channel ID. When this occurs, we will need the user to provide the channel ID manually. To find the channel ID: 1. Open the Slack channel where you would like Loadmill to send notifications. 2. Click on the channel name to open the **channel details**. 3. Scroll down and click the copy icon next to the channel ID.
3. Once copied, paste the channel ID into the Channel ID field in Loadmill.
More about our privacy policy can be found [here](https://app.loadmill.com/assets/privacy-policy.pdf) --- [Next Page](/llms-full.txt/1)