# Loadmill Overview Loadmill helps teams create, run, and manage automated tests across mobile apps, web interfaces, APIs, and performance scenarios. Start with the testing approach that matches the behavior you need to validate; the different approaches can be used independently or combined as your coverage grows. ## Droid mobile testing Droid is Loadmill's AI-powered mobile testing agent. Describe a user journey in natural language and Droid operates the app from the visible screen, producing reusable tests that can run from the desktop app, CLI, or CI. {% embed url="" %} [Run your first Droid test](/get-started/getting-started) ## API-first end-to-end testing Capture the API traffic created by real user behavior and turn it into stable end-to-end test flows. API-first tests are useful for validating business processes quickly and repeatedly without relying on the UI for every run. [Start API-first end-to-end testing](/get-started/quick-start-guide) ## Web UI and hybrid testing Use Playwright or the UI Agent to validate browser behavior. Combine UI steps with API requests in the same flow when you want realistic browser coverage with faster, more deterministic setup and validation. [Create your first web UI test](/get-started/quickstart) ## Performance testing Reuse API scenarios as performance tests to validate throughput, concurrency, response times, and system behavior under load. [Run your first performance test](/get-started/getting-started-1) ## Not sure where to begin? [Choose how you want to test](/get-started/choose-testing-approach), or review the [Loadmill POC setup](/poc-guides/loadmill-poc-setup) when planning an evaluation. # Choose How You Want to Test Loadmill supports several testing approaches. Choose the one closest to the behavior you want to validate. You can start with a single approach and combine them later. | Approach | Best for | How tests are created | Where tests run | | ---------------------------- | -------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------- | | Droid mobile testing | Mobile user journeys and changing mobile interfaces | Plain-English instructions saved as reusable `.dcua` tests | Local devices, emulators, iOS simulators, cloud devices, CLI, and CI | | API-first end-to-end testing | Business flows that cross APIs and services | Capture real application traffic or author API requests in the Test Editor | Loadmill-managed or private execution agents | | Web UI and hybrid testing | Browser behavior, cross-browser coverage, and flows that combine API and UI validation | Playwright code, recorded browser actions, or UI Agent instructions | Desktop, private agents, and managed browser execution | | Performance testing | Capacity, throughput, latency, and behavior under load | Create a load scenario or reuse an API flow | Loadmill's distributed load-testing infrastructure | ## Test a mobile app through the visible UI Choose [Droid mobile testing](/droid-mobile-testing/droid-cua) when the mobile experience itself is what you need to validate. Droid acts through the screen instead of depending on selectors and coordinates. ## Validate a business flow at the API layer Choose [API-first end-to-end testing](/api-first-end-to-end-testing/overview) when you want fast, deterministic coverage of the requests and services behind a real user journey. For mobile applications, [Mobile API Testing](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing) captures the API traffic produced by the app. This is different from Droid: the capture happens on a device, but the generated test runs at the API layer. ## Validate browser behavior Choose [web UI and hybrid testing](/web-ui-and-hybrid-testing/overview) when the browser interface matters. Playwright steps validate exact browser interactions, while UI Agent steps handle goal-oriented tasks in natural language. Hybrid flows can use API steps for setup and Playwright for the UI behavior that must be observed. ## Validate capacity and reliability under load Choose [performance testing](/performance-testing/overview) when you need to understand how a system behaves under concurrent traffic or sustained demand. ## Combine approaches A test strategy does not need to use only one approach. For example, a team can use API-first tests for fast regression coverage, Droid for critical mobile journeys, Playwright for browser-specific behavior, and performance tests before high-traffic events. # Create Account Go to [loadmill.com](https://loadmill.com) and sign up for a new account. You can sign up through your Google or GitHub account. To request a guided walkthrough or discuss a POC with our team, [book a demo](https://www.loadmill.com/book-a-demo). For plans and customized pricing, [contact sales](https://www.loadmill.com/contact-sales). # Droid Mobile Testing Quickstart This is the A-to-Z guide for running your first Droid test. It takes you from installing Droid CUA to saving and reviewing the result. By the end, you will have a connected target, a project with shared app context, a focused `.dcua` test, and a result you can review in the desktop app and Loadmill. *** ## Before you start You will need: * A [Loadmill account](https://app.loadmill.com/app/signup). * The [Droid CUA desktop app](https://www.loadmill.com/mobile-testing-agent). * An app or website you are allowed to test. * A target: an Android device or emulator, an iOS simulator on macOS, or Chrome or Edge for web testing. * A stable internet connection. Droid uses Loadmill's cloud service while it runs. * The platform tools required for your target. The setup wizard will check these for you. For the complete platform requirements, see [Setup](/droid-mobile-testing/setup). *** ## Install Droid CUA and sign in Download the desktop app from the [Loadmill Mobile Testing Agent page](https://www.loadmill.com/mobile-testing-agent), or use a direct installer: * [Mac](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA.dmg) * [Mac Intel](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-intel.dmg) * [Windows](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-Setup.exe) Install and open Droid CUA, then sign in with your Loadmill account. *** ## Complete the setup wizard The first time you open Droid CUA, the setup wizard prepares the computer and target. It has four stages: 1. **Welcome** — choose Android, iOS, or Web. 2. **Checks** — verify the tools Droid needs. Resolve any required check that fails, then retry it. 3. **Connect** — select a device, simulator, or browser and run the setup probe. 4. **Success** — continue to the app after Droid confirms the target is ready. ![Droid CUA setup wizard welcome screen](/files/WEbhr9fwsjC4B53NLtG8) You can run the wizard again later from **Help**. For manual setup and troubleshooting, see [Setup](/droid-mobile-testing/setup) and [Setup troubleshooting](/droid-mobile-testing/setup/setup-troubleshooting). *** ## Create your first project Open **Projects**, select **New project**, and choose a folder in the application repository. A Droid project keeps the tests and shared product knowledge for one application together. A simple project can look like this: ``` tests/droid/ ├── context.md ├── login.dcua ├── test-data.md └── .secrets ``` Only `context.md` and the `.dcua` test are needed for a basic first run. Add `test-data.md` when the same journey needs reusable non-secret data, and add `.secrets` when it needs credentials. Keep `.secrets` out of version control. *** ## Start `context.md` while you create the test You do not need to document the entire app before writing a test. Begin with the first journey and add shared knowledge as you discover what the agent needs. From the project, select **Open app context**. If `context.md` does not exist, Droid can create a starter file with sections for the app overview, screens, actions, and business entities. For a first login test, the context might begin with: ```md # Overview This is a mobile banking app. The main navigation appears at the bottom after sign-in. # Screens The home screen is titled "Accounts" and shows the signed-in user's account cards. # Actions After a successful sign-in, wait for the loading indicator to disappear before using the bottom navigation. ``` Keep information in the place where it will be most useful: * Put steps for this journey in the `.dcua` test. * Put reusable product knowledge in `context.md`. * Put non-secret values that vary between runs in `test-data.md`. * Put credentials and sensitive values in `.secrets`. Continue refining the context as you build and run the test. See [Writing Reliable Droid CUA Tests](/droid-mobile-testing/best-practices#build-context-alongside-the-test) for the full approach. *** ## Confirm the target Open **Devices** and select the target prepared by the wizard. * For Android, choose a connected physical device or emulator. * For iOS, choose an installed simulator. iOS simulator testing is available on macOS only. * For web, choose Chrome or Edge and select a persistent or fresh browser session. ![Droid CUA device selection screen](/files/CdchvuXyOCgBB7i8vG7b) If you are testing a mobile build that is not already installed, add the `.apk`, `.ipa`, or `.app` from the **Apps** page before starting the run. *** ## Create your first test Choose a short, stable flow with an obvious result. Signing in and verifying the first screen is a good starting point. The easiest way to draft the journey is **Design Mode**. Select the project so the agent can use its context, then describe the result you want: ``` Sign in with the standard test account and verify that the Accounts screen is visible. ``` ![Droid CUA Design Mode screen](/files/SizinEnBsPcoCZ9xirgq) Design Mode explores the app and creates a draft `.dcua` file. Review the draft before saving it. A useful first test might read: ``` Open the app and sign in with the standard test account. Verify that the "Accounts" screen is visible. ``` For a web test, begin with a known page: ``` navigate: https://example.test/login Sign in with the standard test account. Verify that the dashboard is visible. ``` You can also create and edit `.dcua` files directly. Each non-empty line is an instruction or assertion, and `//` starts a comment for human readers. ![Example Droid CUA test file](/files/pmOtypXv1YJqIPW4Wlfe) *** ## Run the test and watch what happens Start the test from the desktop app and follow the live execution log. Droid sees the current screen, chooses an action, executes it, and checks the updated screen before continuing. If the run diverges from your intent, find the first meaningful difference: * Clarify the instruction when more than one route is reasonable. * Add reusable, non-obvious product knowledge to `context.md`. * Fix the target or setup when the run never reaches the app. * Stop and report a product regression instead of teaching Droid to work around it. Do not rely on repeated retries. One unchanged rerun can confirm a temporary device or network problem, but ambiguous tests should be improved using the evidence from the run. *** ## Review the result Open the completed run and review its screenshots and instruction timeline. A passing verdict is a starting point: also look for repeated typing, reopened screens, resubmitted actions, or other unexplained recovery. Team members can review uploaded desktop and CI results from the [Droid Runs dashboard](/droid-mobile-testing/runs-dashboard). It provides shared filters, trends, statuses, and links to detailed run reports. *** ## Next steps Keep the first version focused on one coherent journey. Once it completes cleanly and its assertions prove the expected result, continue with: * [Writing Reliable Droid CUA Tests](/droid-mobile-testing/best-practices) * [Common Droid Testing Mistakes](/droid-mobile-testing/common-mistakes) * [Web Testing with Droid](/droid-mobile-testing/web-testing) * [CLI](/droid-mobile-testing/cli) and [Running Droid Tests in CI](/droid-mobile-testing/ci) # API-First E2E Quickstart ## Record and Run Your First Test This guide will help you record and run your **first end-to-end test in Loadmill** using the Maker–Checker demo app.\ Time needed: **10–15 minutes** *** ### What You’ll Need * A Loadmill trial account (logged in) * Google Chrome (latest version) * [Loadmill Test Composer Chrome Extension](https://chromewebstore.google.com/detail/gdkmnfehipofdefhpegbgkkocinlaofd) installed and visible in DevTools * 10–15 minutes of focus *** ### Step 1 — Install the Composer 1. Install the **Loadmill Test Composer** from the Chrome Web Store. 2. If your Chrome is managed, ask IT to approve the extension.
*** ### Step 2 — Verify the Recorder 1. Start by navigating to 2. Open Chrome DevTools (**⌘⌥J** on macOS / **Ctrl+Shift+J** on Windows/Linux). 3. Select the **Loadmill Composer** tab. 4. Make sure the **Record** button is available. 5. Set **Filter Settings** to the demo AUT domain. 👉 Only after this step should you continue to the demo AUT.
*** ### About the Demo AUT — Maker–Checker Bank The demo application is a simple **bank transfer system** showing the classic *Maker–Checker approval workflow* (the “four-eyes principle”). It gives you a realistic but lightweight environment to test Loadmill. * **Maker** logs in, creates a new funds transfer (amount, recipient, description). The transfer is saved with status *Pending*. * **Checker** logs in separately, reviews pending transfers, and either approves or rejects them. * **Maker** can then log back in to see the updated status. **Demo URL & Credentials** * URL: * Maker: `maker / maker1234!` * Checker: `checker / checker1234!` *** ### Step 3 — Record the Maker & Checker Flow Together 1. In the Composer, **create a new Test Suite** by typing the name (e.g., *Maker–Checker Demo Flow*) into the suite selector and pressing **Enter**. 2. Click **Record**. 3. Log in as **Maker** and create a new transfer. 4. Confirm the transfer shows *Pending*. 5. **Click Pause** on the recorder (this ensures both roles are captured cleanly). 6. Log out (or open a private window). 7. Log in as **Checker**. 8. Approve the transfer created earlier. 9. Confirm the status updates to *Approved* (or *Rejected*). 10. Stop the recording only after both Maker and Checker steps are complete. 👉 This demonstrates a **real end-to-end business process** across two roles.
*** ### Step 4 — Analyze Requests 1. Click **Analyze Requests** in the Composer. 2. (Optional) Check the **“Show filtered steps”** box to see which calls were removed. You can override AI filtering decisions here. 👉 This produces a **leaner, faster, and more stable test**.
*** ### Step 5 — Create the Test 1. Once analysis is complete, click **Create Test**. 2. You’ll be redirected to the Loadmill platform. 3. Review the test steps generated from your recording.
*** ### Step 6 — Finalize and Enhance 1. **Review & Rename** your test for clarity. 2. **Check Parameters & Correlations:** Notice how inputs like *amount* and *recipient* were auto-parameterized, and dynamic values like *transaction ID* are automatically reused in later steps. * Rename parameters to readable keys if needed. * 👉 This reduces maintenance and makes tests reusable. 3. **Try an AI Refactor:** Click the **AI Refactor** button and type: ``` change recipient name to test-name ``` * 👉 This shows how Loadmill AI can refactor many steps at once, cutting down manual edits. *** ### Step 7 — Debug and Run the Test 1. Toggle **Debug Mode** (gear icon). 2. Add a **breakpoint** on one of the steps. 3. Run the test and watch it stop at the breakpoint. 👉 Debugging lets you inspect intermediate results and quickly validate logic. *** ### Step 8 — Run the Suite 1. Click **Run Suite** to execute the full flow. 2. Verify that the Maker–Checker flow passes end-to-end.
*** ### Step 9 — Run as Load Test 1. Click the **small arrow** next to **Run Suite** and select **Run as Load Test**. 2. Set the test duration to **2 minutes**. 3. Run with **5 users** (trial accounts are limited to small load tests). 👉 This shows how the **same flow** doubles as both a functional and performance test.
*** ### Step 10 — (Optional) Explore AI Configuration Want to make Loadmill’s AI match your team’s language? 1. Go to **Settings → Algorithm → AI Prompts**. 2. Customize how Loadmill: * Describes steps (tone, style, emojis, business terms) * Groups related steps using AI * Explains failures (e.g., known API errors) * Suggests next steps in a flow 👉 Tailoring AI output makes results clearer, saves triage time, and fits better into your workflow. **Explore here:** [AI Prompts Settings](https://app.loadmill.com/app/user/settings/algorithm?tab=ai-prompts) *** ### Troubleshooting * **Can’t find the recorder?** It may be hidden in Chrome’s toolbar. Pin it. * **No requests?** Check that recording is on and filters are correct. * **Managed Chrome?** Ask IT to install the extension. * **Security warning on AUT?** Expected, safe to proceed (browser flags it as a banking app). # Web UI and Playwright Quickstart This guide creates a small browser test with a Playwright step and runs it as a Loadmill flow. ## Before you start You will need: * A Loadmill account. * A test suite with at least one flow. * A Loadmill Desktop App or private agent with UI testing enabled. * A web page that the execution environment can access. ## Add a Playwright step 1. Open a test suite and select a flow. 2. Add a new step and choose **Playwright**. 3. Enter the contents of a Playwright `test()` block. Do not include the `test()` declaration itself. ```ts await page.goto('https://www.loadmill.com'); await expect(page).toHaveTitle(/Loadmill/); ``` 4. Save the flow. 5. Run the test suite and inspect the step result. The Playwright step receives a `page`, browser `context`, and `testInfo`. Suite and flow parameters are available directly as variables. ## Record instead of writing the first version With the Loadmill Desktop App running, use **Record** in the Playwright step. Interact with the browser window that opens, then close it when the journey is complete. Loadmill adds the generated Playwright actions to the step so you can review and refine them. ## Continue * Learn the supported [Playwright step syntax](/test-editor-reference/steps/playwright-step). * Run the flow across [multiple browsers](/web-ui-and-hybrid-testing/cloud-cross-browser-execution). * Combine browser actions with API requests in a [hybrid flow](/web-ui-and-hybrid-testing/hybrid-api-ui-tutorial). * Use [debug mode](/web-ui-and-hybrid-testing/debugging-playwright-tests) to step through a test locally. # Performance Testing Quickstart Running your first load test. The Loadmill's Load Testing module allows users **to replay their API tests at scale** and use real traffic with the ability to target specific countries or go global. ![](/files/ZL4CLj98Vg2C2ksEXe1c) In this quick guide to Loadmill we will learn how to: 1. Create and run a very simple load test in just a few seconds. 2. Create an API test by using our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder) and run it as a load test in Loadmill. 3. Run load tests as DRY RUN or at scale. ## Let's create and run your first Load test together. There are two ways to do that: * Creating and running a load test from scratch in Loadmill. * Recording an API test by using our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder) and running it as a load test in Loadmill. ### Creating and running a load test from scratch in Loadmill: {% embed url="" %} The simplest load test you can create is an HTTP GET request to your web server. {% hint style="warning" %} :brain: Load test may include **100 requests maximum**. {% endhint %} All you need to do is: 1. Click the **NEW TEST** button on the top navigation menu. This will get you to our test editor. 2. Give your new test a short description `My First Test` 3. Copy and paste the website URL into the **URL** text box. In our case, it's going to be `https://loadmill-test-blog.herokuapp.com`. This is a ghost blogging server we are using for testing and demos, so you can use it for tests. 4. Click the **DRY RUN** button as we recommend running your load test for the first time as a [trial test](https://docs.loadmill.com/getting-started/getting-started-1#running-a-trial-test). ![](/files/5FdWjobbC8AlwsfMMvSm) That's it! You have just created and run your first load test in under a minute 🎉 See how to run it at scale [here](https://docs.loadmill.com/getting-started/getting-started-1#running-a-load-test-at-scale). ### Recording an API test by using [our Chrome recorder extension](https://chrome.google.com/webstore/detail/loadmill-recorder/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en) and running it as a load test in Loadmill: 1. Create an API test as explained in the [API-First End-to-End Quickstart](/get-started/quick-start-guide). 2. Go to Loadmill and open the recorded test in your Test Suite. 3. Click on three dots next to the run icon - Click **Convert To Load Test**. 4. To run the load test for the first time, we recommend running it as a trial test. See how to do that below. ### Running a trial test Before we run a massive load test, we usually want to run just a single iteration of the scenario, to make sure everything works as expected. This is what we call a **DRY RUN**. It's also a very easy way to test your API and can be exported to a **Functional Test** which you can later [use in your CI](https://docs.loadmill.com/integrations/npm-modal). When we are creating a new test or extending/maintaining an old one, we like to iterate several times - modifying the test and trying it out - until we get the result we want. To start a trial run, click the **DRY RUN** button at the bottom of the editor. ![](/files/-MNOl2TQuaHSF8rKCw_Z) ### Running a load test at scale Now that we know that our test is configured correctly, we can run it at scale. To do that: 1. Click the **RUN TEST** button at the bottom of the test editor. 2. Set Test Duration (minutes) and **Maximum Concurrent Sessions OR Maximum Requests Per Second** you would like this test flow to run. 3. Click on **Advanced** to set additional parameters. For example, you can set **Ramp-up** **Duration** (1 minute by default) hence a number of concurrent sessions will increase linearly during the ramp-up. 4. Click **RUN**. ![](/files/VufxX0zFCHVBLFemvOtZ) {% hint style="info" %} high-volume:man\_mage: **Note**: At some point you may notice a message stating that your domain is not *verified* - this only means that you may not run high-volume tests before proving ownership of your domain/hostname. You can learn how to do that [**here**](https://docs.loadmill.com/load-testing/setup/domain-verification) (it's actually pretty easy). {% endhint %} As the test progresses we can see the number of concurrent sessions **ramping up** linearly and see how **response time metrics** change over time. ![](/files/-MNOnaYkgPfUE5jbBYw_) In addition, you can **adjust concurrency** **during a test run**. This is a very powerful feature that allows to see how the test performs with dynamic concurrency changes in real time. ![](/files/-MNSNWvW-a2253t7erkD) Loadmill keeps track of the request **error rate** and fails the test if it passes a certain threshold (%50 by default). We can see the response time for each request and are able to drill down and see the error statistics for each one. {% hint style="info" %} :brain: You can easily run your API Test Suite as load tests by clicking on the drop-down icon within the **RUN SUITE** button - **Run as load test**. {% endhint %} ![](/files/7nNWpas6b4MBa1B2tF0Q) Keep exploring the guides to learn how to analyze load test results, set thresholds and more. ### Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # Droid Mobile Testing Overview Droid CUA is Loadmill's AI-powered desktop app for mobile and web testing. It helps you create, run, and manage tests for Android devices and emulators, iOS simulators on macOS, cloud mobile devices, and browser flows. Instead of writing traditional automation code with selectors and locators, you describe the user flow in natural language and let the agent operate the app from the screen, the same way a person would. With Droid CUA you can: * Connect to a real Android device, Android emulator, iOS simulator, cloud mobile device, or browser. * Create mobile and web tests from plain-English instructions. * Watch the agent execute each action with live logs. * Save reusable `.dcua` test scripts. * Run saved tests again from the desktop app or the CLI. * Use local reports and the shared Droid Runs dashboard to review what happened. *** ## How it helps Droid CUA is useful when you want to validate real mobile flows without spending time building and maintaining fragile selector-based scripts. For example, you can ask the agent to sign in, search for an item, add it to the cart, submit a form, or verify that a confirmation screen is visible. Droid CUA captures the device screen, sends the screen and your instructions to the AI model, receives an action such as tap, scroll, type, or wait, and executes that action on the device. The result is a test workflow that is easier to draft, easier to understand, and closer to how a real user interacts with your app. *** ## Platforms ### Android Droid CUA supports physical Android devices and Android emulators. ### iOS Droid CUA supports iOS simulators on macOS. ### Web Droid CUA supports browser testing with installed Chrome or Edge. ### Cloud devices Droid CUA supports cloud mobile runs from the CLI, including Loadmill Cloud and LambdaTest. Cloud runs use real Android or iOS devices and require the provider credentials plus an app build. ### Desktop support * macOS: Android, iOS simulator, and web workflows * Windows: Android and web workflows *** ## Learn more * [Droid Mobile Testing Quickstart](/get-started/getting-started) * [Writing Reliable Droid CUA Tests](/droid-mobile-testing/best-practices) * [Common Droid Testing Mistakes](/droid-mobile-testing/common-mistakes) * [Web Testing with Droid](/droid-mobile-testing/web-testing) * [Droid Runs Dashboard](/droid-mobile-testing/runs-dashboard) * [Setup](/droid-mobile-testing/setup) * [CLI](/droid-mobile-testing/cli) * [Running Droid Tests in CI](/droid-mobile-testing/ci) * [Agent Skill](/droid-mobile-testing/skill) # Setup This page covers the basic setup required before running Droid CUA tests on mobile devices, cloud devices, and browsers. *** ## Desktop app Download and install the Droid CUA desktop app: * [Mac](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA.dmg) * [Mac Intel](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-intel.dmg) * [Windows](https://github.com/loadmill/droid-cua-release/releases/latest/download/Loadmill-Droid-CUA-Setup.exe) After installation, launch Droid CUA and sign in with your Loadmill account. When you open Droid CUA for the first time, the setup wizard helps you choose a platform and run the required checks. ![Droid CUA setup wizard welcome screen](/files/WEbhr9fwsjC4B53NLtG8) *** ## Android setup To test Android apps, Droid CUA needs access to a physical Android device or an Android emulator. Install and configure: * Android Debug Bridge (ADB) * Android Emulator CLI, if you want Droid CUA to launch emulators * USB debugging on physical Android devices To confirm ADB is available, run: ```sh adb version ``` To confirm a device or emulator is visible, run: ```sh adb devices ``` If the device appears in the list, Droid CUA should be able to connect to it from the desktop app. Use the **Devices** page to choose the device source, platform, and target device. ![Droid CUA devices page with a connected Android emulator](/files/CdchvuXyOCgBB7i8vG7b) *** ## iOS simulator setup iOS simulator testing is available on macOS only. Install and configure: * Xcode * iOS Simulator * Appium * XCUITest driver After setup, open Droid CUA and choose an installed simulator from the device picker. Use the platform selector on the **Devices** page to switch between Android, iOS, and web targets. *** ## Web setup To test web flows, install a supported browser: * Google Chrome * Microsoft Edge Droid CUA uses Playwright with the installed browser for web execution. In the CLI, use `--target web` with `--instructions`. ```sh droid-cua --target web --browser chrome --instructions tests/search.dcua ``` Use `--browser edge` to run against Microsoft Edge. For browser session modes, the page-frame boundary, cloud browsers, and guidance on choosing Droid Web, Playwright, or API testing, see [Web Testing with Droid](/droid-mobile-testing/web-testing). *** ## Cloud device setup Droid CUA supports cloud mobile runs from the CLI. ### Loadmill Cloud Loadmill Cloud is a paid feature. Contact [Loadmill Support](mailto:support@loadmill.com) to discuss access and have it enabled for your account before using it. After access is enabled, you need: * A Loadmill API token. * A target platform: `android` or `ios`. * The cloud device name and OS version. * An app build: `.apk` for Android or `.ipa` for iOS. Set your API token in your shell or CI environment: ```sh export LOADMILL_API_TOKEN=your-loadmill-api-token ``` Use `--device-source loadmill-cloud` with the platform, device, app, and saved test when you run the CLI. See [CLI](/droid-mobile-testing/cli) for an example. ### LambdaTest LambdaTest is another cloud-device option available from the CLI. You will need: * `LAMBDATEST_USERNAME` * `LAMBDATEST_ACCESS_KEY` * A target platform, such as `android` or `ios` * A cloud device name and OS version * An app build: `.apk` for Android or `.ipa` for iOS Set the credentials in your shell or CI environment: ```sh export LAMBDATEST_USERNAME=your-username export LAMBDATEST_ACCESS_KEY=your-access-key ``` Then run Droid CUA with `--device-source lambdatest`. *** ## Project setup Create a Droid CUA project and choose: * A tests folder for saved `.dcua` files. * A results folder for run history, reports, and logs. Keeping `.dcua` files in your app repository makes it easier to review them, commit them, and run the same tests later with the CLI. If your test needs a mobile app build, add it from the **Apps** page. Droid CUA can manage `.apk`, `.ipa`, and `.app` builds. *** ## App context For real app testing, create a short `context.md` file next to your tests. You do not need to complete it before creating the first test; build it alongside the journey as you discover reusable product knowledge. Use it to explain details the agent cannot infer from the screen alone, such as: * What the app does. * Which test accounts to use. * Important screen names and navigation paths. * Common success messages. * Any confusing or similar-looking buttons. Good context makes tests more reliable because the agent has the same product knowledge a teammate would need before testing the app. See the [Droid Mobile Testing Quickstart](/get-started/getting-started#start-contextmd-while-you-create-the-test) for a first-project example and [Writing Reliable Droid CUA Tests](/droid-mobile-testing/best-practices#build-context-alongside-the-test) for guidance on what belongs in each project file. *** ## Setup troubleshooting If setup does not work as expected, see [Setup troubleshooting](/droid-mobile-testing/setup/setup-troubleshooting). # Setup Troubleshooting Use this page when Droid CUA cannot connect to a device, cannot find a required tool, or fails before the test starts. *** ## The app cannot find ADB Make sure Android Debug Bridge is installed and available in your system PATH. Run: ```sh adb version ``` If the command is not found, install Android Platform Tools and restart Droid CUA after updating your PATH. *** ## Android device does not appear Run: ```sh adb devices ``` If the device is not listed: * Confirm the device is connected over USB. * Enable USB debugging on the device. * Approve the debugging prompt on the device screen. * Try reconnecting the cable or restarting ADB. If the device is listed as `unauthorized`, unlock the device and approve the USB debugging prompt. *** ## Android emulator does not start Confirm that Android Emulator CLI is installed and that the emulator can be started outside Droid CUA. If the emulator starts manually but not from Droid CUA, restart the desktop app and select the emulator again from the device picker. *** ## iOS simulator does not appear iOS simulator support requires macOS. Confirm that: * Xcode is installed. * The simulator exists in Xcode. * Appium is installed. * The XCUITest driver is installed. After installing missing tools, restart Droid CUA and open the device picker again. *** ## Web browser is not found For web tests, install Google Chrome or Microsoft Edge and restart Droid CUA. For CLI runs, confirm the selected browser is supported: ```sh droid-cua --target web --browser chrome --instructions tests/search.dcua ``` Use `--browser edge` if you want to run against Microsoft Edge. *** ## LambdaTest cloud device run fails before connecting Check that the required LambdaTest inputs are set: * `LAMBDATEST_USERNAME` * `LAMBDATEST_ACCESS_KEY` * `--device-source lambdatest` * `--platform android` or `--platform ios` * `--device-name` * `--os-version` * `--app` * `--instructions` Android cloud runs require an `.apk` file. iOS cloud runs require an `.ipa` file. *** ## The test starts but the agent gets confused This is usually not a setup problem. Improve the test input instead: * Use the exact labels shown in the app. * Keep the first test short. * Add or improve `context.md`. * Add an assertion that describes something visible on screen. * Split long flows into smaller tests. *** ## CLI run fails before execution Check the most common inputs: * `--instructions` points to an existing `.dcua` file or folder. * `--avd` matches the selected Android device, emulator, or iOS simulator. * `--platform ios` is used for iOS simulator runs. * `--target web` is used for browser runs. * `--config` points to valid JSON. Run with `--debug` to create detailed artifacts for troubleshooting: ```sh droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` # Best Practices If you are arriving from Playwright, Appium, Selenium, or another locator-based automation framework, the most important thing to understand is that Droid CUA is not a faster way to write the same kind of test. It is a different layer of automation. Traditional frameworks run scripts against implementation-level targets such as selectors, element IDs, and DOM structures. Droid CUA gives a computer-using agent a goal, a screen, and product-specific guidance, then lets it operate the product the way a person would. That shift changes what a test is, how you write one, and what makes it reliable. This guide focuses on that craft. Installation, target setup, and command-line options are covered separately in [Setup](/droid-mobile-testing/setup) and [CLI](/droid-mobile-testing/cli). *** ## Brief a teammate, not a robot The most useful mental model for Droid CUA is that you are handing a task to a smart new teammate. They can read your instruction, see the current screen, and use the knowledge supplied by the project. They do not automatically know your product, environment conventions, account roles, or team shorthand. If that teammate would reasonably ask a follow-up question before acting, the instruction is not ready yet. This model tells you when an instruction is too vague, when it is too detailed, what belongs in the test, what belongs in project context, and why assertions must describe something the agent can actually observe. ### Find the useful middle There are roughly three levels at which you can write a Droid instruction. At the highest level is the complete goal: *buy a product and verify the order is confirmed*. At the lowest level are exact interactions: tap this field, type this value, open this menu, tap this button. Between them are short-term goals: *search for the product*, *add it to the cart*, *complete checkout*, *verify the confirmation*. Most reliable tests live in that middle layer. A complete goal gives the agent a great deal of freedom. That can be useful in Design Mode or exploratory work, but repeatable execution may force the agent to rediscover the route every time or let it choose a path you did not intend to test. Exact interaction instructions have the opposite problem: they couple the test to incidental details of the current layout and turn natural language into a brittle script. Short-term goals preserve intent while giving the journey enough shape to stay focused and repeatable. Move higher when the route truly does not matter. Move lower when the screen is ambiguous, when an intermediate state is part of the behavior under test, or when run evidence shows that the agent needs more help. You are not choosing one level for the whole test; you are giving each part of the journey the amount of guidance it needs. ### Precision still matters Natural language is the interface, but precision is still the discipline. Instead of precise selectors, you need an unambiguous goal, the right product language, and an observable result. Use an exact control label when it distinguishes one action from another, but do not prescribe every visible control when a short-term goal is already clear. *Check that checkout works* is not precise. *Complete checkout and verify the Select address screen is visible* is. Too high-level: *Order something from the store.* Balanced: *Search for "milk", add "Amul Toned Milk 1L" to the cart, and verify it appears in the cart.* Too detailed: *Tap the search field, type milk, press Enter, tap the first card, and tap the button below the price.* The detailed form is justified only when those interactions are important or the balanced instruction has proved unreliable. *** ## Understand what the agent sees The agent experiences the interface as a sequence of screenshots rather than continuous motion. At each turn it receives the current screen, the current instruction, relevant project knowledge, and the working history Droid maintains for the run. It chooses an action, Droid executes it, and a new screenshot comes back. This explains why the agent may take another screenshot to confirm a state change, why something remembered from earlier may no longer be actionable, and why it must find a control again after it scrolls out of view. Animations, short-lived toast messages, loading transitions, and temporary empty states can happen between screenshots. Prefer assertions about the durable state that follows them. An explicit wait can be appropriate when the product genuinely needs time, but it should explain a known transition rather than hide an unexplained failure. ### Visible state is evidence The agent works through the interactions available on the current target: taps or clicks, typing, scrolling, key presses, and waits. A visible state change is evidence that a mutating action such as submit, create, approve, reject, login, logout, or send succeeded. If the only immediate confirmation that a form was submitted is that its fields clear and the button returns to its idle state, describe that stable product behavior in context. Then add an assertion for the business result the test actually cares about. Repeated actions are worth noticing even when the run eventually passes. Reopening a screen, typing the same value again, or resubmitting a form may be harmless recovery, but it may also reveal an ambiguous instruction, unstable product behavior, or a bug the agent managed to work around. A pass tells you where to begin reviewing the evidence; it does not make that evidence irrelevant. ### Assertions are the verification contract An assertion is its own instruction where the agent's job changes from acting to verifying. It examines the screen, decides whether the expected result is visible, and returns a clear pass or fail verdict. You can write an explicit `assert:` line or simply say *verify that...* in natural language. Strong assertions name a badge, label, screen title, status, value, or message. *Verify it worked* is weak. *Verify the cart badge shows "1"* names an observable result. Add assertions after important state changes, at the end of a business flow, at meaningful checkpoints in a longer journey, and after expected error paths. *** ## Give project knowledge a clear home A useful Droid project separates the journey, the team's shared knowledge, the data for a run, and sensitive values instead of forcing all of them into one file. Think of the `.dcua` test as the assignment you hand to a teammate. `context.md` is the product handbook they already know. `test-data.md` is a run sheet containing non-secret people, products, roles, and scenario combinations. `.secrets` is the locked envelope containing credentials or other sensitive values. `.env` contains the infrastructure credentials Droid itself needs. A checked-in Droid config records the shared execution settings that make headless runs reproducible. Each non-empty line in a `.dcua` file is one instruction or assertion, and `//` starts a comment for human readers. One instruction is not the same as one tap. Keep the journey readable from top to bottom, state the starting condition when the environment does not guarantee it, and keep secret values out of the file. Use the names of keys from `.secrets`, such as `USER_EMAIL` and `USER_PASSWORD`, so the agent can request those values without placing them in its prompt or the run log. Keep both `.secrets` and `.env` out of version control. A project should normally have one shared context file and one shared Droid config. Reuse them across the project's tests instead of creating a special context or config for every scenario. ### Write context for a new teammate `context.md` should answer one question: *what would a capable new teammate need to know before testing this product for the first time?* Start with an overview of the product, its main screens, and its top-level navigation. Explain the roles and what each one can do without including passwords or other secrets. Add non-obvious navigation, environment banners, product terms that are easy to confuse, and recurring success signals that help the agent recognize what happened. Visual cues deserve their own line. If new items appear at the bottom of a separately scrolling list, say so. If a list refreshes in place rather than opening a new screen, say so. If two controls look nearly identical, explain how to tell them apart. Context is not a copy of the test. Avoid scenario-specific steps, values that belong in `test-data.md`, UI text the agent can already see, speculative workarounds, and stale information. A context file that silently lies about the product is worse than no context file at all. ### Build context alongside the test You do not need to describe the entire application before writing the first test. Start with a focused journey, create the test and `context.md` together, and add knowledge when the work reveals that the agent needs it. When you encounter a missing detail, decide where it belongs: * If it is part of only this journey, add it to the `.dcua` test. * If it is stable product knowledge that other tests or a new teammate would reuse, add it to `context.md`. * If it changes between runs, add it to `test-data.md`. * If it is sensitive, add it to `.secrets` and refer to the secret by name. For example, the test can say *create a transfer as maker and approve it as checker*. The context can explain what the maker and checker roles mean, where pending transfers appear, and that an approved transfer disappears from the pending list. Those facts are useful beyond one scenario; the exact transfer journey is not. After a failure, add context only when the missing knowledge is true, stable, and reusable. Do not encode a product bug, temporary state, or one-off workaround as if it were normal application behavior. This iterative approach keeps context useful without turning it into another test script. *** ## Design once, then make the journey repeatable Design Mode turns a testing idea into a draft test. Bind the session to the relevant project so the agent can use its context during exploration, then describe the business goal rather than prescribing the route. *Create a transfer as maker, approve it as checker, and verify the approved transaction disappears from the pending list* is a useful Design Mode prompt. A detailed tap-by-tap script is not; the point of Design Mode is to discover the route for you. The generated `.dcua` file is a first draft. Review whether it uses short-term goals, remove accidental exploration, clarify ambiguous transitions, and strengthen the assertions before treating it as ready. Design Mode accelerates the craft of writing a test; it does not replace it. ### Close the loop with Jira A Jira ticket can be the starting point for a Design Mode session, but it is rarely an executable test as written. Tickets often omit the account and environment assumptions, concrete data, visible labels, and explicit success criteria the agent needs. Treat the generated journey as a draft that still requires the same review as any other test. The loop also works in the other direction. After a failed run, Droid can prepare a Jira ticket containing the failing instruction, recent execution context, target details, and a summary of what went wrong. Review that draft before filing it. The goal is to preserve useful evidence and save typing, not to turn every failed run into an automatic bug report. ### Prepare state through APIs Many end-to-end tests need data that is slow or awkward to create through the interface. An insurance claim may require an active policy. An approval test may need a pending transaction. Driving through unrelated screens to create that state makes the test longer and less reliable without adding useful coverage. A `.dcua` test can invoke an existing Loadmill flow before continuing through the UI. Write `loadmill:` followed by a distinctive description of the saved flow, or use a natural instruction such as *Run the Loadmill create premium customer flow with country=US.* Droid finds the best matching flow, waits for it to finish, and makes relevant returned information available to later instructions. Refer to that information by meaning, as in *sign in as the customer created by the Loadmill flow*, rather than inventing placeholder syntax. Use an API flow to establish the state the scenario needs, then use Droid to verify the behavior the user experiences. ### Run one journey under different conditions Data-driven testing in Droid does not require turning a readable journey into a parameterized program. A normal run executes the `.dcua` test once, exactly as written. A Scenario Run combines the unchanged test with a plain-language request such as *run checkout with each buyer account and each in-stock product*. Reusable, non-secret accounts, products, roles, regions, and scenario combinations can live in `test-data.md`. Droid previews the cases it plans to run so you can check that it understood the request and notice an unexpectedly large fan-out. Each case receives only the relevant data for that run. The generated cases are execution results, not new test files, and the original journey stays free of `{email}` or `{{product}}` placeholders. Use a normal run when you want one known journey. Use a Scenario Run when the variation is the point of the exercise. *** ## Start with Smart, then earn the switch to Pulse Begin with Loadmill Smart while you are designing and stabilizing a new test. Smart gives a new or difficult journey the strongest starting point. Once the test is clear, repeatable, and no longer relying on recovery, compare the same journey with Loadmill Pulse. Pulse is faster and more efficient for routine execution. Keep the change only if the test remains stable on the same target and application state. Changing models is a comparison, not a cure for an ambiguous instruction, stale context, target problem, or product regression. Beacon is experimental and is better treated as an intentional evaluation. *** ## Know the local web boundary Droid can connect to a locally installed Chrome or Edge browser. The same instruction-writing principles apply, but the agent operates inside the web page frame. It can interact with the site and navigate as part of the test. It cannot use the browser's address bar, settings, bookmarks, extensions, or other browser chrome. Write web tests as journeys through the application, not as instructions for operating the browser itself. For setup and execution commands, see [Setup](/droid-mobile-testing/setup#web-setup) and [CLI](/droid-mobile-testing/cli#run-a-saved-web-test). For a complete local and cloud browser workflow, see [Web Testing with Droid](/droid-mobile-testing/web-testing). *** ## Keep tests focused and independent Short tests are easier to understand and maintain, but splitting is safe only when every resulting test can run independently. Separate unrelated flows such as signing in, editing a profile, and checking out when each has its own setup and observable result. Keep a maker-create/checker-approve journey together when the second half depends on state created by the first. Droid does not guarantee that separate test files will run consecutively or share state, so ordered files are not a substitute for one independently runnable scenario. If setup dominates the behavior under test, consider preparing that state through a Loadmill flow rather than breaking one dependent journey into several files. *** ## Classify a failure before editing the test Find the first place where the run diverged from your intent. Everything after it is usually downstream. A missing credential or dependency is setup. An unavailable device or browser is a target problem. Multiple reasonable interpretations point to the test or stale project knowledge. A temporary provider or network problem may justify one unchanged rerun. A crash, product error, or consistent violation of a clear assertion is an application regression and should stop the healing loop. Do not turn broken product behavior into new context, add a broad wait to cover it, or weaken the assertion until it passes. Fix the smallest evidence-supported set of inputs. If the agent selected the wrong account because two similar accounts were visible, the instruction, context, or test data may need to disambiguate them. If it completed the action but the test did not prove the result, strengthen the assertion. Rewriting the entire journey should be the last thing you try. ### A pass is a baseline, not automatic readiness Review whether the agent repeated actions, reopened screens, recovered from losing state, or took far longer than expected. Those behaviors may not change the final verdict, but they affect whether the test is trustworthy in CI. A straightforward journey may need only one clean representative pass after refinement. A flaky, state-sensitive, or materially changed test deserves another unchanged run. Call it ready when its assertions pass, its important behavior is still covered, and no unexplained recovery loop remains. *** ## Treat the evidence as part of the test A run produces more than a green or red result. Screenshots show what the agent could see at important moments. The HTML report reconstructs the instruction and action timeline in a form you can share. A replay video makes the journey easy to review when motion or timing matters. Explain Mode provides short explanations for non-obvious decisions, while debug artifacts provide a deeper record for diagnosis. Use the lightest artifact that answers the question. The report is the normal handoff. Screenshots are the fastest way to locate a visible divergence. Video helps when timing between frames matters. Device, Appium, or browser logs help separate product behavior from platform or automation behavior. Artifacts should support the conclusion, not force another person to discover it for themselves. ### Working through a coding agent The [Droid CUA Agent Skill](/droid-mobile-testing/skill) lets coding agents such as Codex, Cursor, and Claude Code inspect a project, author or improve a `.dcua` test, run it through the CLI, and return the evidence in the same conversation where development work is happening. Brief the coding agent on the behavior you want checked. It should inspect the existing project conventions, propose the test and target, and ask for confirmation before starting a run. During execution it can show safe progress screenshots. Afterward it should give you the result, the first meaningful problem it found, what changed during any improvement loop, and a link to the final report. For a new, flaky, or slow test, ask the coding agent to stabilize or optimize it rather than merely make it pass once. That authorizes a baseline followed by evidence-driven edits and representative reruns on the same target. It does not authorize changing application code, weakening assertions, or silently switching providers or models. *** ## What this enables Teams that adopt these practices stop writing scripts and start briefing a teammate. They use short-term goals to give a journey shape without coupling it to every interaction. They keep product knowledge, test data, and secrets in the right places. They use Design Mode to discover a route, API flows to prepare state, Scenario Runs to vary conditions, and coding agents to bring the workflow into development. Most importantly, they treat reports and recovery behavior as evidence rather than stopping at a green verdict. Reliability comes from the quality of the brief, the clarity of its assertions, and the discipline to examine what actually happened. When you are ready to put these practices into use, continue with the [Droid Mobile Testing Quickstart](/get-started/getting-started), review the [Common Droid Testing Mistakes](/droid-mobile-testing/common-mistakes), or move on to [Setup](/droid-mobile-testing/setup) and the [CLI reference](/droid-mobile-testing/cli). # Common Testing Mistakes Droid tests are most effective when they validate a focused user journey through the visible product. Many unreliable tests begin by asking Droid to do work that belongs in a different test, a shared project file, or another testing layer. Use this page as a quick review when a test is slow, difficult to understand, or inconsistent between runs. For the reasoning behind these recommendations, see [Writing Reliable Droid CUA Tests](/droid-mobile-testing/best-practices). *** ## Putting the whole journey in one instruction An instruction such as *buy a product, apply a coupon, change the delivery address, pay, and verify the order* gives the agent too many intermediate decisions without useful checkpoints. Break the journey into short-term goals and verify the important state changes: ``` Search for "Amul Toned Milk 1L" and add it to the cart. Verify that the cart badge shows "1". Open the cart and continue to checkout. Apply the coupon "WELCOME10". Verify that the discount appears in the order summary. Complete checkout and verify that the confirmation screen is visible. ``` One instruction does not need to equal one tap. It should express one meaningful piece of the journey that the agent can complete and evaluate before moving on. *** ## Putting every business path in one test A long test that signs in, edits a profile, searches, checks out, changes settings, and signs out is difficult to diagnose and expensive to repeat. Split unrelated behaviors into independently runnable tests. Keep dependent actions together when they form one business journey. A maker creating a transaction and a checker approving that same transaction may belong in one test because the second part depends on the first. Separate files should not depend on running in a particular order. *** ## Testing backend permutations through the UI Before adding many variations, ask what you are trying to prove. Use Droid when the visible client behavior matters: the user can complete checkout, the correct validation appears, or a role sees the correct controls. Use a Droid Scenario Run when a small set of meaningful user-facing conditions should exercise the same journey. Use Loadmill API testing when the real goal is to validate a business rule across dozens or hundreds of inputs. Repeating the same frontend journey for every country, price, permission combination, or response shape is slower and provides little additional UI evidence. | What needs validation | Best starting point | | ------------------------------------------------------- | ------------------------------------- | | A user can complete a visible mobile or web journey | Droid | | The same journey under a few meaningful user conditions | Droid Scenario Run | | Many data combinations or backend rules | Loadmill API testing | | Data or account state needed before a UI journey | A Loadmill API flow followed by Droid | The strongest test may combine the layers: prepare exact state through an API flow, then let Droid validate what the user experiences. *** ## Creating all setup through the interface Driving through unrelated screens to create policies, accounts, transactions, or catalog data makes the test longer without improving coverage of the behavior under test. When setup is not part of the scenario, create it with an existing Loadmill flow and continue through the UI with the returned state. Use the interface for setup only when that setup experience is itself what the test needs to validate. *** ## Rewriting natural language as a brittle script Detailed instructions such as *tap the third icon, click the first card, then press the button below the price* depend on the current layout and hide the purpose of the test. Prefer a short-term goal such as *search for the product and add the 1L package to the cart*. Add an exact label or interaction only when it disambiguates the route or when that interaction is part of the behavior being tested. *** ## Using vague assertions *Verify it worked* does not define what success looks like. Name the stable, visible evidence: * Verify that the order status is **Confirmed**. * Verify that the cart badge shows **1**. * Verify that the **Select address** screen is visible. Prefer durable results over animations, loading states, and short-lived toast messages. *** ## Turning `context.md` into another test script Project context is shared product knowledge, not a hidden list of steps for one scenario. Do not copy the current test into `context.md`, add one-off test values, or document temporary workarounds just to make a failing run pass. Keep journey-specific instructions in the `.dcua` test, reusable non-secret variation data in `test-data.md`, and credentials in `.secrets`. Add something to `context.md` when another test or a new teammate would benefit from knowing it too. *** ## Overloading context with irrelevant information More context does not automatically make a test more effective. Unrelated details, exhaustive screen descriptions, implementation history, and stale workarounds make it harder for Droid to identify the information that matters. Aim for the minimum context that produces the maximum efficiency. Include stable product terminology, important navigation conventions, non-obvious interactions, and reusable constraints that help Droid make better decisions. Leave out anything that does not affect how the current tests should understand or use the product. Build context as you create and refine tests. Add a detail when a run shows that Droid needs it, then keep it only if it is likely to help again. *** ## Rerunning without learning from the failure An unchanged rerun is reasonable for a clearly temporary device, provider, or network problem. Repeated retries are not a substitute for diagnosing an ambiguous instruction, stale context, unavailable target, or product regression. Find the first meaningful divergence, classify it, and change the smallest evidence-supported input. Do not weaken an assertion or document broken product behavior as context merely to produce a passing result. *** ## Treating one pass as proof of readiness A passing run may still contain repeated typing, reopening screens, resubmitting actions, or unexplained recovery. Review the report and make sure the path was direct enough to trust in routine execution. Start a new test with Loadmill Smart. Once the journey is stable and its evidence is clean, compare it with Loadmill Pulse for faster routine runs. *** ## A quick review before saving Before calling a test ready, check that: * Each instruction expresses a clear short-term goal. * Assertions name visible evidence. * The test covers one coherent, independently runnable journey. * Backend setup and large data matrices are not being forced through the UI. * Shared product knowledge, run data, and secrets are stored in the right files, without irrelevant context. * A representative run completed without unexplained recovery. # Web Testing with Droid Droid is primarily designed for mobile testing, but the same goal-oriented approach can operate a web application in Chrome or Edge. Web testing is useful when you want the agent to follow a realistic browser journey without maintaining selectors for every interaction. Use Droid Web for flows where the visible experience and the route through the product matter. Use [Playwright](/web-ui-and-hybrid-testing/overview) when you need precise DOM-level control, and use [API-first testing](/api-first-end-to-end-testing/overview) when the browser is not part of what you need to validate. *** ## Before you start For local execution you need: * The Droid CUA desktop app. * Google Chrome or Microsoft Edge installed on the same computer. * A Droid project containing your `.dcua` tests and optional `context.md`. The setup wizard can verify that Droid detects the browser and can launch it through Playwright. Select **Web** on the wizard's Welcome step, complete the system checks, and run the browser setup probe. For more information about the wizard, see the [Droid Mobile Testing Quickstart](/get-started/getting-started#complete-the-setup-wizard). *** ## Connect a local browser Open **Devices** in the desktop app and: 1. Select **Web** as the platform. 2. Select **Local** as the source. 3. Choose an installed Chrome or Edge browser. 4. Choose a session mode. 5. Connect the browser. | Session mode | Behavior | Good for | | -------------- | ------------------------------------------------- | ----------------------------------------------------------------------- | | **Persistent** | Reuses the Droid browser profile between sessions | Local development and flows that benefit from retained login state | | **Fresh** | Starts with a clean temporary profile | Independent tests, authentication coverage, and repeatable CI-like runs | Persistent mode does not remove the need to make the test's starting state clear. If login state is part of the behavior under test, use a fresh session and perform the login in the journey. Accounts with Loadmill Cloud web access can also choose a hosted Chrome or Firefox browser and a browser version from the Devices page. Cloud availability depends on the account plan. *** ## Write a first web test Start a saved web test with a `navigate:` instruction. This changes the page directly without asking the agent to operate the browser address bar. ``` navigate: https://bank-demo.loadmill.com Sign in as the maker account described in the project context. Open the transfers page and create a transfer for $25. Verify that the new transfer appears with the status "Pending". ``` The same instruction-writing guidance used for mobile applies to web tests: describe short-term goals, use visible product language, and define observable assertions. ### Understand the web boundary The agent is active within the web page frame. It can interact with the site, follow links, type, scroll, and navigate as part of the test. It cannot use the browser's address bar, settings, bookmarks, extensions, or other browser chrome. Use `navigate:` for a known starting URL and write the rest of the test as a journey through the application itself. *** ## Add web-specific context Most shared product knowledge can remain in the same `context.md` used by mobile tests. Add web-specific guidance only when the browser experience differs in a non-obvious way, for example: * Navigation moves from a bottom bar on mobile to a left sidebar on web. * A menu opens only after hovering or clicking an account control. * A table scrolls independently from the page. * The responsive layout changes the names or locations of important controls. Do not document browser settings or address-bar operations in project context because Droid cannot use those surfaces during the test. *** ## Run a saved web test from the CLI Use `--target web`, choose the installed browser, and point `--instructions` at the test: ```sh droid-cua \ --target web \ --browser chrome \ --session-mode fresh \ --instructions tests/transfer.dcua ``` Supported local browser values are `chrome` and `edge`. The CLI runs saved web tests; it does not provide an interactive web shell. See the [CLI reference](/droid-mobile-testing/cli#run-a-saved-web-test) for additional options. *** ## Choose the right web testing layer Start with Droid Web when the test should read like a user journey and adapt to the visible interface. Choose another layer when the test's purpose is different: | Need | Recommended approach | | ------------------------------------------------------------------- | ----------------------------------- | | Validate a realistic goal through the visible browser experience | Droid Web | | Combine an adaptive UI journey with API state preparation | Loadmill flow followed by Droid Web | | Assert exact DOM behavior or use precise locator-level interactions | Playwright | | Exercise many backend inputs without validating the page | Loadmill API testing | For more guidance on scope and data variation, see [Common Droid Testing Mistakes](/droid-mobile-testing/common-mistakes#testing-backend-permutations-through-the-ui). # Droid Runs Dashboard The Droid Runs dashboard brings desktop and CI execution results into the Loadmill web application. Use it to review activity across projects, understand pass rates, find a particular run, and open the detailed report without access to the computer that launched the test. Open [Droid Runs in Loadmill](https://app.loadmill.com/app/api-tests/droid-runs), or sign in to Loadmill and select **Droid Runs** from the main navigation. *** ## Understand the dashboard The top of the page summarizes the selected period: * **Success Rate** shows how many runs passed out of the total. * **Droid Runs Per Day** shows passed, failed, and stopped runs over time, with the daily average for comparison. * The time-period selector changes how many recent days are included. * **Refresh** reloads the summary and run list. ![Droid Runs dashboard](/files/jxNg63fbf5XyNK9h71N4) Your dashboard shows the projects, platforms, users, and results from your Loadmill workspace. The dashboard uses the same filters for the summary and the table, so the charts describe the runs currently in view. *** ## Filter the run history Open **Filters** to narrow the page by: * **Project** — the Droid project that owns the test. * **Source** — Desktop or CI. * **Platform** — the mobile or web platform reported by the run. Filters are useful when comparing local stabilization runs with CI execution, reviewing one application's results, or investigating a platform-specific problem. Clear the filters to return to the complete team view. *** ## Read the runs table Each row describes a test or project run: | Column | Meaning | | --------------- | ------------------------------------------------------------ | | **Test** | The `.dcua` test name, or the project name for a project run | | **Project** | The Droid project that produced the result | | **Source** | Whether the run came from the desktop app or CI | | **Platform** | The target platform used for the run | | **#Cases** | The number of scenario cases included, when applicable | | **Launched by** | The Loadmill user associated with the run | | **Launched on** | When the run completed | | **Duration** | Total execution time | | **Status** | Passed, failed, or stopped | A folder icon identifies a project run, while a document icon identifies a single test run. *** ## Open a run report Select a row with an available report to open the complete Droid execution report. The report reconstructs the instruction and action timeline and includes the screenshots captured during the run. Use the report to find the first meaningful divergence rather than looking only at the final status. A failed assertion may be the consequence of an earlier navigation or state problem. A passing run can still reveal repeated actions or recovery that should be cleaned up before the test moves to CI. Older run reports may expire. When a report is no longer retained, the dashboard can still preserve the run summary while the detailed report is unavailable. For guidance on interpreting execution evidence, see [Writing Reliable Droid CUA Tests](/droid-mobile-testing/best-practices#treat-the-evidence-as-part-of-the-test). *** ## Use the dashboard during test development A practical workflow is: 1. Create and stabilize the test in the Droid desktop app. 2. Review the local report after each meaningful iteration. 3. Move the stable test to the CLI or CI. 4. Use Droid Runs to compare ongoing results across projects, targets, and execution sources. 5. Open the detailed report when a trend or individual failure needs investigation. The desktop app remains the primary place to author and debug a test. The web dashboard gives the team a shared view of what ran and what happened afterward. # CLI The Droid CUA desktop app is the main place to create and debug tests. The CLI lets you run saved `.dcua` tests from a terminal, CI pipeline, or other automation workflow across mobile, cloud device, and web targets. *** ## Install the CLI Install the package globally: ```sh npm install -g @loadmill/droid-cua ``` Then run: ```sh droid-cua ``` *** ## Choose an LLM provider and CUA model ### Use Loadmill (default) The CLI uses Loadmill by default, with `loadmill-smart` as the recommended, most robust model. You do not need to pass `--llm-provider` or `--cua-model` for a standard Loadmill run. ```sh droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua ``` ### Use Loadmill Pulse for faster execution For a faster, lower-cost Loadmill run, use `loadmill-pulse`. Loadmill is already the default provider, so you only need to set the model: ```sh droid-cua \ --cua-model loadmill-pulse \ --avd adb:emulator-5554 \ --instructions tests/login.dcua ``` `loadmill-beacon` is also available as an experimental Loadmill model. ### Use your own OpenAI API key To use OpenAI instead of Loadmill, set `OPENAI_API_KEY` and explicitly pass `--llm-provider openai`. Passing an OpenAI model to `--cua-model` alone does not change the provider. Choose `gpt-5.6-terra`, `gpt-5.6-luna`, or `gpt-5.4`. ```sh OPENAI_API_KEY=your-openai-api-key \ droid-cua \ --llm-provider openai \ --cua-model gpt-5.4 \ --avd adb:emulator-5554 \ --instructions tests/login.dcua ``` For CI or a config file, use `"llmProviderMode": "openai"` to select OpenAI. You can also set `DROID_CUA_LLM_PROVIDER=openai` as the default provider for a shell environment. *** ## Run a saved Android test Use `--instructions` to point at a `.dcua` file and `--avd` to select the target device or emulator. ```sh droid-cua --avd adb:emulator-5554 --instructions tests/login.dcua ``` You can also point `--instructions` at a folder. Droid CUA will run the `.dcua` files in that folder. ```sh droid-cua --avd adb:emulator-5554 --instructions tests ``` *** ## Run a saved iOS simulator test On macOS, use `--platform ios` and pass the simulator name: ```sh droid-cua --platform ios --avd "iPhone 16" --instructions tests/login.dcua ``` *** ## Run a saved web test Use `--target web` to run a `.dcua` test against an installed browser. ```sh droid-cua --target web --browser chrome --instructions tests/search.dcua ``` Supported browser values are `chrome` and `edge`. Local web runs use a persistent browser profile by default. Pass `--session-mode fresh` for an independent temporary profile: ```sh droid-cua \ --target web \ --browser chrome \ --session-mode fresh \ --instructions tests/search.dcua ``` See [Web Testing with Droid](/droid-mobile-testing/web-testing) for browser setup, saved-test examples, and the boundary between the web page and browser controls. *** ## Run on a LambdaTest cloud device Set your LambdaTest credentials: ```sh export LAMBDATEST_USERNAME=your-username export LAMBDATEST_ACCESS_KEY=your-access-key ``` Then run with `--device-source lambdatest` and pass the target device details. ```sh droid-cua \ --device-source lambdatest \ --platform android \ --device-name "Galaxy S24" \ --os-version "14" \ --app ./app-debug.apk \ --instructions tests/login.dcua ``` Android cloud runs require an `.apk` file. iOS cloud runs require an `.ipa` file. *** ## Run on Loadmill Cloud Loadmill Cloud lets you run saved Droid CUA tests on cloud-hosted Android or iOS devices. It is a paid feature; contact [Loadmill Support](mailto:support@loadmill.com) to enable it for your account before setting it up. After Loadmill Cloud is enabled, create a Loadmill API token and make it available to your local shell or CI environment: ```sh export LOADMILL_API_TOKEN=your-loadmill-api-token ``` Start a run with `--device-source loadmill-cloud`, the target platform, device name, OS version, app build, and a saved test: ```sh droid-cua \ --device-source loadmill-cloud \ --platform android \ --device-name "Galaxy S24" \ --os-version "14" \ --app ./app-debug.apk \ --instructions tests/login.dcua ``` Use an `.apk` app build for Android or an `.ipa` app build for iOS. *** ## Use a config file For CI, it is recommended to keep a small JSON config file in the repository and run tests with `--config`. ```sh droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` Example config: ```json { "llmProviderMode": "loadmill", "cuaModel": "loadmill-smart", "promptCustomizations": { "basePromptInstructions": "", "designModeInstructions": "", "executionModeInstructions": "" }, "appContextEnabled": true, "appContextPath": "../tests/context.md", "contextOptimizationEnabled": true, "contextOptimizationThreshold": 30000 } ``` The config file keeps prompt settings and app context consistent between local runs and CI runs. *** ## Common CLI options | Option | Description | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--target` | Target kind, such as `mobile` or `web`. | | `--instructions` | Path to a `.dcua` test file or a folder of `.dcua` files. | | `--avd` | Android device, Android emulator, or iOS simulator name. | | `--platform` | Target platform, such as `android` or `ios`. | | `--browser` | Browser for web runs, such as `chrome` or `edge`. | | `--session-mode` | Local web session mode: `persistent` (default) or `fresh`. | | `--device-source` | Mobile device source, such as `local`, `lambdatest`, or `loadmill-cloud`. | | `--device-name` | Cloud device name for LambdaTest or Loadmill Cloud. | | `--os-version` | Cloud device OS version for LambdaTest or Loadmill Cloud. | | `--app` | App build path for cloud runs. Use `.apk` for Android or `.ipa` for iOS. | | `--config` | Path to a Droid CUA headless config file. | | `--llm-provider` | AI provider for the run: `loadmill` (default) or `openai`. Set this to `openai` when using your own OpenAI API key. | | `--cua-model` | Model to use for the run. With Loadmill, use `loadmill-smart` (recommended and most robust), `loadmill-pulse` (faster and lower cost), or experimental `loadmill-beacon`. With OpenAI, use `gpt-5.6-terra`, `gpt-5.6-luna`, or `gpt-5.4`. | | `--context` | Path to an app context file. | | `--no-context` | Disable app context for the run. | | `--record` | Save screenshots from the run. | | `--debug` | Write detailed debug artifacts for troubleshooting. | *** ## CI basics In CI, install the CLI, provide the required credentials as secrets, and run the saved test file with a config file. ```yaml jobs: droid-cua: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 - run: npm install -g @loadmill/droid-cua - run: | droid-cua \ --avd adb:emulator-5554 \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` A successful run exits with code `0`. A failed run exits with code `1`, so CI can fail the build when the mobile test fails. # Running Droid Tests in CI Saved Droid tests can run from a CI pipeline through the Droid CUA CLI. Build and stabilize a test in the desktop app first, commit the `.dcua` test and its context files to your repository, and then run the same test headlessly in CI. ## Before you start You will need: * A saved `.dcua` test that passes reliably. * The Droid CUA CLI installed in the CI environment. * A device target. Cloud devices are usually the simplest choice for hosted CI runners. * Any required Loadmill, device-provider, or AI-provider credentials stored as CI secrets. * The tested `.apk` for Android cloud runs or `.ipa` for iOS cloud runs. See the [CLI guide](/droid-mobile-testing/cli) for all device sources, credentials, and command options. ## Keep CI configuration in the repository Use a small JSON configuration file to keep the model, application context, and prompt settings consistent between local and CI runs. ```json { "llmProviderMode": "loadmill", "cuaModel": "loadmill-smart", "promptCustomizations": { "basePromptInstructions": "", "designModeInstructions": "", "executionModeInstructions": "" }, "appContextEnabled": true, "appContextPath": "../tests/context.md", "contextOptimizationEnabled": true, "contextOptimizationThreshold": 30000 } ``` Start with `loadmill-smart` while creating and stabilizing a test. After the test is reliable, consider switching routine runs to `loadmill-pulse` for faster, lower-cost execution. ## Example cloud run Store the Loadmill API token as a protected CI secret and expose it to the command as `LOADMILL_API_TOKEN`. ```sh npm install -g @loadmill/droid-cua droid-cua \ --device-source loadmill-cloud \ --platform android \ --device-name "Galaxy S24" \ --os-version "14" \ --app ./app-debug.apk \ --instructions tests/login.dcua \ --config ci/droid-cua.json \ --debug ``` Loadmill Cloud must be enabled for the account. You can also use a configured LambdaTest device or a local device available to a self-hosted runner. ## Use the process exit code A successful Droid run exits with code `0`. A failed run exits with code `1`, allowing the CI job to fail when the test does not complete successfully. Keep detailed artifacts for failed runs by using `--debug`. Use `--record` when screenshots from the complete run are useful for investigation. Uploaded CI results also appear in the shared [Droid Runs dashboard](/droid-mobile-testing/runs-dashboard), where the team can filter by project, source, and platform and open the detailed report. # Agent Skill The bundled Droid CUA Agent Skill gives supported coding agents the instructions they need to work safely with Droid CUA projects. It helps an agent inspect a `.dcua` project, author or update tests and context, check the selected target, and run or diagnose tests with the Droid CUA CLI. Use the skill when you want an agent to work on a Droid CUA test project. It is intentionally limited to Droid CUA work; it is not a general browser automation or mobile-development skill. *** ## Install the skill Install the latest Droid CUA package first: ```sh npm install -g @loadmill/droid-cua ``` You can install the skill for Cursor, Claude Code, or Codex. ```sh droid-cua skill install cursor droid-cua skill install claude-code droid-cua skill install codex ``` By default, the skill is installed globally for your user account. To install it only in the current project, add `--local`: ```sh droid-cua skill install codex --local ``` In the Droid CUA desktop app, you can also choose **Help → Install Droid CUA Skill**, then select your coding agent. The desktop app installs the skill globally. *** ## Use the skill After installation, ask your coding agent to work with Droid CUA or `@loadmill/droid-cua`. The skill directs the agent to: * Inspect the active project without reading or exposing secret values. * Work with existing `.dcua` tests, `context.md`, test data, and CLI configuration. * Confirm the intended test and target before starting a run. * Use Loadmill by default, unless you explicitly ask to use your own OpenAI API key. * Run the narrowest relevant test and report its outcome, screenshots, and report artifacts. For the CLI commands and supported test targets, see [CLI](/droid-mobile-testing/cli). *** ## Manage an installation Check whether a skill is installed: ```sh droid-cua skill status codex ``` Remove an installed skill: ```sh droid-cua skill uninstall codex ``` After upgrading `@loadmill/droid-cua`, explicitly update the installed skill: ```sh droid-cua skill install codex --force ``` The installer does not overwrite or remove locally modified skill files unless you supply `--force`. # Overview API-first end-to-end testing validates complete business processes through the network interactions produced by real user or system behavior. Instead of relying on the UI for every execution, Loadmill captures the underlying API traffic, identifies dependencies between requests, and turns the behavior into a reusable test flow. This approach is useful when a journey crosses several services or applications. The generated flow represents the business process while running quickly and deterministically at the API layer. ## How it works 1. Perform a real workflow in a web or mobile application. 2. [Capture the resulting API traffic](/api-first-end-to-end-testing/working-with-the-recorder). 3. Filter requests that do not belong to the scenario. 4. Let Loadmill identify values that must be extracted and reused between requests. 5. Review the generated flow in the [Test Editor](/test-editor-reference/layout). 6. Run the flow manually, from CI, or as part of a test plan. ## Ways to create API tests You can record browser traffic with the [Loadmill Test Composer](/api-first-end-to-end-testing/working-with-the-recorder/quickstart), capture traffic from a mobile app with [Mobile API Testing](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing), import recorded traffic, or create request steps directly in the Test Editor. ## Where to go next * Follow the [API-first E2E quickstart](/get-started/quick-start-guide) for a complete example. * [Register your first API flow](/api-first-end-to-end-testing/register-your-first-api-flow). * Learn how [contract testing](/api-first-end-to-end-testing/contract-testing) and [regression testing](/api-first-end-to-end-testing/regression-testing) fit into API coverage. * Use the [Test Editor reference](/test-editor-reference/layout) when building more advanced flows. # Register Your First API Flow Navigate to your app and start registering any CRUD (Create/Read/Update/Delete) operation. **To do that:** 1. Navigate to the app/platform of your choice. 2. Go to the Loadmill Test Composer tab inside your dev tools. 3. Create any CRUD operation flow (create and delete any type of operation). 4. Click on Analyze Requests to analyze and optimize your flow from the APIs registered for your test case. 5. Click on Create Test to upload your flow to the Loadmill platform. **Voila! You just registered your first API flow!** Clicking on “Create Test” sends the registered API flow to the Loadmill AI that, through its complex algorithm generates a fully dynamic ready-to-use automated test. The next steps will walk you through the Loadmill platform understanding what and how the Loadmill AI generated the automated test. # Running Your API Test Congrats! You managed to upload your test scenario to the Loadmill platform. Before running your test, let's walk through the key sections of your automated test. #### **Naming Your Flow** Naming your flow is (or should always be) your first step in creating and managing your automated test cases. It allows you to know what your flow is about and what should it do so never forget to name every flow/scenario you compose. When uploading your flow/test scenario to Loadmill, you’ll see the flow added to your test suite on the left side of the app. By default, the name is the date and time you uploaded your test but make sure to rename it to, most commonly, the name of the operation you composed to a test. {% embed url="" %} #### **Generated Test Code** The generated test code from the scenario you registered is the most interesting part of Loadmill. After uploading your flow/scenario to Loadmill, its sophisticated AI generates a complete automated dynamic test build that you can easily view, edit and integrate. The generated code is located at the top of the editor and is written and saved in `.yaml` {% embed url="" %} #### API Components From the generated automated test to your API components, Loadmill makes it easy for you to edit each API component related to your test case. Every component is expandable with various features such as extracting and validating dynamic values from the server response, running custom code to be used in each API call, and managing conditions for every API used for your test case. {% embed url="" %} #### **Running Your Test** Loadmill offers multiple ways of running your test(s) like running your tests manually through the Loadmill app or running them through terminal but the most common one is the CI connection where, once connected to Loadmill, all you need to do is push your code to your CI and Loadmill will run the chosen tests automatically which gives you the freedom to concentrate more on your work rather than going through the hassle of managing multiple tabs and flows manually. # Capture API Traffic Loadmill can capture the API traffic produced by real user sessions and turn it into reusable test flows. The generated flows can be reviewed in the Test Editor, run repeatedly, and integrated into a continuous delivery pipeline. For web applications, use the **Loadmill Test Composer** in Chrome or Edge. For mobile applications, use [Mobile API Testing](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing) with the proxy in the Loadmill Desktop App. Loadmill also provides a node recorder for capturing traffic in a test or production environment. ## Loadmill Chrome recorder extension 1. Download the extension [here](https://chrome.google.com/webstore/detail/loadmill-recorder/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en). 2. Go to [Test Suites](https://app.loadmill.com/app/api-tests/test-suites) in Loadmill and create a Test Suite. 3. Open the Chrome Developer Tools => switch to the **Loadmill Rec** tab and select name of the created Test Suite in the extension. ![](/files/tHa0G74W5OAH6UpumwBn) {% hint style="warning" %} :brain: If you are not logged in to Loadmill, you will be prompted to the login page. Please note that sometimes other [installed extensions](https://docs.loadmill.com/working-with-the-recorder/recording-troubleshooting) like Adblock may cause issues, thus we recommend switching them off while recording. {% endhint %} 4\. Click on the "Clear" button within the extension to clear the captured requests list and navigate to the tested website. 5\. Start recording your test. By default, the recorder captures only requests from the main domain. In case you have more domains to record, click on FILTER SETTINGS and add them, also you can filter irrelevant strings out in these settings. ![](/files/3fb4T9EEKSryzZodY4w8) Note, you can also configure URL Filters within [the Recordings Settings](https://docs.loadmill.com/working-with-the-recorder/recorder-settings#url-filters) so that the entire team will have the same filters. 6\. To make sure your recorded session includes only relevant requests, you can click on the expanding arrow within requests to see their request and response body. Delete irrelevant requests by clicking on the "Delete" icon. 7\. When you are done recording your session, click the "Upload" button at the bottom. 8\. Click on the small box next to the Test Suite name to open the recorded test in your Test Suite. {% hint style="info" %} :mage: For each test flow Loadmill automatically creates [**extractions**](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) and [**assertions**](https://docs.loadmill.com/api-testing/test-suite-editor/assertions): 1. Default assertions - for any extraction we find in the recorded test, we add a default assertion to it (example: "id" exists). 2. Specific assertions - for any parameter with given user key (for example, "success") we extract it and assign to it an assertion (example: "success" equals "true"). {% endhint %} 9\. Run the Test Suite by pressing the "RUN SUITE" button and see [your results](https://docs.loadmill.com/api-testing/analyzing-an-api-test-results). 🥳 ## Loadmill node recorder We've developed **a smart recorder** that allows users to capture traffic in the QA or Production environment and replay it as API tests. ![The Recordings tabs](/files/qPi2LuIjsUkZei6zHd82) ![](/files/-LpTJchKQFqAbttAcw6m) 1. Create a new recording from the [Recordings tab](https://app.loadmill.com/app/recordings/my-recordings) in Loadmill. 2. Play your test scenarios in the recorded app. By default, the traffic is split to different sessions by its source (different IPs) and intermissions between user actions. (If you wish to manually split a recorded session in your test environment, use the [Loadmill session splitter](https://chrome.google.com/webstore/detail/loadmill-session-splitter/beknfelcpakgnojjfcdpjddhnckekhni), else you can use our [advanced Recordings settings](https://docs.loadmill.com/working-with-the-recorder/recorder-settings)) 3. When you're ready, stop the recording and [generate a Test Suite from it](https://docs.loadmill.com/working-with-the-recorder/working-with-the-recorder). See how to set the recorder up in [the next guide](https://docs.loadmill.com/working-with-the-recorder/setting-up-the-recorder). ## Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # Download Test Composer Once your account is created, download the [**Loadmill Test Composer Chrome Extension**](https://chrome.google.com/webstore/detail/loadmill-recorder/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en)**.** This extension allows you to compose your test scenario by registering your API requests while you navigate the site. It will register any type of API requests being sent and received from the server. You can also apply specific filters if you want to register requests relevant to your test case flow. As well as adding domains in case you want to register our-of-domain APIs such as third-party APIs that is necessary for completing your flow. See the [**Test Composer Setup**](/api-first-end-to-end-testing/working-with-the-recorder/quickstart) for more info on the composer features. # Test Composer Setup Loadmill has several ways to compose full API scenarios - the most common is [Loadmill Composer Chrome Extension](https://chrome.google.com/webstore/detail/loadmill-test-composer/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en). This extension allows you to compose your API test scenario while browsing your application. It can register any type of incoming and ongoing API calls from the server. {% hint style="warning" %} Make sure to create an account if you want to use it. {% endhint %} Once installed, you can locate the Loadmill Composer as part of the dev tools tab. Click Command+Option+J (Mac)/ Ctr+Shift+J (Windows) to open your dev tools and select the Loadmill Composer tab. {% embed url="" %} {% hint style="info" %} **You can drag the Loadmill Composer tab for it to be more visible for you every time you open your dev tools** {% endhint %} {% embed url="" %} [![image-text](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LHDbUNdi3wPd9vSolzU%2Fuploads%2FNRrejIpGspdMBzdTlYHH%2FDownload%20Chrome%20Extention.png?alt=media\&token=8f06b3fe-6ad1-4a1b-95e3-ce0bdcc78853)](https://chrome.google.com/webstore/detail/loadmill-test-composer/gdkmnfehipofdefhpegbgkkocinlaofd?hl=en) # Test Composer Layout Here's a quick walkthrough of the Loadmill Composer: 1. "Start" to start composing your test. 2. API registered in your flow. Click to expand the API to see request/response data. 3. "Analyze Request" for the AI to analyze and correlate your flow. 4. "Create Test" registers the flow in your Loadmill suite. 5. "Filter Settings" set specific URL filters. 6. Input for creating a new Loadmill suite. 7. Downloads scenario as a `.har` file. ![](/files/ZzJLs2HmM5hdoT3hDVeV) The composer detects all API calls sent by your application as you navigate the platform. Each API is registered with its method, URL, request body, and response. You can click on any API to view the contents of the request registered. {% embed url="" %} #### Analyzing Requests What makes the Loadmill Composer unique is the “Analyze Requests” feature. It allows the AI to process and analyze how each request will affect your API test and what kind of parameters you need in order to make it dynamic, allowing the test to be ready for any environment. To start correlating your flow, click “Analyze Requests”. {% hint style="info" %} **After creating a flow, hit “Pause”, so it does not register any incoming API calls. This will keep your API resources from being used unnecessarily.** {% endhint %} {% embed url="" %} #### Creating Tests Hit the “Create Test” button to upload a scenario. Loadmill generates the code for that uploaded scenario, parameterizing the correlated values in the corresponding flow. {% embed url="" %} # Filter Settings You can adjust your filter settings to allow you to personalize the API test. #### **Include Domains** "Include Domains" lets you register API calls from outside of the current domain. You'll find it useful when registering API flows that require external domains for authentication, purchase flows**,** etc. {% embed url="" %} #### Exclude Domains Exclude Domains registers every domain but the inputted domain you type in. {% embed url="" %} #### Filter Containing Strings The filter will remove strings containing any input given (such as ".js", “.img”,”css”, etc..) so that the URLs for these strings don't affect your API flow. This is an essential process for ensuring that your test is composed reliably. {% embed url="" %} #### Downloading Flow The `.har` registered scenario/flow file can be downloaded by clicking on the bottom-left icon. {% embed url="" %} # Algorithm Configuration When creating a test in Loadmill, you can configure a rich set of rules that determine how your test scenario is analyzed, interpreted, and converted into an executable flow. These settings control assertions, filtering, data cleanup, parameter extraction, uniqueness detection, and various advanced behaviors of the test-generation algorithm. This guide explains every configuration option, what it does, when to use it, and how it affects the resulting test. To apply or modify these configurations, go to the Algorithm page in Loadmill and make changes through the UI or click `Edit Algorithm Settings` to edit the JSON directly. ![](/files/HGZfdcZzMhPpcr7BIYmi) Then, edit or add the relevant rules in the appropriate JSON properties. ## Table of Contents #### 1. [Assertions](#assertions) * [Automatic Assertions](#automatic-assertions) * [Automatic Assertions by URL](#automatic-assertions-by-url) * [Automatic Polling by URL](#automatic-polling-by-url) * [Strict Response Validation](#strict-response-validation) #### 2. [Filtering & Control](#filtering-and-control) * [Filter Irrelevant Requests](#filter-irrelevant-requests) * [Filter Request by Body](#filter-request-by-body) * [Headers Filters](#headers-filters) * [Ignored Keys](#ignored-keys) * [Ignored Values](#ignored-values) * [Irrelevant Aura Actions](#irrelevant-aura-actions) * [Irrelevant GraphQL Operations](#irrelevant-graphql-operations) * [Irrelevant SOAP Actions](#irrelevant-soap-actions) * [Keep All MIME Types](#keep-all-mime-types) * [Relevant URLs](#relevant-urls) * [Session Cut URLs](#session-cut-urls) * [URL Blacklist](#url-blacklist) #### 3. [Data-cleanup](#data-cleanup) * [Don't Extract From Response by URL Regex](#dont-extract-from-response-by-url-regex) * [Keep Original Values](#keep-original-values) * [Max Entities of Same Type](#max-entities-of-same-type) * [Reduce Global Parameters](#reduce-global-parameters) #### 4. [Structure & Semantics](#structure-and-semantics) * [Fields Hierarchy](#fields-hierarchy) * [JQuery Ignored Elements](#jquery-ignored-elements) * [JSONPath Ignored Keys](#jsonpath-ignored-keys) #### 5. [Parameter Detection](#parameter-detection) * [Custom Identifiers](#custom-identifiers) * [Custom Uniqueness](#custom-uniqueness) * [Default Values](#default-values) * [Aggregate Request Body Parameters](#aggregate-request-body-parameters) * [Include Failed Requests](#include-failed-requests) * [Keys Value as Key](#keys-value-as-key) * [Max Response Content Byte Size](#max-response-content-byte-size) * [Nested Value Decoding](#nested-value-decoding) * [Non Secret Keys](#non-secret-keys) * [ResponseRegexExtractors](#response-regex-extractors) * [Search Unique Values](#search-unique-values) * [Special Keys](#special-keys) * [Synonyms](#synonyms) * [Value Holder Key Map](#value-holder-key-map) #### 6. [Advanced & Serialization Settings](#advanced-and-serialization-settings) * [Allow Dots](#allow-dots) * [Array Format](#array-format) * [Decode URL-Encoded Body](#decode-url-encoded-body) * [Decode URL-Encoded Query](#decode-url-encoded-query) * [Prettify Request Body](#prettify-request-body) * [Split Origin](#split-origin) * [Xml Decode](#xml-decode) #### 7. [Uniqueness & Extraction Controls](#uniqueness-and-extraction-controls) * [ID Prioritization](#id-prioritization) * [Max Uniqueness Length](#max-uniqueness-length) #### 8. [Post Processing Editing](#post-processing-editing) * [Yaml Regex Replacers](#yaml-regex-replacers) * [Regex Replacers](#regex-replacers) *** ## Assertions ### Automatic Assertions Defines **automatic assertions** that Loadmill applies to extracted parameters. **When to use:** * When your API contains stable response fields that should always be validated. * When you want to detect silent failures (`200 OK` but the operation logically failed). **Structure:** **`assertions`** *object* An object that defines how to generate assertions for extracted parameters. The object includes the following properties: * **`isDefaultAssertion`** *boolean, Defaults to `true`* Defines basic automatic assertions of *Exists* applied to all extracted parameters. * **`specificAssertions`** *array, Defaults to \[ ]* Define specific assertion applied to parameters with defined keys. Each item in the array contains: * **`key`** *string* The key that, when encountered, should create the automatic assertion. * **`assertion`** *{ \[assertionType: string]: string }* Defines the assertion to apply. The object must contain a single assertion operator (e.g., equals, matches, contains) and its expected value. Example: ```json "assertions": { "isDefaultAssertion": true, "specificAssertions": [ { "key": "success", "assertion": { "equals": "yes" } } ] } ``` *** ### Automatic Assertions by URL Extracts a value from responses whose URL matches a pattern, then asserts it. **When to use:** * When a specific endpoint always returns a value you want to verify. **Structure:** **`autoAssertionsByUrl`** *array* Defines automatic assertions for specific endpoints. Each item specifies a URL pattern, a JSONPath to extract the value, and the assertion to apply. Each item in the array contains: * **`url`** *string* Regex pattern to match the request URL. * **`jsonpath`** *string* JSONPath expression to extract the value from the response. * **`assertion`** *{ \[assertionType: string]: string }, Optional* Assertion operator and expected value (e.g., `{ "equals": "processed" }`). * **`loop`** *object, Optional* Adds a polling loop for the matched URL. The loop includes: * **`iterations`** *number* Number of polling attempts. * **`assert`** *{ \[assertionType: string]: string }* Assertion to apply on each polling response. * **`wait`** *number* Wait time (ms) between iterations. Example: ```json "autoAssertionsByUrl": [ { "url": "/api/orders/[0-9]{8}$", "jsonpath": "$.data.status", "assertion": { "equals": "processed" } }, { "url": "/api/orders/[0-9]{8}/status$", "jsonpath": "$.data.status", "loop": { "iterations": 5, "assert": { "equals": "completed" }, "wait": 1000 } } ] ``` *** ### Automatic Polling by URL Defines **automatic polling loops** for endpoints matching specific URL patterns. **When to use:** * When your system performs asynchronous operations requiring repeated polling. * For status endpoints that eventually reach a final state. * When testing processes that aren't instantaneous (e.g., onboarding, file processing). **Structure:** **`automaticLoops`** *array* An array where each item defines the conditions under which an automatic loop should be added. Each item in the array contains: * **`url`** *string* Regex pattern to match the request URL. * **`loop`** *object* The loop includes the following properties: * **`iterations`** *number* Number of polling attempts. * **`assert`** *{ \[assertionType: string]: string }* Assertion to apply on each polling response. * **`wait`** *number* Wait time (ms) between iterations. * **`expectedStatus`** *string, Optional* Expected HTTP status code (e.g., `"ANY"`). Example: ```json "automaticLoops": [ { "url": "https://example.com/orders/[0-9]{9}/status\\?subId=[0-9a-z]{40}", "loop": { "iterations": 20, "assert": { "check": "__status", "equals": "200" }, "wait": 2000 }, "expectedStatus": "ANY" } ] ``` *** ### Strict Response Validation Defines **strict validation** rules for API responses using JSON Schema or "JSON contains" logic. **When to use:** * When your API response structure must be validated (schema-based testing). * When responses contain dynamic values that shouldn't break validation (use `ignoreKeys`). **Structure:** **`strictResponseValidation`** *object* Controls whether Loadmill should automatically generate strict validation rules (JSON Schema or JSON Contains) for each API response. The object includes the following properties: * **`jsonSchema`** *boolean* Enables strict validation of the response against an auto-generated JSON Schema. Ensures the structure, types, and required fields match the expected schema. * **`jsonContains`** *boolean* Enables validation that the response contains specific fields/values. * **`ignoreKeys`** *array* A list of keys to exclude from validation. Useful for dynamic fields (e.g., id, timestamp, uuid) that change between runs. * **`shouldValidateArrayItems`** *boolean* If true, validates each item in arrays individually. Examples: ```json "strictResponseValidation": { "jsonSchema": true, "ignoreKeys": ["id"] } ``` ```json "strictResponseValidation": { "jsonContains": true, "ignoreKeys": ["teamLabelsToIgnore", "name"], "shouldValidateArrayItems": true } ``` *** ## Filtering & Control ### Filter Irrelevant Requests Controls whether to automatically remove requests that don’t impact the outcome of the test, such as telemetry, health checks, and other noise. **When to use:** * When HAR contains a lot of browser noise (Mixpanel, GA, LinkedIn, etc.). * When you want cleaner tests with fewer steps. * When focusing only on business logic requests. **Structure:** **`filterIrrelevantRequests`** *boolean, Defaults to true* When enabled, the generated test is cleaner, more stable, and easier to debug. Example: ```json "filterIrrelevantRequests": true ``` *** ### Filter Request by Body Filters requests according to JSONPath rules. **When to use:** * When URL-based filtering is insufficient. * When the request body is JSON. * When the only reliable way to identify irrelevant requests is by inspecting the request body, for example, in GraphQL requests. **Structure:** **`filterRequestByBody`** *array* Each item in the array contains: * **`query`** *string* JSONPath expression to match a value in the request body. * **`equals`** *string* Value to compare against the result of the JSONPath query. Example: ```json "filterRequestByBody": [ { "query": "$.type", "equals": "telemetry" } ] ``` *** ### Headers Filters Removes specific headers during processing. **When to use:** * To avoid extracting irrelevant or unstable headers. * When headers contain high-frequency noise (Cloudflare, Chromium, WebSocket). * When cleaning request data for readability. **Structure:** **`headersFilters`** *string \[ ]* An array of header names or prefixes to exclude from extraction. Example: ```json "headersFilters": [ "x-", "cf-", "accept", "Sec-WebSocket-Key", "cookie" ] ``` *** ### Ignored Keys Skips extraction for specific keys during parameter extraction. **When to use:** * When certain keys are known to be irrelevant or noisy. **Structure:** **`ignoredKeys`** *string \[ ]* Array of key names to ignore during extraction. Example: ```json "ignoredKeys": ["irrelevantKey", "unusedField"] ``` *** ### Ignored Values Skips extraction of trivial or unhelpful values. **When to use:** * To suppress useless values that appear frequently and create irrelevant parameters. **Structure:** **`ignoredValues`** *array, Defaults to `["undefined", null, "null", true, "true", false, "false"]`* List of values to ignore during parameter extraction. Example: ```json "ignoredValues": ["undefined", null, "null", true, "true", false, "false"] ``` *** ### Irrelevant Aura Actions Filters out Aura actions that are not relevant to the generated test. An Aura action is a request in the Salesforce Lightning framework that performs a specific operation. **When to use:** * When your HAR contains Aura requests with actions that are not relevant to your test scenario. * To remove noise and focus on business-critical Aura actions. **Structure:** **`irrelevantAuraActions`** *string \[ ]* Array of regex patterns. Any Aura action that matches a pattern in this array and has no extracted values will be excluded from the generated test. ```json "irrelevantAuraActions": ["^get"] ``` *** ### Irrelevant GraphQL Operations Filters out irrelevant GraphQL operations that should not be included in the generated test. **When to use:** * When your HAR contains GraphQL requests with operations that are not relevant to your test scenario. * To remove noise and focus on business-critical GraphQL operations. **Structure:** **`irrelevantGraphQLOperations`** *string \[ ]* Array of GraphQL operation names. Any operation that matches a name in this array and has no extracted values will be excluded from the generated test. Example: ```json "irrelevantGraphQLOperations": [ "IntrospectionQuery", "TelemetryEvent" ] ``` *** ### Irrelevant SOAP Actions Filters out SOAP actions that should not be included in the generated test. **When to use:** * When your HAR contains SOAP requests with actions that are not relevant to your test scenario. * To remove noise and focus on business-critical SOAP actions. **Structure:** **`irrelevantSoapActions`** *string \[ ]* Array of SOAP action names. Any action that matches a name in this array and has no extracted values will be excluded from the generated test. Example: ```json "irrelevantSoapActions": [ "GetServerTime", "Ping" ] ``` *** ### Keep All MIME Types Controls whether all MIME types are preserved during extraction. **When to use:** * When you need to extract parameters from non-standard or binary MIME types. * To ensure no data is filtered out based on MIME type. **Structure:** **`keepAllMimeTypes`** *boolean* If true, all MIME types are kept during extraction, including binary and custom types. Example: ```json "keepAllMimeTypes": true ``` *** ### Relevant URLs Forces these URLs not to be filtered. **When to use** * When your flow depends on URLs that look noisy but are actually important. * To override aggressive filtering and ensure specific endpoints are always included. **Structure** **`relevantUrls`** *string \[ ]* An array of URL substrings or patterns. Any request URL containing one of these will be included in the generated test, even if it would otherwise be filtered out. Example: ```json "relevantUrls": [ "banking/sessions/create", "v1/cities" ] ``` *** ### Session Cut URLs Defines URL patterns where a new session should be started during test generation. **When to use:** * When using a backend recordings. * To split flows into distinct sessions for better isolation and analysis. **Structure:** **`sessionCutUrls`** *string \[ ]* Array of URL substrings or regex patterns. When a request matches any of these, a new session is started. Example: ```json "sessionCutUrls": [ "/api/login", "/logout" ] ``` *** ### URL Blacklist Removes requests containing any of the specified substrings. **When to use** * To filter out entire domains. * To eliminate 3rd-party scripts (CDN, tracking, ads). **Structure** **`urlBlackList`** *string \[ ]* an array of substrings. Any request URL containing one of these substrings will be excluded from the generated test. Example: ```json { "urlBlackList": ["your-domain.com", "another-domain"] } ``` *** ## Data-Cleanup ### Don't Extract From Response by URL Regex Skips extraction for responses whose request URL matches a pattern. **When to use:** * When some endpoints return noisy or irrelevant payloads. **Structure:** **`dontExtractFromResponseByURLRegex`** *string \[ ]* Array of regex patterns. Extraction is skipped for responses matching these patterns. Example: ```json "dontExtractFromResponseByURLRegex": ["pattern1", "pattern2"] ``` *** ### Keep Original Values Prevents extraction for matching URLs. **When to use:** * When there are fixed values that you don't want to replace with parameters. **Structure:** **`keepOriginalValues`** *string \[ ]* Array of URL patterns for which values should not be replaced with parameters. Example: ```json "keepOriginalValues": ["/internal/status"] ``` *** ### Max Entities of Same Type Limits how many identical keys are extracted from a single request. **When to use** * When responses include many repetitive objects. **Structure:** **`maxEntitiesOfSameType`** *number, Defaults to 15* Maximum number of identical keys to extract per request. Example: ```json "maxEntitiesOfSameType": 15 ``` *** ### Reduce Global Parameters Controls extraction of global parameters to prevent excessive or irrelevant parameterization. **When to use:** * When too many global parameters are being extracted, leading to noisy or unstable tests. * To focus extraction only on meaningful parameters. **Structure:** **`reduceGlobalParameters`** *boolean, Defaults to false* When enabled, limits extraction of global parameters to only those that are relevant. Example: ```json "reduceGlobalParameters": true ``` *** ## Structure & Semantics ### Fields Hierarchy Defines how extracted JSONPaths use attribute-based selectors instead of index-based selectors for improved stability. **When to use:** * When the entity type can be reliably identified by a specific key. * To avoid unstable extraction paths that depend on array indices. **Structure:** **`fieldsHierarchy`** *string \[ ]* Array of field names. When present, extraction paths will use attribute-based selectors for these fields. Example: ```json "fieldsHierarchy": ["name", "title"] ``` *** ### jQuery Ignored Elements Skips extraction for specific HTML elements when generating jQuery selectors. **When to use:** * When invalid HTML causes Loadmill to fix the markup and generate selectors on the corrected structure. * When you want to avoid extracting or interacting with certain elements, even after HTML correction. **Structure:** **`jqueryIgnoredElements`** *string \[ ]* Array of HTML element names to ignore during jQuery selector generation. Example: ```json "jqueryIgnoredElements": ["tbody"] ``` *** ### JSONPath Ignored Keys Defines patterns for dynamic key segments (such as UUIDs or timestamps) that should be replaced with recursive descent (`..`) in generated JSONPath expressions. **When to use:** * When dynamic map keys (such as UUIDs or timestamps) make JSONPath expressions unstable. * To simplify extraction paths and avoid brittle selectors. **Structure:** **`jsonpathIgnoredKeys`** *string \[ ]* Array of regex patterns. Any key matching a pattern will be replaced with recursive descent in JSONPath. Example: ```json "jsonpathIgnoredKeys": [ "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$" ] ``` *** ## Parameter Detection ### Custom Identifiers Adds custom identifier field names (case-insensitive) for parameter extraction. **When to use:** * When your system uses nonstandard naming for ID fields. * To ensure custom identifier fields are prioritized during extraction and correlation. **Structure:** **`customIdentifiers`** *string \[ ]* Array of field names (case-insensitive) to treat as identifiers during extraction. Example: ```json "customIdentifiers": ["testSuiteCard", "orderNumber"] ``` *** ### Custom Uniqueness Detects unique values and their reoccurrences by specifying keys and regex patterns. Without any additional conditions. **When to use:** * When a key contains a unique and important value that should be replaced everywhere it appears. **Structure:** **`customUniqueness`** *array* An array where each item contains rules that define unique values. Each item in the array contains: * **`key`** *string, Optional* The key whose value should be detected as unique. * **`regex`** *string* Regex pattern to match the unique value. Example: ```json "customUniqueness": [ { "key": "patientNumber", "regex": "^[a-z0-9]{25}$" } ] ``` *** ### Default Values Assigns fallback values to specific fields during test generation. **When to use:** * When you need to extract a specific value from the request body and create correlations for it. * When fixed values must be replaced with dynamic ones (e.g., timestamps, dates, random IDs). **Structure:** **`defaultValues`** *array* an array where each item contains rules that define how to extract parameters and assign them a default value. Each item in the array contains: * **`selector`** *string* The key to identify the field. * **`value`** *string* The fallback value to assign. * **`name`** *string* The parameter name to use for correlation. Example: ```json "defaultValues": [ { "selector": "userId", "value": "123", "name": "userId" } ] ``` *** ### Aggregate Request Body Parameters Aggregates matched request-body objects into a single parameter for correlation instead of extracting only primitive values. **When to use:** * When you want to correlate request-body objects (not just primitive values). * To reduce the number of generated parameters by treating an object as one correlated value. **Structure:** **`aggregateRequestBodyParameters`** *boolean, Defaults to false* When enabled, if a match is found in the request body and the value is an object, the algorithm extracts the object as a single parameter and correlates it across requests, which reduces the number of generated parameters. Example: ```json "aggregateRequestBodyParameters": true ``` *** ### Include Failed Requests Includes failed HTTP requests in the generated test. **When to use:** * For debugging flows where failures are expected or important. * When you want to analyze and reproduce error scenarios. **Structure:** **`includeFailedRequests`** *boolean, Defaults to false* When enabled, failed HTTP requests (non-2xx responses) are included in the generated test. Example: ```json "includeFailedRequests": true ``` *** ### Keys Value as Key Treats the value of a key as a parameter. **When to use:** * When you need to extract a value from a key and use it as a parameter for correlation or substitution. **Structure:** **`keysValueAsKey`** *string \[ ]* Array of key names whose values should be treated as parameters. Example: ```json "keysValueAsKey": ["customTokenField"] ``` *** ### Max Response Content Byte Size Controls the maximum response size (in bytes) that the system will process for parameter extraction. **When to use:** * Increase the default value when you need to extract parameters from very large responses. **Structure:** **`maxResponseContentByteSize`** *number, Defaults of 3000000* Maximum response size (in bytes) that will be processed for extraction. Example: ```json "maxResponseContentByteSize": 3000000 ``` *** ### Nested Value Decoding Parses and extracts parameters from fields containing stringified JSON objects. **When to use:** * When a field contains a stringified JSON object that needs to be parsed for parameter extraction and correlation. **Structure:** **`nestedValueDecoding`** *array* an array where each item contains rules specifying which keys in the response body should undergo additional parsing. Each item in the array contains: * **`key`** *string* The key whose value is a stringified JSON object. * **`type`** *string* The format of the nested value (e.g., `"JSON"`). Example: ```json "nestedValueDecoding": [ { "key": "value", "type": "JSON" } ] ``` *** ### Non-Secret Keys Defines keys whose values are considered non-sensitive and will be extracted and recorded even when operating in secure mode. **When to use:** * When using a backend recordings. * When you want to ensure certain values are always extracted, even in secure recording. * When specific keys do not contain secrets and should be included for correlation or debugging. **Structure:** **`nonSecretKeys`** *string \[ ]* Array of key names whose values are safe to extract and record. Example: ```json "nonSecretKeys": ["sessionId", "userId"] ``` *** ### Response Regex Extractors Extracts values from response bodies using regex capturing groups. **When to use:** * When you need to extract and correlate a value from the response body that cannot be accessed using standard extraction methods (e.g., not available via JSONPath). **Structure:** **`responseRegexExtractors`** *array* an array where each item contains rules for extracting parameters from the response body using regex. Each item in the array contains: * **`regex`** *string* Regex pattern with capturing groups to extract the desired value from the response body. * **`name`** *string* The parameter name to assign to the extracted value. * **`forceExtract`** *boolean, Optional, Defaults to false* If true, always extract the value even if it only appears once and has no correlations. Example: ```json "responseRegexExtractors": [ { "name": "refresh_token_id", "forceExtract": false } ] ``` *** ### Search Unique Values Detects unique values and their reoccurrences based on their values, not just their keys or context. **When to use:** * When you want to identify unique values according to their actual content, regardless of their field names or context. **Structure:** **`searchUniqValues`** *boolean, Defaults to false* When enabled, the algorithm will detect and correlate unique values wherever they appear. Example: ```json "searchUniqValues": true ``` *** ### Special Keys Always extracts these keys from the request body, regardless of other extraction settings. **When to use:** * When `reduceGlobalParameters` is off and you still want to extract certain keys from the request body. **Structure:** **`specialKeys`** *string \[ ]* Array of key names to always extract from the request body. Example: ```json "specialKeys": [ "totalAmount", "delivery", "customerQuoteTime", "subtotalAmount" ] ``` *** ### Synonyms Maps different key names to unify correlation and extraction. **When to use:** * When different endpoints use different naming conventions for the same logical identifier. * To unify extraction and correlation of values across APIs with inconsistent field names. **Structure:** **`synonyms`** *array* Each item is an array of key names that should be treated as synonyms for correlation and extraction. Example: ```json "synonyms": [ ["sku", "product"], ["amountPaid", "totalAmountPaid", "totalAmount"], ["customerQuoteTime", "delivery"] ] ``` *** ### Value Holder Key Map Extracts nested values when a key contains an object instead of a primitive. **When to use:** * When a field stores a nested object instead of a primitive. **Structure:** **`valueHolderKeyMap`** *array* an array where each item contains rules for extracting parameters whose value is located inside one of their nested fields. Each item in the array contains: * **`containerKey`** *string* The key whose value is an object containing the actual value. * **`valueKey`** *string* The key inside the object that holds the value to extract. Example: ```json "valueHolderKeyMap": [ { "containerKey": "password", "valueKey": "text" }, { "containerKey": "id", "valueKey": "value" } ] ``` *** ## Advanced & Serialization Settings ### Allow Dots Controls whether dots (`.`) are allowed in parameter names. **When to use:** * When parameter names may safely contain dots (e.g., `user.name`). * To support APIs or systems that use dotted notation for keys. **Structure:** **`allowDots`** *boolean, Defaults to false* If true, allows dots in parameter names during extraction and correlation. Example: ```json "allowDots": false ``` *** ### Array Format Controls how arrays are serialized in URL-encoded form. **When to use:** * When URL-encoded arrays must follow a specific backend format. * To ensure compatibility with server-side array parsing. **Structure:** **`arrayFormat`** *string, Defaults to "indices"* **Allowed values:** "indices" | "brackets" | "repeat" Specifies the serialization format for arrays in URL-encoded bodies. Example: ```json "arrayFormat": "indices" ``` *** ### Decode URL-Encoded Body Enables decoding of request bodies with `application/x-www-form-urlencoded` content type. **When to use:** * For POST requests with `application/x-www-form-urlencoded`. * When you need to extract parameters from URL-encoded request bodies. **Structure:** **`decodeUrlEncodedBody`** *boolean, Defaults to false* If true, decodes URL-encoded request bodies for parameter extraction. Example: ```json "decodeUrlEncodedBody": true ``` *** ### Decode URL-Encoded Query Enables decoding of URL query strings to improve readability in the generated test. **When to use:** * When your server does not require query parameters to remain encoded. * When you want URL query parameters to appear in a human-readable format for easier review and debugging. **Structure:** **`decodeUrlEncodedQuery`** *boolean, Defaults to true* If true, decodes URL-encoded query strings for better readability in the output. Example: ```json "decodeUrlEncodedQuery": true ``` *** ### Prettify Request Body Controls formatting of request bodies in the generated YAML output. **When to use:** * When you want readable, formatted JSON bodies in your test output. * When performance snapshots require compact bodies (disable prettification). **Structure:** **`prettifyRequestBody`** *boolean, Defaults to true* If true, pretty-prints large JSON bodies in output YAML for readability. If false, leaves request bodies compact (not formatted). Example: ```json "prettifyRequestBody": true ``` *** ### Split Origin Separates origins into host and protocol components. **When to use:** * When combining HAR files from backend recordings. * To ensure requests are grouped and processed by their actual origin. **Structure:** **`splitOrigin`** *boolean, Defaults to false* If enabled, splits the origin into host and protocol for more granular processing. Example: ```json "splitOrigin": false ``` *** ### XML Decode Enables decoding of special formats from systems like Priority ERP (not standard XML). **When to use:** * When working with Priority ERP or systems returning non-standard XML encodings. **Structure:** **`xmlDecode`** *boolean, Defaults to false* If true, enables decoding and extraction from non-standard XML formats. Example: ```json "xmlDecode": true ``` *** ## Uniqueness & Extraction Controls ### ID Prioritization Prioritizes extraction of parameters whose keys match known or custom identifier fields (such as `id`, `uuid`, or those defined in `customIdentifiers`) when multiple candidates exist in an object or array. **When to use:** * When your data contains multiple possible identifiers and you want to ensure the most meaningful one is extracted. * To improve correlation and stability by always preferring standard or custom ID fields. **Structure:** **`idPrioritization`** *boolean, Defaults to false* If true, extraction will prioritize keys matching identifier fields (e.g., `id`, `uuid`, or those in `customIdentifiers`) over other candidates. Example: ```json "idPrioritization": false ``` *** ### Max Uniqueness Length Limits the maximum length of a value considered for uniqueness detection. **When to use:** * To avoid extracting long strings (such as logs, SQL queries, or HTML) as parameters. * When you want to prevent noisy or irrelevant values from being treated as unique identifiers. **Structure:** **`maxUniquenessLength`** *number, Defaults to 40* Maximum length (in characters) of a value to be considered for uniqueness detection. Values longer than this will be ignored. Example: ```json "maxUniquenessLength": 40 ``` *** ## Post Processing Editing ### Regex Replacers Extracts values from requests using regex capturing groups. **When to use:** * When values do not appear in JSON at all (e.g., tokens inside headers or raw bodies). * When there is a repeated value that was not extracted into a parameter. **Structure:** **`regexReplacers`** *array* an array where each item contains a regex that, if matched, should create a parameter for it. Each item in the array contains: * **`name`** *string* The parameter name to assign to the extracted value. * **`regex`** *string* Regex pattern with capturing groups to extract the desired value from the request. * **`reuse`** *boolean, Optional, Defaults to false* If true, reuses an existing parameter even when the values are different. Example: ```json "regexReplacers": [ { "name": "auth_token", "regex": "eyJhbGciOi.*", "reuse": true } ] ``` *** ### YAML Regex Replacers post-processing regex replacements on the generated YAML output. **When to use:** * When you need to mask, transform, or enhance YAML after generation. **Structure:** **`yamlRegexReplacers`** *array* an array where each item contains rules specifying which changes should be applied to the generated YAML. Each item in the array contains: * **`regex`** *string* Regex pattern to match in the YAML output. * **`replace`** *string* Replacement string to substitute for matches. * **`flags`** *string, Defaults to ""* Regex flags (e.g., `"g"` for global, `"i"` for case-insensitive). Example: ```json "yamlRegexReplacers": [ { "regex": "password: \".*\"", "replace": "password: \"***\"", "flags": "g" } ] ``` *** # Mobile API Testing Mobile API testing captures the network traffic produced while someone uses a mobile app and turns that traffic into reusable API-first test flows. The initial scenario is performed on a device, but the generated test validates the underlying services directly and does not need to repeat every action through the mobile UI. This is different from [Droid mobile testing](/droid-mobile-testing/droid-cua). Droid operates and validates the visible mobile interface. Mobile API testing uses a proxy to observe the app's requests and responses, then replays the business flow at the API layer. ## When to use mobile API testing Use this approach when you want to: * Build fast regression coverage from real mobile behavior. * Validate a business journey across several backend services. * Capture dynamic IDs, tokens, and other values passed between requests. * Reuse the resulting API flow in CI or as the basis for a performance test. Use Droid instead when the mobile interface, visual state, or device interaction is the behavior you need to validate. ## How it works 1. The Loadmill Desktop App runs a proxy on your computer. 2. A mobile device is configured to send its network traffic through that proxy. 3. You perform the scenario in the mobile app. 4. Loadmill records the relevant requests and responses. 5. The Desktop Recorder analyzes the capture and generates a parameterized API test flow. 6. You review, run, and maintain the flow in Loadmill.
Mobile application traffic captured through the Loadmill proxy

Mobile application traffic is captured and converted into an API-first test flow.

## Before you start You will need: * A mobile app and device you can use for the capture. * A Mac or Windows computer running the Loadmill Desktop App. * The computer and mobile device connected to the same network. * Permission to install the Loadmill certificate and configure the device's Wi-Fi proxy. ## Capture your first flow 1. [Install the Loadmill Desktop App](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/installing-loadmill-desktop-app). 2. [Install the Loadmill certificate](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/installing-certificate-on-mobile-devices) on the mobile device. 3. [Configure the mobile proxy](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/configuring-proxy-on-mobile-devices). 4. [Capture the mobile API traffic](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/capturing-traffic-with-loadmill-mitm-proxy). 5. Use the [Loadmill Desktop Recorder](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/loadmill-desktop-recorder) to filter the capture and [generate a test flow](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/loadmill-desktop-recorder/generating-test-flows). If traffic is not appearing or the app cannot connect, see [Mobile Capture Troubleshooting](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/troubleshooting). # Capturing Mobile API Traffic Loadmill's Desktop App uses a MITM (Man-in-the-Middle) proxy to capture API traffic from your mobile device. This records the requests and responses produced by a real mobile journey so Loadmill can generate a reusable API test flow. {% hint style="info" %} It is also possible to use 3rd party MITM proxies as well (ie. Charles proxy and proxyman) and export the captured traffic into loadmill. {% endhint %} The MITM proxy intercepts the communication between your mobile device and the backend servers, allowing Loadmill to analyze and record the API calls and responses. This approach ensures that you capture the most accurate and complete data for your testing scenarios. To use the MITM proxy within Loadmill's Desktop App, follow these steps: 1. **Start the Loadmill Desktop App:** Switch to the proxy section\ ![](/files/sjRv7b9iIyz4qN5sd5Oq)\ at this stage the proxy will be paused without content. 2. **Install a Certificate**: Installing the Loadmill certificate on your device establishes a secure trust relationship between your device and the proxy, enabling it to decrypt the traffic for analysis and testing purposes.\ In loadmill proxy, look for the following button ![](/files/EW7v8kc5q6sG4GMg6IcZ) to download the certificate. 3. **Set up your mobile device** to use the proxy server provided by the Loadmill Desktop App. This typically involves configuring your device's Wi-Fi settings to use the proxy server's IP address and port number. 4. **Start the proxy:** By clicking the button 5. **Interact with your application:** you will see new requests accumulating in loadmill.\ ![](/files/62iYsv3idDLhWpmVgc2i) # Installing the Loadmill Desktop App The Loadmill Desktop App is our official application, available for Windows and macOS. It bundles almost everything that Loadmill has to offer, the testing platform UI, the MITM proxy for capturing application traffic, a dedicated private Loadmill Agent for running tests locally, Hybrid test support, and more. To get started with Loadmill Desktop App, follow the steps below. ## Downloading the Loadmill Desktop App 1. Visit the Latest [Official Loadmill Release](https://github.com/loadmill/desktop-app/releases/latest). 2. Choose the appropriate version for your operating system (Windows or macOS). * For Windows, the installer is named `Loadmill-desktop-app--Setup.exe`. * For macOS, the installer is named `Loadmill--x64.dmg`. 3. Click it to download the installer to your computer. ## Installing the Loadmill Desktop App On Windows To install the Loadmill Desktop App on Windows, follow the instructions in the [Install Windows desktop app](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/installing-loadmill-desktop-app/install-windows-desktop-app) guide. # Install Windows Desktop App This document outlines the required prerequisites and setup steps before installing the Loadmill Desktop App on a Windows system. Ensure these requirements are met to avoid installation or runtime issues. *** ## ✅ System Requirements * **Operating System:** Windows 10 or Windows 11 (64-bit) * **Processor:** Intel i3 or higher / AMD equivalent * **Memory:** Minimum 4 GB RAM (8 GB recommended) * **Disk Space:** At least 500 MB free space for installation plus additional space for captured HAR data and logs * **Network:** Stable internet connection required to communicate with Loadmill cloud services *** ## ⚙️ Software & Access Requirements * Administrative privileges to install software and security certificates * Optional: Corporate proxy credentials if behind enterprise firewall (the app supports proxy configuration) *** ## 🖥 Installation on Windows 1. Download the latest version of Loadmill desktop app for Windows from the [latest release page](https://github.com/loadmill/desktop-app/releases/latest) 2. Double-click to run the setup 3. App will get installed in the following folder:\ `C:\Users\\AppData\Local\desktop_app` ![Loadmill Desktop App](/files/hSv9eW0xTJVMB65CMhwA) *** ## 🔐 Certificate Installation (for HTTPS Traffic Capture) * When capturing HTTPS traffic (from browsers or mobile devices), Loadmill installs a local root certificate to decrypt HTTPS traffic * During installation, accept the certificate prompt to allow HTTPS capture * If using for mobile capture: export and install the certificate on the mobile device via Wi-Fi proxy configuration * Ensure both desktop and mobile device are on the same Wi-Fi network for proper capture ![Proxy Tab](/files/bCrYm34FqFO6dM9hpd8m) *** ## 🌐 Network Configuration * Allow outbound HTTPS (TCP port 443) to Loadmill’s cloud endpoints * Ensure that any corporate VPNs or firewalls do not block the Loadmill app executable or proxy ports * For capturing mobile traffic, verify the device proxy is configured to the desktop app’s IP and port (shown within the app) *** ## 📁 Log and Configuration Paths * **Log files:**\ `C:\Users\\AppData\Roaming\Loadmill\logs` * **Configuration and captured data:**\ Stored under the app’s user data directory in `AppData\Roaming\Loadmill` ![Download Logs](/files/V6bDOm50Er5ElsMFUR1z) *** ## 📞 Support For installation or connectivity issues, contact Loadmill Support via or refer to the official documentation at # Loadmill Desktop Recorder Loadmill Desktop Recorder captures mobile application interactions at the API level and turns them into end-to-end API test flows. There are three main components that work together to provide a powerful and efficient testing solution: 1. **Loadmill UI**: Same as the web interface for managing your testing process. With the Loadmill UI, you can edit and execute test flows, and access reporting and run results.\ ![](/files/Cn1qpfwHO9AbwUkyLqq2) 2. **MITM Proxy**: The integrated Man-in-the-Middle (MITM) proxy allows you to capture API traffic from your mobile device by intercepting the communication between the device and the backend servers.\ ![](/files/nSfTYV5mazUSiFSh17sk) 3. **Loadmill Agent**: The Loadmill Agent enables local execution of test scenarios, allowing you to run tests directly from your computer within your network. This feature simplifies the testing process and makes it more accessible, allowing you to quickly and easily debug tests and obtain results. {% hint style="info" %} The agent is controlled by the play/stop button integrated to the top right corner of the application. {% endhint %} ## **Proxy area (Recorder section)** **General controls:**
At the top, you can find controls for: * starting/pausing the proxy * Importing har file, to continue the work * Clearing the list * Listening IP+Port information * Certificate download * Filtering area * Destination suite for generated tests **Proxy entries area:**
Once started and configured, the proxied requests will be collected in the list.\ The controls in the list allow:\ \- Viewing a request details by clicking on it\ \- Deleting requests by selecting them\ \- Exporting har file **Test creation controls:**\ ![](/files/n2hEdl76550HMcT2XO9w)\\ Bottom right area * Analyze button - will mark out irrelevant requests, the user can revert this decision by clicking the button at the top of the entries area. * Create test button - will process the requests and generate parameterized test in loadmill.\ (The test will be found according to the suite selector by switching to loadmill area) # Generating API Test Flows Loadmill allows streamlining tests generation by capturing user behavior on a mobile device. To generate a test follow the following steps: 1. Ensure your device is up and configured correctly with [mitm certificate](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/installing-certificate-on-mobile-devices) and [proxy settings](/api-first-end-to-end-testing/working-with-the-recorder/deviceless-mobile-testing/configuring-proxy-on-mobile-devices). 2. Start your application and position it's state to the beginning of a scenario 3. Start loadmill proxy by clicking,\ Tip: make sure you start from an empty list, you can clear the residues by clicking . 4. Drive the application on your device by the scenario you want to capture, make sure requests are captured in the proxy 5. If needed, adjust the filterto focus or relevant domain(s) to reduce potential noise from irrelevant requests. 6. Once finished driving the scenario, click on button.\ loadmill will filter out requests that are highly suspected as irrelevant and strikes them out. You can review the filtered requests and if needed restore them by selecting and clicking. 7. Adjust the destination suite in the suite field: 8. To generate the test, click on 9. To review the test navigate to Loadmill section go to Test suite\ \ And select your test suite,\ The recently created flow in the list will be marked with the recent timestamp. # Installing Certificates on Mobile Devices Depend on the platform and your application, there are various steps that might be required to install new certificate.\ In the next sections you will find the most common methods to deploy new certificate. # iOS Certificate Installation To deploy a certificate into an iOS device, follow these steps: 1. **Download the certificate**: In loadmill desktop app, go to the proxy section, download the certificate by clicking on![](/files/JdRy4guzbTPxQ4UgUv44)button. Transfer the certificate file (usually in `.cer` or `.crt` format) to your device (ie. via AirDrop or Google drive). 2. **Install the certificate**: Once the certificate file is downloaded, a prompt will appear asking to review and install a profile.\ ![](/files/rzNPU3kRAAerEmLsQaUs) 3. Go to device settings and select "Profile Downloaded"\ ![](/files/vJUXJofitQ3oLAupITYM) 4. Click on install,\ You will be prompted for your device password, proceed to completion.\ ![](/files/mhNNLukN3QxdTzPLovio)\\ 5. **Trust the certificate**: After the certificate is installed, you need to manually trust it. To do this, go to `Settings` > `General` > `About` > `Certificate Trust Settings`. Under the "Enable full trust for root certificates" section, find the Loadmill certificate you just installed and toggle the switch to enable trust for the certificate.\ ![](/files/qHd78Wm1T7UI5b7YvAsq) After completing these steps, the iOS device can work with the Loadmill MITM proxy. This enables Loadmill to capture and analyze encrypted API traffic from the device for mobile API testing. # Android Certificate Installation More information about certificate installation on android can be found [in our blog here](https://www.loadmill.com/blog/behavior-driven-mobile-testing) # Configuring the Mobile Proxy # iOS Wi-Fi Settings {% hint style="info" %} Before following the next steps, make sure you are connected to the same network as the machine hosting loadmill desktop app proxy. {% endhint %} To set the proxy on your wifi connection: 1. Go to the main `Settings` menu and select `Wi-Fi`.\ ![](/files/lEmiuFYoRZKzJ6shhBtP) 2. Tap the "i" icon next to your connected Wi-Fi network.\ ![](/files/WAebG0p1bvNKIJ1Wn7yC) 3. Scroll down to the `HTTP Proxy` section and select "Manual."\ ![](/files/ZG41jGXX9fSkM0DTiKMv) 4. Enter the IP address and port number provided by the Loadmill Desktop App for the MITM proxy server.\ ![](/files/TAs6zKqmQLFe1n8EU6k7)\\ *The IP and the port number can be found at the upper area of the loadmill desktop app in the proxy section.*
5. Save the settings and ensure that your device is still connected to the Wi-Fi network. # Android Wi-Fi Settings Setting proxy in wifi setting on android can be found in [our blog here](https://www.loadmill.com/blog/behavior-driven-mobile-testing). # Mobile Capture Troubleshooting Q: My application utilize ssl-pinning, as a result I don't see encrypted traffic.\ A: In that case you must build a development version of your application with one of the options:\ (a) disabling ssl-pinning\ (b) whitelisting loadmill-mitm-proxy certificate as part of ssl-pinning\ \ Q: After configuring everything, my application is not responding anymore, neither I see traffic.\ A: Make sure the following:\ (a) The proxy is started\ (b) The proxy port is whitelisted in machine's firewall\ (c) Your device and proxy host machine are connected to the same network and the port is allowed by the network.\ (d) The application does not utilize ssl-pinning or loadmill-ssl-certificate is whitelisted. # Additional Capture Methods Up to now, we've reviewed the following recording methods: * The [Loadmill Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder#loadmill-chrome-recorder-extension) * The [Loadmill Recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder) In case you're already using a recording solution and would like Loadmill to generate an API Test Suite from it, the following recording methods can be imported to Loadmill : 1. [GoReplay](https://goreplay.org/): An open-source network monitoring tool which can record your live traffic, and use it for shadowing, load testing, or detailed analysis and monitoring. You can import Gor files in the recording page by clicking IMPORT RECORDING. 2. [HAR file](https://en.wikipedia.org/wiki/HAR_%28file_format%29): Use the browser's Dev Tools to record a HAR file. You can import HAR files in the API and Load Test Editor. 3. [Charles proxy](https://www.charlesproxy.com/): An HTTP proxy that allows a developer to view all of the HTTP and SSL / HTTPS traffic between their machine and the Internet. Export sessions to HAR files and import them as Loadmill Test Flows. # Set Up Application Recording It only takes a few minutes to install the Loadmill recorder on your website. Once installed, you’ll be able to record user behavior on your website and generate a replay-able Test Suite. ## Create an Application Before you'll start recording users flows, you'll need to set up an application to record from: 1. Go to the **Recordings** tab , located in the navigation area at the left, and click on the **App Setup** tab. 2. In the "Setup" tab, click on + "ADD APPLICATION" button. 3. Name the application in the description and fill in its URL. ![](/files/-MNZ-bsiZ3pO4u34O5Kl) 4\. Click on "GET TRACKING ID" button. ![](/files/-MNZ1Jj_1iYmtyTlYBDy) 5\. There are three options how to install the recorder: * Option 1: [Server side](https://www.npmjs.com/package/@loadmill/node-recorder) aka Node * Option 2: Client side aka HTML * Option 3: Testing in Developer Tools aka DEVTOOLS Choose a relevant option for you and right after you've installed the recorder - the application will appear in the applications table. ![](/files/-MNZ3_1g-As3hih-LADb) ## # Embed the Recording Service Worker ### What is a service worker A service worker is a script that your browser runs in the background, separate from a web page, opening the door to features that don't need a web page or user interaction. Service workers essentially act as proxy servers that sit between web applications, the browser, and the network. They are intended, among other things, to capture and inspect network requests. ### Recording website traffic using Loadmill ![Loadmill service-worker architecture](/files/TMy2nMUUbVFvkyPV8VK6) As shown in the diagram above, to record a website using the Loadmill service worker, you will have to embed the Loadmill SW on your page. However, due to security concerns, service workers can **not** be loaded from another domain. This means that you will have to host and serve the service worker file from your recorded server. Another thing to consider is that service workers will only listen to events triggered in the same scope you have served the service worker from. For example — if you registered your Service Worker from *my-website.com/aaa* the Service Worker will intercept all the *my-website.com/aaa/\** requests. Requests for *my-website.com/bbb/\** won’t be recorded! ### The TL;DR * Make sure that your service worker registration script is embedded in every page your users might visit. * A good place to embed the recorder script is usually where you injected your google analytics script tag. * The service worker script file has to be loaded from the recorded application domain. (usually hosted on your server) * It is best to load the service worker from the root path of the recorded application, i.e. *my-website.com/loadmill-worker.js **and not** from my-website.com/wubba/lubba/dub/dub/loadmill-worker.js* * If your website is using a strict [Content Security Policy (CSP)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP), you should add these two domains to the list of allowed domains `https://sdk.amazonaws.com` and `https://*.loadmill.com`. In case the Loadmill recorder fails to load because of a CSP problem, you should see the following error message in the console logs- ![CAP console error](/files/jibiZjWUUNinHxsLaRcJ) # Recorder Settings Use the advanced Recording Settings to create meaningful replay-able tests that are easy to configure. Team Admins can control these settings [on this page](https://app.loadmill.com/app/user/settings/recordings). ![](/files/odZwNU5R4CnnCGSO9w28) {% hint style="info" %} **Reduce Global Parameters** is the only setting enabled by default that makes sure only relevant parameters will be created during the recording import. :point\_down: These are recording rules you can add & configure for our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder#loadmill-chrome-recorder-extension) & [recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder). {% endhint %} ### Split sessions by URLs (for the [Recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder) use only) In general we want the test cases created based on recorded data, to be short and focused as possible. By default, Loadmill will split a recorded user session into two separate Loadmill test flow, if the user hasn't been active for 5 minutes. In many cases, you will want to control when a session is split into separate test cases. Using this setting, you can add API URLs that will define the beginning of a new test case each time they are called. {% hint style="warning" %} :brain: **Note:** You can also use raw JavaScript to fire an event from your code to split a recorded session:`navigator.serviceWorker.controller.postMessage({ type: "reset-session" });` {% endhint %} ### URL Filters Use this setting to filter API calls that are not relevant for your tests i.e. marketing, monitoring, analytics and other external integrations. ### Special Keys Use this setting to manage important Special Keys located in your POST requests body. This way, we can make sure they won’t be missed during the recording processing phase. ### Entities Synonyms Entities Synonyms help us detect same entities with different names like a 'Jet' and an 'Airplane'. Here you can add and manage entities synonyms. This way, we can make sure they won’t be missed during the recording processing phase. Synonyms must be entered in a list and separated by commas. i.e. Plane, Aircraft, Jet. ### Ignored Keys Ignored keys are a list of keys that will be ignored while generating tests from recordings.\ Keys in this list will never be extracted to a parameter in the created tests. ### Default values Default values are a list of keys that once detected will be assigned with a default value. This value may be overridden on runtime. ### Non-Secret Keys (for the [Recorder](https://docs.loadmill.com/working-with-the-recorder#loadmill-node-recorder) use only) Non-secret keys will be used to identify and record values even when recording in secure mode.\ Keys and paths to identify non-secret values are given in a JSONPath-like syntax. {% hint style="warning" %} :brain: **Note** : Do not record private user data. It is recommended to test these keys locally before recording real users. {% endhint %} ### Regex replacers Regex replacers will be used to identify any strings that match the regex and extract them as a parameter. ### Strict Response Validation When enabled, this option will add JSON Schema assertion to every recorded JSON response. ### Automatic extractions and assertions Configure which JSON keys should be extracted into parameters and which assertions should be created for them. For example, I've recorded a test flow that includes the "total\_score" JSON key and it should be equal to 100 at the beginning of the test flow, so I can add an automatic extraction and assertion for it: ![](/files/-MZwhSYlzE06okrNmE2s) Thus, when uploading a recording, Loadmill will automatically create the extraction and assertion in the test flow in case there is such a JSON key. ![The automatic extraction in the flow](/files/-Ml915ChuNjkVueXkHw9) ![The automatic assertion in the flow](/files/-Ml91DvEwKgkywqFRCy7) # Recording Troubleshooting ### I'm trying to record my tests by using the Chrome recorder extension but getting error that I need to login to Loadmill however I'm logged in... Any idea on how to fix it? There are a few ones 🙂: * It might be related to the 3rd party blocking policy, see how to disable it [here](https://support.cloudhq.net/how-to-enable-3rd-party-cookies-in-google-chrome-browser/). * If the above option didn't help, you might have the "don't track" option enabled in Chrome and you will need to whitelist app.loadmill.com, see where to do this whitelisting [here](https://support.google.com/chrome/answer/2790761?hl=en\&co=GENIE.Platform%3DDesktop). * Sometimes other installed extensions like Adblock may cause this, thus we recommend switching them off while recording. # How to Work with Recordings ## Start Recording After you're all set with your application, it's time to start recording your users session: 1. Go to the "Recordings" tab and click on + NEW RECORDING. 2. In the dialog, select the wanted application from the list. 3. Click START RECORDING. 4. Once the recording starts, you'll be able to see its progress in the recordings table. **Note:** you can have 1 active recording per app. 5. Click on the running recording in order to review its progress. 6. In order to stop a recording, click on the STOP button. ## Awesome! I've recorded my application, now what? ![](/files/H0mywt9i4q3biE5ZCPb5) When you stop the recording process, you can review its list of tracked API's (method, path, Avg response time & Throughput). The next step will be Generating a repayable Test Suite containing your test flows: Click "GENERATE TEST SUITE". Check out more cool things you can do with recordings in the [Recordings Settings](https://docs.loadmill.com/working-with-the-recorder/recorder-settings) guide. # Contract Testing #### Intro Contract testing is a testing approach used in software development that involves testing the interactions between two different services or systems by defining and verifying the contracts or agreements between them. In a typical scenario, when two services communicate with each other through an API, the contract testing approach involves defining the expected input and output data for each API endpoint, and then writing tests to verify that the API endpoint behaves as expected. Contract testing helps in detecting and preventing issues that may arise due to changes made to the API, without having to run end-to-end tests. This allows for faster and more efficient testing of software systems, reducing the time and effort required for testing. Overall, contract testing helps improve the reliability, stability, and resilience of software systems by ensuring that the different components of the system work together as intended, and are compatible with each other. #### Contract testing with Loadmill Contract testing with Loadmill is easy. To create contract validation of an API call, use [JSON schema](/test-editor-reference/assertions#json-schema-validate-that-a-parameter-comply-with-given-json-schema) or [JSON contains](/test-editor-reference/assertions#json-contains-validates-that-a-json-contains-a-subset-json-in-such-way-dynamic-fields-can-be-omitted) as following: * **Enforcing JSON Schema of an API:**\ With 'JSON Schema' you can explicitly enforce multiple of aspects of your json data.\ In the request editor assertions insert new entry, use '\_\_responseBody' built-in parameter and ensure the values are of the type "JSON Schema", then insert a valid json-schema.\\

An example of schema validation request in loadmill

In the following table you can see an example of the schema and a typical response that comply with the schema.
SchemaResponse
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "integer"
    },
    "title": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "price": {
      "type": "integer"
    },
    "discountPercentage": {
      "type": "number"
    },
    "rating": {
      "type": "number"
    },
    "stock": {
      "type": "integer"
    },
    "brand": {
      "type": "string"
    },
    "category": {
      "type": "string"
    },
    "thumbnail": {
      "type": "string"
    },
    "images": {
      "type": "array",
      "items": [
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        }
      ]
    }
  },
  "required": [
    "id",
    "title",
    "description",
    "price",
    "discountPercentage",
    "rating",
    "stock",
    "brand",
    "category",
    "thumbnail",
    "images"
  ]
}

{
   "id":1,
   "title":"iPhone 9",
   "description":"An apple mobile which is nothing like apple",
   "price":549,
   "discountPercentage":12.96,
   "rating":4.69,
   "stock":94,
   "brand":"Apple",
   "category":"smartphones",
   "thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg",
   "images":[
      "https://i.dummyjson.com/data/products/1/1.jpg",
      "https://i.dummyjson.com/data/products/1/2.jpg",
      "https://i.dummyjson.com/data/products/1/3.jpg",
      "https://i.dummyjson.com/data/products/1/4.jpg",
      "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
   ]
}
* **Enforcing JSON structure using JSON-Contains:**\ \&#xNAN;**'**Json-contains' is a lightweight capability in Loadmill to enforce JSON data.\ In the request editor assertions insert new entry, use '\_\_responseBody' built-in parameter and ensure the values are of the type "JSON Contains", then insert an expected json or a subset that can contain less fields.\ Note, to ensure existence of a field ignoring it's value use `"*"` notation.

Json-Contains example in loadmill - Enforcing json content

In the following table you can see an example of the expected JSON-Contains object and a typical response that comply with it.
JSON-Contains objectResponse
{
    "id": 1,
    "title": "iPhone 9",
    "description": "An apple mobile which is nothing like apple",
    "price": 549,
    "discountPercentage": "*",
    "stock": 94,
    "brand": "Apple",
    "category": "smartphones",
    "thumbnail": "https://i.dummyjson.com/data/products/1/thumbnail.jpg",
    "images": [
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
    ]
}

{
    "id": 1,
    "title": "iPhone 9",
    "description": "An apple mobile which is nothing like apple",
    "price": 549,
    "discountPercentage": 12.96,
    "rating": 4.69,
    "stock": 94,
    "brand": "Apple",
    "category": "smartphones",
    "thumbnail": "https://i.dummyjson.com/data/products/1/thumbnail.jpg",
    "images": [
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
    ]
}
``` ``` **Automatic assertions**\ Loadmill can generate 'JSON schema' or 'JSON contains' assertions automatically during test composition. Such assertions can serve are a quick baseline that can be adjusted later on.\ To turn automatic JSON validations. 1. head to settings icon -> recordings:\ 2. Locate the 'JSON Validation section in the list', make sure the toggle turned on and adjust the type according to the needs.
if 'JSON Validation' section doesn't exist, try adding it by clicking on ![](/files/9PsKEWmU0JucFmCh6H6p) at the top right corner and selecting it from the list.
4. (optional) Click on the '+' to add ignored keys.\ ![](/files/Gog7Z1ZSqUoQwBnjpCmr) 5. From now on, every generated test with loadmill composers will cary json validation assertions. # Regression Testing #### How do developers create their regression tests? Regression testing is a process of testing changes to software to make sure that the new code doesn't break existing functionality. It's an important part of the software development process, but it can be time-consuming and expensive. There are a few different ways that developers can create regression test suites for APIs. One common way is to use a tool like SoapUI or Postman. These tools allow developers to create test cases and run them against an API to ensure that the API is functioning correctly. Another way to create a regression test suite is to manually write code that makes calls to the API and verifies the results. This approach requires more work up front, but it can be more flexible and allow for more comprehensive testing. #### The issue many companies have in testing their servers Today, many organizations are struggling with regression testing. They're finding that it's taking longer and longer to test their software, and they're not able to keep up with the pace of change. As a result, they're missing bugs and releasing software with serious defects. There are several reasons for this problem. One is that software is becoming increasingly complex. Another is that the pace of change is accelerating, making it harder to keep track of all the changes that need to be tested. Finally, many organizations are under pressure to release new features and products quickly, which leaves less time for regression testing. As a result, they're forced to cut corners and take shortcuts that can lead to serious defects in their software. # Testing with CORS ## Testing with CORS For high volume load tests, [Loadmill](https://www.loadmill.com) uses real browser sessions to test your application/web server. This requires that you allow [CORS](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing) in your server code (just for us - no third parties need to be allowed). [Enabling CORS](https://enable-cors.org/server.html) mainly involves reading request headers and writing response headers accordingly on your server. The following example code shows exactly how to enable CORS for Loadmill tests in an [express.js](http://expressjs.com) application. Express users **don't need to copy this example**, simply use our npm module: [express-loadmill](https://www.npmjs.com/package/express-loadmill) ```javascript var app = require("express")(); // Handle CORS before processing requests: app.use(function (req, res, next) { // This request header contains the host of the CORS client: var origin = req.header("Origin"); // This request header contains the HTTP request method: var requestMethod = req.header("Access-Control-Request-Method"); if (origin === "http://app.loadmill.com" || origin === "https://app.loadmill.com") { // This response header allows CORS from loadmill.com: res.header("Access-Control-Allow-Origin", origin); // This response header is required only if you use cookies in your tests: res.header("Access-Control-Allow-Credentials", 'true'); if (req.method === 'OPTIONS' && origin && requestMethod) { // It's a pre-flight request: // This request header contains the request headers of the pre-flighted request: var requestHeaders = req.header("Access-Control-Request-Headers"); setPreFlightHeaders(res, requestMethod || "", requestHeaders || ""); return res.sendStatus(204); } else { // If your test scenario involves reading response headers, // we automatically include them in this request header: var exposedHeaders = req.header("Loadmill-Request-Expose-Headers") || ""; // This response header allows the test client to read the desired headers from the response: res.header("Access-Control-Expose-Headers", exposedHeaders); } } return next(); }); function setPreFlightHeaders(res, allowedMethod, allowedHeaders) { res.header({ // This response header asks the browser not to pre-flight // the same request URL again for the next 24 hours: "Access-Control-Max-Age": "86400", // These response headers approve the request method and headers specified: "Access-Control-Allow-Methods": allowedMethod, "Access-Control-Allow-Headers": allowedHeaders }); } ``` If you are not sure how to enable CORS in your application or would like to see examples for different languages/frameworks, please contact us at . ## Express Middleware If you *are* using [node.js](https://nodejs.org) and [express](https://expressjs.com) you can simply use our npm module: [express-loadmill](https://www.npmjs.com/package/express-loadmill). It may also be used for quick and easy [domain verification](/performance-testing/domain-verification). # API Testing FAQs ### I've got a JSON response where I want to extract all book titles from the array by using JSON Path expression - $.store.book\[\*].title but I'm getting only the first title. WHY? Loadmill uses a framework where only the first value gets retrieved in case you are using the \* sign. Use the [`__jsonpath_all`](https://docs.loadmill.com/api-testing/test-suite-editor/functions#__jsonpath_all-target-jsonpath-default) function instead so your extractions will look this way: ![](/files/-Ml5BVm4krGm8xabRTr6) ### **I'm testing entity creation where entity Name should be unique. Can I assign a set of random letters to it?** Sure, create a parameter with using the `__random_letters([length=10])` function for that: ![](/files/-MXCV10KqsLPcOKkpuxP) Find more great functions on [this page](https://docs.loadmill.com/api-testing/test-suite-editor/functions#randomization-functions) and more information about default parameters [here](https://docs.loadmill.com/api-testing/test-suite-editor/test-suite-parameters). ### **I'm getting ID of my resource and I want to make sure it is in the format of UUID. How can I do that?** You can easily achieve that: 1. Extract the ID into a parameter by using JSONPath. 2. Create another parameter by using the extracted ID and [function `__random_uuid()`](https://docs.loadmill.com/api-testing/test-suite-editor/functions#__random_uuid). The output would be "true" if the ID is in the format of UUID. Else, it would be "false". ![](/files/-Ml5CAL42Cv6XMI0zkdB) 3\. Now, assert the second parameter you've created earlier. ![](/files/-Ml5CixnW1tAL_C-yiGl) ### Is there a way to run an API test with a combination of values? In my case, I want to create tickets where each ticket has a specific type, there are 10 ticket types available. There is a feature in Loadmill that allows to do that. First, create a CSV file containing all ticket types, like below: ![CSV file](/files/-MXbbZqxSBTxGkHQj0zM) Then, go to Loadmill, upload the file by using the "Import CSV" button within a test flow and use the "ticket\_type" parameter in your request body: ![](/files/-Ml5DMNzdiy_ecoEtudi) Thus, when running the Test Suite, each test iteration will get a relevant ticket type from the CSV file. Find more information about using CSV files in API tests [here](https://docs.loadmill.com/api-testing/test-suite-editor/api-tests-data-from-csv-files). ### # API Testing Troubleshooting ### I'm getting an error that my JSON Path expression is invalid, although the path is correct. What should I do? ![](/files/-Ml5DmoheBLoA9qioS5g) That happens due to the underscore in the JSON Path expression, change it to be like shown below and it will work: ![](/files/-Ml5DvpdwRLnZipGw8zK) # Overview Loadmill supports browser testing with Playwright and natural-language UI Agent steps. You can create UI-only tests or combine browser actions with API requests in the same flow. Hybrid testing keeps UI coverage focused on behavior that must be observed in the browser. API steps can prepare data, authenticate users, or validate backend state, while Playwright or the UI Agent handles the visible user journey. ## Choose a UI step Use a [Playwright step](/test-editor-reference/steps/playwright-step) when you want exact, reviewable browser automation code and direct control over interactions and assertions. Use a [UI Agent step](/test-editor-reference/steps/ui-agent-step) when you want to describe a browser goal and its validation in natural language. The agent interprets the visible page and works toward that goal. ## Why combine API and UI steps? Browser tests become slower and more fragile when every setup action must happen through the UI. A hybrid flow can create the required state through an API, validate the user-facing behavior in the browser, and verify the final state through another API request. This keeps the test close to the business journey while reducing unnecessary UI interactions. ## Get started * Follow the [Web UI and Playwright Quickstart](/get-started/quickstart). * Build a combined flow with the [Hybrid API and UI Tutorial](/web-ui-and-hybrid-testing/hybrid-api-ui-tutorial). * Review the complete [Playwright integration capabilities](/web-ui-and-hybrid-testing/capabilities). * Use the [Agent Testing Quickstart](/web-ui-and-hybrid-testing/agent-testing-quick-guide) for a natural-language web flow. # Playwright Integration Capabilities ## Record Playwright Tests Easily create Playwright tests by recording your actions in the browser. This feature makes test creation accessible to everyone, even those without coding experience, and speeds up the process for QA and product teams. **Note:** To record Playwright tests, you must have the Loadmill Desktop App installed. ### How to use: 1. Open a flow in the Test Editor. 2. Click the "Record" button in the Playwright step (Desktop App required). ![Record Playwright Button](/files/EmBLBeZfoA5jeZ4ZOqcC) 3. Interact with your application in the browser window that opens. 4. When finished, close the browser. Loadmill will generate Playwright code based on your actions. **Tip:** You can edit the generated script for advanced scenarios. ## Cross-Browser Testing Run your Playwright tests across multiple browsers and devices directly in Loadmill. This ensures your application works as expected for all users, regardless of their browser or device. By automating cross-browser validation, you save time and catch compatibility issues early. ### How to use: 1. In the Test Editor, add a Playwright step to your flow. ![Add Playwright Step](/files/PcNQrtRZslvBHcca8mDo) 2. In the Playwright step settings, select the devices you want to test (e.g., Desktop Chrome, Firefox, Mobile Safari) ![Select Browsers](/files/kGvK13RGLqVLhpRZU5Oe) 3. Save and run your test suite. Loadmill will automatically provision and execute tests across the selected browsers. ![Select Browsers](/files/6dwGmnu6qa3aCKQzVure) ## Debugging Run your Playwright tests in debug mode and step through the Playwright script inside the flow context in Loadmill. This lets you view exactly what's happening in the test, analyze issues, and recreate bugs more effectively. Debug mode provides a step-by-step view, making it easier to write, troubleshoot, and refine your Playwright scripts. **Note:** To debug Playwright steps, you must have the Loadmill Desktop App installed. ### How to use: 1. Open your flow in the Test Editor. 2. Run the test in debug mode. [Learn more about Debug Mode](https://docs.loadmill.com/test-editor/flows/flow-controls#debug-mode) 3. Step through the Playwright script, viewing each action and its effect in real time. ![View Playwright Step Logs](/files/JpncK6HBbFKEplY9RmRQ) 4. Use this interactive view to analyze issues, recreate bugs, and improve your test scripts. ## Automated Orchestration & Infrastructure Loadmill automatically provisions browsers, executes tests in parallel, and scales your test runs. You don’t need to manage infrastructure or runners, making your testing pipeline faster, more reliable, and maintenance-free. ### How to use: Simply run your test suite with Playwright steps. Loadmill automatically manages browser provisioning, parallel execution, and scaling. No additional setup or runners required. **Note:** For large test suites, Loadmill will automatically balance load and optimize execution speed. # Agent Testing Quickstart This quick guide shows how to get access to Loadmill Agent Testing, create a simple agent-based test, and run it against a real web UI from the desktop app. *** ## Before you start You will need: * A Loadmill account * A plan that supports AI Agent capabilities (if you have a promo code, you can use it during sign-up to activate access) * The Loadmill desktop app installed *** ## Step 1: Sign up and activate access Create your account here: [Sign up to Loadmill](https://app.loadmill.com/app/signup) *If you already have a Loadmill account, you can skip the sign-up part of this step.* If you received a promo code by email, open [Billing](https://app.loadmill.com/app/user/settings/billing) after signing in and apply the code there. This enables access to the Agent step for eligible accounts. Open the Billing page from your account settings. ![Billing page showing the subscription plan section](/files/XQCdncwI2iycL4f1E0k3) Enter the promo code and activate it. ![Billing page showing the subscription code activation field](/files/yCdBuRvqihuUMMzdjHmj) *** ## Step 2: Download the desktop app Download the desktop app here: [Loadmill desktop app](https://app.loadmill.com/app/user/settings/desktop-app) **Important:** The desktop app is required to run Agent steps. *If you already have the desktop app installed, you can skip this step.* *** ## Step 3: Create your first agentic test 1. Open the Loadmill desktop app and create a new test suite. 2. Add a new step and choose **UI Agent**. ![Add a new UI Agent step](/files/UaJRv2m3bmXFEwZgsYZf) 3. Enter a short prompt that describes the user flow you want the agent to perform on your site and what it should validate. ![Example of a UI Agent step configured in a test](/files/fu9vyEw4uWnpi8QYc6Ak) For more details on configuring the UI Agent step, including the Navigate to URL, Agent Instructions, and AI-Driven UI Validation fields, see the [UI Agent step documentation](/test-editor-reference/steps/ui-agent-step). 4. Run the test you created. ![Example of a UI Agent step run result](/files/RKQKWzm72dRkrPqcL04p) This is the fastest way to get started: give the agent a short, real UI task and validate that it can complete it reliably. {% hint style="info" %} Keep the first run simple. Start with one short path on your site so you can watch how the agent navigates the UI and validate the result. {% endhint %} *** ## Learn more about creating tests in Loadmill Agent Testing is a fast way to validate real user flows, but it works best as part of Loadmill's broader API-first testing approach. If you are new to Loadmill, you can also learn how to create tests with the Loadmill Composer and generate API tests from real user activity. See the [Quick Start Guide](/get-started/quick-start-guide) for a step-by-step walkthrough. *** ## Need help? If you have any questions or need help, contact us at . If you would like to book a demo, click [here](https://www.loadmill.com/book-a-demo). # Hybrid API and UI Tutorial A hybrid flow uses API steps for fast setup or backend validation and UI steps for behavior that must be observed in the browser. This example outlines a common pattern: create data through an API, verify it through the UI, and confirm the resulting state through another API request. ## Before you start Create a test suite and a flow, then make sure a Loadmill Desktop App or private agent with UI testing enabled is available. ## Step 1: Prepare state through an API Add a [Request step](/test-editor-reference/steps/request-editor) that creates the data needed by the scenario. Extract any value the UI step needs, such as an item ID, email address, or generated name. API setup is usually faster and less fragile than navigating several browser screens before reaching the behavior under test. ## Step 2: Validate the browser journey Add a [Playwright step](/test-editor-reference/steps/playwright-step) after the request. Open the relevant page, locate the data created by the API step, perform the important user action, and assert the visible result. Suite and flow parameters are available directly as variables in the Playwright code. Cookies created by earlier steps are passed to the browser context automatically. ```ts await page.goto(targetUrl); await page.getByText(createdItemName).click(); await expect(page.getByText('Completed')).toBeVisible(); ``` You can use a [UI Agent step](/test-editor-reference/steps/ui-agent-step) instead when the browser portion is better described as a goal in natural language. ## Step 3: Verify the final state Add another Request step to retrieve the updated resource and assert the backend state. This separates visible UI validation from deeper data validation while keeping both in one end-to-end flow. ## Step 4: Run and refine Run the complete suite. If the browser step needs investigation, use [debug mode](/web-ui-and-hybrid-testing/debugging-playwright-tests). Once the flow is stable, connect it to your [CI workflow](/integrations-and-ci-cd/npm-modal). # Cloud and Cross-Browser Execution Loadmill can execute the same Playwright flow across multiple browser configurations. This is useful when a journey must behave consistently across browser engines, desktop sizes, or mobile browser profiles. ## Select browser coverage 1. Open a flow containing a Playwright step. 2. Open the Playwright step settings. 3. Select the browser and device configurations you want to test. 4. Save and run the test suite. Loadmill provisions the selected browser environments and includes each execution in the run results. ## Start with purposeful coverage Use the smallest browser set that represents the environments your users depend on. Add configurations when they protect a real compatibility risk rather than selecting every available browser for every run. A common pattern is to use one primary browser for pull-request feedback and a broader browser matrix for scheduled or release runs. ## Keep setup outside the browser when possible Cross-browser suites multiply the cost of repeated UI actions. Use API steps to create data or establish state, then keep the Playwright portion focused on the browser-specific behavior you need to validate. See the [Hybrid API and UI Tutorial](/web-ui-and-hybrid-testing/hybrid-api-ui-tutorial). For the complete set of browser features, see [Playwright Integration Capabilities](/web-ui-and-hybrid-testing/capabilities). # Git-Based Playwright Workflows Loadmill test suites can be synchronized with a Git repository so test changes are versioned, reviewed, and tied to the same delivery workflow as application changes. A suite containing Playwright steps is stored as Loadmill suite configuration, including its Playwright code. ## Connect a repository Connect Loadmill to GitHub from **Settings → Integrations**, then select the repository used for test synchronization. Team administrator access is required to configure the connection. See [GitHub Data Sync](/integrations-and-ci-cd/github-integration/data-sync-connection-to-github) for the complete setup. ## Review test changes in Git Use the **GitHub Sync** tab in a test suite to choose a branch and commit the current suite configuration. Loadmill stores synchronized suites under `loadmill-suites/` in the selected repository. From that point, test changes can follow a branch and pull-request workflow. Review the Playwright code and surrounding API steps together so the complete business flow remains understandable. ## Run a versioned test configuration When running a Test Plan, select the branch under **Versioned Test Configuration**. Loadmill uses the committed suite configuration from that branch rather than uncommitted changes in the editor. You can also use the [Loadmill MCP test authoring workflow](/ai-and-developer-workflows/test-authoring) to update synchronized suites from a compatible AI development tool, validate the changes, and open them for review. ## Avoid overwriting unsynchronized work Before checking out a branch or replacing the editor's working copy, make sure any changes that should be kept have already been synchronized. Checking out a committed suite version can replace unsynchronized editor changes. # Debugging Playwright Tests Debug mode runs a Playwright step interactively through the Loadmill Desktop App. Use it while creating a test or investigating a failure so you can follow each action and inspect the browser state. ## Before you start Install and start the Loadmill Desktop App. Open the test suite and flow that contain the Playwright step you want to inspect. ## Start a debug run 1. Open the flow in the Test Editor. 2. Select **Debug** from the flow execution controls. 3. Step through the Playwright actions while watching the browser. 4. Review the step output and update the script where necessary. 5. Run the complete flow again after the debug session succeeds. ![Playwright test running in debug mode](/files/JpncK6HBbFKEplY9RmRQ) ## What to inspect When a Playwright step fails, check: * Whether the page reached the expected state before the action. * Whether the locator identifies a stable user-facing element. * Whether data created by earlier API steps is available to the browser step. * Whether the action needs a more specific assertion or timeout. * Whether the failure occurs only in a particular browser configuration. Use the [Flow Controls](/test-editor-reference/flows/flow-controls) reference for more information about debug mode and the [Playwright Step](/test-editor-reference/steps/playwright-step) reference for supported variables and examples. # Overview Loadmill performance testing replays API scenarios at scale so you can measure how a system behaves under concurrent traffic. A performance test can be created directly or derived from an API flow that already represents real application behavior. ## What you can validate Use performance tests to: * Measure response times, throughput, and error rates under load. * Validate expected capacity before a release or high-traffic event. * Find requests or services that become bottlenecks as concurrency increases. * Compare system behavior across environments or application versions. * Add performance thresholds to a repeatable delivery workflow. ## Typical workflow 1. Create a scenario or convert an existing API flow. 2. Run a dry run to confirm that one iteration succeeds. 3. Configure duration, concurrency or request rate, ramp-up behavior, and execution locations. 4. Verify ownership of the tested domains before running at higher volume. 5. Run the test and review the results for latency, throughput, and failures. ## Get started Follow the [Performance Testing Quickstart](/get-started/getting-started-1), learn how to [configure a load test](/performance-testing/working-with-the-test-editor), or review how to [analyze performance test results](/performance-testing/analyzing-load-test-results). # Configure a Load Test ![](/files/FYnehjmXBkrkmwk80NBM) Let's take a look at the Load Test Editor. ![](/files/QjBxZLZQhS11Ek7ouMIP) | Option | Description | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 The requests' area | The main area of your load test. Double-click a request to manage its headers, parameters and assertions. [Learn More](https://docs.loadmill.com/api-testing/test-suite-editor/request-editor) | | 2. Tested Domains | Manage domains/hosts tested by the load test. See your domain in red? Check how to verify it [here](https://docs.loadmill.com/load-testing/setup/domain-verification). | | 3. DRY RUN | Run your load test as [a single iteration](https://docs.loadmill.com/load-testing/getting-started#running-a-trial-test). | | RUN TEST | Run your load test [at scale](https://docs.loadmill.com/getting-started/getting-started-1#running-a-load-test-at-scale). | | 4. Advanced Settings | Configure Iterations, E-mail notifications, enable Cookies, [Import CSV](/performance-testing/creating-a-parameterized-load-test) and more. | | 5. Replace Parameter | Easily replace the parameter you wish so all of the flow's requests will be affected. | | Extract Parameter | [Extract a value to a parameter](/test-editor-reference/flows/flow-controls#extract-parameter) from all across the test requests. | # Analyzing Performance Test Results Load tests results display several types of data series over the duration of the test. The data is divided into two graphs: ![](/files/-Lwm27tkNucbDzpEGh-u) ### Performance Over Time In the graph the main x-axis is the running time of the test, the y-axis on the left is response time measured in milliseconds, and the right y-axis is the number of active users. On this graph you can track how the number of active users correlates with your application performance using the following metrics: * **Concurrency** (black) - the number of active users in the test. * **Avg. Response Time** (green) - the average response time for all successful requests in a given moment in time. * **50% Response Time** **(purple)** - the 50th percentile for response time, also known as Median. This is the response time value, half of the requests measured below it, and half of the request measured above it. * **95% Response Time** **(orange)** - the 95th percentile for response time. 5% of the requests measured slower response time than this value. * **Error rate (red)** - The percentage of failed requests out of the total number of attempted requests for the given moment in time. In case the load test reaches a certain threshold (%50 by default), it will fail. The difference between average, 50th percentile and 95th percentile, also known as Standard Deviation, can be a useful tool to identify bottlenecks in your applications performance. ### Throughput This graph represents the number of successful requests per second. As the number of active sessions increases, the number of attempted requests will go higher. When the number of active sessions is constant for some time, changes in throughput will indicate changes in your applications ability to respond fast to requests. # Parameterized Load Tests Creating a load test by using parameters and extracting response values In our [first simple test](https://docs.loadmill.com/load-testing/getting-started#creating-and-running-a-load-test-from-scratch-in-loadmill) we have simulated users loading the homepage of our blog, our second test is going to simulate several users publishing new entries to the blog. * First create a new empty test. Only logged in users can post to our blog, so let's start by executing a login request and store the authentication token returned by it into a parameter for later use. * Give your new test a short description `Login & Create a blog post` * Update the request description to `Blog Login` * Set the Request #1 method to `POST` and its URL to - `https://loadmill-test-blog.herokuapp.com/ghost/api/v0.1/authentication/token/` * Click the "ADVANCED" button to expand the advanced section of the request. * Set the Content-Type to `application/x-www-form-urlencoded` (last in the list). * And paste this as the request body - `grant_type=password&username=a@b.com&password=Test1234&client_id=ghost-admin&client_secret=b91601629baf` * Expand the “Extractions - Set Parameters” card and set a value to a parameter named `access_token` using a `JSONPath` extractor querying for `access_token`. This will create a value extractor that will execute a [JSONPath query](http://goessner.net/articles/JsonPath/) against the JSON response of the request and extract the authentication token into a parameter named `access_token`. * Verify that a value was set to the parameter `access_token` by the `JSONPath` extractor by expanding the ‘Verify Response’ card of the request and creating an assertion. Add an assertion that states that the `access_token` parameter `Is True` (Use the "Exists" assertion type). ![](/files/-MNSi4P0GO61k5y_XlvU) * Click **DRY RUN** and execute a trial run to make sure it is working. * Now that we have the authentication token stored in `access_token` we can use it to publish on our blog * Create a new request by clicking the "+ ADD REQUEST" button below the first request. * Set its description to `Publish Blog Post` Set it as a `POST` request to `https://loadmill-test-blog.herokuapp.com/ghost/api/v0.1/posts/` * Click on the "ADVANCED" button and set the Content-Type to `application/json` and the body to this JSON- ```javascript { "posts": [ { "title": "Title ${__random_chars}", "slug": "${__random_chars}", "markdown": "Text ${__random_chars}", "status": "published" } ] } ``` * This JSON body defines the blog post that we are going to publish. Notice that we have used one of [Loadmill's functions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions) to make it a little more interesting - `${__random_chars}`. The `__random_chars` parameter will change to 10 random characters during test/trial execution. * Now, let's use the `access_token` parameter value we extracted from the login response to authenticate this request. Expand the header card of publish request and add a header with the name `Authorization` and the value `Bearer ${access_token}` * Run a trial of the whole scenario and go to `https://loadmill-test-blog.herokuapp.com/` to see that our blog posts are getting published. Now that we know that our full scenario is working we can run it as a load test. Click the "RUN TEST" button at the bottom of the test and run the test 🎉 # Domain Verification **Domain verification** is the process of verifying that your domain or hostname actually belongs to you, so we can load-test it. For example, if you have an app hosted on `myapp.com` you would like test requests from [Loadmill](https://www.loadmill.com) to be sent to it during a load test. However, if we allow ***anyone*** to send requests to `myapp.com` it could be abused by a malicious user to attack your servers by flooding them with requests. The way this problem is solved is by using **domain verification**, a simple process in which you are asked to: 1. Go to **Settings > Domains**. 2. Enter the hostname or sub-domain you would like to test in the text area at the top of the page and click **Verify**. 3. You will be prompted and asked to host a small text file, containing a **Verification Token**, on your server in order to prove you own your domain, e.g. `myapp.com/loadmill-challenge/aSnd5K8L86Pggg1rGLPgLlf6guK.txt`. 4. Once the file is hosted on your server, click **Verify** to complete the process. If you have access to your site's **DNS configuration**, you can [prove ownership without changing code on your server](#dns-verification). {% hint style="warning" %} :brain: **Note:** There is no need to prove ownership of a sub-domain once its parent is verified. For example, if you have already completed the verification process for `myapp.com` you can load-test `www.myapp.com` or `blog.myapp.com` as well. {% endhint %} ## Express Middleware If you are using [node.js](https://nodejs.org) and [express](https://expressjs.com), you can easily serve the verification file using our npm module: [express-loadmill](https://www.npmjs.com/package/express-loadmill). It may also be used to enable [crowdsourced load testing](/api-first-end-to-end-testing/testing-with-cors) for high volume load tests. ## DNS Verification If you do not want or unable to serve a file on your server, you can prove ownership of your domain via DNS. According to the documentation of your hosting service or DNS provider, add a DNS record of the following format: * **Type:** `TXT` * **Name:** The sub-domain you would like to test, e.g. `www` or `myapp.com` * **Value:** `loadmill-challenge=` where `` stands for the **Verification Token** you were asked to serve in a text file. E.g. `loadmill-challenge=aSnd5K8L86Pggg1rGLPgLlf6guK` You can make sure that the text record was properly added by looking it up using DNS lookup tools like this one - [MxToolbox](https://mxtoolbox.com/TXTLookup.aspx). Once the DNS record is set, click **Verify** to complete the process. {% hint style="info" %} :man\_mage: **Note:** DNS records may take a while to be propagated, so if the verification initially fails, try again after a few minutes. {% endhint %} # Configuration Files Learn how to work with test configuration files Loadmill test configuration files are an extension of the [YAML file format](https://en.wikipedia.org/wiki/YAML). The test configuration example below was converted into JSON to be used in [our REST API](https://docs.loadmill.com/integrations/rest-api#create-load-test) and demonstrates some of the key elements such as - [extractions](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions), [assertions](/test-editor-reference/assertions) and parameter defaults. {% code title="example.json" %} ```javascript { "meta": { "description": "New Flow" }, "requests": [ { "description": "Basic flow", "method": "POST", "url": "httpbin.org/anything", "postData": { "text": "{\"parameter\":\"value\"}", "mimeType": "application/json" }, "extract": [ { "parameter_value": { "jsonPath": "$.json.parameter" } } ], "assert": [ { "check": "parameter_value" } ] } ], "parameters": [], "skipLoginFlow": true, "useCookies": true, "authenticationHeaders": [], "notifications": [], "concurrency": 50, "duration": 120000, "rampUp": 60000, "iterationDelay": 1000, "maxIterations": "5", "targetedCountries": [] } ``` {% endcode %} # Performance Testing FAQs ### **Is there a way to run a load test by throughput instead of setting the maximum number of concurrent sessions?** Sure, click the arrow next to **Max Concurrent Sessions**, select **Max Requests Per Second** and set the number you wish. ![](/files/-MXWJu-dK2FYG_BRosx8) ### **I've got an error message below\... what should I do?** ![Unable to run load test](/files/-MMGEWvKJqxrsswqBkWg) The message you've got means that you've reached your plan's maximum credit. Please go to **Settings -> Billing -> Change Plan** and upgrade. If you don't have access to the Billing tab, contact your team admin who will be able to buy more credit. ### **Can I add multiple conditions to a flow control step?** Yes, you can combine multiple conditions by using an **assignment extraction** in the **Extractions - Set Parameters** section. Use the following syntax when assigning a value to a parameter: ``` ${condition1 && condition2} ``` This ensures that the combined condition is evaluated before being used in the flow control step. Do not place this expression directly in the loop input parameter itself. # Performance Testing Troubleshooting Debugging some common issues with load tests ### Load test does not start If your test is stuck at "Pending" state, it is possible that some of its requirements are not being met. For example, if your test is using crowdsourced traffic from a specific country, but there not enough simulation devices are available from this location, your tests will wait in "Pending" state until those will become available. ### **I’m getting an error ‘Unverified domains are already under test’ but our domain has been verified. What do I miss here?** Please make sure your tests are not using other domains like facebook, salesforce etc. Contact support at if you need [an additional domain to get verified](https://docs.loadmill.com/load-testing/setup/domain-verification) or any other assistance is required. ### **Error message: domain name \[domain] is not on white list** As mentioned in the [domain verification ](/performance-testing/domain-verification)page, you are only allowed to test websites that you own. Before your test starts, the list of participating domains (white list) is calculated based on all of the test requests. In case one of your request uses a dynamic parameter to generate one of the test request's URL, we might miss this domain, and not add it to the test's white list. ![HTTP request domain name \[random-website-name.com\] is not on white list](/files/-LuZRsOmQRUGtcVYnK9A) A simple workaround would be to add at least one request for the dynamic domain in one of the other requests. (If you want you can even use a skip condition to disable this request from actually running). # Common Load Test Issues Debugging some common issues with load tests #### Load test does not start If your test is stuck at "Pending" state, it is possible that some of its requirements are not being met. For example, if your test is using crowdsourced traffic from a specific country, but there not enough simulation devices are available from this location, your tests will wait in "Pending" state until those will becomes available. #### Error message: domain name \[domain] is not on white list As mentioned on the [Domain Verification](/performance-testing/domain-verification) page, you are only allowed to test websites that you own. Before your test starts, the list of participating domains (white list) is calculated based on all of the test requests. In case one of you request uses a dynamic parameter to generate one of the test request's URL, we might miss this domain, and not add it to the test's white list. ![HTTP request domain name \[random-website-name.com\] is not on white list](/files/oUj6MIJ8AuJHPWui1811) A simple workaround would be to add at least one request for the dynamic domain in one of the other requests. (If you want you can even use a skip condition disable this request from actually running) # Layout Welcome to the Test Editor Layout. Here you’ll tour all the editor components to understand better and customize your tests. Loadmill’s Test Editor brings ease of access and customization to API testing. Upload a test scenario and generate dynamic code with multiple, expandable API components including extracting and validating dynamic values from the server response, running custom code for every API call, and managing conditional results for every API used in your test case. You can run your tests locally, or in your CI/CD pipeline with a single click. Here’s a tour of the Test Editor to guide you to all its functionalities:
* In the Flows section on the left column, you can rename or delete your flows. * The API editor in the center are all the expandable API components. * On the Flow Runs (or Trial Runs) in the right column, you can see a single test run per flow that includes all its activity logs. # Flows A flow is a composed set of correlated API requests registered from an operation made by the user. A flow can represent any type of operation like adding and deleting a user to a group chat or simply going through a checkout process. ![](/files/FrfJl6XSmeEma8iUnhI5) You can register flows through the [Test Composer Chrome Extension](/api-first-end-to-end-testing/working-with-the-recorder/quickstart) or adding the Loadmill User Behavior Module on your server. When uploading your flow/test scenario to Loadmill, you’ll see the flow added to your test suite on the left side of the platform. By default, the name is the date and time you uploaded your test but make sure to rename it to, most commonly, the name of the operation you composed to a test. {% embed url="" %} # Generated Flow Code After uploading your flow/scenario to Loadmill, its sophisticated AI generates a complete automated dynamic test build that you can easily view, edit and integrate. The generated code is located at the top of the editor and is written and saved in `.yaml` You can edit your test code through the Loadmill platform or through your IDE by setting up Data Sync to your repository. See here how to enable Data Sync to your Test Suite. {% embed url="" %} # Test Flow Editor API Test flow is a series of API calls (HTTP requests) that represent **a single "user" flow you would like to test**. These requests will be executed sequentially until completion or until the first failure. ![](/files/QEJ1mktw7JmWv4sCuv4K) {% hint style="info" %} 🧠 Control visibility of the flows' list area by clicking on Flows in the Test Suite navigation panel. This way you can better focus on the flow you are currently working on. {% endhint %} ## The Test Flow Toolbar
| Option | Description | | ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | AutoSave | Auto saves the test flow unless its configuration is invalid. | | Login toggle | Toggle this switch to enable the flow to use [the Global Login](https://docs.loadmill.com/api-testing/test-suite-editor/global-login-flow) settings. | | CSV toggle | Easily re-run your test flows by using different parameters values by uploading a [CSV file](https://docs.loadmill.com/api-testing/test-suite-editor/api-tests-data-from-csv-files) containing a list of the values. | | Flow status (default: ![](/files/5RT0s8svV9roWXSotBZj)) | Set the [status of the flow](https://docs.loadmill.com/general/api-testing1/analyzing-an-api-test-results#flow-status). | | Run Flow button (![](/files/HuEf7uBdq9peO008MFWj)) | Run the edited flow as a "Trial run" to debug and validate it. | | Run with overrides (![](/files/FbaGJSD5cy1MWYg1f6hz)) | Run the edited flow as usual, but [override some parameters](/test-editor-reference/flows/flow-execution#running-with-overrides). | | Debug mode toggle (![](/files/XxZdBGk4sN0C280F8p9B)) | Turn the [Debug Mode](https://docs.loadmill.com/test-editor/flows/flow-controls#debug-mode) on/off. | | Flow scope parameters (![](/files/YltJ7ZZc2XI1unP8U6EK)) | Set parameters which can be used in a specific flow. | | Edit flow code (![](/files/3Ntn9WC9gO1xnfVgVkea)) | Edit the raw YAML code of the saved flow. | | Edit using AI (![](/files/kHVVcDjIR0ih10D7eEOd)) | Describe the desired changes and get a suggestion made by AI. | | Restore deleted requests (![](/files/zX27kkp7f3nG0BD2Gb7k)) | Allows to restore previously deleted requests (up to 10 requests), and the restoration stack clears upon refreshing the page. | | Add request | Add en empty request to the flow | | Paste requests | Paste a request - accepts CURL, and copied requests from the flow. |
| Option | Description | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Export Flow | Save the flow to a YAML file. You can then import the flow into another test suite or as a load test. | | Duplicate Flow | Create a copy of the original flow within the Test Suite. | | Convert to Load Test | Converts the flow to a [Load Test](#converting-api-flow-to-load-test). | | Replace Parameter | Replace the usage of a parameter in this flow with another parameter. (Find more information about parameters [here](https://docs.loadmill.com/api-testing/test-suite-editor/parameters)). | | Extract Parameter | Extract a value to a parameter from all across the test requests. | | Copy Flow to Suite | Copy the flow into another Test Suite. | | Generate Descriptions | Generate a suitable description to all requests in a flow using AI. | | Configure service Definition | For GRPC, allows selecting the service that will be used in the flow. | | Get Last Trial Run | Open the last run's results window. | ### **Converting API flow to Load Test** Easily convert an API flow to Load test by clicking on the Flow options and select "Convert to Load Test". Once you've selected ![](/files/Yfj5y4GZ5kPqkJlB6uJY), a new Load test will open containing all the requests and properties taken from the original API test. # Flow Controls Flows provides additional features and functions that can be found in the top right corner of the editor. {% embed url="" %} ### Debug Mode Working on a flow often requires checking or verifying certain things during execution. For example, if the flow involves adding then removing an item from an online shopping cart, we might want to verify that the item was indeed added to the cart during the execution. Loadmill provides the functionality to execute tests in debug mode. When the debug toggle (![](/files/XxZdBGk4sN0C280F8p9B)) is turned on (![](/files/LAxh1OeljDgD29BXCS6h)), a small checkbox will appear next to each request in the flow, allowing you to set a request as the breakpoint for debugging. Once the flow is executed, the test will be paused right before the checked request is made. The results window will already show the statuses of completed requests, with a 'pending' status indicating the upcoming request, which serves as the breakpoint. {% embed url="" %} #### Export Flow Exports the current flow you're viewing (in `.yaml`). #### Duplicate Flow Duplicates the current flow on your Test Suite. #### Convert to Load Test Coverts the current flow into a Load Test. See here how to setup Load testing with Loadmill. #### Replace Parameters Replaces the given parameter generated by Loadmill or by the user to a new parameter name. **ex:** Loadmill uses `${origin_`*`1}` to define the base url of the test.* By applying Replace Parameters you can replace `${origin1}` to `${base_url}`. {% hint style="info" %} This only changes the name of the parameter without affecting its value. {% endhint %} {% embed url="" %} #### Extract Parameter Extracts a given value from the Test Flow into a parameter. **ex:** Say you have a unique value that you want to parameterize like a visitor id, a specific name or even a phone number. You can extract those values and parameterize them by applying the Extract Parameter tool. {% embed url="" %} {% hint style="info" %} All parameterized values are located in the PARAMETERS tab of the suite. {% endhint %} #### Copy Flow To Suite Copies the current flow to the chosen suite. #### Suggest Failure Once a flow is executed and fails, this button will appear ![](/files/37DZJuF21YyN8fqr5N5V) which, using AI, provides you with a possible reason and solution for the test, according to the tests agenda. For example:

In this example, it seems like one of the request used the wrong method

#### Fix It For Me As AI suggests whats wrong with the test, and gives a possible way to solve it, why wouldn't it just solve it by itself?\ The 'FIX IT FOR ME' button, leads to the edit-using-AI panel prompt prefilled with the suggestion AI provided before.
Then it acts like the regular 'Edit Using AI' feature explained [here](https://docs.loadmill.com/test-editor/flows/test-flow-editor#the-test-flow-toolbar). # Groups A Group lets you bundle multiple steps together under a shared label. This keeps your flow organized and makes it easier to understand at a glance what a set of steps is doing. Groups also unlock features like **Repeat Group**, which lets you run the same set of steps multiple times automatically. ![Steps group in the flow editor](/files/HQR3yHQISoshcHI7Fb71) ## Creating a Group To create a group: 1. In the flow editor, select the steps you want to group together using the checkboxes next to each step. 2. Click the **Group** action that appears at the top. 3. Give your group a descriptive name (e.g., "Create user and add to team"). The grouped steps will now appear together under a collapsible section with your description. ![Group steps in the flow editor](/files/FbofSzEONRR0kMMiRGJA) ## Repeat Group Repeat Group lets you run the same group of steps once for each item in an array. ### How It Works When you configure a Repeat Group, you provide: 1. **An 'array parameter'** — an array to iterate over (e.g., a list of user IDs). 2. **A 'set value to' parameter** — the name of the parameter that will hold the current item on each iteration (e.g., `user_id`) On each iteration, Loadmill takes the next item from the array and assigns it to your chosen parameter. All steps in the group execute with that value before moving on to the next item. You can also set an optional **delay** (in milliseconds) between iterations if needed. ![Repeat group configuration in the flow editor](/files/KZ42NIZ8oL1UeFU10gdO) ### Example: Adding Users to a Team Use Repeat Group when you need to run the same steps (e.g., create user, add to team) once for each item in a list returned by an earlier step. * You have a parameter `userIds` with the value: ```json ["user_1", "user_2", "user_3"] ``` * In the Repeat Group settings: * **Array parameter:** `userIds` * **Set value to:** `user_id` * **Delay:** `300` ms **What happens when the flow runs:** | Iteration | `${user_id}` value | | --------- | ------------------ | | 1 | `user_1` | | 2 | `user_2` | | 3 | `user_3` | Each iteration, the group's steps execute using the current `${user_id}` value. For example, if one of the steps sends a POST request to add a user to a team, the request body might look like: ```json { "userId": "${user_id}", "teamId": "${team_id}" } ``` On the first iteration, `${user_id}` will be `user_1`. On the second, `user_2`. And so on until all items in the list have been processed. ![Repeat group report](/files/FdsezsWHqp8eZ4wImkHJ) # Add CSV to Flow When you're ready to take your automation testing to the next level, it's time to start thinking about testing your flow with multiple variations. This means creating different versions of your test cases and running them against different versions of your software. Doing this can help you find bugs that only show up in specific combinations of your software. This is important because it helps to ensure that your tests are comprehensive and that they cover all possible scenarios. Creating multiple variations of your tests also allows you to test different parts of the API in isolation, which can be helpful in identifying issues. Loadmill provides a quick, easy method in testing different iterations of your flow by simply uploading a CSV file containing the different iteration values. Here's an example for a user management application. This app allows you to add and remove users from your dashboard. {% embed url="" %} To start with, let's try 5 different email addresses, as well as 5 first names and 5 last names. We'll divide the task into 4 easy steps. ### Create a CSV file. The following table shows the variations we want to test. {% hint style="info" %} The **expected\_outcome** will not be uploaded to the test. This is simply to show you the expected outcome for each variation tested. {% endhint %}
emailfirst_namelast_nameexpected_outcome
variation1@loadmill.comJohnDoepass
variation2@random.comDoe2pass
variation3VariationThreefail > wrong email
variation4@John4Doe_4fail > wrong email
variation45@companyJohn_$fail > wrong email
### Update parameter names. The *"Create a user"* API contains hard-coded values that need to be substituted with the parameter names from the CSV headers (***email**, **first\_name** and **last\_name***). To do that, replace the string value to ${...parameter name...}. The ${} wraps the values you set on your CSV file by pointing the parameter name to the column header name. Request Body (before) ```json { "email": "${__escape_quotes(email_in_request_body_1)}", "firstName": "John", "lastName": "Doe", "customId": "johndoe@loadmill.com", "environmentId": "da7167ae9563468a9bcb6eb157011745", "tenants": [ "e7f403f4c30d4bc3840456a7953bedb4" ] } ``` Request Body (After) ```json { "email": "${email}", "firstName": "${first_name}", "lastName": "${last_name}", "customId": "johndoe@loadmill.com", "environmentId": "da7167ae9563468a9bcb6eb157011745", "tenants": [ "e7f403f4c30d4bc3840456a7953bedb4" ] } ``` ### Upload CSV file. Now that you're done, the last step is to upload your CSV file by clicking on the CSV button. {% embed url="" %} ### Run the suite. By running the suite, we see the 5 iterations of the flow we created, each sending the server the values from the uploaded CSV. {% embed url="" %} We notice that the last two test iterations have passed when the expected outcome would have been to fail. That is a crucial part in testing your server as it allows you to understand how your API behaves in those scenarios. Overall, creating multiple variations of your API tests is important because it helps to ensure that the API is functioning correctly and that it meets all expectations. By doing so, you can avoid potential issues and ensure that your API is ready for production. # Flow Execution ### Running with Overrides When running a flow within the flow editor for debugging purposes, it can be very convenient to replace certain parameters for a single run. Instead of changing the parameter value in the global parameters window, you can simply click the 'run with overrides' button (![](/files/2vTqpUSzrw8k5slIAEEM)), and then set values for a specific execution. This won't affect the global parameters, providing a quick and simple way to debug your flows. i.e.: This is the global parameter: ![](/files/09CDVwsKSIZn0nO3ZVdA), and we want to check something specific and change the value of `email` for only one run. We'll use the 'run with overrides' button to execute it and reset its value. For example:

Good to know: the override parameters in this window are saved for subsequent 'runs with overrides' as well.

### Assigning Labels Flow Labeling is an efficient way to save time and enhance productivity during application testing. By executing only relevant flows, you can avoid redundant testing and concentrate on critical aspects. To utilize Flow Labeling, simply label your flows with specific keywords for identification. Then, assign these labels to relevant sections of your application's CI. This ensures that only relevant flows are executed when changes are made. Testing and validation are crucial in product and software development, but maintaining this process amidst diverse requirements can be challenging. Flow labeling is essential for developers to effectively test every aspect of their product. You may need to test multiple operations or ensure your application loads correctly. However, each update or bug fix undergoes a categorical test scenario that may not encompass all test cases from your Test Suite/Plan. Creating and assigning labels to flows is a straightforward process with Loadmill. Just select the flow and click on the label icon ![](/files/pXxY57tcjmOX37W3SxPY) to assign a label of your choice. You can either assign a pre-made label or create a new one yourself. The following Test Suite represents a user management application with 6 different flows. Each flow is labeled ***"operation"*** or ***"sanity"*** or both. {% embed url="" %} {% hint style="info" %} Make sure your flow is attached to the CI in order to run it based on labels. {% endhint %} ### Installing Loadmill NPM Package Before you start, install the [Loadmill NPM Package](https://www.npmjs.com/package/loadmill). ### Running Labeled Flows via CLI Now that we've labeled our flows correctly, we'll run them according to label we pass in to the command line. We want to run all flows within a test suite that are labeled ***"sanity"***. Let's quickly tour what values you need before running our tests.
parameterwhere to find
<test-suite-id>Navigate to your test suite page and the suite id is located inside the url
<token>The token is generated from the access token page
Open your CLI and run the following command: ``` loadmill --test-suite -w -t --labels --report "label1,label2" ``` {% embed url="" %} To conclude, flow labels allow us to focus our tests based on our needs regardless of the amount of test cases we have on our test suites or test plans. We could schedule daily test runs on ***"sanity"*** labeled flows and run "operation" flows for every time we push an update to our code. # Steps Steps are the core components of flows, made up of one or more steps of varying types. They generate, process, and share data. In the code editor, steps can be copied, created, edited, and deleted. #### Steps views There are two views in the flow editor. 1. Summary View: Displays a scrollable area containing all the steps, with each step collapsed to show summarized details.\\

Summary view - Collapsed steps example

2. Expanded view: Displays single steps at a time expanded with all the available functionality.\ note, some function will require additional expansion.

Expanded step example

{% hint style="info" %} Navigating between steps in Expended view can be done by keyboard up/down keys and by clicking on the step name on the left index list (when available on wide views). {% endhint %} **Switching between the views** can be dome by clicking on the title area of a step in non active areas as illustrated below
#### Adding a new step In Summary view, Click on the dotted menu (![](/files/OAb5T8H7O1jmvLk4K1bR)) located in the separation line between any two steps and in the beginning and the end of the flow, depends on where you'd like to introduce the new step.
Select the `+ option`Then click on the desired step type ![](/files/45puyv8cZRUJQTyFkJmP) A new empty step will be added. #### Copying step(s) * To copy a single step click on the copy button (![](/files/nQG9xvLFtqGK41jCEC3n)) located on the top right corner of every Step
To paste the step in the desired place, locate the divider between any two requests or at the beginning or the end of the flow and click on the dotted menu (![](/files/q3HezUu0X2kpXzuYKvSR)) then on the paste button ![](/files/StMeuVajk8aCs4fVXQ8a) * To copy multiple steps, use the checkboxes located left to the description of each step then click on the copy button located at the top.
to paste the copied steps, locate the divider between any two requests or at the beginning or the end of the flow and click on the dotted menu (![](/files/q3HezUu0X2kpXzuYKvSR)) then on the paste button ![](/files/StMeuVajk8aCs4fVXQ8a) same as individual flow. {% hint style="info" %} Tip1: To copy all the requests click on the checkbox at the top then use copy/paste as already demonstrated.\ ![](/files/bDoHgXGLbjEg6szhOP0c) {% endhint %} {% hint style="info" %} On empty flows, this top menu and the dividers will be missing, but you still paste steps by clicking the "paste" label at the top right corner of the screen![](/files/oUj6MIJ8AuJHPWui1811) {% endhint %} # Request Step Request is the main building block of tests. It represents a single API call or a user action. ![](/files/-Ml4iL1e0ky9rMeHBILr) ## Request sections When we expand the request editor by clicking at the top-right of the request, we can see that a test request includes two main sections - **Request and Response Handling sections**. ![](/files/-Ml4lQnkGb7kR4glnph6) ![](/files/-Ml4kWzOqsz0DHlxPB9w) ### The Request Handling section ![The request handling section](/files/-Ml4mPc7Zo_SNpLVh1RM) * The **Method** and **URL** fields are the most basic requirements for a valid request. * Black input fields are fields in which you can use [parameters](https://docs.loadmill.com/api-testing/test-suite-editor/parameters) and [functions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions). * It is recommended to set the Request description in a way that describes its action. This will make it easier for you to debug your tests later if this request fails. * Some requests require a **body**. Selecting the right **content-type** for your request will help us highlight the syntax of your request body (i.e. JSON or XML). * The **Headers** section allows to see and set the request headers. You can add header to all requests or as a Global Header so that it will be added to all of the test flow requests and appear in [the Global Login tab](https://docs.loadmill.com/api-testing/test-suite-editor/global-login-flow). #### GraphQL requests A standard GraphQL POST request should use the `application/json` content type, and include a JSON-encoded body of the following form: ```json { "query": "...", "operationName": "...", "variables": { "var1": "value", } } ``` `operationName` and `variables` are optional fields. `operationName` is only required if multiple operations are present in the query. The response should return the queried data in a JSON. i.e., ```json { "data": { ... }, "errors": [ ... ] } ``` Based on ([graphql.org](https://graphql.org/learn/serving-over-http/)). ### The Response Handling section ![](/files/-Ml4nYb5LN9t8YgmOMgP) ![The response handling section](/files/-Ml4naxy0fC4hflfFBkN) The response handling section includes 3 sections: [**Extractions - Set Parameters**](/test-editor-reference/set-parameters-extractions), [**Postscript - run code**](/test-editor-reference/postscript) (disabled by default), [**Assertions - Verify Response**](/test-editor-reference/assertions) and Flow control. #### The **Extractions - Set Parameters** section allows you: * To extract values from the response body into an existing or a new parameter using JSONPath, Closure, jQuery, RegExp. * To extract an HTTP header value into a parameter using the Header extractor. * To assign a static value or [**apply a function on another parameter**](https://docs.loadmill.com/api-testing/test-suite-editor/functions) using the "Assign" option. * To add an extraction from [the Suggestions repository](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions#suggestions). Team admins configure the repository within Settings - Suggestions. ![](/files/-Ml4p1yW4r0domoc1bB8) The **Assertions - Verify Response** section allows you to validate the request response. ![](/files/-Ml4pH1lRzMW7AKhSt9L) The **Flow Control** section allows to repeat the request until the parameter's value meets the requirement or number of iteration reached: ![](/files/-Ml4r93brFyxkz-8L5cR) You can also set Skip and Stop conditions between requests within this section: ![](/files/-Ml4rTipI-4BcR58NI9i) # Code Step The code steps allows running JavaScript code snippets at any point without spinning an entire request. Among it's usages are introducing initialization logic at first step of a flow, processing specific data right after a generic shared flow, and providing more visibility and clarity in flows and it's special logic steps.
The code step consists of two areas: 1. Code editor - a place to write your JavaScript snippets 2. Flow control - Allows skipping to another step, stopping the flow and add wait time after step execution. # Extraction and Assertion Step The extraction & assertion step allows extracting, assigning parameters and add validations without spinning an entire request. Among it's usages are assigning or allocating values at the beginning of a flow of after calling a generic shared flow and we need to extract/re-assign/validate additional flow specific data.
The Extract & Assert step consists of three main areas: 1. Extraction section - a place to assign and extract the parameters.\ Note that the extractions will refer to the last response that was introduced, ie. the previous response.\\
Each extraction consists of three parts: 1. Parameter name - the name that the parameter will be referenced when used, this filed can accept letters, numbers and undescores (\`\_\`). 2. Assignment method/type - A selection of methods that can be used to acquire the value for the parameter. The available methods are: Assign, JSONPath, Clojure, xPath, RegExp, Header and WebSocket. For detailed explanation about each one of the methods please refer to the [Extractions documentation](/test-editor-reference/set-parameters-extractions). 2. Assertions area - A place for data validations (Assertions)
The assertions section consists of three areas: 1. The parameter that carries the value for the validation. 2. The comparison method (the operator), also referred as the value, the available operators are\ Exists, Doesn't exists, Equals, Doesn't equals, Contains, Doesn't contains, Matches, Greater than, Less than, JSON Schema, JSON Contains, XML Contains.\ For detailed explanation about each one of the comparison methods please refer to the [Assertions documentation](/test-editor-reference/assertions). 3. The expression - when applicable, this value will carry the compared value or expression for the validation. 3. Flow control area - Allows skipping to another step, stopping the flow and add wait time after step execution. # WebSocket Step The WebSocket step enables real-time, bidirectional communication testing by allowing you to connect to WebSocket endpoints, send messages, and validate responses. This step is particularly useful for testing applications that require persistent connections and real-time data exchange. ## WebSocket Step Operations A WebSocket step can perform three main operations: 1. **Connect** - Establishes a WebSocket connection to the specified endpoint 2. **Send** - Transmits messages through an established WebSocket connection 3. **Receive** - Listens for and validates incoming WebSocket messages ## Step Configuration ### Connection ![](/files/meRaImRnhlwSh8DkPG3n) When establishing a connection: * The URL format follows: `wss://` * Headers can be added to the WebSocket handshake request for authentication or custom protocols * A timeout value (in seconds) can be set for the connection attempt ### Sending Messages ![](/files/m936MgTnxjDlf6EUGMDQ) For sending messages: * Supports two data types: * Text (JSON, plain text) * Binary data * Messages can include dynamic parameters * Headers can be included with the message * Used for both sending data and subscribing to channels * A timeout can be specified for the send operation Note: A Send step without a preceding Connect step will automatically create a new WebSocket connection. ### Receiving Messages ![](/files/AKW8CaCWh1d9nlJWWzYo) When receiving messages: * Messages are fetched based on the **WebSocket URL** * Use URL query parameters (e.g., `wss://example.com/ws?userId=1`) to send and filter messages by specific properties * **Message retrieval options**: * Last - Returns the most recent single message * Last 10 - Returns an array of the last 10 messages * Last 50 - Returns an array of the last 50 messages * **Extractions** support all standard Loadmill extraction methods * Example JSONPath for array filtering: `$[?(@.id == "${my_id}")]` * [Learn more about extractions](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) * **Wait for extractions to be resolved** - Since WebSocket connections are asynchronous, this setting defines how long to wait for messages that match your extraction criteria. Default is 5 seconds, but you may need to adjust this based on your application's behavior. * **Assertions** - Use the same powerful assertion capabilities as HTTP requests to validate your WebSocket messages, including JSON/XML validation, value comparisons, and regex matching. [Learn more about assertions](https://docs.loadmill.com/api-testing/test-suite-editor/assertions) * The **Flow Control** section allows to repeat the request until the parameter's value meets the requirement or number of iteration reached. ## Example Run Let's look at a simple WebSocket test flow in action: ![](/files/pOLXeMJ7eegswZ6qGY9w) This example demonstrates a complete WebSocket interaction: #### 1. Connect to WebSocket: * Establishes connection to a WebSocket server #### 2. Send Some Data: * Sends a message on the connection #### 3. Assert message received: * Waits for all extractions to be resolved * Once resolved, validates that the extracted `${message}` exists * Received messages are displayed for easy debugging # Playwright Step The **Playwright step** allows you to run UI automation scripts using [Playwright](https://playwright.dev/) as part of your Loadmill flow. This is especially useful when validating UI behavior alongside your API steps — for example, logging in through the UI, verifying elements on the page, or simulating full end-user workflows. ![Playwright step editor](/files/hPsnaKZMCKzK7i8GIpxv) ## Usage The Playwright step contains a single **code editor** where you write your Playwright script. > 🧠 Only write the contents of a `test()` block — do **not** include the test declaration itself. ✅ **Correct:** ```ts await page.goto('https://www.loadmill.com'); await expect(page).toHaveTitle(/Loadmill/); await page.getByRole('textbox', { name: 'Email' }).fill(email); ``` ❌ **Incorrect:** ```ts test('Login test', async ({ page }) => { await page.goto('https://www.loadmill.com'); await expect(page).toHaveTitle(/Loadmill/); await page.getByRole('textbox', { name: 'Email' }).fill(email); }); ``` ## Parameters and context In the Playwright step, **all suite and flow parameters** are available directly as variables — no need to use `${param}` syntax like in other inputs. ✅ **Correct:** ```ts await page.getByRole('textbox', { name: 'Email' }).fill(email); ``` ❌ **Incorrect:** ```ts await page.getByRole('textbox', { name: 'Email' }).fill(${email}); ``` ## Cookies Session cookies are passed automatically into the Playwright browser context, so you can stay authenticated between API and UI steps. To manually set cookies, you can use Playwright’s built-in [`context.addCookies`](https://playwright.dev/docs/api/class-browsercontext#browser-context-add-cookies) method: ```ts await context.addCookies([ { name: 'my_cookie', value: '12345', domain: 'your-app.com', path: '/', httpOnly: true, secure: true, sameSite: 'Lax', }, ]); ``` ## Attaching screenshots to the report You can attach screenshots directly to your test report using Playwright’s `testInfo.attach` method. ✅ Example: ```ts await page.goto('https://loadmill.com'); const screenshot = await page.screenshot(); testInfo.attach('Loadmill Page', { body: screenshot, contentType: 'image/png', }); ``` ## Changing timeout You can change the default timeout for Playwright actions using the [`setDefaultTimeout`](https://playwright.dev/docs/api/class-page#page-set-default-timeout) method: ```ts page.setDefaultTimeout(10000); // 10 seconds ``` ## Changing view resolution To set the browser viewport size, use the [`setViewportSize`](https://playwright.dev/docs/api/class-page#page-set-viewport-size) method: ```ts await page.setViewportSize({ width: 1280, height: 720 }); ``` ## Requirements To use Playwright steps, a [private agent](https://docs.loadmill.com/integrations/testing-localhost-application) with UI tests enabled is required. You can enable UI testing using one of the following options: * **Desktop App** - UI testing is supported out of the box with no additional configuration required. * **Docker** - Set the following environment variable when running the container: ```bash docker run -it --rm -e LOADMILL_AGENT_TOKEN= -e UI_TESTS_ENABLED=true loadmill/agent ``` * **NPM Package** - Run the agent with the `--ui-tests` flag: ```bash loadmill-agent start -t INSERT_TOKEN_HERE --ui-tests ``` > ⚠️ This option requires additional dependencies and lacks the isolation benefits of Docker or the Desktop app. # UI Agent Step The **UI Agent step** allows you to automate UI interactions using natural language prompts. Instead of writing code, you describe what you want the agent to do, and it autonomously navigates your application, performs actions, and validates results. This step is especially useful for: * **Hybrid testing** — If most of your test logic is in API steps but you need to validate something in the UI, you can add a UI Agent step to open the app and verify a specific element or state without building a full end-to-end UI test. * **Hard-to-automate actions** — When certain actions are difficult to replicate via API (e.g., complex UI interactions, OAuth flows, or third-party widgets), you can use the UI Agent step to perform those actions and then continue with your API-based test approach. * **End-to-end UI testing** — Quickly create full UI tests by describing user flows in natural language. ![UI Agent step editor](/files/fu9vyEw4uWnpi8QYc6Ak) ## Step structure The UI Agent step consists of three main parts: ### 1. Navigate to URL The starting point for the agent. This is the URL the browser navigates to when the step begins. **Example:** ``` https://example.com/login ``` ### 2. Agent Instructions (optional) Instructions describing the actions the agent should perform before reaching the UI state you want to validate. This field is optional — if you only need to validate something on the page without performing any actions, you can leave it empty. Each action should be written on a separate line. Use this to describe interactions such as: * Clicking buttons or links * Filling out forms * Navigating through menus * Scrolling or hovering **Example:** ``` Enter "user@example.com" in the email field. Enter "password123" in the password field. Click the Login button. ``` ### 3. AI-Driven UI Validation Describe what you want to verify on the UI state after the actions are complete. This is where you specify the expected outcome of the test. Common validations include: * Verifying an element is visible * Checking that a button is clickable * Confirming an item was deleted or removed * Validating that specific text or content is displayed * Ensuring error messages appear (or don't appear) **Example:** ``` Verify that the dashboard page is displayed and the welcome message shows "Hello, User". ``` ## Advanced settings ### Timeout Configure the maximum time (in seconds) the agent is allowed to complete the step. If the step takes longer than the specified timeout, it will fail. Set a reasonable timeout based on the complexity of your flow and the expected response time of your application. ## Parameters You can use parameters in any of the UI Agent step fields. Parameters work the same way as in other parts of Loadmill — use the `${parameterName}` syntax, and the value will be resolved at runtime. **Example:** ``` Enter "${userEmail}" in the email field. Enter "${userPassword}" in the password field. ``` This allows you to: * Reuse the same test with different data sets * Reference values extracted from previous steps * Use suite-level or flow-level parameters ## Viewing results After running a test with UI Agent steps, you can view: * **Validated screenshot** — A screenshot of the UI state that was validated * **Step-by-step actions** the agent performed * **Validation results** showing what passed or failed If the test failed, a **session recording** is also provided. This allows you to watch exactly what the agent did during execution and identify where it went wrong, making it easy to debug and tweak your prompts to improve stability. ![UI Agent step run result](/files/RKQKWzm72dRkrPqcL04p) ## Best practices * **Be specific in your instructions** — The more precise your prompts, the more reliable the test. Instead of "click the button", say "click the Submit button" or "click the blue button labeled Continue". * **One action per line** — Write each instruction on a separate line for clarity and consistency. * **Keep validations focused** — Use a single, clear validation statement. Avoid combining multiple unrelated checks in one validation. * **Start simple** — Begin with short, focused flows and expand as you gain confidence in the agent's behavior. * **Use parameters for variable data** — Avoid hardcoding values that may change; use parameters instead to make tests reusable. ## Requirements To use UI Agent steps, you need: * A Loadmill account with AI Agent capabilities enabled * The [Loadmill desktop app](https://app.loadmill.com/app/user/settings/desktop-app) installed and running {% hint style="info" %} UI Agent steps require the desktop app because the agent needs to control a real browser to interact with your application's UI. {% endhint %} ## Related documentation * [Agent Testing Quick Guide](/web-ui-and-hybrid-testing/agent-testing-quick-guide) - Get started with agent testing * [Playwright step](/test-editor-reference/steps/playwright-step) - For code-based UI automation # Extractions Use the **Extractions - Set Parameters** section of the request to extract values from the responses. These extracted values can be use in any of the following requests or to assert and verify your success criteria. ![The request Extractions - Set Parameters section](/files/-Ml9SIw6apzelQsYPuvq) ## Extraction Types ### Assign Use this to assign any combination of existing parameter or response values to another parameter ![](/files/-Ml4wxHOLpnEkj6LomQZ) ### JSONPath Use this extractor to extract values from JSON responses. JSONPath is a query language for JSON objects, much like XPath for XMLs. ![](/files/-Ml4xTxazsrtLhgsH-Pj) You can easily find a relevant JSON Path expression by using our JSON Path Finder feature. To use it, just run your test flow, go to the Response body section, click on JSONPATH and copy the JSON Path expression you need. ![](/files/-MZHL8kEPdzyenDocHew) ![Copying a JSON Path expression from the finder](/files/-MZHLLZDiwG--47lse6Q) Also find helpful JSON Path expressions [here](https://goessner.net/articles/JsonPath/index.html#e2). ### Clojure Use this extractor to extract values from Clojure (EDN content type) responses. You need to enter JSONPath Query, Clojure queried as JSON. ![](/files/-Ml4xqLtYAdC5jlqkRlN) ### jQuery Use this extractor to extract values from HTML or XML responses. jQuery extraction syntax is very similar to XPath and may be very useful to extract values from HTML pages (such as [security tokens](https://portswigger.net/web-security/csrf/tokens)). ![](/files/-Ml4y8IEMq0TepNcmdjG) ### RegExp Use this extractor to extract values from any type of response using RegExp. The first value captured by the regex (in parentheses) will be set as the parameter's value. You can test your regex expressions using this useful online tool - [link](https://regex101.com). ![](/files/-Ml4yOojCAMr1gVPkRMa) ### Header Use this extractor to extract values from any of the response headers. ![](/files/-Ml4ygdP5i_ge2e3i0Z0) ## Suggestions In many cases Loadmill users use the same or similar extractions and [assertions](https://docs.loadmill.com/api-testing/test-suite-editor/assertions). We've implemented the Suggestions feature that allows team admins to configure a repository of extractions and assertions within **Settings - Suggestions**. Then, each user can add the extractions from the repository by clicking **+ SUGGESTIONS**. ![](/files/-Ml4ysn3rQNF_J71IsXR) By default, there are a few common extraction and assertion examples in the repository. Team admins can also navigate to the Suggestions Settings directly from within the suggestion dialog window. ![](/files/-M_G7232wBDcWdKiNqM3) ## Autocomplete {% hint style="info" %} :man\_mage: Use the autocomplete option to see a list of existing parameters. To see it, press Shift + Cmd (Ctrl for Windows) + Space. See how it works below. {% endhint %} ![](/files/1vy5dZarpZ7rgqGoKVlR) # Assertions When testing an API, it is usually not enough to verify that all our HTTP requests completed successfully. Often, it is necessary to make sure that we got the correct response from the server by examining the **response status, body or headers**. With [Loadmill](https://www.loadmill.com), this is made easy by using **Assertions**. Assertions are used in conjunction with [parameters](/test-editor-reference/parameters) to do just that: examine the server's response and assert its correctness. You may have an arbitrary number of assertions executed after each successful request. If an assertion fails, the next request will **not be executed** and the test scenario will be marked as **failed**, but the subsequent assertions for the current request **will be executed** nonetheless. ## Assertions editor Assertions editor can be found in loadmill under each request cubical on it's expanded form.\\

Assertion section in request editor

Furthermore, there is an option to disable an assertion by clicking the three dots (![](/files/aHyW1ye2vxBOGm5ffYPd)) in the assertion line. A disabled assertion will still be evaluated, but it won't cause the test to fail.
## Assertions Creation from Response It is also possible to create assertions directly from the response panel from the 'JSONPATH' window (only when the response is a JSON). After executing the test, click on one of the responses to expand it, which will open the response panel, as follows:
Click on ![](/files/TicC5ocbItUyBaxmBben), then navigate to the desired parameter. For example, let's find the 'content-length':
Now, simply click on the plus sign located in the top right corner, set the extraction, and add the desired assertion right from there!
You can then see both the extraction and the assertion in the request editor panel:
## Assertion Types The target of an assertion is always a parameter value. You may use built-in parameters, default parameters or any parameter extracted from the current or previous requests in the **current scenario** as the target. ![The request Assertions - Verify Response section](/files/AHGrkGD6YwCjp89VsVTC) When using assertions, it will usually require to use [extractions](/test-editor-reference/set-parameters-extractions) first to set the parameter with proper data and only then to attempt to validate it's form. There are several types of assertions: #### **Exists** - Validates that a parameter exists and not null. * Example 1:\ Assigning empty term to `param1` will result in assertion failure\ ![Click to enlarge the image](/files/AytStooaHpofsBv1vRDz)![](/files/wYEOq5U1tMZl25O0tgUq) * Example 2:\ Assigning any value to `param1` will result in passed assertion\ ![Click to enlarge the image](/files/mR8N4TSZAZpqGRaeDa2F)![](/files/lHTdef4Vu1Hr3hqqfh6y) #### **Doesn't exist** - Validates that a parameter doesn't exists or null. * Example 1:\ Assigning any value to `param1` will result in assertion failure\ ![Click to enlarge the image](/files/y4JEJpe77IMRnNRHTisi)![](/files/VUszTrxUNkFNb3Qkj6fV) * Example 2:\ Not setting `param1` entirely will result in passed assetion\ ![Click to enlarge the image](/files/NWv3A3PxdYSgIg4PDZ8u)![](/files/VYgSeJeRIOSXjKN0zs3Y) #### **Equals** - Validates that a parameter is equal to the given expression. The equality check is **case sensitive.** * Example 1:\ The assigned `param1` doesn't equals to loadmill hence will fail in assertion\ ![Click to enlarge the image](/files/VXXerdgTmy4jbzsvD0Q7)![](/files/Od2RTLZfZr9uPOVrVMf7) * Example 2:\ Extracting `$.args.app` json path field to `param1`(from the result that is not displayed) should be equal to "Demo" and will result the assertion to pass\ Assigning numeric value to `param2` of 11.0 in postscript should be equal to "11", and the assertion will pass.\ ![Click to enlarge the image](/files/eSamPaQZOjzFmACiIema)![](/files/mLCFop4OSQNvIzkJXr6t)\ \&#xNAN;***Note** that all values are implicitly compared as strings.* #### **Doesn't equal** - Validates that a parameter is NOT equal to the given expression. The equality check is **case sensitive**. * Example 1:\ `param1` shouldn't be equals to 'demo' but it is, hence the assertion will fail\ ![Click to enlarge the image](/files/gQjaj2p6DSAddWszqx0y)![](/files/MXZo4ca7TRYHkcFecZhe) * Example 2:\ `param1` doesn't equals to 'loadmill' hence the assertion will pass\ ![Click to enlarge the image](/files/xJm99VgVFBlUR08vW3LV)![](/files/eFB66jd3pBqa4h9nq5pN) #### **Contains** - Validates that a parameter contains a sub-string. The containment check is **case sensitive**. * Example 1:\ When `param1` set with the value 'loadmill-demo' and doesn't contains the string 'example' the assertion will fail.\ ![Click to enlarge the image](/files/C1UKMWTkBo9pe8TFVSBM)![](/files/vJzqSp8xVSsU1NOWvdO3) * Example 2:\ When `param1` set with the value 'loadmill-demo' and contains the string 'demo', the assertion will pass.\ ![Click to enlarge the image](/files/bRldyTBf7vrJugDqv6G7)![](/files/k6bDSDgB4vbRJpsS05z3) #### **Doesn't contain** - Validates that a parameter DOESN'T contain the given expression. The containment check is **case sensitive**. * Example 1:\ When `param1` set with the value 'loadmill-demo' and contains the string 'demo', the assertion will fail.\ ![Click to enlarge the image](/files/IWfl6e2CEDhlROTjbn8b)![](/files/vpLRcE70Jhn6pDeRN7zQ) * Example 2:\ When `param1` set with the value 'loadmill-demo' and doesn't contain the string 'example', the assertion will pass.\ ![Click to enlarge the image](/files/TCWWpeHsuGIz3128J8CJ)![](/files/QaQyDlJkKprkt3GSTWEw) #### **Matches** - Validates that a parameter matches the given regular expression. * Example 1:\ The assigned epoch representation of the current time to `param1` will contain 13 digits would not match the regex (`^\d{10}$`), hence the assertion will fail\ ![Click to enlarge the image](/files/04ZAf9VIaMni1RiTvQKk)![](/files/GfjGr5BZyPcSagiEHkbI) * Example 2:\ The assigned epoch representation of the current time to `param1` will contain 13 digits and would match the regex (`^\d{13}$`), hence the assertion will pass\ ![Click to enlarge the image](/files/YH6l7XF6Anj5ZWir6pza)![](/files/K4A1dORgUyycbjNhwHb1) #### **Greater than** - Validates that a parameter is greater than the given expression. * Example 1:\ `param1` is assigned with the value 31 that is not greater than 40 hence the assertion will fail.\ ![Click to enlarge the image](/files/fN2IRr0hn1ay4D1v8VjU)![](/files/EPf0fd9miUcSahkw1pea) * Example 2:\ `param1` is assigned with the value 31 which is greater than 30 will make the assertion to pass\ ![Click to enlarge the image](/files/zuma6YdYSQaLJFRvyh5m)![](/files/RPEMmLXOahPQPhoRnCMx) #### **Less than** - Validates that a parameter is less than the given expression. * Example 1:\ `param1` is assigned with the value 30 that is not less than 31 hence the assertion will fail.\ ![Click to enlarge the image](/files/gytaFPlctV14xBOqlSc2)![](/files/O2CdLyfFIv74MtQNR3Bw) * Example 2:\ `param1` is assigned with the value 31 which is less than 40 will make the assertion to pass\ ![Click to enlarge the image](/files/i0zphUifu15KJzOhPn9w)![](/files/edFNEzz2zYmxAPP3HIp9) #### **JSON Schema** - Validate that a parameter comply with given [JSON Schema](https://json-schema.org/). * Example 1:\ Given `param1` was set with a json as below, opposed to the expected schema the field "name" is not numeric hence the assertion will fail.\ ![Click to enlarge the image](/files/OtSxrxdbvPorpJi65QoM)![](/files/KkzF3VANzORSXyvgCFOw)\ Note, the assertion will suggests to exclude the failing field from the validation schema. * Example 2:\ Given `param1` was set with a json as below and the expected schema describes it correctly, the assertion will pass.\ ![Click to enlarge the image](/files/uDoa481cKTgOJ2YTHwwp)![](/files/kIC3iWMentq6ntFR6CL8) {% hint style="info" %} To prepare initial JSON schema out of a JSON you can search for 'from json to schema' in google or use a [tool like this](https://transform.tools/json-to-json-schema). {% endhint %} #### **JSON Contains -** Validates that a json contains a subset json, in such way dynamic fields can be omitted to avoid false negatives. Accepting any value for individual field will accept "\*" notation\ Note, the subset json object should preserve the same hierarchal structure as the containing object while fields can be omitted. * Example 1:\ Given `param1` was set with a json as below, any mismatch\ in a key or value on same hierarchy will result in assertion error:\ ![Click to enlarge the image](/files/1IonoVYmgE0zPN9Dk1LK)![](/files/um46p7eZl1lzxYOFaNjZ)\ Note to the "Exclude ..." notations, clicking on them will adjust the value of the assertion to exclude the failing field. * Example 2:\ Given `param1` was set with a json as below, any partial subset in json-contains validation value with same structure will result pass in assertion:\ ![Click to enlarge the image](/files/wILT9qro8LGJjXKLJYHd)![](/files/nH3rBs7Dq2RyJM4hPGM2)\ \- Using star notation to accept any value for `latitude` and `country` fields.\ ![Click to enlarge the image](/files/IBWkKpjtJOWo05isJdP1)\ Note, the "\*" notation works only on simple values (strings/numerics/booleans) #### **XML Contains -** Validates that an xml contains a subset xml, in such way dynamic fields can be omitted to avoid false negatives. Accepting any value for individual field will accept "\*" notation\ Note, the subset xml object should preserve the same hierarchal structure as the containing object while fields can be omitted. * Example 1:\ Given `param1` was set with an xml as below, any mismatch\ in a key or value on same hierarchy will result in assertion error: ![](/files/yZAfXDcuaKmkUttn6a28) \\ * Example 2:\ Given `param1` was set with an xml as below, any partial subset in xml-contains validation value with same structure will result pass in assertion:\ ![](/files/xmGsM8u871zQTIn0UeoP)![](/files/QKbY6RWWIeyHVLiuMBnD)\ \ \- Using star notation to accept any value for `latitude` and `country` fields.\ ![](/files/c5NMw4ybXSfSzlXKyUsE)\ Note, the "\*" notation works only on simple values (strings/numerics/booleans) You may embed parameters in any assertion expression. These parameters will be evaluated right before the assertion is executed. See **an example of the assertion** that validates the generated ID is in [the format of UUID](/test-editor-reference/functions#is_uuid-target) below: ![](/files/R7ncXB2e3h333VvLC9mR) {% hint style="info" %} :man\_mage: When creating tests via our [Chrome recorder extension](https://docs.loadmill.com/working-with-the-recorder), Loadmill will create **two automatic assertion types** for you: 1. Default assertions - for any extraction we find in the recorded test, we add a default assertion to it (example: "id" exists). 2. Specific assertions - for any parameter with given user key (for example, "success") we extract it and assign to it an assertion (example: "success" equals "true"). Optional automatic JSON Schema assertion: you can enable [this assertion](https://docs.loadmill.com/working-with-the-recorder/recorder-settings#strict-response-validation) within Settings - Recordings. {% endhint %} ## Suggestions In many cases Loadmill users use the same or similar extractions and assertions. We've implemented the Suggestions feature that allows team admins to configure a repository of [extractions](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) and assertions within **Settings - Suggestions**. Then, each user can add the assertions from the repository by clicking on **+ SUGGESTIONS**. ![](/files/-Ml5-FKwVzDSE-IyLRTR) By default, there are a few common extraction and assertion examples in the repository. Team admins can also navigate to the Suggestions Settings directly from within the suggestion dialog window. ![](/files/-M_G8F1z_TeFwD_IfSS2) ## Caveats Keep in mind that all parameter values are **textual**, i.e. a parameter has **no type** such as `Number` or `Array` that we know from common programming languages. This is important in order to avoid confusion when using parameter extractors such as **JSONPath**. For example, consider the following scenario: 1. Extract the value for `books` via the JSONPath query `student.books` on ```javascript { "student": { "books": [] } } ``` 2. Assert `books` is **Not Empty**. You may expect this assertion to fail but, in fact, it will succeed. This is because the parameter `books` is evaluated to `[]` and therefore is considered as a non-empty string. One possible way to correct this is to use a **RegExp** assertion on the `books` parameter instead: `\[[^\s]+\]`. # Parameters Working with [Loadmill](https://www.loadmill.com) makes it very easy to convert recorded browser/network sessions (via HAR files) into test scenarios. But once a scenario has been created, it often needs to be **parameterized** in order to denote the dynamic parts of your API. This is best explained by example, so let's have a look at the following test scenario: 1. User adds a blog post: `curl https://www.myblog.com/posts --data "Hello World!"` 2. User likes his own post: `curl -X PUT https://www.myblog.com/posts/123/like` The `123` part is the identifier of the blog post created by the user - it cannot be known in advance because it is generated by the server at runtime. This is a very common use case for **parameters** and is handled by two simple steps: 1. [Extract](#parameter-extraction) the ID from the first response into a parameter, e.g. `postId` 2. Embed the parameter in the second request URL: `https://www.myblog.com/posts/${postId}/like` You may embed parameters with the `${}` syntax in the request URL as well as the request body, request headers, [extractions](#parameter-extraction), [assertions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/assertions) and more. Note that any parameterized expression such as `posts/${postId}` will remain as-is if no such parameter is defined or extracted before the expression is evaluated. ## Default Parameters Another common use case for parametrization is when you want to **reuse test scenarios** on different environments or with small adjustments. Let's extend our first example by requiring that the user provide credentials via **basic authentication**. This means our new URLs could look like this: 1. `https://testUser:testPassword@www.myblog.com/posts` 2. `https://testUser:testPassword@www.myblog.com/posts/${postId}/like` If you would like to use different credentials for every test run, you may replace the username and password with parameters and set their values in the [Test Suite Parameters](https://docs.loadmill.com/api-testing/test-suite-editor/test-suite-parameters) tab whenever you reuse the test. So now the URLs will look like: 1. `https://${user}:${pass}@www.myblog.com/posts` 2. `https://${user}:${pass}@www.myblog.com/posts/${postId}/like` Using parameter defaults is especially useful for automated testing and CI where you may be testing a different server every time you run a test. Please refer to the [Loadmill CLI and npm module](https://www.npmjs.com/package/loadmill#parameters) for more information about how to inject parameters dynamically in such scenarios. ## Parameter Extraction Parameters can be defined and populated with values dynamically after each request in your test scenario. ![](/files/-Ml4uy9yMuihq1xt7j9C) There are several **extraction query types** that may be used: * **JSONPath** - used for extracting values from a JSON response. For example, the query `$.post.id` will extract the value `123` from this JSON response: ```javascript { "post": { "id": 123, "text": "Hello World!" } } ``` ![](/files/-Ml4vF6b33EbcWRkRTRE) * **JQuery (Cheerio)** - used for extracting values from XML/HTML responses. We use a subset of the JQuery selector syntax called [Cheerio](https://cheerio.js.org). You may add an optional (but very useful) **attribute** input to your query that selects an attribute value from the first element found by the jQuery. If you do not provide an attribute to select, the query will simply output the inner content of said element. * **JS RegExp** - used for extracting arbitrary values from any kind of textual response via regular expressions with capture groups. For example, we can extract the `id` field from the same JSON response we've seen above using a regular expression: `.*"id":\s*([0-9]*)`. * **Header** - used for extracting response header values via header names. * **Assignment** - used for assigning an explicit value to a parameter or [a function](https://docs.loadmill.com/api-testing/test-suite-editor/functions) to an existing parameter. Previously defined or built-in parameters may be embedded within the string, e.g. `https://${host}/path/to/glory` or `The time is ${__now}`. * **Clojure** - used for extracting values from Clojure (EDN content type) responses. Querying the data is done using JSONPath. For example, the query `$[":user"][":id"]` will extract the value `56` from this EDN response: ``` {:user {:role :viewer, :name "Rivi", :teams nil, :id "56"}} ``` Previously defined or built-in parameters may be embedded within **any kind of extraction query**. These parameters will be evaluated right before the query itself is evaluated. ## Built-in Parameters There are several **built-in** parameters that you can use in your test scenario. They are: * `__status` The status code of the last HTTP response. * `__statusText` The status text of the last HTTP response. * `__responseTime` The total response time (in milliseconds) of the last HTTP response. * `__testRunId` - The test run id: (Suite / Flow / Load) * `__testStartTime` - The test run start time (UTC in milliseconds) * `__launchedBy` - The name of the user running the test. **Note:** some previous built-in parameters are now defined as no-argument [parameter functions](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions) and can still be used in the same way. ## Advanced Usage So far, we've only seen how to inject a simple parameter value into an arbitrary expression, e.g. `Hello ${name}`. However, it is also possible to inject a ***computed value*** using [Parameter Operators](#operators) or [Parameter Functions](https://docs.loadmill.com/api-testing/test-suite-editor/functions), e.g. `The total price is ${price + __mult(price,tax)}`. You may use operators or functions anywhere parameters may be used. Computed values can be extremely useful when you need to introduce conditional behavior to your test. Say you want to skip **Purchase Request** if the preceding **Get Price Request** response returns a price above the current budget. This could be accomplished by extracting the price to a parameter and setting the **Purchase Request** skip condition to `${budget <= price}`. ![](/files/-MlFgzMp-Gg8mHMv6fD3) You may also use ***literal values*** within expressions, e.g. `${__if_then_else(is_good,'Success!',':_(')}` - but be aware there are some [syntactic limitations](#notes-and-limitations). Function calls without arguments can be used with or without parentheses, e.g. `__random_uuid` can be used instead of `__random_uuid()`. See below the full list of supported [operators](#operators) and [functions](https://docs.loadmill.com/api-testing/test-suite-editor/functions). ### Notes And Limitations Current syntax has some limitations. Syntax errors are easy to spot in the GUI - a malformed expression will simply not be highlighted. If an expression is invalid due to its syntax it will simply remain as-is and will not be computed nor replaced at run-time. If, however, an operator or a function receives invalid input (e.g. division by zero or any of the parameter arguments not having a value) the test will fail with an error. Note that predefined parameter values (AKA [Parameter Defaults](#default-parameters)) are computed whenever a test configuration is validated - thus computation related errors will render the test configuration invalid. Current syntax limitations are: * Operators ***must*** be separated from their arguments by spaces, e.g. `${x + y}` is fine but `${x+y}` will not be computed. * Spaces are ***not allowed*** anywhere else within an expression, e.g. both `${__add(x, y)}` and `${fullName == 'John Doe'}` will not be computed. * **Literal values** may not contain whitespace characters, commas (`,`) or single quotes (`'`). These cannot be escaped - simply define a previous parameter with the desired value when the need arises. Note that numeric literal values need to be quoted the same as any other value, e.g. `${x > '0'}` is OK but `${x > 0}` will not be computed. * You may chain multiple operations together, e.g. `${x * y + z}` but you may ***not*** use parentheses to set precedence, e.g. `${(x * y) + z}` will not be computed. This can usually be worked around using functions though, e.g. `${__mult(x,y) + z}` * All operators have the ***same precedence*** - computations always conform to right-associativity, i.e. `${x * y + z - j + k}` will be computed as `x * (y + (z - (j + k)))`. * Computations may ***not be nested***, i.e. you may not pass a computed value as an argument to function, e.g. `${__mult(x,y) + z}` is OK but neither `${__mult(x + y,z)}` nor `${__mult(__add(x,y),z)}` will be computed. * Unary operators, e.g. `${-x}` are ***not*** supported. This can be overcome using functions such as [\_\_neg](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions#__neg-p-1) or [\_\_not](https://docs.loadmill.com/api-testing/test-suite-editor/parameters/functions#__not-p-1). ### Operators The currently supported operators are: #### Textual Operators * `=` Strict equals operator. Aliases: `==` and `===`. * `!=` Strict not-equals operator. Alias: `!==`. May be applied to any two parameters which have values. #### Boolean Operators * `|` Logical OR operator. Alias: `||`. * `&` Logical AND operator. Alias: `&&`. May be applied to any two parameters which have values. #### **True Semantics** A parameter translates to boolean `true` if and only if * It has a value ***and*** * The value is not an empty string ***and*** * The value is not equal to `false`, `FALSE`, `FaLsE` or any other combination of upper-case and lower-case letters that forms the word `false`. The computed value of a valid boolean operation is either exactly `true` or exactly `false`. #### Numeric Operators * `+` Addition operator. * `-` Subtraction operator. * `*` Multiplication operator. * `/` Division operator. * `<` Less-then operator. * `<=` Less-then-or-equals operator. * `>` Greater-then operator. * `>=` Greater-then-or-equals operator. May be applied to any two parameters which have values that translate to ***finite numbers***. Computed values are ***not*** rounded to integers. ### # Parameter Execution Order When using parameters in Loadmill tests, it is important to understand their hierarchy. To figure it out, let's look at a basic example below: The user has a test flow with a GET request to his application home page. ![](/files/-Ml4vmiAEkyqOfJ6k6zb) The "base\_url" parameter is used to store URLs and choose them depending on which environment the user would like to run tests against. This parameter can be defined by several ways, let's review them: ![](/files/-MZIBxk_UWTCbBGu90Ho) 1. **Test Suite Parameters tab** - this is a place where the user defines a default value for the parameters he is going to use in the Test Suite. In our case - "base\_url". ![](/files/NYDf5U9Kmk3ztHcs24TW) 2\. **Test Plan parameter value** - once the user has a few Test Suites, he groups them into a higher level called [Test Plan](https://docs.loadmill.com/api-testing/test-plan) per a specific task or feature they belong to. Then, the user can configure the "base\_url" parameter value that will be sent to all Test Suites during the Test Plan run. This value will override the value from the Test Suite Parameters tab (if there is any). ![](/files/UpZyMSdkVgfdwyqwOdVE) 3\. **npm module parameter value** - when running tests in CI/CD, the user initiates their execution by using [our npm module](https://www.npmjs.com/package/loadmill) and he can add a relevant "base\_url" parameter value to the command and **this value will** be taken for the run and **override the parameter value set in #2 and #1 above** (if there is any)**.** ``` loadmill --test-plan -w -v -t --report base_url=your_current_url ``` 4\. **CSV file parameter value** - in some cases the user may need to run the same request multiple times using different parameter values, so he can use a [CSV file for that](https://docs.loadmill.com/api-testing/test-suite-editor/api-tests-data-from-csv-files). ![](/files/-MZILUTBTbuUsQthSWio) **The value from the CSV file will** be taken during the test run and **override the parameter value from sections above** (if there is any). 5\. **Extraction parameter value** - when the user extracts value into a parameter within the request's [Extractions - Set parameters](https://docs.loadmill.com/api-testing/test-suite-editor/set-parameters-extractions) section, **this value will override the parameter value from all sections above during the test run**. # Test Suite Parameters The Test Suite Parameters tab allows to configure default parameters which are relevant to the execution context. e.g. target host, login credentials, and user input. Default parameters can be used in requests using the template strings notation: `${parameter_name}`. ![](/files/X8Iyz5ZAMbfftmMXBkLl) ![](/files/aWgjfGzgszdm4ZSeQXhf) ### Parameters scope - *DEPRECATED* ⚠️ Note: This feature is deprecated. Use [Suite Execution Mode](/test-editor-reference/parameters/suite-execution-mode) instead. By default, Test Suite parameters have the **Entire Test Suite Run** execution scope. The alternative option is to select the **Single Flow Run** scope. ![](/files/XWQPRbAL9Test7MYQyeF) Look at the example below to better understand what execution scopes mean. Let's say, I have a test flow where I test creating and editing a blog post. I've extracted ID of the post that has been created and now I want to use it in the following requests of the same flow. This is a classic example of a self-contained test and It will work regardless of the execution scope selected. But in case I would like to use the post ID parameter in another flow of the Test Suite I should use the "Entire Test Suite Run" scope, otherwise, Loadmill won't recognize that parameter. ### Support We are always here if you need any help! Click on the bubble chat button in the lower-right corner of the screen or drop us a line at . # Suite Execution Mode Loadmill lets you control how flows inside a test suite interact with each other and how failures are handled. The *Execution Mode* determines whether flows share state, how retries behave, and whether the suite stops on the first failed flow. *** ## **Where to configure Execution Mode** You can configure Execution Mode from the suite’s **Run Settings**: 1. Open your test suite. 2. Navigate to **Run Settings** in the top navigation bar. 3. Scroll to the **Execution Mode** section. 4. Select either **Dependent** or **Independent**. ## ![](/files/j2PbbmOGhibpAPUUKSe3) ## **Modes** ### **Dependent Mode** Flows run sequentially and share execution state. **Characteristics:** * **Shared cookies & parameters** — any extracted values from one flow are available in the next. * **Stops on first failure** — if one flow fails, the entire suite ends immediately. * **Retry reruns the entire suite** — to rebuild all shared context correctly. **Best for:** * Multi-step flows where later tests rely on earlier outputs. * Ensuring consistent global state. *** ### **Independent Mode** Each flow runs in complete isolation (except `Before All`). **Characteristics:** * **No shared state** — every flow receives a clean environment. * **Suite continues even if a flow fails** — all flows execute regardless of failures. * **Retry reruns only failed flows** — much faster iteration when debugging. **Best for:** * Collections of unrelated flows. * Wide coverage testing where independence improves stability. * Re-tring specific failures without rerunning everything. *** ## **How Execution Mode Works** * **Before All** always runs before the suite starts. * In **Dependent mode**, all flows share one continuous execution context (cookies and parameters). * In **Independent mode**, each flow gets its own isolated environment (cookies, parameters, and context are not reused). * Retries follow the selected mode to ensure deterministic behavior. *** # Parameter Sets Inside the Parameters page, there is an option to insert a parameters set. This adds each parameter defined in the desired set to the current parameters list. ### Creating a Parameters Set The page is located under TEST DESIGN -> Parameters Sets.
Creation of a new set: {% embed url="" %} A set that hasn't been published even once will be saved as a 'draft':
Changed but unpublished sets will be saved as 'modified' (). ### Importing/Exporting Parameters Sets A Parameters Set can also be represented in a YAML file with the following structure: ```yaml meta: description: Example Parameters Set parameters: - name1: "Foo" - name2: "Bar" ``` An example use case is to duplicate a set and modify it (![](/files/GEnayTfHK6NiLx9kreqY), ![](/files/QWKTrkAn0EqTB71PZVy9)). ### Exporting as a key-value file From inside the parameters sets editor, it is possible to download only the parameters as a key-value .txt file, e.g.: ``` name1=Foo name2=Bar ``` It is good for Loadmill's CI integration, where inline parameters can be overridden or added by providing a key-value file instead (See "Parameters" in [Loadmill's NPM README](https://www.npmjs.com/package/loadmill?activeTab=readme)).
# Built-in Functions Loadmill has builtin functions available to use while creating a test. These functions will be evaluated in run time as part of the test. ## Numeric Functions ### `__abs(p1)` Returns the absolute value of `p1`. * `${__abs('-42')} // returns 42` * `${__abs(p1)} // returns 42, assuming parameter p1 equals 42` ### `__add(p1,[p2,[...]])` Same as the `+` operator, applied to any number of arguments. * `${__add('42','3')} // returns 45` * `${__add(p1,p2)} // returns 10, assuming p1 is 7 and p2 is 3` * `${__add(p1,p2,p3)} // returns 15, assuming p1 is 7, p2 is 3 and p3 is 5` ### `__sub(p1,p2)` Same as the `-` operator. * `${__sub('42','3')} // returns 39` * `${__sub(p1,p2)} // returns 4, assuming p1 is 7 and p2 is 3` * `${__sub(p1,p2,p3)} // returns 3, assuming p1 is 7, p2 is 3 and p3 is 1` ### `__neg(p1)` Unary minus, equivalent to `__sub('0',p1)`. * `${__neg('4')} // returns -4` * `${__neg('-4')} // returns 4` * `${__neg(p1)} // returns -4, assuming p1 is 4` ### `__mult(p1,[p2,[...]])` Same as the `*` operator, applied to any number of arguments. * `${__mult('2','3')} // returns 6` * `${__mult(p1,p2,p3)} // returns 24 assuming p1 is 2, p2 is 3 and p3 is 4` ### `__div(p1,p2)` Same as the `/` operator. * `${__div('6','3')} // returns 2` * `${__div(p1,p2)} // returns 4 assuming p1 is 24 and p2 is 6` ## Conditional Functions ### `__true()` Always returns `true`. ### `__false()` Always returns `false`. ### `__and(p1,[p2,[...]])` * `${__and('true','true')} // returns true` * `${__and('true','false')} // returns false` * `${__and('true','')} // returns false` * `${__and(p1,p2)} // returns true, assuming p1 and p2 aren't false nor ''` * `${__and()} // returns true` * `${__and('-42')} // returns true` Logical AND (same as the `&` operator), applied to any number of arguments. ### `__or(p1,[p2,[...]])` Logical OR (same as the `|` operator), applied to any number of arguments. * `${__or('true','true')} // returns true` * `${__or('true','false')} // returns true` * `${__or('false','false')} // returns false` * `${__or('','false')} // returns false` * `${__or(p1,p2)} // returns true, assuming p1 or p2 aren't false nor ''` * `${__or()} // returns true` * `${__or('-42')} // returns true` ### `__not(p1)` Logical NOT. See also [True Semantics](/test-editor-reference/parameters#true-semantics). * `${__not('false')} // returns true` * `${__not('true')} // returns false` * `${__not('')} // returns true` * `${__not(p1)} // returns true, assuming p1 is't false nor ''` * `${__not('-42')} // returns false` ### `__eq(p1,p2)` Same as the `==` operator. * `${__eq('42','3')} //returns false` * `${__eq('false','false')} //returns true` * `${__eq('FaLsE','false')} //returns false` * `${__eq('','')} //returns true` * `${__eq(p1,p2)} //returns true if both p1 and p2 have the same value` ### `__neq(p1,p2)` Same as the `!=` operator. * `${__neq('42','3')} //returns true` * `${__neq('false','false')} //returns false` * `${__neq('FaLsE','false')} //returns true` * `${__neq('','')} //returns false` * `${__neq(p1,p2)} //returns true if both p1 and p2 have a different value` ### `__eqi(p1,p2)` Same as `__eq` but case-insensitive. * `${__eqi('42','3')} //returns false` * `${__eqi('false','false')} //returns true` * `${__eqi('FaLsE','false')} //returns true` * `${__eqi('','')} //returns true` * `${__eqi(p1,p2)} //returns true if both p1 and p2 have the same value` ### `__neqi(p1,p2)` Same as `__neq` but case-insensitive. * `${__neqi('42','3')} //returns true` * `${__neqi('false','false')} //returns false` * `${__neqi('FaLsE','false')} //returns false` * `${__neqi('','')} //returns false` * `${__neqi(p1,p2)} //returns true if both p1 and p2 have a different value` ### `__lt(p1,p2)` Same as the `<` operator. * `${__lt('42','3')} //returns false` * `${__lt('3','42')} //returns true` * `${__lt('42','42')} //returns false` * `${__lt(p1,p2)} //returns true, assuming p1 is 3 and p2 is 42` ### `__lte(p1,p2)` Same as the `<=` operator. * `${__lte('42','3')} //returns false` * `${__lte('3','42')} //returns true` * `${__lte('42','42')} //returns true` * `${__lte(p1,p2)} //returns true, assuming p1 is 3 and p2 is 42` ### `__gt(p1,p2)` Same as the `>` operator. * `${__gt('42','3')} //returns true` * `${__gt('3','42')} //returns false` * `${__gt('42','42')} //returns false` * `${__gt(p1,p2)} //returns true, assuming p1 is 42 and p2 is 3` ### `__gte(p1,p2)` Same as the `>=` operator. * `${__gte('42','3')} //returns true` * `${__gte('3','42')} //returns false` * `${__gte('42','42')} //returns true` * `${__gte(p1,p2)} //returns true, assuming p1 is 42 and p2 is 3` ### `__matches(target,regex)` Returns `true` if and only if the `target` matches the `regex`. * `${__matches(target,'.*search.*')} // returns true, assuming target is 'A text to search in'` ### `__contains(target,search)` Returns `true` if and only if the `target` contains the string `search`. * `${__contains(target,'search')} // returns true, assuming target is 'A text to search in'` * `${__contains(target,'SEARCH')} // returns false, assuming target is 'A text to search in'` * `${__contains(target,'.*search.*')} // returns false, assuming target is 'A text to search in'` ### `__containsi(target,search)` Same as `__contains` but case-insensitive. * `${__containsi(target,'search')} // returns true, assuming target is 'A text to search in'` * `${__containsi(target,'SEARCH')} // returns true, assuming target is 'A text to search in'` * `${__containsi(target,'.*search.*')} // returns false, assuming target is 'A text to search in'` ### `__if_then_else(condition,then,else)` Returns `then` if `condition` is [semantically true](/test-editor-reference/parameters#true-semantics), otherwise returns `else`. * `${__if_then_else(p1,'good','bad')} // returns 'good', assuming p1 is true or has a value` * `${__if_then_else(p1,'good','bad')} // returns 'bad', assuming p1 is false or ''` ### `__is_number(target)` Returns true if target is not empty and not a NAN. * `${__is_number('42')} // returns true` * `${__is_number('')} // returns false` * `${__is_number('Hi')} // returns false` * `${__is_number(p1)} // returns true assuming p1 is a number i.e. 42` ### `__is_uuid(target)` Returns true if target is in the format of UUID. * `${__is_uuid('123e4567-e89b-12d3-a456-426614174000')} // returns true` * `${__is_uuid('42')} // returns false` ### `__switch(target,case1,value1,[case2,value2,[...],[default]])` Returns `value1` if `target` equals `case1`, otherwise returns `value2` if `target` equals `case2` and so on. If no match is made, the returned value will be an empty string - a default value may be given as the last argument. * `${__switch(p1,'3','Hi','42','towel','none')} // returns 'Hi' if p1 is 3, 'towel' if p1 is 42 and 'none' if p1 is neither 3 nor 42` ### `__switchi(target,case1,value1,[case2,value2,[...],[default]])` Same as `__switch` but case-insensitive. * `${__switchi(p1,'this','Hi','that','towel','none')} // returns 'Hi' if p1 is 'this' or 'THIS', 'towel' if p1 is 'that' or 'THAT' and 'none' if p1 is neither of those` ### `__pick(selection,p1,[p2,[...]])` Returns one of `p1` or `p2` or `p3`, etc. according to the `selection` - either a zero-based index (e.g. `2` picks `p3`) or the word `random` in order to pick a random value. * `${__pick('0','A','B','C')} // returns 'A'` * `${__pick('1','A','B','C')} // returns 'B'` * `${__pick(p1,'A','B','C')} // returns 'C', assuming p1 is 2` * `${__pick('random','A','B','C')} // returns one of 'A', 'B' and 'C'` ### `__pick_random(p1,[p2,[...]])` Same as `__pick('random',p1,[p2,[...]])`. ### `__split_pick(target,delim,[selection=0])` Splits the value of `target` into multiple strings separated by `delim` and returns one of them according to the `selection` as defined by the `__pick` function. If a `selection` argument is not provided, the first value will be returned. * `${__split_pick(p1,'API')} // returns 'loadmill' , assuming p1 value is 'loadmill API testing'` * `${__split_pick(p1,'API','0')} // returns 'loadmill' , assuming p1 value is 'loadmill API testing'` * `${__split_pick(p1,'API','1')} // returns 'testing' , assuming p1 value is 'loadmill API testing'` ## Textual Functions ### `__usd()` Returns the `$` character. ### `__length(target)` Counts the number of characters in `target`. * `${__length('loadmill')} // returns 8` * `${__length('')} // returns 0` ### `__escape_regexp(target)` Returns the value of `target` after escaping special RegExp characters. * `${__escape_regexp('.*search.*')} // returns \.\*search\.\*` ### `__encode_url(target)` Returns the value of `target` after [URL encoding](https://en.wikipedia.org/wiki/Percent-encoding) special characters. * `${__encode_url(p1)} // returns 'this%20is%20url' assuming p1 is 'this is url'` ### `__decode_url(target)` Returns the value of `target` after [URL decoding](https://en.wikipedia.org/wiki/Percent-encoding). * `${__decode_url(p1)} // returns 'this is url' assuming p1 is 'this%20is%20url'` ### `__encode_base64(target)` Returns the value of `target` after [Base64](https://en.wikipedia.org/wiki/Base64) encoding. * `${__encode_base64(p1)} // returns 'ZXhhbXBsZQ==' assuming p1 is 'example'` ### `__decode_base64(target)` Returns the value of `target` after [Base64](https://en.wikipedia.org/wiki/Base64) decoding. * `${__decode_base64(p1)} // returns 'example' assuming p1 is 'ZXhhbXBsZQ=='` ### `__hex_to_base64(target)` Accepts a string representation of a hexadecimal number and returns its value in its Base64 representation. * `${__hex_to_base64(p1)} // returns 'q83vmHZUMg==' assuming p1 is 'abcdef987654321'` ### `__escape_quotes(target)` Returns the value of `target` after escaping special characters. This function is like escape quotes in JavaScript and will escape characters like \n \r \t and \\". * `${__escape_quotes('"escapeMe"')} // returns \"escapeMe\"` * `${__escape_quotes(P1)} // returns \"escapeMe\", assuming p is "escapeMe"` ### `__lower(target)` Returns the value of `target` after converting all characters to lower case. * `${__lower('LoadMill')} // returns loadmill` ### `__upper(target)` Returns the value of `target` after converting all characters to upper case. * `${__upper('LoadMill')} // returns LOADMILL` ### `__slice(target,[begin=0,[end]])` Returns a sub-string of `target` which starts at `begin` index (inclusive) and ends at `end` index (exclusive). Both indexes are zero-based. * `${__slice('Loadmill')} // returns 'Loadmill'` * `${__slice('Loadmill','0')} // returns 'Loadmill'` * `${__slice('Loadmill','2')} // returns 'admill'` * `${__slice('Loadmill','2','4')} // returns 'ad'` * `${__slice('Loadmill',p1,p2)} // returns 'mill', assuming p1 is 4 and p2 is 8` * `${__slice(p1,'0','2')} // returns 'Lo', assuming p1 is Loadmill` ### `__replace(target,[begin=0,[end]])` Returns a new string with one, some, or all matches of a `searchValue` replaced by a `replaceValue`. `searchValue` can be either a string or a regex. If it's a regex, then the `options` argument can be used as the regex options. {% hint style="info" %} Regex tokens must be escaped in `searchValue`. (`searchValue='+'` is bad, `searchValue='\+'` is good) {% endhint %} * `${__replace(p1,'a','x')} // returns 'xaa' assuming p1 is 'aaa'` * `${__replace('AAA','a','x','i')} // returns 'xAA' (here we used the i = case insensitive option)` ### `__replace_all(target, searchValue, replaceValue)` Returns a new string with all matches of a `searchValue` replaced by a `replaceValue` (Same as `__replace`, but the g \[= global] option is always on). * `${__replace_all(p1,'a','x')} // returns 'xxx' assuming p1 is 'aaa'` ## Array functions ### `__array_length(target)` Counts the number of elements in `target` array. * `${__array_length('[]')} // returns 0` * `${__array_length(p1)} // returns 4, assuming p1 is [1,"str",true,null]` ### `__array_matches(target,regex)` Returns a boolean string of whether all the elements in the target array match the given regex. * `${__array_matches(array,'.*a.*')} // returns true, assuming array is ["apple","banana","orange"]` * `${__array_matches(array,'.*Z.*')} // returns false, assuming array is ["apple","banana","orange"]` ### `__array_in_range(target,[min=0,max=2^32])` Returns a boolean string of whether all the elements in the target array match the given range. * `${__array_in_range(array,'0','100')} // returns true, assuming array is ["100","99","1"]` * `${__array_in_range(array,'0','100')} // returns true, assuming array is [100,99,1]` ### **`__array_includes(src,target)`** Returns a boolean string of whether each element in the target array is also in the src array. * `${__array_includes(srcArray,targetArray)} // returns true, assuming srcArray is ["apple","banana","orange"] and targetArray is ["apple","banana"]` * `${__array_includes(srcArray,targetArray)} // returns false, assuming srcArray is ["apple","banana","orange"] and targetArray is ["apple","banana","lemon"]` ### `__array_sum(target)` Returns the sum of all the elements in the target array. * `${__array_sum(targetArray)} // returns 401, assuming targetArray is ["400","0","1"]` * `${__array_sum(targetArray)} // returns 188, assuming targetArray is [100,80,8]` * `${__array_sum(targetArray)} // returns NaN, assuming targetArray is [100,"some-word",8]` ### `__array_sort(target)` Returns the sorted array in the ascending order. * `${__array_sort(targetArray) // returns ["apple","banana","orange"], assuming targetArray is ["banana","apple","orange"]` ### `__array_sort_numbers(target)` Returns the sorted array of numbers in the ascending order. * `${__array_sort_numbers(targetArray) // returns [2,19,34,55,90,109,136,156,188,190], assuming targetArray is [109, 136, 156, 188, 19, 190, 2, 34, 55, 90]` ### `__array_is_unique(target)` Returns a boolean string of whether all elements in the array are unique. * `${__array_is_unique(targetArray) // returns true, assuming targetArray is ["cat","dog","bird"]` * `${__array_is_unique(targetArray) // returns false, assuming targetArray is ["cat","cat","dog"]` ### `__array_pluck(target,'key1','key2','key3:newKeyName',['key4',[...]])` Returns an array of objects with specified JSON keys with an option to rename JSON keys. Returns an empty array in case the target array is undefined. * `${__array_pluck(array,'author','book')} // returns [{"author":"Bulgakov","book":"The Master and Margarita"},{"author":"Bulgakov","book":"The Fatal Eggs"}], assuming array is [{"author":"Bulgakov","book":"The Master and Margarita","year":"1967"},{"author":"Bulgakov","book":"The Fatal Eggs","year":"1925"}]` * `${__array_pluck(array,'author','book','year:publishment_year')} // returns [{"author":"Bulgakov","book":"The Master and Margarita","publishment_year":"1967"},{"author":"Bulgakov","book":"The Fatal Eggs","publishment_year":"1925"}], assuming array is [{"author":"Bulgakov","book":"The Master and Margarita","year":"1967"},{"author":"Bulgakov","book":"The Fatal Eggs","year":"1925"}]` ### `__sequence_array(p1)` Returns an array containing a sequence of numbers from 0 to `p1`. * `${__sequence_array('5')} // returns [0,1,2,3,4,5]` * `${__sequence_array(p1)} // returns [0,1,2,3,4,5,6,7,8,9,10], assuming p1 is '10'` ### `__array_missing_element(sub_array,full_array)` Returns the next suggested number from `full_array` which is not in `sub_array.` * `${__array_missing_element(sub_array,full_array) // returns 2 assuming sub_array is [0,1,3,4] and full_array is [0,1,2,3,4,5,6,7,8,9,10]` * `${__array_missing_element(sub_array,full_array) // returns "banana", assuming sub_array is ["apple","mellon"] and full_array is ["apple","banana","mellon","strawberry"]` * `${__array_missing_element(sub_array,full_array} // returns 3 assuming sub_array is [0,1,2,4,5,6,7] and full_array is [1,2,3,4]` ## **Extraction Functions** See also [Parameter Extractions](/test-editor-reference/parameters#parameter-extraction). ### `__regexp(target,regexp,[default])` Extracts a value from `target` using `regexp` as a JS RegExp. If there is no match, an empty string will be returned or, if present, the given `default` value. * `${__regexp(p1,'.*(search).*')} // returns 'search' assuming p1 is 'A text to search in'` * `${__regexp(p1,'(.*search.*)')} // returns 'A text to search in' assuming p1 is 'A text to search in'` * `${__regexp(p1,'.*(bad).*','none')} // returns 'none' assuming p1 is 'A text to search in'` ### `__json_keys(target,[default])` Returns the keys of target object in an array. Works similar to [Object.keys](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys). If `target` is empty, an empty array will be returned or, if present, the given `default` value. * `${__json_keys('{"key1":"val1"}')}// returns ["key1"]` * `${__json_keys(target)} // returns ["key1","key2"] assuming target is {"key1":"val1","key2":"val2"}` ### `__jsonpath(target,jsonpath,[default])` Extracts a value from `target` using `jsonpath` as a JSONPath query. If there is no match, an empty string will be returned or, if present, the given `default` value. * `${__jsonpath('{"key":"val"}','$.key')} // returns 'val'` * `${__jsonpath(target,jsonpath)} // returns 'val' assuming target is '{"key":"val"}' and jsonpath is '$.key'` * `${__jsonpath('{"key":"val"}','$.notHere','none')} // returns 'none'` ### `__jsonpath_all(target,jsonpath)` Similar to `__jsonpath` but returns **all results from the JSONPath** and not only one as happens when using `__jsonpath.` ### `__xpath(target,xpath)` Extracts a value from `target` using `xpath` as an XPath query. If there is no match, an empty string will be returned. * `${__xpath('val','//key/text()')} // returns 'val'` * `${__xpath(target,xpath)} // returns 'val' assuming target is 'val' and xpath is '//key/text()'` * `${__xpath('val','//missing/text()')} // returns ''` ### `__jsonpath_keys(target,jsonpath)` Returns the keys of the extracted value from `target` queried by `jsonpath` in an array. Works similar to [Object.keys](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/keys). * `${__jsonpath_keys('{"key1":"val1"}','$')}// returns ["key1"]` * `${__jsonpath_keys(target,jsonpath)} // returns ["key1","key2"] assuming target is {"data": {"key1":"val1","key2":"val2"}} and jsonpath is '$.data'` ### `__jsonpath_apply(target,jsonpath,new_value)` Returns an object or an array of objects with the updated key value as set in `new_value` on specific key as set in `jsonpath`. * `${__jsonpath_apply(target,jsonpath,'USD')} // returns '[{"country":"Japan","currency":"USD"},{"country":"China","currency":"Yuan"}]' assuming target is '[{"country":"Japan","currency":"Yen"},{"country":"China","currency":"Yuan"}]', jsonpath is '$[0].currency' and new_value is 'USD'.` ### `__jquery(target,jquery,[selection=0,[attribute,[default]]])` Extracts a value from `target` using `jquery` as a jQuery selector. If multiple elements are matched, `selection` is applied as in `__pick` with the first element being selected by default. If a non-empty `attribute` is given, the returned value will be the attribute value of the selected element, otherwise, the inner content of the element will be returned. If there is no match, an empty string will be returned or, if present, the given `default` value. * `${__jquery('

Hello world

','h2.title')} //returns Hello world` * `${__jquery(p1,jquery)} //returns Hello world, assuming p1 is '

Hello world

' and jquery is h2.title` ## Randomization Functions ### `__random_uuid()` Returns a random v4 UUID string. ### `__random_boolean([probability=50])` Returns a random boolean value. You may pass an integer between 0 and 100 as the `probability` to get `true` - defaults to 50%. * `${__random_boolean()} // 50% true / false` * `${__random_boolean('70')} // returns true 70% of the times` * `${__random_boolean(p1)} // returns true 20% of the times, assuming p1 is 20` ### `__random_number([max],[min=0,max=2^32])` Returns a random integer between 0 and 2^32. By passing a positive integer you can set a lower maximum, e.g. `__random_number('30')` will resolve to a number between 0 and 30, inclusive. You can also set the minimum, e.g. `__random_number('10','30')` will be between 10 and 30, inclusive. * `${__random_number('30')} // returns a number between 0 and 30` * `${__random_number('10','30')} // returns a number between 10 and 30` * `${__random_number(p1)} // returns a number between 0 and the value of p1` ### `__random_chars([length=10])` Returns a random string of `length` alpha-numeric characters. * `${__random_chars('5')} // returns g2Niu` * `${__random_chars()} // returns YIeT6JXgbJ` * `${__random_chars(p1)} // returns YXgbT, assuming p1 is 5` ### `__random_digits([length=10])` Returns a random string of `length` decimal digits. * `${__random_digits('5')} // returns 83547` * `${__random_digits()} // returns 8354718624` * `${__random_digits(p1)} // returns 83524, assuming p1 is 5` ### `__random_letters([length=10])` Returns a random string of `length` english letters. * `${__random_letters()} // returns YMfvLoYhjh` * `${__random_letters('5')} // returns diMdf` * `${__random_letters(p1)} // returns diMdf, assuming p1 is 5` ### `__random_uppers([length=10])` Returns a random string of `length` upper-case english letters. * `${__random_uppers()} // returns STNOMFSLZC` * `${__random_uppers('5')} // returns WPVRA` * `${__random_uppers(p1)} // returns WPVRA, assuming p1 is 5` ### `__random_lowers([length=10])` Returns a random string of `length` lower-case english letters. * `${__random_lowers()} // returns osmcjtudhe` * `${__random_lowers('5')} // returns alrmv` * `${__random_lowers(p1)} // returns alrmv, assuming p1 is5` ### `__random_hex([length=10])` Returns a random string of `length` hexadecimal digits. * `${__random_hex()} // returns d8c3f7ce22` * `${__random_hex('5')} // returns e5a4b` * `${__random_hex(p1)} // returns e5a4b, assuming p1 is 5` ### `__random_from(chars,[length=10])` Returns a random string of `length` characters present in `chars`. For example, you may generate a random **uppercase** hexadecimal string using `__random_from('0123456789ABCDEF')`. * `__random_from('0123456789ABCDEF') // returns 313249C4FA` * `__random_from('Loadmill') // returns lilodaaLod` * `__random_from(p1) // returns lilodaaLod, assuming p1 is 'Loadmill'` * `__random_from('Loadmill','3') // returns ido` ### `__random_seeded_number([seed='',length=10])` Returns a random string of up to `length` (but no more than 10) decimal digits based on a given seed. Same seed will return the same number. * `__random_seeded_number() // returns 994027583` * `__random_seeded_number('0123456789ABCDEF') // returns 2082602645` * `__random_seeded_number('0123456789ABCDEF','5') // returns 20826` ## Hash Functions ### `__sha1(secret,[secretFormat='utf-8'])` Creating hash digests of data using an algorithm of `sha1`. The function returns a hash in hexadecimal digits. Supported secretFormats are `utf-8` and `hex`. * `${__sha1('mySecret')} alias ${__sha1('mySecret','utf-8')}` * `${__sha1('mySecret','hex')}` * `${__sha1(p1,p2) assuming p1 contains your secret and p2 - your secretFormat.` ### `__sha256(secret,[secretFormat='utf-8'])` Creating hash digests of data using an algorithm of `sha256`. The function returns a hash in a hexadecimal digits. Supported secretFormats are `utf-8` and `hex`. * `${__sha256('mySecret')} alias ${__sha256('mySecret','utf-8')}` * `${__sha256('mySecret','hex')}` * `${__sha256(p1,p2) assuming p1 contains your secret and p2 - your secretFormat.` ## XML/JSON Conversion Functions ### `__xml_to_json(xml, [normalize])` Converts an XML string to its JSON representation. By default, normalization is enabled unless `normalize` is explicitly set to `'false'`. *normalize*: Trims whitespace at the beginning and end of text nodes, and also inside text nodes. * `${__xml_to_json('value')} // returns {"root":{"item":"value"}}` * `${__xml_to_json(xml,'false')} // disables normalization` ### `__json_to_xml(json)` Converts a JSON string to its XML representation. The output is headless (no XML declaration). * `${__json_to_xml('{"root":{"item":"value"}}')} // returns value` ## Time Functions ### `__now([length=13])` Returns the current time (of evaluation) given as UTC milliseconds of `length` digits. Alias: `_now_ms`. * `${__now()} // returns 1604422320167` * `${__now('5')} // returns 16044` * `${'Yo'} ${__now()} // returns 'Yo 1604422320167'` ### `__now_iso([addedMinutes=0])` The same as `__now` but given in ISO-8601 format while adding `addedMinutes` . For example, you may generate the current time + 15 minutes using `__now_iso('15')`. * `${__now_iso()} // returns 2020-11-03T16:54:24.526Z` * `${__now_iso('15')} // returns 2020-11-03T17:09:24.526Z` * `${__now_iso('-15')} // returns 2020-11-03T16:39:24.526Z` ### **`__date_iso()`** The same as `__now_iso` but given in a "date only" format - i.e. `2020-03-03` * `${__date_iso()} // returns 2020-11-03` * `${__date_iso('15')} // returns 2020-11-18` * `${__date_iso('-15')} // returns 2020-10-19` ### **`__formatted_date([format='',addedMinutes=0])`** Formatted date according to the supplied format while adding `addedMinutes`. * `${__formatted_date()} // returns 2024-03-12T13:41:29+00:00` * `${__formatted_date('DD/MM/YYYY')} // returns 12/03/2024` * `${__formatted_date('DD/MM/YY','60000')} // returns 13/3/24` ### **`__format_unit_date(unixTimestamp, [format])`** Formats a given UNIX timestamp according to the supplied format. * `${__format_unix_date('1741534304753','DD/MM/YYYY')} // returns 09/03/2025` * `${__format_unix_date('1741534962788')} // returns 2025-03-09T15:42:42+00:00` For more formatting options, have a look [here](https://day.js.org/docs/en/display/format) # Postscript Create and run various assertions by adding custom codes. Developers are always looking for ways to make their lives easier and writing code is no different. In this section, we'll take a look at how post-request scripts (or postscripts) can help you streamline your HTTP requests. Postscripts are small pieces of code that you can use to automate parts of the request-response cycle. ### Writing Postscripts With Loadmill, writing Postscript code is as easy as writing plain javascript. The server response is stored into an identifier shown as a dollar sign (`$`). To access specific parameter values we would use the same method as we would when accessing an object: ```javascript const apple = { "genus": "Malus", "name": "Apple", "id": 6, "family": "Rosaceae", "order": "Rosales", ... } const appleFamily = apple.family; // returns "Rosaceae" ``` Similarly, when receiving a server response, we would view it the same: ```javascript const response = $; //returns the complete server response const responseProperty = response.propertyName; // or response["propertyNameavascript"] // returns the accessed property value inside the response ``` ### Accessing Postscript Extractions The Postscript feature is very useful when you want to use coded variables on different API requests. With this feature, you can access then throughout your automated flow. This is helpful when you need to store and use information from a previous API request in subsequent requests. In this test case we have an API returning a list of fruits as a `JSON` object.\ From that list, we'll be extracting the name and the calories from the "Apple" object and use it on our next requests: ```javascript const apple = $.find((fruit)=>fruit.name == "Apple"); // the $ represend the whole json object // we then use an array method to find the "Apple" object ``` We can now access this object and it's attributes such that: ```javascript const appleName = apple.name; // returns "Apple" const appleCalories = apple.nutritions.calories; //returns 52 ``` To test this out we can use the console.log method to print out appleName and appleCategories {% embed url="" %} You can now access those variables within the rest of your flow. {% embed url="" %} ### Freedom of Coding Postscript is a powerful tool that gives you complete control over the API. With just a few lines of code, you can create conditions, loops and functions for your API tests. Developers find it useful because they can test their APIs the same way they would in a standard programming language. In our previous example, we tested Fruits API to better understand the Postscript execution. We will now explore the freedom of building code from our API. Here, we'll create a new array containing all the fruits family. ```javascript const fruits = $; //returns all fruits in JSON const fruitsFamily = []; for(var i = 0; ifruit.name == "Orange") // returns true ``` {% endcode %} Similarly, when an API returns an array of fruits and their properties, you can validate the existence of a specific fruit with the exact same code. {% hint style="info" %} **Note: The $ (dollar sign) returns the entire API response in object type.** {% endhint %} {% embed url="" %} ```javascript const fruits = $; // returns the entire json response in object type. const orangeOnList = fruits.some((fruit)=>fruit.name == "Orange") // returns true ``` {% embed url="" %} # Accessing with Postscript The Postscript feature is very useful when you want to use coded variables on different API requests. With this feature, you can access then throughout your automated flow. This is helpful when you need to store and use information from a previous API request in subsequent requests. In this test case we have an API returning a list of fruits as a `JSON` object.\ From that list, we'll be extracting the name and the calories from the "Apple" object and use it on our next requests. ### Extracting w/ Postscript ```javascript const apple = $.find((fruit)=>fruit.name == "Apple"); // the $ represend the whole json response object // we then use an array method to find the "Apple" object ``` We can now access this object and it's attributes such that: ```javascript const appleName = apple.name; // returns "Apple" const appleCalories = apple.nutritions.calories; //returns 52 ``` To test this out we can use the `console.log` method to print out **appleName** and **appleCategories** {% embed url="" %} You can now access those variables within the rest of your flow. {% embed url="" %} ### Using Postscript with Pre-Extracted Values You can use Postscript print out values from the Extractions section used across the flow. Here are the steps: 1. Create any extraction from the Extractions section. 2. Open your Postscript section. 3. Assign the extracted parameter to postscript.\ ie: `const name = extractedParameterName;` {% embed url="" %} # Validating Postscript Validating postscript values Most scenarios with postscript require you to validate specific values to ensure a successful API test flow. Loadmill provides 2 methods in validating postscript values: Using Mocha or Assertions. #### Postscript & Mocha Mocha is a JavaScript test framework making asynchronous testing very simple. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases. In the previous article we showed how to verify if a value exists in an array or API response. In our example we tested with a fruits API that returns a list of fruits with their properties and verified if a specific fruit was part of that list. ```javascript const fruits = $; // returns the entire json response in object type. const orangeOnList = fruits.some((fruit)=>fruit.name == "Orange") // returns true ``` In order to validate the API based on the value of orangeOnList we would add the following line of code: ```javascript //assert.equal(parameter, value) assert.equal(orangeOnList, true) ``` {% embed url="" %} Furthermore, a validation error would result the test to fail providing a comprehensive error message: {% embed url="" %} You can reference yourself with the Mocha documentation for more useful functions to better test your API flows at #### Postscript & Assertion Another way to validate a value from postscript is via the Assertion section of the API container. Once your value is saved in a variable, you can use that variable for validation. {% hint style="warning" %} Make sure to convert any variable to a string type while using the Assertion section. {% endhint %} In this case our code should look like the following: ```javascript const fruits = $; // returns the entire json response in object type. const orangeOnList = fruits.some((fruit)=>fruit.name == "Orange").toString() // returns "true" ``` This now converts the boolean value `true` to a string type. To validate your API based on orangeOnList: 1. Add an Assertion. 2. Select/type isOrangeOnList on the parameters list. 3. Select “Equals” from the Values dropdown. 4. Type `true` on the field. 5. Run your API test. {% embed url="" %} # Using Built-in Functions in Postscript Using built-in functions in Postscript You can use [Loadmill's built-in functions](/test-editor-reference/functions) in your Postscript code to perform common operations. You can invoke these functions like you would call it in Loadmill's extractions or assertions. For example, to add two numbers: ```javascript const result = __add(1, 2); console.log(result); // 3 ``` Obviously, the `__add` example is a bit simplistic and you can use the JavaScript `+` operator directly, but it illustrates how to use built-in functions. To really leverage the power of built-in functions, you can use them for more complex operations which are not easily achievable with basic JavaScript. ## More Interesting Examples Some of Loadmill's built-in functions are very useful and powerful and can help you save time and effort in your Postscript code. Here are a few examples: ```javascript // Generate a random UUID const id = __random_uuid(); // Hash a string using SHA-256 const hash = __sha256('mySecret', 'hex'); // Get the next suggested element from full_array which is not in sub_array. const nextMissingElement = __array_missing_element([0,1,3,4], [0,1,2,3,4,5,6,7,8,9,10]) // returns 2 ``` And many more complex operations are available. You can find the full list of built-in functions in the [built-in functions documentation](/test-editor-reference/functions). ### ⚠️ String Inputs and Outputs in Built-in Functions When working with Loadmill's built-in functions in Postscript, **remember that all inputs and outputs must be strings**. This is a common source of confusion, especially when dealing with complex types like arrays or objects. For example, this will **not** work as expected: ```javascript // ❌ Bad: passing arrays directly const nextMissingElement = __array_missing_element([0,1,3,4], [0,1,2,3,4,5,6,7,8,9,10]); ``` That’s because the function expects string representations of the arrays. To fix this, convert them using `JSON.stringify`: ```javascript // ✅ Good: Correct usage with JSON.stringify const array1 = JSON.stringify([0, 1, 3, 4]); const array2 = JSON.stringify([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]); const nextMissingElement = __array_missing_element(array1, array2); const result = JSON.parse(nextMissingElement); console.log(result); // output: 2 ``` Another example: ```javascript const arrAsString = JSON.stringify([109, 136, 156, 188, 19, 190, 2, 34, 55, 90]) const sortedArr = __array_sort_numbers(arrAsString) console.log(sortedArr); // output: [2, 19, 34, 55, 90, 109, 136, 156, 188, 190] ``` 🔁 Tip: Use `JSON.stringify` for inputs and `JSON.parse` for outputs when working with arrays, objects, numbers or booleans. Never assume the return value is a native type, it will always be a string. # Authentication Flows Within the Login tab of each Test Suite, you can define a short flow that will run **before your test flows**. This setup flow is most commonly used to execute requests that will authenticate the test flow itself via cookies or authorization tokens. This flow can also be used to retrieve and extract data necessary for the execution of your test flows. Cookies received and parameters extracted by the login flow requests will be accessible by the requests of the actual test flows of that suite. Global authentication headers configured below can use parameters extracted by the login flow requests (In a `${param}` syntax), and will override any matching headers in the test flow requests of that suite. ![](/files/JdzfGyu90ABEuASsbCn8) ![](/files/5QY2rE6pNgCwt2n2QB94) You can create your login flow from scratch within the Login tab OR use a better way - create it as a [shared flow](https://docs.loadmill.com/collaboration/shared-flows) so you can re-use it in other test suites. # Before and After Hooks Configure tests to run before and after each/all Test Suite flows. Often while writing tests, you have some setup work that needs to happen before tests run, and you have some teardown work that needs to happen after tests run. From now on, you can do it and even more by using our before & after hooks. There are **4 optional hook types** that can be configured: * Before all test flows * Before each test flow * After each test flow * After all test flows ## How it works Let's take a simple yet comprehensive example on how to use the hooks. For example, I have a few tests aka test flows in Loadmill that have a shared setup and teardown logic. First, I need to create a new user and login. Then, my test flow/s should run but it is important to get user data before each test flow. Eventually, I need to delete the user that has been created. Let's configure this logic together. Navigate to/create a Test Suite containing relevant tests -> BEFORE/AFTER tab. On this page, you can create before & after hooks from scratch or use pre-defined [shared flows](https://docs.loadmill.com/collaboration/shared-flows). ![](/files/qh1J4pXnUDyQtUM1UEwE) {% hint style="info" %} :brain: Make sure all test flows residing in the Test Suite are aligned with the Before & After logic. You may configure all hooks or just one depending on your use case. {% endhint %} In our case, we will create the "Before All" flow that will include requests to create a new user. Then, we will use our well-known [Login tab](https://docs.loadmill.com/api-testing/test-suite-editor/global-login-flow) to create a login flow. The "Before Each" flow will include a request to get the user data. After that, our test flow/s run and eventually, we delete the created user within the "After All" flow. Let's run the test flow and see results. ![](/files/-MbemKq0mLiGJtI8GYiC) If we enter the test flow itself, we will see our Login and Before Each flows as well. ![](/files/-MbempbJIUlLqNh0i_jm) # Loadmill MCP Overview ## Why Loadmill MCP? Loadmill MCP brings the power of AI-native workflows to E2E and behavioral testing. Inspired by the latest advances in agentic workflows, Loadmill MCP lets you: * **Generate new E2E tests in seconds from natural language.** * **Run tests as you develop, with instant feedback and AI-powered insights.** * **Automate repetitive QA tasks and empower your team to ship with confidence.** Test generation is also supported for hybrid testing (combining API and UI steps). [Learn more in the Hybrid Testing section.](/web-ui-and-hybrid-testing/overview) *** ## Core Features * **AI-Native Test Authoring:** Describe what you want to test in plain text. Instantly get a ready-to-run E2E suite—no boilerplate, no manual scripting. * **Continuous Feedback Loop:** Run tests directly from your IDE or any AI tool, see results and logs in real time, and get actionable suggestions for failures. * **Self-Verifying Workflows:** Loadmill MCP enables agents to generate, run, and validate tests in a closed loop—ensuring your E2E flows work as intended after every change. * **Collaboration for All:** Anyone on your team, regardless of experience, can contribute meaningful tests and improve coverage. No prior testing knowledge required. *** ## Real-World Workflow: Main Use Cases ### 1. Run E2E Tests & Analyze Results While Developing With Loadmill MCP, you can instantly validate your changes as you code—without leaving your IDE. This approach keeps you focused, reduces context switching, and ensures that every update is automatically checked against your real-world E2E scenarios. The result: faster feedback, fewer bugs, and greater confidence in every release. This example shows how you can use Loadmill MCP to quickly validate your changes and build with confidence. The workflow below is shown with VS Code and GitHub Copilot, but you can achieve the same results with any compatible IDE or AI tool: **Step 1: Make changes and run tests** After updating your code (for example, changing a function or fixing a bug), trigger all relevant E2E tests. Test results appear instantly—if there are failures, you’ll see them right away. ![Run tests and see failures](/files/7ZpbpTllXaxtyJXqEX04) **Step 2: Analyze failures and fix the issue** Use Loadmill MCP to get and analyze the failures, leveraging its insights and suggestions to help you find a solution. Update your code to address the issues. ![Analyze failures and fix code](/files/Hr2gt3DLIlHvQJjIjFfS) **Step 3: Re-run tests and confirm success** Re-run the tests. This time, all tests pass, giving you confidence to continue development. ![All tests pass](/files/2PJ1pkIFz0KI3Io1DAv4) *** ### 2. Generate New E2E or Hybrid Tests in Seconds Using Free Text With Loadmill MCP, creating new tests is as simple as describing what you want to check. Just type your scenario in plain text and let the AI instantly generate a ready-to-run E2E test flow or suite for you. ![Generating a test from free text](/files/Jtvcg5bzZPdY31I7jBIQ) This workflow is designed for speed and flexibility: * **Instant test generation:** Go from a free-text idea to a fully structured test in seconds. No manual scripting, no previous testing experience needed. * **Flexible integration:** Once generated, you can either: * **Import the test file directly into Loadmill** and run it as part of your existing suites, or * **Sync the test via git**—push it to GitHub (or any version control system) and have it run automatically as part of your CI/CD pipeline. * **Always up-to-date and spec-compliant:** Every test generated by Loadmill MCP is built using the latest Loadmill test-suite schema, so you can be sure your tests are always aligned with current Loadmill standards—no outdated formats, no compatibility worries. *** ## Getting Started ### Installation ```bash npm install -g @loadmill/mcp ``` ### Connecting Your IDE **First, generate a Loadmill API token from your** [**account settings**](https://app.loadmill.com/app/user/settings/security)**.** #### VS Code Add a server configuration to your IDE: ```json { "servers": { "loadmill": { "type": "stdio", "command": "npx", "args": ["@loadmill/mcp"], "env": { "LOADMILL_API_TOKEN": "${input:loadmill-api-token}" } } }, "inputs": [ { "id": "loadmill-api-token", "type": "promptString", "description": "Loadmill API token", "password": true } ] } ``` #### Cursor Setup Steps: 1. Open the Cursor application. 2. Navigate to **Cursor Settings → MCP & Integrations → MCP Tools → New MCP Server**. 3. This opens `MCP.json` where you paste the following configuration: ```json { "mcpServers": { "loadmill": { "command": "npx", "args": ["@loadmill/mcp"], "env": { "LOADMILL_API_TOKEN": "REPLACE_WITH_YOUR_LOADMILL_API_TOKEN" } } } } ``` Set your `LOADMILL_API_TOKEN` environment variable before launching Cursor. *** ## Resources * [Loadmill MCP official npm page](https://www.npmjs.com/package/@loadmill/mcp) # Test Authoring Workflow ### 🚀 Overview This guide explains how to create and update Loadmill test suites directly in your IDE (Cursor, VSCode, or any other editor) using the Loadmill MCP. With the MCP workflow you can: * Create and update Loadmill tests directly from your IDE * Validate changes instantly * Run tests to ensure they work as expected * Integrate new tests into your pipeline > ⚠️ Before you get started, make sure you meet the [Prerequisites](#prerequisites) below for this workflow. ### 🎥 Video Guide Watch this demo to see how to create a new test suite using the Loadmill MCP: {% file src="/files/SdHjGBp0Y0ZVwQ4uwpAO" %} ### 📦 Where the Work Happens All Loadmill suites live in a Git repository, and this repository is the **entry point** for creating and updating tests. By working directly in this repo: * The actual changes happen alongside the suite files, just like any other code change. * You are able to leverage GitHub MCP to pull context from anywhere in your system—PRs, API files, spec documents—but the resulting test updates remain local to the suite repo. * The familiar Git workflow (branches → commits → PRs → review) applies naturally. ### 🧩 The `AGENTS.md` File Each repository that contains Loadmill suites should include an `AGENTS.md` file that guides the IDE agent. This file defines: * Where suites and flows live * How the agent should structure and modify them * When validation or execution should occur * When to consult Loadmill Docs or GitHub MCP `AGENTS.md` acts as a **baseline behavior model**, and you may extend it with project-specific conventions—for example: * Naming standards * Which Loadmill shared flows to use * Common authentication patterns * Internal testing best practices With this file in place, the agent handles the workflow automatically. You only need to describe *what* to build or change. *** ## 🔧 Main Workflows There are two primary workflows when developing Loadmill tests in your IDE. ### Updating an Existing Test Suite Use this process when adding flows or modifying existing test logic. #### High-Level Steps 1. **Identify the suite to update**: Locate the relevant YAML file under `loadmill-suites/`. 2. **Apply the required changes**: Describe the new flow or modifications, the agent updates the YAML. 3. **Ensure the suite is valid**: The agent validates the suite and resolves any issues. 4. **Commit the changes to a branch**: A new Git branch is created, changes are committed, and the branch is pushed. 5. **Run the suite via MCP**: The suite is executed using that branch, and a link to the run is provided. 6. **Adjust if needed**: Update the suite and re-run until everything passes. 7. **Open a pull request**: Review and merge your changes. 8. *(Optional)* **Sync the suite in Loadmill UI**: Use the GitHub Sync tab to check out the correct commit to view or edit it in the Loadmill UI. **Important:** This last action will overwrite any unsynced changes in this suite in Loadmill, so be sure to sync first if needed. ### Creating a New Test Suite Use this workflow when introducing a new suite for a feature or area. #### High-Level Steps 1. **Create an empty suite**: The agent uses Loadmill MCP to generate a new suite in Loadmill. 2. **Create the actual test**: A new YAML file is created under `loadmill-suites/` with the generated content. 3. **Ensure the test is valid**: Validation is performed and issues are fixed. 4. **Commit to a new branch**: The suite is committed and pushed. 5. **Run the suite**: The agent executes it and provides a link to the run results. 6. **Adjust and re-run if necessary**: Continue until the suite passes. 7. **Open a pull request and merge**: Complete the standard Git workflow. 8. *(Optional)* **Sync the suite in the Loadmill UI**: Checkout the commit using the GitHub Sync tab to view or edit it in the Loadmill UI. ### 🧭 Your Role as the Developer The agent handles the lifecycle—validation, execution, branching, committing.\ You are responsible for: * Describing the tests or updates you want * Providing relevant context if needed (API changes, feature behavior, etc.) * Reviewing the generated test logic * Merging the resulting pull requests The intent comes from you; the workflow is handled automatically. ### ✅ Summary * Loadmill MCP enables you to build and update tests directly inside your IDE. * Test logic lives in the repository and follows your normal development workflow. * Loadmill MCP validates and executes tests; GitHub MCP manages branches and commits; Loadmill Docs MCP provides reference material. * The `AGENTS.md` file defines your automation model and can be customized. * Two main workflows are supported at the moment: updating an existing suite, or creating a new one. **Note:** This functionality is currently in beta. Please provide feedback to help us improve it! *** ### 📘 Prerequisites #### 1. Your Loadmill Suites Must Be Synced to Git Your Loadmill test suites must be connected and synced to a Git repository. This enables version control, collaboration, and integration with the MCP workflow. For setup instructions, see: [Syncing Loadmill with GitHub](/integrations-and-ci-cd/github-integration/data-sync-connection-to-github) #### 2. MCP Configuration To use this workflow, you need to configure your MCP setup to include Loadmill and GitHub servers. Example Cursor config: ```json { "mcpServers": { "loadmill": { "command": "npx", "args": ["@loadmill/mcp"], "env": { "LOADMILL_API_TOKEN": "REPLACE_WITH_YOUR_LOADMILL_API_TOKEN" } }, "loadmill-docs": { "url": "https://docs.loadmill.com/~gitbook/mcp", }, "github": { "url": "https://api.githubcopilot.com/mcp/", "headers": { "Authorization": "Bearer REPLACE_WITH_YOUR_GITHUB_TOKEN" } } } } ``` #### 3. `AGENTS.md` File Your Loadmill tests repository should include an `AGENTS.md` file at the root level to guide the IDE agent. This file defines conventions, structure, and automation rules for your Loadmill suites and can be customized to fit your project's needs. Download the file and add it to your repository to get started: {% file src="/files/D4aHTekWaijC16OCvfUX" %} # Loadmill Agent Skill ## Overview Agent Skills are a new open standard for giving AI coding agents specialized capabilities. Tools like **Cursor**, **Claude Code**, **OpenAI Codex**, and **Goose** all support skills—portable instruction packages that agents discover and load automatically when a relevant task comes up. The **Loadmill Agent Skill** gives your IDE agent everything it needs to create, maintain, and run Loadmill E2E tests as a natural part of your development workflow. Instead of context-switching to a separate testing tool, you describe what you need and the agent handles the rest—authoring YAML test files, validating them, running them via Loadmill MCP, and managing the Git workflow around them. > **Prerequisite:** The Loadmill Agent Skill requires the **Loadmill MCP** server to be installed and configured in your IDE. See the [Installation guide](https://github.com/loadmill/loadmill-docs/tree/master/loadmill-mcp/loadmill-mcp/README.md#installation) for setup instructions. *** ## Use Cases ### 1. Creating New Tests Describe the scenario you want to test in plain text and the agent takes it from there. It creates a new Loadmill test suite, writes the flows, validates the YAML against the Loadmill schema, commits to a branch, runs the suite, and hands you a link to the results—all without leaving your IDE. You can also point the agent at a **pull request for a new feature** and have it generate tests that cover the new behavior end-to-end. Or, direct it to a **buggy or uncovered section of your codebase** and let it create tests that fill the gap—turning blind spots into well-tested flows. No prior Loadmill experience is needed; the skill encodes the structure, conventions, and best practices for you. ### 2. Test Maintenance When your application code changes, tests need to change too. The Loadmill Agent Skill helps you keep your test suites in sync with your codebase. Point the agent at a PR, a diff, or simply describe what changed, and it will: * Identify which test suites and flows are affected. * Update assertions, request payloads, or flow logic to match the new behavior. * Validate and re-run the updated tests to confirm everything passes. This turns what used to be a manual, error-prone maintenance task into a quick conversation with your agent. ### 3. Running Regression Tests During Development While building a new feature, you want continuous confidence that existing E2E flows still work—without waiting for CI. With the skill installed, you can ask your agent to run the relevant test suites or a full test plan against your **localhost** as you develop. The agent will: * Execute the tests via Loadmill MCP against your local environment. * Analyze failures and surface actionable insights. * Help you fix any broken tests caused by intentional changes. * Re-run until all regressions are resolved. This creates a tight feedback loop—write code, run tests against your local server, fix issues—all inside your IDE, long before you push. *** ## Getting Started ### 1. Install Loadmill MCP Follow the [Installation guide](https://docs.loadmill.com/loadmill-mcp/loadmill-mcp#installation) to configure the Loadmill MCP server in your IDE. ### 2. Download the Skill [Download the latest Loadmill Agent Skill](https://github.com/loadmill/loadmill-agent-skill/releases/latest/download/loadmill.zip) and extract it into your project's skills directory: | IDE / Agent | Directory | | ------------ | ------------------------- | | Cursor | `.cursor/skills/loadmill` | | Claude Code | `.claude/skills/loadmill` | | OpenAI Codex | `.agents/skills/loadmill` | | Goose | `.goose/skills/loadmill` | The download is updated automatically whenever a new skill version is published. Download `loadmill.zip` using the link above rather than GitHub's generic **Source code** archives, which are not packaged for installation. To inspect the skill before installing it, visit the [Loadmill Agent Skill source repository](https://github.com/loadmill/loadmill-agent-skill). ### 3. Start Using It Once the skill is in place, your agent will automatically pick it up. Try prompts like: * *"Create an E2E test suite for the user registration flow"* * *"Update the checkout tests to reflect the new payment API"* * *"Run all regression tests for the billing module"* The agent will follow the workflows defined in the skill—validating, committing, running, and reporting results—while keeping you in control at every step. # Database Testing Loadmill allows users to execute various queries directly to their DB in order to validate data. This is a very powerful feature as it allows users to check data integrity and accuracy. ### Usage #### Postgres To execute a database query: 1. Go to the relevant test flow within your Test Suite. 2. Request Method: POST. 3. Request URL: `https://db-relay-service.loadmill.com/api/postgres` 4. Content Type: application/json 5. Request Body: ```json { "connectionString": "postgres://...", "query": "SELECT * FROM USERS" } ``` See the request example below:
Running this request will return a JSON response with the required data. #### MongoDB To execute MongoDB queries: 1. Apply the same steps above. 2. Request URL: `https://db-relay-service.loadmill.com/api/mongo` 3. Request Body: {% hint style="success" %} In this example we're searching for a user with the name "John" {% endhint %} ```json { connectionString: 'mongodb://...', collection: 'users', command: 'find', query: { "name":"John" } } ``` {% hint style="info" %} The MongoDB service is initially meant for "read-only" purposes (i.e. find). However, using a Docker image privately allows you to use the environment variable `ALLOW_ALTERING=true` and by doing so the following options are available:\ `insertOne`, `insertMany`, `updateOne`, `updateMany`, `deleteOne` and `deleteMany`. {% endhint %} #### MongoDB Altering Example Options **InsertOne:** ```json { connectionString: 'mongodb://...', collection: 'users', command: 'insertOne', query: { "name": "John", "age": "56" } } ``` **updateOne:** ```json { connectionString: 'mongodb://...', collection: 'users', command: 'updateOne', query: { "name":"John" }, update: { "$set": { "age": "67" } } } ``` **deleteOne:** ```json { connectionString: 'mongodb://...', collection: 'users', command: 'deleteOne', query: { "name":"John" } } ``` Additional help regarding update operators can be found [here](https://www.mongodb.com/docs/manual/reference/operator/update). #### Redis To execute Redis queries: 1. Request Method: POST. 2. Request URL: `https://db-relay-service.loadmill.com/api/redis` 3. Request Body: 
{
    connectionString: "redis://...", 
    command:"get | hget | hgetall",
    key:"any-key",
    field: "any-field"
   }
#### MySQL To execute queries directly to MySQL 5.7: 1. Request Method: POST. 2. Request URL: `https://db-relay-service.loadmill.com/api/mysql` 3. Request Body: ```json { "connectionString": "mysql://...", "query": "SELECT * FROM TASKS" } ``` #### Oracle 1. Request Method: POST. 2. Request URL: `https://db-relay-service.loadmill.com/api/oracle` 3. Request Body: `https:///api/oracle` ``` body: { connectionString: 'host[:port][/service_name]' user: 'database-user' password: 'mypw' query: 'SELECT * FROM table_name' } ``` ### DB relay service static IPs Executing queries to your environment DB may require VPN. You can easily overcome this by whitelisting the DB relay service static IPs: `[52.42.51.230, 54.190.108.53]` in your firewall. ### Docker image You can use a Docker image for DB relay service to deploy it in a specific environment. Find more information [here](https://hub.docker.com/r/loadmill/db-relay-service). --- [Next Page](/llms-full.txt/1)