Loadmill
Learn moreSchedule a demo
  • Introduction
    • Loadmill - AI - Powered Solution
    • Deviceless mobile testing
      • Capturing traffic with Loadmill MITM Proxy
      • Loadmill desktop recorder
        • Generating test flows
      • Installing certificate on mobile devices
        • iOS certificate installation
        • Android certificate installation
      • Configuring proxy on mobile devices
        • iOS Wi-Fi settings
        • Android Wi-Fi settings
      • Troubleshooting
    • What is an API
      • API - Data Fetching
      • Quick examples of API requests
      • What is an API endpoint?
    • API Server Testing
      • Contract testing
      • Regression Testing
  • Quick Guide
    • Create Account
    • Download Test Composer
    • Register your first API flow
    • Running Your API Test
  • Loadmill Test Composer
    • Quickstart
    • Composer Layout
    • Filter Settings
  • Test Editor
    • Layout
    • Flows
      • Generated Flow Code
      • Test Flow editor
      • Flow Controls
      • Add CSV to Flow
      • Flow Execution
    • Steps
      • Request step
      • Code step
      • Extraction & Assertion step
      • Web Socket step
    • Extractions - Set Parameters
    • Assertions - Verify Response
    • Parameters
      • Parameter Execution Order
      • Test Suite Parameters
      • Parameters Sets
    • ⨍(⨯) FUNCTIONS
    • Postscript
      • Running Postscript
      • Accessing w/ Postscript
      • Validating Postscript
    • Login/Authentication Flow
    • Before & After Hooks
  • Load Testing
    • Load Test Editor
    • Load Testing Guide
    • Analyzing Load Test Results
    • Parameterized Load Test
    • Domain Verification
    • Configuration Files
    • Load Testing FAQs
    • Load Testing Troubleshooting
  • User Behavior Testing
    • Overview
    • Setup
    • Recording troubleshooting
    • Additional recording methods
    • Recording settings
    • How to work with Recordings
  • Auth
    • Okta SSO integration
    • API Tokens
    • Testing with CORS
    • REST API
  • Integrations
    • Loadmill Agent
    • CI integration
    • GitHub
      • CI integration
      • Data sync
    • GitLab
    • Bitbucket
    • Jira
    • New Relic
    • Slack integration
    • TestRail integration
    • Database Testing
    • Kafka Testing
    • Datadog Integration
    • ✉️Email Testing
    • Webhook Testing
    • Integrations FAQs
    • XRay
    • TestRail
    • gRPC Support
  • Collaboration
    • Collaboration
    • Teams
    • Groups & Reports
    • Test Suite Collaboration
    • Reviews
    • Shared Flows
    • Labels
  • Reporting
    • API Catalog & Coverage
      • API Catalog
        • Unique Entity ID's Mapping
        • Domain Mapping and grouping
        • Endpoints grouping
        • OpenAPI upload
      • Test Coverage
        • Generating API test coverage report
  • General
    • Billing
      • Usage report
    • Settings
      • 📈Analytics
        • Flow Run History
      • 🧳Import & Export
    • General FAQs
    • General troubleshooting
    • Comparisons
      • Loadmill vs. SoapUI
      • Loadmill vs. JMeter
      • Loadmill vs. Blazemeter
      • Loadmill vs. WebdriverIO
      • Loadmill vs. Potato
    • Miscellaneous
      • Running a Test Suite
      • Test Plan
      • API Testing troubleshooting
      • API Testing FAQs
      • Test Editor
        • API Tests - Data from CSV files
Powered by GitBook
On this page
  1. Introduction
  2. API Server Testing

Contract testing

PreviousAPI Server TestingNextRegression Testing

Last updated 2 years ago

Intro

Contract testing is a testing approach used in software development that involves testing the interactions between two different services or systems by defining and verifying the contracts or agreements between them.

In a typical scenario, when two services communicate with each other through an API, the contract testing approach involves defining the expected input and output data for each API endpoint, and then writing tests to verify that the API endpoint behaves as expected.

Contract testing helps in detecting and preventing issues that may arise due to changes made to the API, without having to run end-to-end tests. This allows for faster and more efficient testing of software systems, reducing the time and effort required for testing.

Overall, contract testing helps improve the reliability, stability, and resilience of software systems by ensuring that the different components of the system work together as intended, and are compatible with each other.

Contract testing with Loadmill

Contract testing with Loadmill is easy. To create contract validation of an API call, use or as following:

  • Enforcing JSON Schema of an API: With 'JSON Schema' you can explicitly enforce multiple of aspects of your json data. In the request editor assertions insert new entry, use '__responseBody' built-in parameter and ensure the values are of the type "JSON Schema", then insert a valid json-schema.

In the following table you can see an example of the schema and a typical response that comply with the schema.

Schema
Response
  • Enforcing JSON structure using JSON-Contains: 'Json-contains' is a lightweight capability in Loadmill to enforce JSON data. In the request editor assertions insert new entry, use '__responseBody' built-in parameter and ensure the values are of the type "JSON Contains", then insert an expected json or a subset that can contain less fields. Note, to ensure existence of a field ignoring it's value use "*" notation.

In the following table you can see an example of the expected JSON-Contains object and a typical response that comply with it.

JSON-Contains object
Response

Automatic assertions Loadmill can generate 'JSON schema' or 'JSON contains' assertions automatically during test composition. Such assertions can serve are a quick baseline that can be adjusted later on. To turn automatic JSON validations.

  1. Locate the 'JSON Validation section in the list', make sure the toggle turned on and adjust the type according to the needs.

  1. From now on, every generated test with loadmill composers will cary json validation assertions.

head to settings icon -> recordings:

if 'JSON Validation' section doesn't exist, try adding it by clicking on at the top right corner and selecting it from the list.

(optional) Click on the '+' to add ignored keys.

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "integer"
    },
    "title": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "price": {
      "type": "integer"
    },
    "discountPercentage": {
      "type": "number"
    },
    "rating": {
      "type": "number"
    },
    "stock": {
      "type": "integer"
    },
    "brand": {
      "type": "string"
    },
    "category": {
      "type": "string"
    },
    "thumbnail": {
      "type": "string"
    },
    "images": {
      "type": "array",
      "items": [
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        },
        {
          "type": "string"
        }
      ]
    }
  },
  "required": [
    "id",
    "title",
    "description",
    "price",
    "discountPercentage",
    "rating",
    "stock",
    "brand",
    "category",
    "thumbnail",
    "images"
  ]
}
{
   "id":1,
   "title":"iPhone 9",
   "description":"An apple mobile which is nothing like apple",
   "price":549,
   "discountPercentage":12.96,
   "rating":4.69,
   "stock":94,
   "brand":"Apple",
   "category":"smartphones",
   "thumbnail":"https://i.dummyjson.com/data/products/1/thumbnail.jpg",
   "images":[
      "https://i.dummyjson.com/data/products/1/1.jpg",
      "https://i.dummyjson.com/data/products/1/2.jpg",
      "https://i.dummyjson.com/data/products/1/3.jpg",
      "https://i.dummyjson.com/data/products/1/4.jpg",
      "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
   ]
}
{
    "id": 1,
    "title": "iPhone 9",
    "description": "An apple mobile which is nothing like apple",
    "price": 549,
    "discountPercentage": "*",
    "stock": 94,
    "brand": "Apple",
    "category": "smartphones",
    "thumbnail": "https://i.dummyjson.com/data/products/1/thumbnail.jpg",
    "images": [
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
    ]
}
{
    "id": 1,
    "title": "iPhone 9",
    "description": "An apple mobile which is nothing like apple",
    "price": 549,
    "discountPercentage": 12.96,
    "rating": 4.69,
    "stock": 94,
    "brand": "Apple",
    "category": "smartphones",
    "thumbnail": "https://i.dummyjson.com/data/products/1/thumbnail.jpg",
    "images": [
        "https://i.dummyjson.com/data/products/1/1.jpg",
        "https://i.dummyjson.com/data/products/1/2.jpg",
        "https://i.dummyjson.com/data/products/1/3.jpg",
        "https://i.dummyjson.com/data/products/1/4.jpg",
        "https://i.dummyjson.com/data/products/1/thumbnail.jpg"
    ]
}
JSON schema
JSON contains
An example of schema validation request in loadmill
Json-Contains example in loadmill - Enforcing json content