Assertions - Verify Response
When testing an API, it is usually not enough to verify that all our HTTP requests completed successfully. Often, it is necessary to make sure that we got the correct response from the server by examining the response status, body or headers.
With Loadmill, this is made easy by using Assertions. Assertions are used in conjunction with parameters to do just that: examine the server's response and assert its correctness.
You may have an arbitrary number of assertions executed after each successful request. If an assertion fails, the next request will not be executed and the test scenario will be marked as failed, but the subsequent assertions for the current request will be executed nonetheless.
The target of an assertion is always a parameter value. You may use built-in parameters, default parameters or any parameter extracted from the current or previous requests in the current scenario as the target.
There are several types of assertions:
Exists - the target exists.
Doesn't exist - the target doesn't exist.
Equals - the target is equal to the given expression. The equality check is case sensitive.
Doesn't equal - the target is NOT equal to the given expression. The equality check is case sensitive.
Contains - the target contains the given expression. The containment check is case sensitive.
Doesn't contain - the target DOESN'T contain the given expression. The containment check is case sensitive.
Matches - the target matches the given regular expression.
Greater than - the target is greater than the given expression.
Less than - the target is less than the given expression.
JSON Schema - the target matches the JSON Schema.
You may embed parameters in any assertion expression. These parameters will be evaluated right before the assertion is executed.
See an example of the assertion that validates the generated ID is in the format of UUID below:
.gif?alt=media&token=87d8756d-75a4-4deb-88da-de201b3e54e6)
🧙♂ When creating tests via our Chrome recorder extension, Loadmill will create two automatic assertion types for you:
Default assertions - for any extraction we find in the recorded test, we add a default assertion to it (example: "id" exists).
Specific assertions - for any parameter with given user key (for example, "success") we extract it and assign to it an assertion (example: "success" equals "true").
Optional automatic JSON Schema assertion: you can enable this assertion within Settings - Recordings.
In many cases Loadmill users are using the same extractions and assertions. We've implemented the Suggestions feature that allows team admins to configure a repository of extractions and assertions within Settings - Suggestions. Then, all users can add the assertions from the repository by clicking on + SUGGESTIONS.
By default, there are a few common extraction and assertion examples in the repository. Team admins can also navigate to the Suggestions Settings directly from within the suggestion dialog window.
Keep in mind that all parameter values are textual, i.e. a parameter has no type such as Number or Array that we know from common programming languages.
This is important in order to avoid confusion when using parameter extractors such as JSONPath. For example, consider the following scenario:
Extract the value for
booksvia the JSONPath querystudent.bookson{"student": {"books": []}}Assert
booksis Not Empty.
You may expect this assertion to fail but, in fact, it will succeed. This is because the parameter books is evaluated to [] and therefore is considered as a non-empty string. One possible way to correct this is to use a RegExp assertion on the books parameter instead: \[[^\s]+\].
