Parameters

Working with Loadmill makes it very easy to convert recorded browser/network sessions (via HAR files) into test scenarios. But once a scenario has been created, it often needs to be parameterized in order to denote the dynamic parts of your API.

This is best explained by example, so let's have a look at the following test scenario:

1. User adds a blog post: curl https://www.myblog.com/posts --data "Hello World!"

2. User likes his own post: curl -X PUT https://www.myblog.com/posts/123/like

The 123 part is the identifier of the blog post created by the user - it cannot be known in advance because it is generated by the server at runtime. This is a very common use case for parameters and is handled by two simple steps:

1. Extract the ID from the first response into a parameter, e.g. postId

2. Embed the parameter in the second request URL: https://www.myblog.com/posts/${postId}/like

You may embed parameters with the ${} syntax in the request URL as well as the request body, request headers, extractions, assertions and more.

Note that any parameterized expression such as posts/${postId} will remain as-is if no such parameter is defined or extracted before the expression is evaluated.

Default Parameters

Another common use case for parametrization is when you want to reuse test scenarios on different environments or with small adjustments.

Let's extend our first example by requiring that the user provide credentials via basic authentication. This means our new URLs could look like this:

1. https://testUser:[email protected]/posts

2. https://testUser:[email protected]/posts/${postId}/like

If you would like to use different credentials for every test run, you may replace the username and password with parameters and set their values in the Test Suite Parameters tab whenever you reuse the test. So now the URLs will look like:

1. https://${user}:${pass}@www.myblog.com/posts

2. https://${user}:${pass}@www.myblog.com/posts/${postId}/like

Using parameter defaults is especially useful for automated testing and CI where you may be testing a different server every time you run a test. Please refer to the Loadmill CLI and npm module for more information about how to inject parameters dynamically in such scenarios.

Parameter Extraction

Parameters can be defined and populated with values dynamically after each request in your test scenario.

There are several extraction query types that may be used:

• JSONPath - used for extracting values from a JSON response. For example, the query $.post.id will extract the value 123 from this JSON response:

 {
     "post": {
         "id": 123,
         "text": "Hello World!"
     }
 }

• JQuery (Cheerio) - used for extracting values from XML/HTML responses. We use a subset of the JQuery selector syntax called Cheerio. You may add an optional (but very useful) attribute input to your query that selects an attribute value from the first element found by the jQuery. If you do not provide an attribute to select, the query will simply output the inner content of said element.

• JS RegExp - used for extracting arbitrary values from any kind of textual response via regular expressions with capture groups. For example, we can extract the id field from the same JSON response we've seen above using a regular expression: .*"id":\s*([0-9]*).

• Header - used for extracting response header values via header names.

• Assignment - used for assigning an explicit value to a parameter or a function to an existing parameter. Previously defined or built-in parameters may be embedded within the string, e.g. https://${host}/path/to/glory or The time is ${__now}.

• Clojure - used for extracting values from Clojure (EDN content type) responses. Querying the data is done using JSONPath. For example, the query $[":user"][":id"] will extract the value 56 from this EDN response:

 {:user {:role :viewer, :name "Rivi", :teams nil, :id "56"}}
 

Previously defined or built-in parameters may be embedded within any kind of extraction query. These parameters will be evaluated right before the query itself is evaluated.

Built-in Parameters

There are several built-in parameters that you can use in your test scenario. They are:

• __status The status code of the last HTTP response.

• __statusText The status text of the last HTTP response.

• __responseTime The total response time (in milliseconds) of the last HTTP response.

• __testRunId - The test run id: (Suite / Flow / Load)

• __testStartTime - The test run start time (UTC in milliseconds)

• __launchedBy - The name of the user running the test.

Note: some previous built-in parameters are now defined as no-argument parameter functions and can still be used in the same way.

Advanced Usage

So far, we've only seen how to inject a simple parameter value into an arbitrary expression, e.g. Hello ${name}. However, it is also possible to inject a computed value using Parameter Operators or Parameter Functions, e.g. The total price is ${price + __mult(price,tax)}. You may use operators or functions anywhere parameters may be used.

Computed values can be extremely useful when you need to introduce conditional behavior to your test. Say you want to skip Purchase Request if the preceding Get Price Request response returns a price above the current budget. This could be accomplished by extracting the price to a parameter and setting the Purchase Request skip condition to ${budget <= price}.

You may also use literal values within expressions, e.g. ${__if_then_else(is_good,'Success!',':_(')} - but be aware there are some syntactic limitations.

Function calls without arguments can be used with or without parentheses, e.g. __random_uuid can be used instead of __random_uuid().

See below the full list of supported operators and functions.

Notes And Limitations

Current syntax has some limitations. Syntax errors are easy to spot in the GUI - a malformed expression will simply not be highlighted.

If an expression is invalid due to its syntax it will simply remain as-is and will not be computed nor replaced at run-time. If, however, an operator or a function receives invalid input (e.g. division by zero or any of the parameter arguments not having a value) the test will fail with an error.

Note that predefined parameter values (AKA Parameter Defaults) are computed whenever a test configuration is validated - thus computation related errors will render the test configuration invalid.

Current syntax limitations are:

• Operators must be separated from their arguments by spaces, e.g. ${x + y} is fine but ${x+y} will not be computed.

• Spaces are not allowed anywhere else within an expression, e.g. both ${__add(x, y)} and ${fullName == 'John Doe'} will not be computed.

• Literal values may not contain whitespace characters, commas (,) or single quotes ('). These cannot be escaped - simply define a previous parameter with the desired value when the need arises. Note that numeric literal values need to be quoted the same as any other value, e.g. ${x > '0'} is OK but ${x > 0} will not be computed.

• You may chain multiple operations together, e.g. ${x * y + z} but you may not use parentheses to set precedence, e.g. ${(x * y) + z} will not be computed. This can usually be worked around using functions though, e.g. ${__mult(x,y) + z}

• All operators have the same precedence - computations always conform to right-associativity, i.e. ${x * y + z - j + k} will be computed as x * (y + (z - (j + k))).

• Computations may not be nested, i.e. you may not pass a computed value as an argument to function, e.g. ${__mult(x,y) + z} is OK but neither ${__mult(x + y,z)} nor ${__mult(__add(x,y),z)} will be computed.

• Unary operators, e.g. ${-x} are not supported. This can be overcome using functions such as __neg or __not.

Operators

The currently supported operators are:

Textual Operators

• = Strict equals operator. Aliases: == and ===.

• != Strict not-equals operator. Alias: !==.

May be applied to any two parameters which have values.

Boolean Operators

• | Logical OR operator. Alias: ||.

• & Logical AND operator. Alias: &&.

May be applied to any two parameters which have values.

True Semantics

A parameter translates to boolean true if and only if

• It has a value and

• The value is not an empty string and

• The value is not equal to false, FALSE, FaLsE or any other combination of upper-case and lower-case letters that forms the word false.

The computed value of a valid boolean operation is either exactly true or exactly false.

Numeric Operators

• + Addition operator.

• - Subtraction operator.

• * Multiplication operator.

• / Division operator.

• < Less-then operator.

• <= Less-then-or-equals operator.

• > Greater-then operator.

• >= Greater-then-or-equals operator.

May be applied to any two parameters which have values that translate to finite numbers. Computed values are not rounded to integers.