Assertions
Assertions are used to check whether test results comply with expectations. they can be included in a test or in a phase specification. Both phase assertion and a test assertion is a list of of assert-counter
and assert-histogram
assertions. All of them must be successful for the test or phase to pass.
When assertions are included in a test specification, the histograms used are the representation of the specified metric for the whole test. When assertions are included in a phase specification, the histograms used are the representation of the specified metric for that single phase.
Counter
The assert-counter
action does a check on the value of the specified counter.
Name | Type | Required | Action | Info |
---|---|---|---|---|
id |
string | Yes | No | Counter name as described in Counters. |
metric |
string | Yes | No | Takes the value percent or count . Defines if the assertion is done regarding the counter value or the percentage it represents over the total of requests. |
filter |
filter object |
Yes | No | Filter to assert the success of the test. A filter is a single condition. Applied over the choice of metric . |
The filter
object can use one of the following conditions:
Name | Type | Required | Action | Info |
---|---|---|---|---|
lte |
integer | Yes | No | Counter value less than or equal to lte value. |
lt |
integer | Yes | No | Counter value less than lt value. |
gte |
integer | Yes | No | Counter value greather than or equal to gte value. |
gt |
integer | Yes | No | Counter value greather than gt value. |
eq |
integer | Yes | No | Counter value is equal to eq value. |
\end{tcolorbox} |
Example
{"assert-counter" : {"id" : "http_counter_responseCode_200_localhost",
"metric" : "percent",
"filter" : {"lte" : 75}
}
}
Histogram
The assert-histogram
action does a check on the collection of values in a histogram.
Name | Type | Required | Action | Info |
---|---|---|---|---|
id |
string | Yes | No | Histogram name as described in chapter Counters. |
statistic |
string | Yes | No | Defines the statistic on which to apply the filter. Values in: min , max , mean , median , percentile50 , percentile75 , percentile90 , percentile95 , percentile99 , percentile999 . |
filter |
filter object |
Yes | No | Filter to assert the success of the test. A filter is a single condition. Applied over the choice of statistic . |
The filter
object:
Name | Type | Required | Action | Info |
---|---|---|---|---|
lte |
integer | Yes | No | Counter value less than or equal to lte value. |
lt |
integer | Yes | No | Counter value less than lt value. |
gte |
integer | Yes | No | Counter value greather than or equal to gte value. |
gt |
integer | Yes | No | Counter value greather than gt value. |
eq |
integer | Yes | No | Counter value is equal to eq value. |
Example
{"assert-histogram" : {"id" : "http_counter_responseTime",
"statistic" : "mean",
"filter" : {"lt" : 3000}
}
}