Assertions

Assertions are used to check whether test results comply with expectations. they can be included in a test or in a phase specification. Both phase assertion and a test assertion is a list of of assert-counter and assert-histogram assertions. All of them must be successful for the test or phase to pass.

When assertions are included in a test specification, the histograms used are the representation of the specified metric for the whole test. When assertions are included in a phase specification, the histograms used are the representation of the specified metric for that single phase.

Counter

The assert-counter action does a check on the value of the specified counter.

Name Type Required Action Info
id string Yes No Counter name as described in Counters.
metric string Yes No Takes the value percent or count. Defines if the assertion is done regarding the counter value or the percentage it represents over the total of requests.
filter filter object Yes No Filter to assert the success of the test. A filter is a single condition. Applied over the choice of metric.

The filter object can use one of the following conditions:

Name Type Required Action Info
lte integer Yes No Counter value less than or equal to lte value.
lt integer Yes No Counter value less than lt value.
gte integer Yes No Counter value greather than or equal to gte value.
gt integer Yes No Counter value greather than gt value.
eq integer Yes No Counter value is equal to eq value.
\end{tcolorbox}

Example

{"assert-counter" : {"id" : "http_counter_responseCode_200_localhost",
                     "metric" : "percent",
                     "filter" : {"lte" : 75}
                    }
}

Histogram

The assert-histogram action does a check on the collection of values in a histogram.

Name Type Required Action Info
id string Yes No Histogram name as described in chapter Counters.
statistic string Yes No Defines the statistic on which to apply the filter. Values in: min, max, mean, median, percentile50, percentile75, percentile90, percentile95, percentile99, percentile999.
filter filter object Yes No Filter to assert the success of the test. A filter is a single condition. Applied over the choice of statistic.

The filter object:

Name Type Required Action Info
lte integer Yes No Counter value less than or equal to lte value.
lt integer Yes No Counter value less than lt value.
gte integer Yes No Counter value greather than or equal to gte value.
gt integer Yes No Counter value greather than gt value.
eq integer Yes No Counter value is equal to eq value.

Example

{"assert-histogram" : {"id" : "http_counter_responseTime",
                       "statistic" : "mean",
                       "filter" : {"lt" : 3000}
                    }
}