Limiting Queries

When you deal with large installations with maybe thousands of hosts, then some views might produce a large output - too large to be viewed by a human. Large output also might hang your system quite a time. Even if Livestatus is fast, if you are going to display 25.000 services, that will eat up many ressources while being useless to the user.

In order to avoid such a situation, Multisite imposes a configurable limit on the number of returned data sets of a query. There is a soft and a hard limit, which are preset to 1000 and 5000, resp. You can change those limits in multisite.mk:

soft_query_limit = 2000
hard_query_limit = 10000

If a query would produce more results then the soft limit, than only the first that many data sets are being displayed. But users might have the permission the ignore the soft limit and can repeat the query while ignoring the limit. Then the hard limit comes into action.

Per default, normal users can ignore the soft limit, but only admin users can ignore the hard limit.

In order to take effect and actually save ressources, limits are honored directly at the source of the data - in the MK Livestatus module in the Nagios core. But that means, that the limit is imposed before any sorting is done. That way - if you make a query that reaches the limit, the data sets that are not being displayed might not reflect your sort order. Data sets normally being displayed first might be dropped while other ones might be displayed. In multi site setups - since data is fetched from one site after the other - as soon as the limit is exceeded, no futher sites will be queried at all.

We understand that you might not like that behaviour. But it is crucial for good performance. If we would apply the limit after sorting, we would have to retrieve all data from all sites always - regardless of any limit. The limit would then only save ressources in the webserver rendering the HTML output and in the browser. It would not save Livestatus network traffic and CPU ressources in the Nagios core.

The current implemention, on the other hand, strictly aborts processing data in the core as soon as the limit is reached.