Every now and then, some spurious peaks show up on munin graphs. The peaks are order of magnitude higher than the expected range of the data. This particularly happens with DERIVE plugins, that are notably used for network interfaces.

One way to fix this, as suggested by Steve Schnepp (and in the faq), is to set the maximum straight into the RRD database, and then let it reprocess the data to honour this maximum.

RRD_FILE=/var/lib/munin/example.net/host-if_bytes-send-d.rrd
rrdtool tune ${RRD_FILE} --maximum 42:${MAXIMUM_VALUE}
(rrdtool dump ${RRD_FILE} | rrdtool restore --range-check - ${RRD_FILE}.new) && \
mv -i ${RRD_FILE} ${RRD_FILE}.bak && \
mv ${RRD_FILE}.new ${RRD_FILE}

42 is the datasource name that munin uses.

As for the MAXIMUM_VALUE, a conservative value for network devices is to use the bandwidth of the backplane. So, for a 20Gbps backplane, one could use MAXIMUM_VALUE=$((20*1000*1000*1000/8)) (note the division by 8, as the plugin internally stores bytes, and converts them to bits using CDEFs when graphing).

UPDATE (2021-06-04): Of course, for single interfaces, rather than the total, you should set the maximum value to the speed of the interface.

UPDATE (2021-06-02): Sometimes, occasional peaks within the MAXIMUM_VALUE but orders of magnitude beyond usual use need to be cleared. A bit of seding can help, by replacing the high magnitude numbers with NaNs.

RANGE='[0-9.]\+e+0[6789]'
rrdtool dump ${RRD_FILE} \
  | sed "s/<v>${RANGE}/<v>NaN/" \
  | rrdtool restore --range-check - ${RRD_FILE}.new

UPDATE (2021-06-08): Correct the MAXIMUM_VALUE calculation and add a note about bits and byte conversion.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.