Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.5k views
in Technique[技术] by (71.8m points)

grafana - Prometheus Increase function alert

So my goal is to set up alerting mechanism for network interface drops. I am using node_network_receive_drop_total which is a counter. My goal is to alert when drops occur meaning when count of drops go up. Lets say I have these values for drops over a period of time.

|node_network_receive_drop_total |difference
|0                               |0
|1                               |1
|5                               |4
|5                               |0  
|7                               |2

In this example, how can I use increase function and alert condition in grafana to alert when the difference is non-zero?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

In the Query tab a graph with a name something like node network drops and query something like this: increase(node_network_receive_drop_total[5m]).

When there are no drops, this graph should have a flat line at 0. When there is a drop the graph will show a line at 1 for 5 mins after the drop.

In the Alert tab create an alert with condition of WHEN max() OF query(A, 15m, now) IS ABOVE 0.

As you will only have one value returned by the query you could use max(), min(), or max() (they will all return the same value).

The A in query should match the letter on your query in the Query tab. If you have more than one query displayed on the graph, you may need to change this.

15m and now look at data from the last 15m.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...