Monitoring and Alarm Service Deployment

Overview

The Kato monitoring and alarm system is based on the Alertmanager service. Alertmanager is mainly used to receive the alert information sent by Prometheus, supports a wealth of alert notification channels, and it is easy to perform deduplication, noise reduction, grouping, etc. of the alert information. It is an avant-garde alert notification system.

This document is suitable for users who intend to monitor Kato nodes or Kato components. It describes how to deploy a monitoring and alarm system in Kato.

For cluster monitoring alarm item description, please refer to cluster monitoring alarm item description

Prerequisites

  • There is already a monitoring and alarm service application template in the application market. Online users can install it through the application market, and offline users can download Monitoring and Alarm Service Offline package Import.

Steps

You can deploy Alertmanager, ans Slack to your Kato environment by one-click installation in the application market.

Installation and Deployment

Install Alarm Service

Running Result

Modify Prometheus Configuration in the Cluster

  1. First check the external port of Alertmanager
  1. Modify the environment variables of Alertmanager. In Environment Configuration -> Environment Variables, modify the variable value of WEB_EXTERNAL_URL to the external port of Alertmanager above.
  1. Modify the configuration of the monitor in the cluster, modify the address connected to alertmanager, add the following parameters, and the parameter value is the external port of Alertmanager above.
$ kubectl edit rbdcomponents.kato.io rbd-monitor -n rbd-system
spec:
  args:
  ---alertmanager-address=39.99.54.241:10000
  env:
  -name: REGION_NAME
    value: Kato

Parameter explanation:

--alertmanager-address: the address where rbd-monitor pushes messages to alertmanager;

REGION_NAME: Custom cluster name, used to distinguish different clusters when multiple clusters push messages to a Slack group at the same time.

as follows

After deleting the resource, it will automatically restart to take effect

kubectl delete statefulsets.apps rbd-monitor -n rbd-system

This completes the deployment of monitoring and alarm services.