Notification Methods
Alarm notifications support multiple channels such as email, WeCom, DingTalk, etc. You need to configure the notification methods first before you can select the corresponding channels in the alert rules to receive alert messages.
Email
1. Enable SMTP service
Add the following environment variables in ops.yaml
, and restart the Ops Platform service to enable SMTP email functionality.
ENV_SMTP_ENABLED: "true" # Enable SMTP email capability
ENV_SMTP_HOST: "smtp.example.com" # Replace with the SMTP email server address
ENV_SMTP_PORT: "465" # Replace with the SMTP email server port
ENV_SMTP_USER: "ops@example.com" # Replace with the SMTP email server account
ENV_SMTP_PASSWORD: "************" # Replace with the SMTP email server authorization code
ENV_SMTP_SKIPVERIFY: "true" # Skip SSL/TLS certificate verification for the mail server
ENV_SMTP_FROM_ADDR: "ops@example.com" # Replace with the alert sending email address
ENV_SMTP_FROM_NAME: "HAP Alert" # Alert email subject
2. In Ops Platform --> Alerting --> Notification Methods, select Email and proceed with the editing.
3. In the Addresses field, fill in the email addresses of the alert recipients. Multiple email addresses can be entered using separators ;
\n
,
to send notifications to multiple recipients.
4. After saving, re-enter the editing page and click Test --> Send test notification. After a short wait, the recipient's email address will receive a test email, indicating that the email notification function is working properly.
Group Bot Notifications Configuration (WeCom/DingTalk)
Before configuring the WeCom or DingTalk bots, you need to define alert message templates to ensure that messages are notified in the appropriate format.
Configure Alert Message Templates
1. On the Notification Templates page, click Add notification template to add an alert template.
2. Customize the alert title template.
Fill in the template name in the Template name field: custom_title
The content of the Template is as follows:
{{ define "custom_title" -}}
{{ if gt (.Alerts.Firing | len) 0 }}🚨 {{ .Alerts.Firing | len }} Firing alerts. {{ end }}{{ if gt (.Alerts.Resolved | len) 0 }}✅ {{ .Alerts.Resolved | len }} Resolved alerts.{{ end }}
{{ end -}}
3. Customize the alert content template.
Similar to adding an alert template above, fill in the template name in the Template name field: custom_template
The content of the Template is as follows:
{{ define "custom.alert_labels_and_annotations" -}}
{{ len .Alerts.Firing }} Firing alert(s)
{{ range .Alerts.Firing -}}
{{ template "alert.labels_and_annotations" . -}}
Alert fired at: {{ .StartsAt | date "2006-01-02 15:04:05 MST" }}
{{ end }}
{{ len .Alerts.Resolved }} Resolved alert(s)
{{ range .Alerts.Resolved -}}
{{ template "alert.labels_and_annotations" . -}}
Alert and Recovery Time:
- Alert fired at: {{ .StartsAt | date "2006-01-02 15:04:05 MST" }}
- Alert resolved at: {{ .EndsAt | date "2006-01-02 15:04:05 MST" }}
{{ end -}}
{{ end -}}
{{ define "alert.labels_and_annotations" }}
Alert labels: {{ len .Labels.SortedPairs }}
{{ range .Labels.SortedPairs -}}
- {{ .Name }} = {{ .Value }}
{{ end }}
Alert annotations: {{ len .Annotations.SortedPairs }}
{{ range .Annotations.SortedPairs -}}
- {{ .Name }} = {{ .Value }}
{{ end }}
{{ end -}}
WeCom
1. Obtain the WeCom group alert bot URL, for example:
https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=******************
2. On the Contact points page, click Create contact point to create a new alert channel.
3. Fill in the following information
- Name: Custom name
- Integration: Select WeCom
- Webhook URL: Fill in the WeCom group bot URL
- Optional WeCom settings
- Message Type: Select Markdown as the message type
- Message: Fill in the alert message using the template identifier:
{{ template "custom.alert_labels_and_annotations" . }}
- Title: Fill in the alert title using the template identifier:
{{ template "custom_title" . }}
4. On the Contact points page, select the corresponding newly added alert channel, click Edit, re-enter the editing page, and then click Test --> Send test notification to send a test message.
Receiving a test message sent by the bot in the WeCom group indicates that the WeCom alert channel is configured successfully.
DingTalk
1. Obtain the DingTalk group alert bot URL, for example:
https://oapi.dingtalk.com/robot/send?access_token=******************
2. On the Contact points page, click Create contact point to create a new alert channel.
3. Fill in the following information
- Name: Custom name
- Integration: Select DingDing
- URL: Fill in the DingTalk group bot URL
- Optional DingDing settings
- Message Type: Select ActionCard as the message type
- Title: Fill in the alert title using the template identifier:
{{ template "custom_title" . }}
- Message: Fill in the alert message using the template identifier:
{{ template "custom.alert_labels_and_annotations" . }}
4. On the Contact points page, select the corresponding newly added alert channel, click Edit, re-enter the editing page, and then click Test --> Send test notification to send a test message.
Receiving a test message sent by the bot in the DingTalk group indicates that the DingTalk alert channel is configured successfully.
Webhook
1. On the Contact points page, click Create contact point to create a new alert channel.
2. Fill in the following information:
- Name: Custom name
- Integration: Select Webhook
- URL: Enter the Webhook URL
3. On the Contact points page, select the newly added alert channel, click Edit to re-enter the edit page, then click Test --> Send test notification to send a test message sequentially.
The Webhook will receive the following message body format:
{
"receiver": "test",
"status": "firing",
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "TestAlert",
"instance": "Grafana"
},
"annotations": {
"summary": "Notification test"
},
"startsAt": "2025-03-03T11:47:43.86072525+08:00",
"endsAt": "0001-01-01T00:00:00Z",
"generatorURL": "",
"fingerprint": "57c6d9296de2ad39",
"silenceURL": "http://localhost:3000/prometheus/grafana/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DTestAlert&matcher=instance%3DGrafana",
"dashboardURL": "",
"panelURL": "",
"values": null,
"valueString": "[ metric='foo' labels={instance=bar} value=10 ]"
}
],
"groupLabels": {
"alertname": "TestAlert",
"instance": "Grafana"
},
"commonLabels": {
"alertname": "TestAlert",
"instance": "Grafana"
},
"commonAnnotations": {
"summary": "Notification test"
},
"externalURL": "http://localhost:3000/prometheus/grafana/",
"version": "1",
"groupKey": "test-57c6d9296de2ad39-1740973663",
"truncatedAlerts": 0,
"orgId": 1,
"title": "[FIRING:1] TestAlert Grafana ",
"state": "alerting",
"message": "**Firing**\n\nValue: [no value]\nLabels:\n - alertname = TestAlert\n - instance = Grafana\nAnnotations:\n - summary = Notification test\nSilence: http://localhost:3000/prometheus/grafana/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DTestAlert&matcher=instance%3DGrafana\n"
}
4. Important field analysis
Field Name | Description |
---|---|
status | Alert status, firing means triggered, resolved means recovered |
alerts | List of alerts, containing multiple alert events |
labels | Alert labels, such as alertname identifies the alert rule name, instance identifies the triggering instance of the alert rule |
annotations | Additional alert information, such as summary |
startsAt | Alert trigger time |
endsAt | Alert recovery time (shown as 0001-01-01T00:00:00Z when not recovered) |
fingerprint | Unique identifier for the alert |
title | Title of the alert notification |
message | Detailed alert message content |
Taking HAP WebHook as an example
1. Create a worksheet in HAP containing the following fields:
Field Name | Field Type |
---|---|
Rule Name | Text |
Status | Single choice |
Instance | Text |
Summary | Text |
Trigger Time | Date |
Recovery Time | Date |
Unique Identifier | Text |
Detailed Content | Text |
- The status field presets two options: firing, resolved
2. Create a Webhook workflow in HAP, with the following process configuration:
2.1 Generate parameters, by generating from the JSON data example, the request data example can use the message body content of the test message example above
2.2 Add a [Fetch Multiple Data] node, select [Fetch Array Objects]
Configuration options:
- Data type: Send API request array
- Send API request node: Webhook trigger
- Array selection: alerts
2.3 Add a [Add Record] node, select the previously created worksheet, based on multiple records to add records one by one, and choose to acquire the array of objects as the data source.
Field mapping:
Worksheet Field Name | Corresponding Array Field |
---|---|
Rule Name | labels_alertname |
Status | labels_status |
Instance | labels_instance |
Summary | annotations_summary |
Trigger Time | startsAt |
Recovery Time | endsAt |
Unique Identifier | fingerprint |
Detailed Content | messages (Webhook trigger) |
3. In the Ops Platform, add a notification method, select Webhook type, and enter the Webhook URL created in the HAP workflow.
After configuration, click Test --> Send test notification to send a test message, and a new record will be added to the HAP worksheet.