How to configure Prometheus Alertmanager to send alerts to Telegram and Slack with Docker Compose — a practical step‑by‑step guide

Deploy & DevOps · By · Published on · 9 min read

How to configure Prometheus Alertmanager with Docker Compose to send alerts to Telegram and Slack
Photo: Egor Komarov / Pexels

In this tutorial you'll learn how to configure Prometheus Alertmanager to send alerts to Telegram and Slack using Docker Compose. You will create a Telegram bot, set up a Slack webhook, write the configuration files, bring the stack up with Docker Compose and validate the whole flow with test alerts in a safe lab environment.

Prerequisites and flow architecture

You'll run a minimal stack: Prometheus, Alertmanager and notification targets for Telegram and Slack. Prometheus evaluates rules and sends events to Alertmanager, which groups and routes them by severity.

If you don't have Prometheus and Grafana yet, see the guide to bring up the basic stack: Guide: Prometheus & Grafana with Docker Compose on a VPS.

  • Docker and Docker Compose installed. You need permission to use Docker.
  • A Telegram bot with its token and the chat_id of the channel/group or direct chat.
  • A Slack Incoming Webhook (or app/bot with permissions to post to a channel).
  • Text editor and local ports 9090 (Prometheus) and 9093 (Alertmanager) available.

Create the Telegram bot and get the chat_id

Create a bot by talking to @BotFather and note the token. Add the bot to the group/channel and send any message to initiate the conversation. Then use the API to find the chat_id and test sending messages.

Never share your token. Keep it out of your Git repository.

# 1) Verifique o bot
curl -s "https://api.telegram.org/bot<TELEGRAM_BOT_TOKEN>/getMe" | jq .

# 2) Descubra o chat_id: envie uma mensagem ao bot e depois rode
curl -s "https://api.telegram.org/bot<TELEGRAM_BOT_TOKEN>/getUpdates" | jq .
# Procure por result[].message.chat.id

# 3) Teste o envio manual
curl -s -X POST \
  "https://api.telegram.org/bot<TELEGRAM_BOT_TOKEN>/sendMessage" \
  -d chat_id=<CHAT_ID> \
  -d text="Teste de alerta via Telegram"

Create the Slack webhook/app and test delivery

In Slack, create an Incoming Webhook (or an app with chat:write permission) and copy the webhook URL. Choose the channel that will receive alerts.

Validate delivery with a simple POST. Do not expose the webhook URL publicly.

# Substitua pela sua URL de webhook
SLACK_WEBHOOK="https://hooks.slack.com/services/XXX/YYY/ZZZ"

# Teste básico de mensagem
curl -X POST -H 'Content-type: application/json' \
  --data '{"text":"Teste de alerta via Slack (webhook)"}' \
  "$SLACK_WEBHOOK"

Write the monitoring stack docker-compose

Create a project directory containing prometheus/ and alertmanager/ folders. We'll mount the configuration files via volumes. In the lab, ports 9090 and 9093 are exposed locally.

In production, restrict access with a firewall and require authentication on a reverse proxy. Also check the VPS Linux security checklist.

Service Internal port External port Role
prometheus 9090 9090 Metrics scraping and rule evaluation
alertmanager 9093 9093 Routing and notification delivery 
version: "3.8"
services:
  prometheus:
    image: prom/prometheus:v2.52.0
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --web.enable-lifecycle
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - ./prometheus/alerts.yml:/etc/prometheus/alerts.yml:ro
    ports:
      - "9090:9090"
    networks: [monitoring]

  alertmanager:
    image: prom/alertmanager:v0.27.0
    command:
      - --config.file=/etc/alertmanager/alertmanager.yml
      - --web.route-prefix=/
      - --web.listen-address=:9093
    volumes:
      - ./alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro
      - alertmanager-data:/alertmanager
    ports:
      - "9093:9093"
    networks: [monitoring]

networks:
  monitoring: {}

volumes:
  alertmanager-data: {}

Prometheus: scrape and alerting rules (alerts.yml)

Prometheus must know where the Alertmanager is and load a rules file. The InstanceDown rule detects targets that are down. The Watchdog always fires and is useful to validate the notification pipeline.

Keep Watchdog only for tests to avoid continuous noise in production.

# arquivo: prometheus/prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

alerting:
  alertmanagers:
    - static_configs:
        - targets: ["alertmanager:9093"]

rule_files:
  - /etc/prometheus/alerts.yml

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ["prometheus:9090"]

---
# arquivo: prometheus/alerts.yml
groups:
  - name: basic
    rules:
      - alert: InstanceDown
        expr: up == 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Alvo indisponível"
          description: "O alvo {{ $labels.instance }} está inativo por mais de 1 minuto."

      - alert: Watchdog
        expr: vector(1)
        labels:
          severity: warning
        annotations:
          summary: "Watchdog: canal de alertas ativo"
          description: "Alerta contínuo para verificar o pipeline. Desabilite em produção."

Alertmanager: routing and messages for Telegram and Slack

Define routing by severity: warnings go to Telegram; critical alerts go to Slack. Adjust this to match your teams, schedules and escalation policies.

Tokens and webhooks are stored in the file below. Do not commit that file to Git. Keep secrets in a vault or outside the repository.

# arquivo: alertmanager/alertmanager.yml
global: {}

route:
  receiver: default
  group_by: ["alertname", "severity"]
  group_wait: 10s
  group_interval: 2m
  repeat_interval: 4h
  routes:
    - matchers:
        - severity = "critical"
      receiver: slack
    - matchers:
        - severity = "warning"
      receiver: telegram

receivers:
  - name: default
    slack_configs:
      - api_url: "https://hooks.slack.com/services/REPLACE_WITH_YOUR_WEBHOOK"
        channel: "#alerts"
        send_resolved: true
        title: "[{{ .Status | toUpper }}] {{ .CommonLabels.alertname }}"
        text: >-
          {{ range .Alerts -}}
          • {{ .Annotations.summary }} ({{ .Labels.instance }})
          {{ end }}

  - name: telegram
    telegram_configs:
      - bot_token: "REPLACE_WITH_YOUR_TELEGRAM_BOT_TOKEN"
        chat_id: REPLACE_WITH_YOUR_CHAT_ID
        parse_mode: "Markdown"
        disable_web_page_preview: true
        send_resolved: true
        message: >-
          [{{ .Status | toUpper }}] {{ .CommonLabels.alertname }}\n
          {{ range .Alerts -}}
          • {{ .Annotations.summary }} ({{ .Labels.instance }})
          {{ end }}

  - name: slack
    slack_configs:
      - api_url: "https://hooks.slack.com/services/REPLACE_WITH_YOUR_WEBHOOK"
        channel: "#alerts"
        send_resolved: true
        title: "[{{ .Status | toUpper }}] {{ .CommonLabels.alertname }}"
        text: >-
          {{ range .Alerts -}}
          • {{ .Annotations.summary }} ({{ .Labels.instance }})
          {{ end }}

Validate syntax with promtool and amtool before reload

Validate files to avoid reload errors. Use promtool for Prometheus rules and amtool for Alertmanager configuration.

Run these commands from the project root where the prometheus/ and alertmanager/ folders live.

# Checar regras do Prometheus
docker run --rm -v "$PWD/prometheus":/etc/prometheus \
  prom/prometheus:v2.52.0 promtool check rules /etc/prometheus/alerts.yml

# Checar config do Alertmanager
docker run --rm -v "$PWD/alertmanager":/etc/alertmanager \
  prom/alertmanager:v0.27.0 amtool check-config /etc/alertmanager/alertmanager.yml

Bring up containers and validate the alert pipeline

Now start the stack, reload Prometheus and Alertmanager and generate a test alert. The Watchdog should appear in Alertmanager and reach Telegram within a few seconds.

To exercise Slack, temporarily change the Watchdog severity to critical and reload the configurations.

# Subir o stack
docker compose up -d

# Verifique os serviços
docker compose ps

# Recarregar Prometheus (habilitado por --web.enable-lifecycle)
curl -X POST http://localhost:9090/-/reload

# Recarregar Alertmanager via SIGHUP
docker compose kill -s HUP alertmanager

# Acesse as UIs:
# Prometheus:   http://localhost:9090
# Alertmanager: http://localhost:9093

Verification, fine tuning and troubleshooting

Confirm in Alertmanager that Watchdog is active and routed. Check delivery history in Telegram and Slack. Remove or silence Watchdog after the test.

Common issues and quick fixes:

  • Use Alertmanager silences for maintenance windows.
  • Adjust group_wait and group_interval to reduce noise.
  • Standardize labels like severity, service and environment in your rules.
  • Separate channels per team (e.g. on-call in Slack; NOC in Telegram).

Common issues and fixes (continued)

- No Telegram messages: start a conversation with the bot or use the correct group chat_id.

- No Slack deliveries: invalid webhook, missing app permissions or a private channel.

- Reload has no effect: YAML syntax error. Check logs and indentation.

- Ports already in use: change the 9090/9093 mappings in compose.

- Exposed tokens: move sensitive configs outside Git and restrict directory permissions.

To collect metrics from your PHP application, integrate a specific exporter: Expose PHP-FPM metrics to Prometheus. For general host security, see the VPS Linux security checklist. Want an observability comparison? See Prometheus & Grafana vs Loki & Tempo. And if you want to notify security incidents to Telegram, check the fail2ban webhooks tutorial.

Frequently asked questions

Can I use only Telegram or only Slack?

Yes. Keep a single receiver and remove the other from alertmanager.yml. In the route, direct all severities to the chosen receiver. Prometheus stays the same: it sends events to Alertmanager, which handles routing.

How do I protect tokens and webhooks from being committed to Git?

Keep alertmanager.yml outside the repository or use a private repo. Restrict directory permissions (chmod 600). In larger environments, store secrets in a dedicated vault and generate the configuration file at deploy time.

Why doesn't the InstanceDown rule fire in the example?

Because the only target is Prometheus itself, which is healthy. To test real failures, add nonexistent targets or temporarily stop a monitored service. Use Watchdog only to validate the delivery path.

How do I change the message format?

Edit the message field (Telegram) and title/text fields (Slack) in alertmanager.yml. You can include alert labels and annotations. Keep messages short and contextual: severity, alert name, instance and a concise summary.

Can I silence alerts by schedule or by environment?

Yes. Use matchers with labels like environment and service in Alertmanager routes. For scheduled windows, create silences via the UI or automation. Also label alerts by environment in your rules to separate DEV, STG and PROD.

How to integrate services beyond Telegram and Slack?

Alertmanager supports many receivers (email, webhooks, PagerDuty, etc.). Add receivers to alertmanager.yml and create routes by severity, service or team. Test each integration in isolation before enabling in production.

Ready to take your alerts seriously?

Use this template as a base, standardize labels and routes, and integrate your services. Need help adapting this for production with high availability and less noise? Let's talk.

Talk about observability

See also