[ Série Monitoração ] - 01 - Prometheus, Netdata, cAdvisor, Grafana, Docker e Slack

SIGA O VÍDEO

Nesse primeiro vídeo da série sobre monitoração, vamos mostrar como criar essa stack completa de monitoração com dashboards e integrações fodas!

Ahhh e você conseguirá monitorar tudo, de containers a hosts físico!

Link para o projeto: https://github.com/badtuxx/giropops-monitoring

Howto

Primeira coisa, vamos clonar o nosso repo:

# git clone https://github.com/badtuxx/giropops-monitoring.git

Instalando o Docker e criando o cluster Swarm

# curl -fsSL https://get.docker.com | sh
# docker swarm init

Instalando netdata:

# bash <(curl -Ss https://my-netdata.io/kickstart.sh)

Ajustando o arquivo de configuração do Prometheus para que ele possa coletar as métricas do Netdata :

# vim conf/prometheus/prometheus.yml
...
- job_name: 'netdata'
    metrics_path: '/api/v1/allmetrics'
    params:
      format: [prometheus]
    honor_labels: true
    scrape_interval: 5s
    static_configs:
         - targets: ['YOUR_IP:19999']

Pegando o incoming webhook do Rocket.Chat

1. Deploy giropops stack, only to get the WebHook

# docker stack deploy -c docker-compose.yml giropops

1. Access YOUR_IP:3080 and create your account

2. Login with your user and go to: Administration => Integrations => New Integration => Incoming WebHook

3. Set "Enabled" and "Script Enabled" to "True"

4. Set all channel, icons, etc. as you need

5. Paste contents of rocketchat/incoming-webhook.js into Script field.

6. Create Integration. You;ll see some values apper. Copy WebHook URL and proceed to "Integration between Rocket.Chat and AlertManager" section.

7. Remove giropops stack

# docker stack rm giropops

Rocket.Chat Docs

Integrando o Rocket.Chat e o AlertManager

# vim conf/alertmanager/config.yml

route:
    repeat_interval: 30m
    group_interval: 30m
    receiver: 'rocketchat'

receivers:
    - name: 'rocketchat'
      webhook_configs:
          - send_resolved: false
            # copy below the WEBHOOK that you create before
            url: '${WEBHOOK_URL}'

realizando o deploy do stack em nosso cluster swarm

Realizando o deploy do nosso compose file:

# docker stack deploy -c docker-compose.yml giropops

Creating network giropops_backend
Creating network giropops_frontend
Creating network giropops_default
Creating service giropops_prometheus
Creating service giropops_node-exporter
Creating service giropops_alertmanager
Creating service giropops_cadvisor
Creating service giropops_grafana
Creating service giropops_rocketchat
Creating service giropops_mongo
Creating service giropops_mongo-init-replica

Verificando se os serviços estão ok:

# docker service ls

ID              NAME                          MODE         REPLICAS  IMAGE                                  PORTS
2j5vievon95j    giropops_alertmanager         replicated   1/1       linuxtips/alertmanager_alpine:latest   *:9093->9093/tcp
y1kinszpqzpg    giropops_cadvisor             global       1/1       google/cadvisor:latest                 *:8080->8080/tcp
jol20u8pahlp    giropops_grafana              replicated   1/1       grafana/grafana:latest                 *:3000->3000/tcp
t3635s4xh5cp    giropops_mongo                replicated   1/1       mongo:3.2
t8vnb7xuyfa8    giropops_mongo-init-replica   replicated   0/1       mongo:3.2
usr0jy4jquns    giropops_node-exporter        global       1/1       linuxtips/node-exporter_alpine:latest  *:9100->9100/tcp
zc3qza0bxys7    giropops_prometheus           replicated   1/1       linuxtips/prometheus_alpine:latest     *:9090->9090/tcp
7bgnm0poxbwj    giropops_rocketchat           replicated   1/1       rocketchat/rocket.chat:latest          *:3080->3080/tcp

Acessando nossos serviços através do browser:

http://YOUR_IP:9090

Para acessar o AlertManager:

http://YOUR_IP:9093

Para acessar o Grafana:

http://YOUR_IP:3000
user: admin
passwd: giropops

To add plugs edit file giropops-monitoring/grafana.config
GF_INSTALL_PLUGINS=plug1,plug2
Current plugs grafana-clock-panel,grafana-piechart-panel,camptocamp-prometheus-alertmanager-datasource,vonage-status-panel

Para acessar o Netdata:

http://YOUR_IP:19999

Para acessar as métricas do node_exporter:

http://YOUR_IP:9100/metrics

Para acessar o Rocket.Chat:

http://YOUR_IP:3080
> First to register becomes admin

Vamos derrubar um service para testar se nosso alerta configurado está funcionando:

# docker service rm giropops_node-exporter

Wait some seconds and you will see the integration works fine! Prometheus alerting the AlertManager that alert the Slack that shows it to you! It's so easy and that simple! :D

E claro, crie novos alertar no arquivo:

# vim conf/prometheus/alert.rules

Compartilhem em suas redes sociais! Curta o post aqui no blog e deixe seu comentário!

#VAIIII