Operator监控Ingress-nginx

一、创建ServiceMonitor

yaml文件如下

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    app: ingress-nginx
  name: nginx-ingress-scraping
  namespace: ingress-nginx
spec:
  endpoints:
  - interval: 30s
    path: /metrics
    port: metrics
  jobLabel: app
  namespaceSelector:
    matchNames:
    - ingress-nginx
  selector:
    matchLabels:
      app: ingress-nginx

创建并查看

[root@master servicemonitor]# kubectl apply -f servicemonitor.yaml
[root@master servicemonitor]# kubectl get servicemonitor -n ingress-nginx
NAME                     AGE
nginx-ingress-scraping   4h18m

二、查看Prometheus日志

level=error ts=2020-12-13T09:52:35.565Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:426: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:monitoring:prometheus-k8s\" cannot list resource \"endpoints\" in API group \"\" in the namespace \"ingress-nginx\""

可以看到是prometheus权限不足 需要修改下prometheus的clusterrole

kubectl edit clusterrole prometheus-k8s

#rules的内容修改为以下参数
rules:
- apiGroups:
  - ""
  resources:
  - nodes
  - services
  - endpoints
  - pods
  - nodes/proxy
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  - nodes/metrics
  verbs:
  - get
- nonResourceURLs:
  - /metrics
  verbs:
  - get

再次查看prometheus上已经正常了

[serviceMonitor/ingress-nginx/nginx-ingress-scraping/0 (2/2 up)]

三、Grafana添加模板

ingress-nginx模板id 9614

查看监控