Skip to main content

Based on Kubernetes

Deploy Node Exporter​

When the Ops Platform is deployed within a Kubernetes cluster, the ops-nodeagent container includes the Node Exporter service and runs as a DaemonSet resource on each node in the Kubernetes cluster.

To monitor servers outside the Kubernetes cluster, Node Exporter must be deployed on the external servers to collect monitoring data metrics.

Ops Platform Deployment​

Download Images (Offline Package Download)​

crictl pull registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-gateway:1.1.0
crictl pull registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-prometheus:1.1.0
crictl pull registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-agent:1.1.0
crictl pull registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-nodeagent:1.0.0
  • By default, the ops-gateway, ops-prometheus, and ops-agent images only need to be downloaded on the nodes where the Ops Platform is deployed, while the ops-nodeagent image needs to be downloaded on every node within the Kubernetes cluster.

Create Node Labels​

By default, the Ops Platform is deployed on a fixed node using nodeSelector, and monitoring data from the ops-prometheus service is stored persistently on local disks using the hostPath method.

To create a hap-ops=true label for the node where the Ops Platform services will be deployed:

kubectl label node nodeName hap-ops=true
  • Replace nodeName with the name of the node where the Ops Platform services will be deployed. You can view the node name via kubectl get node -o wide.

Create Ops Platform Service Yaml File​

Create and navigate to the directory where the Yaml files will be stored:

mkdir -p /data/mingdao/script/kubernetes/ops/
cd /data/mingdao/script/kubernetes/ops/

Create the ops.yaml file:

cat > ops.yaml << 'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
  name: ops-config
  namespace: hap-ops
data:
  TZ: "Asia/Shanghai"
  ENV_OPS_TOKEN: "SS9PobGG7SDTpcyfSZ1VVmn3gCmy2P52tYk"          # Adjust for the first deployment; this environment variable is the access token for the Ops Platform
  ENV_PROMETHEUS_HOST: "service_01/192.168.1.5:59100,service_02/192.168.1.6:59100,service_03/192.168.1.7:59100" # Replace with actual Node Exporter service addresses during deployment
  ENV_PROMETHEUS_SERVER: "http://ops-prometheus:9090"
  ENV_PROMETHEUS_GRAFANA: "http://ops-prometheus:3000"
  ENV_PROMETHEUS_ALERT: "http://ops-prometheus:9093"
  ENV_PROMETHEUS_KARMA: "http://ops-prometheus:8080"
  ENV_PROMETHEUS_KAFKA: "kafka_1/ops-agent:9308"
  ENV_PROMETHEUS_ELASTICSEARCH: "elasticsearch_1/ops-agent:9114"
  ENV_PROMETHEUS_REDIS: "redis_1/ops-agent:9121"
  ENV_PROMETHEUS_MONGODB: "mongodb_1/ops-agent:9216"
  ENV_PROMETHEUS_MYSQL: "mysql_1/ops-agent:9104"
  # Below are connection details for storage components. Modify the environment variable values according to the actual deployment environment.
  ENV_MYSQL_HOST: "192.168.1.11"
  ENV_MYSQL_PORT: "3306"
  ENV_MYSQL_USERNAME: "root"
  ENV_MYSQL_PASSWORD: "changeme"
  ENV_MONGODB_URI: "mongodb://root:changeme@192.168.1.12:27017"
  ENV_MONGODB_OPTIONS: "?authSource=admin"
  ENV_REDIS_HOST: "192.168.1.13"
  ENV_REDIS_PORT: "6379"
  ENV_REDIS_PASSWORD: "changeme"
  ENV_KAFKA_ENDPOINTS: "192.168.1.14:9092"
  ENV_ELASTICSEARCH_ENDPOINTS: "http://192.168.1.15:9200"
  ENV_ELASTICSEARCH_PASSWORD: "elastic:changeme"
  ENV_FLINK_URL: "http://flink-jobmanager.flink:8081"

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ops-gateway
  namespace: hap-ops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ops-gateway
  template:
    metadata:
      labels:
        app: ops-gateway
    spec:
      nodeSelector:
        hap-ops: "true"
      containers:
        - name: ops-gateway
          image: registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-gateway:1.1.0
          envFrom:
            - configMapRef:
                name: ops-config
          resources:
            limits:
              cpu: "2"
              memory: "4Gi"
            requests:
              cpu: "0.1"
              memory: "200Mi"

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ops-prometheus
  namespace: hap-ops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ops-prometheus
  template:
    metadata:
      labels:
        app: ops-prometheus
    spec:
      nodeSelector:
        hap-ops: "true"
      containers:
        - name: ops-prometheus
          image: registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-prometheus:1.1.0
          volumeMounts:
            - mountPath: /data/
              name: prometheus-data
          envFrom:
            - configMapRef:
                name: ops-config
          resources:
            limits:
              cpu: "2"
              memory: "4Gi"
            requests:
              cpu: "0.1"
              memory: "200Mi"
      volumes:
        - name: prometheus-data
          hostPath:
            path: /data/ops-prometheus-data  # Path for persistent storage
            type: DirectoryOrCreate      # Create the directory if it doesn't exist

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ops-agent
  namespace: hap-ops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ops-agent
  template:
    metadata:
      labels:
        app: ops-agent
    spec:
      nodeSelector:
        hap-ops: "true"
      containers:
        - name: ops-agent
          image: registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-agent:1.1.0
          envFrom:
            - configMapRef:
                name: ops-config
          resources:
            limits:
              cpu: "2"
              memory: "4Gi"
            requests:
              cpu: "0.1"
              memory: "200Mi"

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: ops-nodeagent
  namespace: hap-ops
spec:
  selector:
    matchLabels:
      app: ops-nodeagent
  template:
    metadata:
      labels:
        app: ops-nodeagent
    spec:
      containers:
        - name: ops-nodeagent
          image: registry.cn-hangzhou.aliyuncs.com/mdpublic/ops-nodeagent:1.0.0
          envFrom:
            - configMapRef:
                name: ops-config
          ports:
            - containerPort: 59100
          resources:
            limits:
              cpu: "2"
              memory: "4Gi"
            requests:
              cpu: "0.1"
              memory: "200Mi"
          volumeMounts:
            - name: host-root
              mountPath: /host
              readOnly: true
              mountPropagation: HostToContainer
      volumes:
        - name: host-root
          hostPath:
            path: /
      hostNetwork: true             # Use host network
      hostPID: true                 # Use host PID namespace   

---

apiVersion: v1
kind: Service
metadata:
  name: ops-prometheus
  namespace: hap-ops
spec:
  selector:
    app: ops-prometheus
  ports:
    - name: server
      port: 9090
      targetPort: 9090
    - name: grafana
      port: 3000
      targetPort: 3000
  type: ClusterIP

---

apiVersion: v1
kind: Service
metadata:
  name: ops-agent
  namespace: hap-ops
spec:
  selector:
    app: ops-agent
  ports:
    - name: prometheus
      port: 9104
      targetPort: 9104
    - name: mongodb
      port: 9216
      targetPort: 9216
    - name: redis
      port: 9121
      targetPort: 9121
    - name: kafka
      port: 9308
      targetPort: 9308
    - name: elasticsearch
      port: 9114
      targetPort: 9114
  type: ClusterIP

---

apiVersion: v1
kind: Service
metadata:
  name: ops-gateway
  namespace: hap-ops
spec:
  selector:
    app: ops-gateway
  ports:
    - name: gateway
      port: 48881
      targetPort: 48881
      nodePort: 30081
  type: NodePort
EOF

Create Namespace​

kubectl create ns hap-ops
  • Default deployment namespace is hap-ops

Launch Ops Platform Services​

kubectl apply -f ops.yaml
  • Stop command: kubectl delete -f ops.yaml

Check Ops Platform Service Status​

kubectl -n hap-ops get pod -o wide
  • Normally, the READY column should show 1/1 status

Configure Nginx Reverse Proxy​

cat > hap-ops.conf << 'EOF'
upstream hap-ops {
    server 172.29.202.34:30081;    # Replace with the IP of the K8S node where the Ops Platform is deployed
}

server {
    listen 48881;
    server_name _;
    access_log /data/logs/weblogs/hap-ops.log main;
    error_log /data/logs/weblogs/hap-ops.error.log;

    underscores_in_headers on;
    client_max_body_size 2048m;
    gzip  on;
    gzip_proxied any;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_min_length 512;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_types text/plain text/css application/json application/x-javascript application/javascript application/octet-stream text/xml application/xml application/xml+rss text/javascript image/jpeg image/gif image/png;

    location / {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_pass http://hap-ops;
   }
}
EOF
  • It is recommended to use port 48881 for the access entry, keeping it consistent with the backend fixed port of the Ops Platform.

Access the Ops Platform​

Using the above Nginx proxy as an example, access the Nginx entry point:

http://hap-ops.demo.com:48881
  • The login token is the value of the ENV_OPS_TOKEN environment variable in ops.yaml