prometheus export取jvm,json指标

jmx-exporter

当Java应用提供jmx端口时可以在不重启应用的情况下使用jmx_exporterh给Prometheus提供指标

  • 下载jmx_exporter
  wget  https://share.vinkdong.com/_category/ops/java/jmx_exporter_1.0.tar.gz
  • 解压jmx_exporter_1.0.tar.gz
  tar -xzf jmx_exporter_1.0.tar.gz
  cd jmx_exporter/
  • 编辑config.yml,hostPort改为对应的jmx地址,javaapp替换为对应app的名字
  ---
  hostPort: 127.0.0.1:9902
  username: 
  password: 
  lowercaseOutputName: true
  blacklistObjectNames: ["com.alibaba.druid:type=DruidDataSourceStat"]
  whitelistObjectNames: ["java.lang:type=Memory","java.lang:type=ClassLoading","java.lang:type=Threading","java.lang:type=OperatingSystem"]
  rules:
    - pattern: 'java.lang<type=(Memory|ClassLoading|Threading|OperatingSystem)><.*>(\w+):'
      name: framework_jvm_$1
      labels: 
        javaapp: uatosp
        name: $2
  • 编辑run.sh,5557可以改为合适的端口
  #!/usr/bin/env bash
  # Script to run a java application for testing jmx4prometheus.

  # Note: You can use localhost:5556 instead of 5556 for configuring socket hostname.
  # -Djava.util.logging.config.file=logging.properties \
  java \
  -jar jmx_export.jar \
  5557 \
  config.yml
  • 启动服务
  chmod a+x run.sh
  ./run.sh 1>jmx_exporter.log 2>&1 &
  • 查看日志
  tail -f jmx_exporter.log
  • prometheus添加scrape配置
  ...
  - job_name: 'framework-jvm-exporter'
    static_configs:
    - targets: 
      - '10.0.0.2:5557'
    relabel_configs:
    - source_labels: [__address__]
      separator: ;
      regex: (.+)(?::\d+)
      target_label: node_name
      replacement: $1
      action: replace
    - source_labels: []
      separator: ;
      regex: (.*)
      target_label: cluster
      replacement: framework
      action: replace
    - source_labels: []
      separator: ;
      regex: (.*)
      target_label: prometheus
      replacement: uat-api
      action: replace
  ...
  • 重新加载Prometheus配置文件
  curl -XPOST prometheus.wenqi.us/-/reload

json exporter

当已有json格式指标时就可以使用json-expoter

  • 下载json-exporter
  https://share.vinkdong.com/_category/ops/json/json_exporter_0.1.tar.gz
  • 解压
  tar -xvzf json_exporter_0.1.tar.gz
  cd json_exporter
  • 编辑json_exporter_prometheus.sh,替换JAVA_APP为对应APP名称,JAVA_APP_URL为对应json-metrics地址,–port指定合适的端口
  ...
  JAVA_APP=${JAVA_APP:-example}
  JAVA_APP_URL=${JAVA_APP_URL:-'http://127.0.0.1:8012/alert/metrics'}
  if [ !-a config.yml_bk ]; then 
    cp config.yml config.yml_bk
  fi

  sed "s/{{JAVA_APP}}/${JAVA_APP}/" -i config.yml
  ./json_exporter \
  --port 5558 \
  ${JAVA_APP_URL} \
  config.yml
  • 启动服务
   ./json_exporter_prometheus.sh 1>json_exporter.log 2>&1 &
  • 查看日志
  tail -f json_exporter.log
  • 添加到Prometheus中
  - job_name: 'framework-json-exporter'
    static_configs:
    - targets: 
      - '10.0.0.1:5558'
    relabel_configs:
    - source_labels: [__address__]
      separator: ;
      regex: (.+)(?::\d+)
      target_label: node_name
      replacement: $1
      action: replace
    - source_labels: []
      separator: ;
      regex: (.*)
      target_label: cluster
      replacement: framework
      action: replace
    - source_labels: []
      separator: ;
      regex: (.*)
      target_label: prometheus
      replacement: framework_json
      action: replace
  • Prometheus重新加载配置
  curl -XPOST prometheus.wenqi.us/-/reload

grafana导入模板

VinkDong

open to open