利用netdata+grafana进行CAE软件性能分析

2020-10-26

netdata的安装

Netdata是一款Linux性能实时监测工具。

Netdata是一个高度优化的Linux守护进程，它为Linux系统，应用程序，SNMP服务等提供实时的性能监测。

它用可视化的手段，将被监测者最细微的细节，展现了出来。这样，你便可以清晰地了解你的系统和应用程序此时的状况。

本次使用的是针对netdata定制开发了插件，可以监控CPU指令集，Infiniband带宽等信息。

在slave节点的安装

cat netedata.conf
[global]
        run as user = root
        history = 3996
        hostname = gv134
        memory mode = none
[web]
        bind to = *
        web files owner = root
        web files group = root
        mode = none
cat stream.conf

# 1. ON SLAVE NETDATA - THE ONE THAT WILL BE SENDING METRICS

[stream]
    # Enable this on slaves, to have them send metrics.
    enabled = yes
    destination = node228:19999
    # The API_KEY to use (as the sender)
    api key = 833289f0-616b-4412-ac1d-cfce09a381a3

    # The timeout to connect and send metrics
    timeout seconds = 60

    # If the destination line above does not specify a port, use this
    default port = 19999

    # filter the charts to be streamed
    send charts matching = *

    # The buffer to use for sending metrics.
    # 1MB is good for 10-20 seconds of data, so increase this if you expect latencies.
    # The buffer is flushed on reconnects (this will not prevent gaps at the charts).
    buffer size bytes = 1048576

    # If the connection fails, or it disconnects,
    # retry after that many seconds.
    reconnect delay seconds = 5

    # Sync the clock of the charts for that many iterations, when starting.
    initial clock resync iterations = 60

在master节点的安装

[global]
        # process scheduling policy = keep
        # OOM score = keep
        run as user = root
        history = 3996
        glibc malloc arena max for plugins = 1
        glibc malloc arena max for netdata = 1
        hostname = node228
        update every = 1
        config directory = /opt/clusmon/netdata/etc/netdata
        stock config directory = /opt/clusmon/netdata/lib/netdata/conf.d
        log directory = /opt/clusmon/netdata/var/log/netdata
        web files directory = /opt/clusmon/netdata/share/netdata/web
        cache directory = /opt/clusmon/netdata/var/cache/netdata
        lib directory = /opt/clusmon/netdata/var/lib/netdata
        home directory = /opt/clusmon/netdata/var/cache/netdata
        plugins directory = "/opt/clusmon/netdata/libexec/netdata/plugins.d" "/opt/clusmon/netdata/etc/netdata/custom-plugins.d"
        memory mode = save
        host access prefix =
        memory deduplication (ksm) = yes
        TZ environment variable = :/etc/localtime
        timezone = Asia/Shanghai
        debug flags = 0x0000000000000000
        debug log = /opt/clusmon/netdata/var/log/netdata/debug.log
        error log = /opt/clusmon/netdata/var/log/netdata/error.log
access log = /opt/clusmon/netdata/var/log/netdata/access.log
        facility log = daemon
        errors flood protection period = 1200
        errors to trigger flood protection = 200
        pthread stack size = 8388608
        cleanup obsolete charts after seconds = 3600
        gap when lost iterations above = 1
        cleanup orphan hosts after seconds = 3600
        delete obsolete charts files = yes
        delete orphan hosts files = yes
        enable zero metrics = no
[web]
        bind to = *
        web files owner = root
        web files group = root

通过10.0.50.228:19999登录

netdata

influxdb数据库安装

yum install -y influxdb-1.5.4.x86_64.rpm

cat /etc/influxdb/influxdb.conf
[[opentsdb]]
  enabled = true
  bind-address = ":4242"
  database = "clusmon"
  retention-policy = ""
  consistency-level = "one"
  tls-enabled = false
  certificate= "/etc/ssl/influxdb.pem"
  
  cat /opt/clusmon/netdata/etc/netdata/netdata.conf 
  [backend]
        host tags =
        enabled = yes
        data source = average
        type = clusmon
        destination = localhost:4242
        prefix = clusmon
        hostname = node228
        update every = 10
        buffer on failures = 10

安装grafana

1	yum install -y grafana-5.1.3-1.x86_64.rpm

登录grafana 10.0.50.228:3000配置数据库

CAE软件Abaqus_2018性能分析

本次算例为240万网格受均布载荷的悬臂梁：

Beam

采用单节点SMP的方式运行，命令行如下：

1	/public/software/Abaqus_2018/DassaultSystemes/SIMULIA/Commands/abq2018 analysis input=Beam.inp job=Beam cpus=48 scratch=./ standard_parallel=all interactive

前处理阶段：

此时CPU资源消耗较少，使用的指令集为双精度Scalar指令集

scalar

由于不涉及到线性方程组的迭代求解，此时内存占用较小

mem

从图中可以看出在前处理阶段，磁盘整体IO压力不大，没有IO性能瓶颈

disk

分析阶段：

从监控图中可看出，分析阶段使用AVX2指令集进行分析计算，Intel Xeon Gold 6226R AVX2理论双精度浮点计算性能为1.4848TFLOPS，从图中可以看出在峰值计算阶段，实际可以达到1.311TFOPS，使用效率可以达到88.3%，表明abaqus隐式求解器属于计算密集型应用。在计算过程中CPU主频基本维持在3.1GHz左右。但是对于6226R支持的AVX512指令集使用率为0，表明商业软件对先进指令集的开发使用滞后于CPU的发展。

avx2

cpu

由于abaqus线性静力学分析采用隐式求解器进行求解，因此在求解过程中需要进行大量线性方程组的迭代求解，对内存的消耗比较大，本次服务器内存通道满配12*16GB=192GB，从图中可以看出，在计算高峰阶段内存的使用率可以达到145G，内存使用率为76%，表明abaqus隐式求解器为访存密集型求解器。

mem

在io带宽方面发现先行静力学分析存在IO性能平静，本次计算使用6Gb SSD固态硬盘，最大读写速率除去开销在520MB/s左右，从图中可以看出，在求解过程中，磁盘写速率直接达到100%，IO利用率也达到100%，表明abaqus静力学分析对磁盘写入性能要求很高，后续准备换成PCIe U.2接口的SSD固态进行测试写入带宽应该可以达到3~4GB/s。

总结：abaqus standard模块为计算密集型、读写密集型、访存密集型应用，对于CPU、磁盘和内存大小和读写带宽均有较高要求，因此建议服务器配置高性能CPU、采用大内存，存储使用高性能SSD本地磁盘，或者分布式并行存储。