JMeter - Perfmon - ServerAgent

Installation

    • Create a directory on the server
    • Copy the archive on it
    • Unpack it
    • Verify the java version (minimum 1.4.*)
    • And start the agent ./startAgent.sh
[[email protected] JMeterServerAgent]$ ./startAgent.sh
INFO    2013-04-17 15:34:57.469 [kg.apc.p] (): Binding UDP to 4444
INFO    2013-04-17 15:34:58.468 [kg.apc.p] (): Binding TCP to 4444
INFO    2013-04-17 15:34:58.471 [kg.apc.p] (): [email protected] Agent v2.2.0 started

Usage and commands

Test it with telnet:

Using username "oracle".
Last login: Wed Apr 17 15:32:18 2013 from 192.168.7.128
[[email protected] ~]$ telnet localhost 4444
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
test
Yep
exit
Connection closed by foreign host.

Possible commands are:

  • exit - terminates current client session and closes connection to agent, no parameters
  • test - test if server is alive, no parameters
  • shutdown - terminate all client connections and shutdown agent process, no parameters
  • interval - change metrics reporting interval used in 'metrics' command, single parameter is integer value in seconds. Interval can be changed in the middle of metrics reporting. Example: interval:5
  • metrics - starts automatic metrics collection, parameters are metrics list to collect, described below. Example: metrics:cpu
  • metrics-single - calls single metric collection iteration.
metrics-single:memory
82.40174605900177
metrics-single:cpu      memory
1.743090990034588       82.40096038578609

cpu and memory are separated with a tab.

Each metric specification consists of several fields, colon-separated. Short example:

metrics-single:cpu:idle memory:free
98.33814099619674       4.8909893632E10

Fields number is metric-type specific. Possible metric types are:

  • cpu
  • memory
  • swap
  • disks
  • network
  • tcp
  • tail
  • exec
  • jmx

exec: ServerAgent can be shell exec vulnerability.

metrics-single:exec:/bin/sh:-c:free | grep Mem | awk '{print $7}'
1152488

PerfMon Metrics Collector Listener

Version 0.5.0 ships new ServerAgent which provide over 75 separate metrics, support per-process CPU and Memory metrics and even custom metrics for measuring whatever you want: file sizes, database row counts, Java heap sizes and garbage collections.

It reads jppm files (JPPM=JMeter Plugins Peformance Monitoring). In command line, you can specify “-Jjppmfile=/path/to/file.jppm”. If nothing is specified, a file is created in the running path of JMeter, named perfmon_yyyyMMdd-HHmmss.jppm.

Metric Parameter

PerfMon Metrics Collector has special “Metric Parameter” column, where user can specify metric subtype to collect, specify which process should be monitored (which filesystem, network interface). Metric parameter string may have several parameters inside it, separated with colon :. To include colon as a char inside parameter, use backslash escaping \:. Make note you cannot use tab characters inside metric parameter string, all tabs will be converted to spaces silently.

Selector / Filter parameter

  • name, pid and ptql for processes
  • core for monitoring specific CPU of multicore systems
  • fs for filesystems
  • iface for network interfaces
Metric to collect Parameter Description
CPU combined measure total CPU usage, equals to 100-idle value
CPU core=2:user measure user process CPU usage for third core in system (core numbering starts at 0)
CPU name=java#2:user will monitor second java process instance for user time spent
CPU pid=14523:percent will monitor process with PID 14523 for total CPU usage percentage
CPU name=httpd omitting metric type will use default 'percent'
Disk IO fs=/home:writes will monitor /home filesystem for number of write operations
Network IO iface=eth0:tx will monitor interface eth0 for transmitted packet rate

Server Agent has special command-line option –sysinfo for printing available processes, filesystems and network interfaces together with their selectors.

Metrics

CPU

Generally it is never possible for user processes to consume ALL of the CPU because as its load increases so does the load from other types of CPU usage.

How a particular server uses its CPU depends on a number of factors but in general the real metric we use to determine CPU usage is the idle time.

This is the metric that alerts are based from. The idle value is the amount of CPU left not doing any work. If CPU has no idle time it is 100% in use - how that is allocated depends on the server architecture and the application. In some cases it might be all User, System, and Steal. If it is doing a massive database backup it might all be in wait. It just depends.

The cpu metrics are a combination of many CPU values and the “total” of those will add up to 100%.

Scope Type CPU Metrics Description
Total Primary combined measure total CPU usage, equals to 100-idle value
Total Primary idle CPU not doing any work
Total Primary system CPU used by the operating system (kernel) itself and its associated processes. (things like RAID rebuilding, and handling network transmission and checksums fall into this category as well)
Total Primary user Time the CPU has spent running users'processes. CPU used by user applications
Total Primary iowait (of wait) CPU waiting for disk IO operations to complete. Time that the CPU is idle and there is at least one input or output operation in progress
Total Additional irq interrupt request - CPU allocated to hardware interrupts
Total Additional nice CPU used to allocate multiple processes demanding more cycles than the CPU can provide
Total Additional softirq CPU servicing soft interrupts
Total Additional stolen
IO

The nature for disk read/write metrics is that OS have them counted in a deferred manner. So you may have no IO figures for some time, then all accumulated operations data. Usually this is seen as spikes at IO graphs. On linux there is some OS options in sysctl to change this behavior.

Documentation / Reference

Support

liblibsigar-x86_64-linux.so: cannot open shared object file

1 [main] DEBUG Sigar  - liblibsigar-x86_64-linux.so: liblibsigar-x86_64-linux.so: cannot open shared object file: No such file or directory
org.hyperic.sigar.SigarException: liblibsigar-x86_64-linux.so: liblibsigar-x86_64-linux.so: cannot open shared object file: No such file or directory
   at org.hyperic.sigar.Sigar.loadLibrary(Sigar.java:172)
   at org.hyperic.sigar.Sigar.<clinit>(Sigar.java:100)
   at java.lang.Class.initializeClass(libgcj.so.7rh)
   at kg.apc.perfmon.PerfMonWorker.<init>(PerfMonWorker.java:54)
   at kg.apc.perfmon.AgentTool.getWorker(AgentTool.java:90)
   at kg.apc.perfmon.AgentTool.processParams(AgentTool.java:23)
   at kg.apc.cmdtools.PluginsCMD.processParams(PluginsCMD.java:63)
   at kg.apc.cmdtools.PluginsCMD.processParams(PluginsCMD.java:23)
   at java.lang.reflect.Method.invoke(libgcj.so.7rh)
   at kg.apc.cmd.UniversalRunner.main(UniversalRunner.java:175)

CPU Architecture Information:

[[email protected] ~]$ uname -p
x86_64

Java (Via JConsole)

Architecture: 
amd64

Java Version

[[email protected] ~]$ java -version
java version "1.4.2"
gij (GNU libgcj) version 4.1.2 20080704 (Red Hat 4.1.2-48)

Solution:

  • copy the file liblibsigar-amd64-linux.so into liblibsigar-x86_64-linux.so

Powered by ComboStrap