In our last article, we have learned about installing and upgrading the **sysstat** package and understanding briefly about the utilities which comes with the package.
注:此文一并附上,在同一个原文更新中
- [Sysstat – Performance and Usage Activity Monitoring Tool For Linux][1]
![20 Sysstat Commands for Linux Monitoring](http://www.tecmint.com/wp-content/uploads/2014/09/sysstat-commands.png)
20 Sysstat Commands for Linux Monitoring
Today, we are going to work with some interesting practical examples of **mpstat, pidstat, iostat** and **sar** utilities, which can help us to identify the issues. We have different options to use these utilities, I mean you can fire the commands manually with different options for different kind of work or you can create your customized scripts according to your requirements. You know Sysadmins are always bit Lazy, and always tried to find out the easy way to do the things with minimum efforts.
### mpstat – Processors Statistics ###
1.Using mpstat command without any option, will display the Global Average Activities by All CPUs.
tecmint@tecmint ~ $ mpstat
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
12:23:57 IST CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
12:23:57 IST all 37.35 0.01 4.72 2.96 0.00 0.07 0.00 0.00 0.00 54.88
2.Using mpstat with option ‘**-P**‘ (Indicate Processor Number) and ‘ALL’, will display statistics about all CPUs one by one starting from 0. 0 will the first one.
tecmint@tecmint ~ $ mpstat -P ALL
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
12:29:26 IST CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
12:29:26 IST all 37.33 0.01 4.57 2.58 0.00 0.07 0.00 0.00 0.00 55.44
### pidstat – Process and Kernel Threads Statistics ###
This is used for process monitoring and current threads, which are being managed by kernel. pidstat can also check the status about child processes and threads.
#### Syntax ####
# pidstat <OPTIONS> [INTERVAL] [COUNT]
6.Using pidstat command without any argument, will display all active tasks.
tecmint@tecmint ~ $ pidstat
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
12:47:24 IST UID PID %usr %system %guest %CPU CPU Command
12:47:24 IST 0 1 0.01 0.12 0.00 0.13 1 init
12:47:24 IST 0 3 0.00 0.01 0.00 0.01 0 ksoftirqd/0
12:47:24 IST 0 9 0.00 0.04 0.00 0.04 0 rcu_sched
12:47:24 IST 0 10 0.00 0.00 0.00 0.00 0 watchdog/0
12:47:24 IST 0 11 0.00 0.00 0.00 0.00 1 watchdog/1
12:47:24 IST 0 12 0.00 0.00 0.00 0.00 1 migration/1
12:47:24 IST 0 13 0.00 0.01 0.00 0.01 1 ksoftirqd/1
12:47:24 IST 0 23 0.00 0.00 0.00 0.00 0 kworker/u9:0
12:47:24 IST 0 29 0.00 0.61 0.00 0.61 0 kworker/0:1
12:47:24 IST 0 30 0.00 0.06 0.00 0.06 1 kworker/1:1
12:47:24 IST 0 224 0.00 0.01 0.00 0.01 1 jbd2/sda1-8
12:47:24 IST 0 360 0.00 0.00 0.00 0.00 1 upstart-udev-br
12:47:24 IST 0 365 0.01 0.00 0.00 0.01 0 systemd-udevd
12:47:24 IST 0 476 0.00 0.00 0.00 0.00 0 kworker/u9:1
7.To print all active and non-active tasks use the option ‘**-p**‘ (processes).
tecmint@tecmint ~ $ pidstat -p ALL
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
12:51:55 IST UID PID %usr %system %guest %CPU CPU Command
12:51:55 IST 0 1 0.01 0.11 0.00 0.12 1 init
12:51:55 IST 0 2 0.00 0.00 0.00 0.00 0 kthreadd
12:51:55 IST 0 3 0.00 0.01 0.00 0.01 0 ksoftirqd/0
12:51:55 IST 0 5 0.00 0.00 0.00 0.00 0 kworker/0:0H
12:51:55 IST 0 7 0.00 0.00 0.00 0.00 0 migration/0
12:51:55 IST 0 8 0.00 0.00 0.00 0.00 0 rcu_bh
12:51:55 IST 0 9 0.00 0.04 0.00 0.04 1 rcu_sched
12:51:55 IST 0 10 0.00 0.00 0.00 0.00 0 watchdog/0
12:51:55 IST 0 11 0.00 0.00 0.00 0.00 1 watchdog/1
12:51:55 IST 0 12 0.00 0.00 0.00 0.00 1 migration/1
12:51:55 IST 0 13 0.00 0.01 0.00 0.01 1 ksoftirqd/1
12:51:55 IST 0 15 0.00 0.00 0.00 0.00 1 kworker/1:0H
12:51:55 IST 0 16 0.00 0.00 0.00 0.00 1 khelper
12:51:55 IST 0 17 0.00 0.00 0.00 0.00 0 kdevtmpfs
12:51:55 IST 0 18 0.00 0.00 0.00 0.00 0 netns
12:51:55 IST 0 19 0.00 0.00 0.00 0.00 0 writeback
12:51:55 IST 0 20 0.00 0.00 0.00 0.00 1 kintegrityd
8.Using pidstat command with ‘**-d 2**‘ option, we can get I/O statistics and 2 is interval in seconds to get refreshed statistics. This option can be handy in situation, where your system is undergoing heavy I/O and you want to get clues about the processes consuming high resources.
tecmint@tecmint ~ $ pidstat -d 2
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
9.To know the cpu statistics along with all threads about the process id **4164** at interval of **2** sec for **3** times use the following command with option ‘-t‘ (display statistics of selected process).
tecmint@tecmint ~ $ pidstat -t -p 4164 2 3
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
01:09:06 IST UID TGID TID %usr %system %guest %CPU CPU Command
10.Use the ‘**-rh**‘ option, to know the about memory utilization of processes which are frequently varying their utilization in **2** second interval.
tecmint@tecmint ~ $ pidstat -rh 2 3
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
# Time UID PID minflt/s majflt/s VSZ RSS %MEM Command
12.To get realtime priority and scheduling information use option ‘**-R**‘ .
tecmint@tecmint ~ $ pidstat -R
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
01:09:06 IST UID PID prio policy Command
01:09:08 IST 1000 3 99 FIFO migration/0
01:09:08 IST 1000 5 99 FIFO migration/0
01:09:08 IST 1000 6 99 FIFO watchdog/0
Here, I am not going to cover about Iostat utility, as we are already covered it. Please have a look on “[Linux Performance Monitoring with Vmstat and Iostat][2]注:此文也一并附上在同一个原文更新中” to get all details about iostat.
### sar – System Activity Reporter ###
Using “**sar**” command, we can get the reports about whole system’s performance. This can help us to locate the system bottleneck and provide the help to find out the solutions to these annoying performance issues.
The Linux Kernel maintains some counter internally, which keeps track of all requests, their completion time and I/O block counts etc. From all these information, sar calculates rates and ratio of these request to find out about bottleneck areas.
The main thing about the sar is that, it reports all activities over a period if time. So, make sure that sar collect data on appropriate time (not on Lunch time or on weekend.:)
13.Following is a basic command to invoke sar. It will create one file named “**sarfile**” in your current directory. The options ‘**-u**‘ is for CPU details and will collect **5** reports at an interval of **2** seconds.
tecmint@tecmint ~ $ sar -u -o sarfile 2 5
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
01:42:28 IST CPU %user %nice %system %iowait %steal %idle
01:42:30 IST all 36.52 0.00 3.02 0.00 0.00 60.45
01:42:32 IST all 43.32 0.00 5.04 0.00 0.00 51.64
01:42:34 IST all 56.46 0.00 4.05 0.00 0.00 39.49
01:42:36 IST all 44.44 0.00 3.79 0.00 0.00 51.77
01:42:38 IST all 50.75 0.00 3.75 0.00 0.00 45.50
Average: all 46.30 0.00 3.93 0.00 0.00 49.77
14.In the above example, we have invoked sar interactively. We also have an option to invoke it non-interactively via cron using scripts **/usr/local/lib/sa1** and **/usr/local/lib/sa2** (If you have used **/usr/local** as prefix during installation time).
- **/usr/local/lib/sa1** is a shell script that we can use for scheduling cron which will create daily binary log file.
- **/usr/local/lib/sa2** is a shell script will change binary log file to human-readable form.
Use the following Cron entries for making this non-interactive:
# Run sa1 shell script every 10 minutes for collecting data
*/2 ** * * /usr/local/lib/sa/sa1 2 10
# Generate a daily report in human readable format at 23:53
53 23 * * * /usr/local/lib/sa/sa2 -A
At the back-end sa1 script will call **sadc** (System Activity Data Collector) utility for fetching the data at a particular interval. **sa2** will call sar for changing binary log file to human readable form.
15.Check run queue length, total number of processes and load average using ‘**-q**‘ option.
tecmint@tecmint ~ $ sar -q 2 5
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
02:00:44 IST runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
02:00:46 IST 1 431 1.67 1.22 0.97 0
02:00:48 IST 4 431 1.70 1.23 0.97 0
02:00:50 IST 2 431 1.70 1.23 0.97 0
02:00:52 IST 2 431 1.70 1.23 0.97 0
02:00:54 IST 0 431 1.64 1.23 0.97 0
Average: 2 431 1.68 1.23 0.97 0
16.Check statistics about the mounted file systems using ‘**-F**‘.
tecmint@tecmint ~ $ sar -F 2 4
Linux 3.11.0-23-generic (tecmint.com) Thursday 04 September 2014 _i686_ (2 CPU)
02:02:31 IST MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
02:02:33 IST 1001 449 30.95 1213790475088.85 18919505 364463 1.89 /dev/sda1
02:02:33 IST MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
02:02:35 IST 1001 449 30.95 1213790475088.85 18919505 364463 1.89 /dev/sda1
02:02:35 IST MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
02:02:37 IST 1001 449 30.95 1213790475088.85 18919505 364463 1.89 /dev/sda1
02:02:37 IST MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
02:02:39 IST 1001 449 30.95 1213790475088.86 18919505 364463 1.89 /dev/sda1