How to identify what processes are generating IO Wait load.
Thursday, July 9th, 2009
An easy way to identify what process is generating your IO Wait load is to enable block I/O debugging. This is done by setting /proc/sys/vm/block_dump to a non zero value like:
echo 1 > /proc/sys/vm/block_dump
This will cause messages like the following to start appearing in dmesg:
bash(6856): dirtied inode 19446664 (ld-2.5.so) on md1
Using the following one-liner will produce a summary output of the dmesg entries:
dmesg | egrep "READ|WRITE|dirtied" | egrep -o '([a-zA-Z]*)' | sort | uniq -c | sort -rn | head
354 md
324 export
288 kjournald
53 irqbalance
45 pdflush
14 portmap
14 bash
10 egrep
10 crond
8 ncftpput
Once you are finished you should disable block I/O debugging by setting /proc/sys/vm/block_dump to a zero value like:
echo 0 > /proc/sys/vm/block_dump