[conspire] How much memory each process is taking up
Rick Moen
rick at linuxmafia.com
Mon Feb 23 15:55:38 PST 2009
It occurs to me that many Linux users have absolutely no idea how to
fruitfully use the "ps" (process status) and "top" commands -- or others
-- to examine the use of memory. So, I thought I'd show a few tricks.
You can get a stratospheric view of system memory using the "free"
command, or looking at the /proc/meminfo pseudofile:
rmoen at borgia:~$ free
total used free shared buffers
cached
Mem: 2074864 2044428 30436 0 80300
1357888
-/+ buffers/cache: 606240 1468624
Swap: 1951888 88 1951800
rmoen at borgia:~$ cat /proc/meminfo
MemTotal: 2074864 kB
MemFree: 30536 kB
Buffers: 80312 kB
Cached: 1357876 kB
SwapCached: 12 kB
Active: 589072 kB
Inactive: 1328784 kB
HighTotal: 1178136 kB
HighFree: 7400 kB
LowTotal: 896728 kB
LowFree: 23136 kB
SwapTotal: 1951888 kB
SwapFree: 1951800 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 479588 kB
Mapped: 59692 kB
Slab: 106924 kB
SReclaimable: 100592 kB
SUnreclaim: 6332 kB
PageTables: 1968 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2989320 kB
Committed_AS: 718068 kB
VmallocTotal: 114680 kB
VmallocUsed: 44220 kB
VmallocChunk: 62964 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB
rmoen at borgia:~$
This is a system with 2GB physical RAM ("MemTotal: 2074864 kB"). The
fact that it reports "MemFree: 30536kB" (30 MB) is _not_ a bad thing:
Unix does its level best to make sure RAM is put to productive use. If
there's nothing better for it to do, the RAM will get allocated to disk
cache, which is mostly what's happening in this case.
It's a Debian "squeeze" (new Debian testing track) P4 workstation with the
xdm display manager running the Window Maker window manager:
Post-installation, I reduced the number of "getty" processes from six to
two by commenting out lines in /etc/inittab[1], then disabling SysVInit
startup of the Avahi mDNS/DNS-SD daemon, the OpenBSD inetd, and the
kerneloops daemon, none of which I currently need.
(Running "telinit Q" makes the system init process re-parse
/etc/inittab, which is handy for putting into effect changes to that
file without rebooting.)
Likewise, the fact that the swap partition is actually in use (if
barely) does not indicate, in any way, system stress. Some swap use
always happens if you provide swap to the kernel. This happens to be an
edge case of a workstation with generous RAM that's running fairly
lightweight processes, hence is barely touching swap. There are
effective ways to measure the "swappiness" of a system, e.g., using the
iostat utility, but "free" isn't really useful for that.
Getting down to the "ps" and "top" commands: "ps" has a default output
that includes a number of default fields (columns) and omits others, but
you can easily customise its reporting, using command options. Here's
a custom "ps" report highlighting process memory stats:
$ ps -eo pid,user,group,%mem,rss,vsz,args
The option "e" is the SysV-style way of saying "include all processes".
Option "o" is "include user-specified columns" as a comma-separated
list. That list then follows:
"pid" = the command's Process ID.
"user" = what user the process is running as.
"group" = what group the process is running as.
"%mem" = What percentage the process's resident set size (to be
explained below) is relative to total physical memory.
"rss" = "the non-swapped physical memory that a task has used (in kiloBytes)".
"vsz" = "virtual memory size of the process in KiB (1024-byte units)".
"arg" = the process name plus all the arguments it started with.
In this case, we get:
ps -eo pid,user,group,%mem,rss,vsz,args
PID USER GROUP %MEM RSS VSZ COMMAND
1 root root 0.0 692 2100 init [2]
2 root root 0.0 0 0 [kthreadd]
3 root root 0.0 0 0 [migration/0]
4 root root 0.0 0 0 [ksoftirqd/0]
5 root root 0.0 0 0 [watchdog/0]
6 root root 0.0 0 0 [migration/1]
7 root root 0.0 0 0 [ksoftirqd/1]
8 root root 0.0 0 0 [watchdog/1]
9 root root 0.0 0 0 [events/0]
10 root root 0.0 0 0 [events/1]
11 root root 0.0 0 0 [khelper]
44 root root 0.0 0 0 [kblockd/0]
45 root root 0.0 0 0 [kblockd/1]
47 root root 0.0 0 0 [kacpid]
48 root root 0.0 0 0 [kacpi_notify]
121 root root 0.0 0 0 [kseriod]
160 root root 0.0 0 0 [pdflush]
161 root root 0.0 0 0 [pdflush]
162 root root 0.0 0 0 [kswapd0]
163 root root 0.0 0 0 [aio/0]
164 root root 0.0 0 0 [aio/1]
624 root root 0.0 0 0 [ksuspend_usbd]
625 root root 0.0 0 0 [khubd]
700 root root 0.0 0 0 [ata/0]
706 root root 0.0 0 0 [ata/1]
708 root root 0.0 0 0 [ata_aux]
788 root root 0.0 0 0 [scsi_eh_0]
789 root root 0.0 0 0 [scsi_eh_1]
932 root root 0.0 0 0 [kjournald]
1008 root root 0.0 1532 2936 udevd --daemon
1544 root root 0.0 0 0 [kpsmoused]
1776 root root 0.0 0 0 [kjournald]
1838 daemon daemon 0.0 512 1892 /sbin/portmap
1849 statd root 0.0 724 1956 /sbin/rpc.statd
2112 root root 0.0 0 0 [kondemand/0]
2114 root root 0.0 0 0 [kondemand/1]
2155 root root 0.0 1516 28164 /usr/sbin/rsyslogd -c3
2166 root root 0.0 648 1764 /usr/sbin/acpid
2176 103 108 0.0 888 2620 /usr/bin/dbus-daemon --system
2223 root root 0.1 2948 6544 /usr/sbin/cupsd
2490 101 105 0.0 936 6296 /usr/sbin/exim4 -bd -q30m
2526 daemon daemon 0.0 440 2048 /usr/sbin/atd
2546 root root 0.0 988 3456 /usr/sbin/cron
2569 root root 0.0 920 5432 /usr/bin/xdm
2575 root root 0.0 504 1764 /sbin/getty 38400 tty1
2578 root root 0.0 504 1764 /sbin/getty 38400 tty2
2593 root root 1.8 37600 306540 /usr/bin/X :0 vt7 -nolisten tcp -auth /var/lib/xdm/authdir/authfiles/A:0-PoA
2598 root root 0.1 3752 6740 -:0
2619 rmoen rmoen 0.0 1208 6816 /usr/lib/WindowMaker/WindowMaker
2640 rmoen rmoen 0.0 604 4756 /usr/bin/ssh-agent /usr/bin/dbus-launch --exit-with-session x-window-manager
2643 rmoen rmoen 0.0 672 3100 /usr/bin/dbus-launch --exit-with-session x-window-manager
2644 rmoen rmoen 0.0 860 2620 /usr/bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
2662 rmoen rmoen 0.2 5204 9856 /usr/lib/WindowMaker/WindowMaker --for-real
2669 rmoen rmoen 0.1 2228 5308 /usr/lib/libgconf2-4/gconfd-2
11
2698 root root 0.0 612 2180 dhclient3 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp3/dhclient.eth0.le
2760 rmoen rmoen 0.0 1256 3868 /bin/sh /usr/bin/icedove
2772 rmoen rmoen 0.0 1276 3908 /bin/sh /usr/lib/icedove/run-mozilla.sh /usr/lib/icedove/icedove-bin
2777 rmoen rmoen 6.1 127228 230032 /usr/lib/icedove/icedove-bin
3526 rmoen utmp 0.2 4372 10656 xterm
3527 rmoen rmoen 0.1 2944 5668 bash
3538 rmoen rmoen 0.0 1412 4364 /bin/sh /usr/local/bin/irc
3539 rmoen rmoen 0.2 4652 9692 irssi --port=7000 --connect=irc.example.com --nick=rmoen --password=hahyouwish
3931 rmoen rmoen 15.4 321364 440068 /usr/lib/iceweasel/firefox-bin
5843 rmoen utmp 0.2 4280 10648 xterm
5844 rmoen rmoen 0.1 2976 5672 bash
10055 rmoen rmoen 0.1 2236 5260 ssh rick at 198.144.195.186
10102 rmoen utmp 0.2 4864 11276 xterm
10103 rmoen rmoen 0.1 2988 5680 bash
10150 root root 0.0 1132 3772 su -
10151 root root 0.0 1728 4228 -su
10327 root root 0.0 1000 3716 ps -eo pid,user,group,%mem,rss,vsz,args
So, before I go further, "resident set size" (RSS) and "virtual size"
(VSZ) are crucial concepts in understanding RAM usage. RSS is the
active parts of an executable -- allocated pages of RAM being actively
used within the process's RAM segment -- as opposed to portions of the
program that were never loaded into RAM in the first place, or portions
that used to be present but were swapped out to disk by the kernel. (A
"page" is the smallest contiguous block of memory that the kernel can
allocate for storing data and code.)
The VSZ is the total "virtual" size of the process and its shared
libraries: resident in-use memory, currently used swap (swapped-out
portions of code or data), memory-mapped files, and portions of the
program that reside currently only on-disk and have not been loaded.
Basically, physical RAM gets taken up by loaded-from-disk executable
code, plus stack size, plus data roaded into RAM. Those comprise the
RSS. RSS plus pages currently in swap equals VSZ.
There's a third crucial concept: shared memory size (SHR), but we'll
encounter that only when we look at "top".
Notice the first process, init. It has process ID 1; in fact, that's
one of its defining characteristics: When your system starts, it loads
the kernel as process number zero. The kernel forks, and the new
process (#1) execs "/bin/init". init then starts up the rest of your
system.
All of the couple-dozen processes in brackets are kernel service
processes. To my knowledge, there's not much you can (or should) do
about deciding which of those to run.
udevd is the 2.6-kernel-ish daemon that manages what device nodes exist
on your system. Notice it has an RSS of about 1.5 megs.
I waste about 1.2MB on two service daemons for NFS (portmapper and
nfs.statd).
rsyslogd (1.5 MB RSS) is a modern revamp of the original BSD system
logging daemon. It's an essential system feature. Like syslog-ng,
it supports across networks (optionally), and encryption.
acpid (1/2 MB RSS) listens for system hardware events (ACPI = Advanced
Control Peripheral Interfae) and can take actions based on them. Looks
like the only things acpid is listening for (/etc/acpi/events/*) are
"power button" signals to shut off system power upon shutdown. I might
consider disabling acpid, as that seems a bit stupid to waste half a meg
of RAM on. Of course, ACPI can potentially do many other things,
including taking scripted actions when you press particular buttons on
the keyboard, and lots of other things.
dbus-daemon is an ugly GNOME-ism that's been allowed to worm its way
into Linux system space, D-Bus being a system facility, running both as
a system daemon and as a per-user daemon, and a related library
(libdbus) to manage dynamic system "events" such as adding/removing a
printer or adding/removing casual storage (e.g., flash drives), and
providing a general inter-process communication mechanism for talking
about system "events" between apps.
I didn't actually want an overengineered "middleware layer" created by a
bunch of desktop-software types, but that's what D-Bus is -- and it's
the camel's nose in the tent for HAL
(http://www.freedesktop.org/wiki/Software/hal), too. Since I actually
don't _want_ something rummaging around in the background mounting and
umounting drives for me, I will probably disable the damned thing.
Meanwhile, at least it's reasonably small, about 2.5 MB scattered across
four processes (including the dbus-launch stuff).
cupds is the CUPS (Common Unix Printing System) daemon, which I need.
3 MB RSS.
exim4 is the system stub MTA for handling local mail, e.g., reports
generated for the admin user. 1MB RSS.
atd is for timed future execution of one-time (as opposed to recurring)
processes. 1/2 MB RSS. crond runs recurring processes, 1 MB RSS.
xdm is the old-school display manager, providing graphical login. 1MB
RSS.
Two getty ("get tty") processes furnish virtual consoles. You need
minimum one. 1/2 MB RSS each.
The next process, "X", is the one where you have to be wary about
overinterpreting memory-usage stats, because physical RAM on the video
card is included. So, the 37MB of RSS reported is vastly overstated.
How much _real_ RSS, then? I'm not sure.
Process "-:0" is somehow related to the "session" of the X server that
the window manager is talking to. There's some X Window System arcana
that I've largely forgotten relating to ":0" (session #0) or ":1"
(session #1), etc. Like, you can start up a second instance on a
different virtual console, like this:
startx -- :1 &
Process WindowMaker is of course my favourite window manager, a mere 1.2
MB of RSS.
Process ssh-agent is a nice little tool to hold private SSH keys and
used for authentication. I haven't yet looked into how it's set up on
this workstation. 1/2MB RSS.
Process gconfd-2 is a poxy bit of GNOME that got implicitly started by
the Debian build of Window Maker. I probably cannot get rid of it
unless I compile Window Maker locally without "GNOME support" hooks.
2 MB of RSS wasted (almost twice as big as my window manager).
Process dhclient3 is, of course, the DHCP client, accepting "leased" IP
addresses and related information.
Finally, we get to serious memory usage:
"icedove" (Mozilla Thunderbird): about 130 MB of RSS across three
processes.
"iceweasel" (Mozilla Firefox): 320 MB of RSS.
And there are a few miscellaneous proceses: Each instance of bash is
about 3 MB of RSS, xterm is 4-5 MB, and my preferred IRC client (irssi)
is 6MB including the little shell script that runs it.
Moving along: "top" defaults to a dynamic display of processes,
refreshing every second and sorting by percentage of CPU consumed.
Once it's doing that, typing "M" will make top re-sort by percentage of
_memory_ consumed, and "c" will toggle between program name (default)
and command-line.
If you just want to see the output once (instead of refreshed every
second), and want command lines (instead of program names), you can do:
# top -b -n 1 -c
top - 15:34:58 up 7 days, 1:10, 5 users, load average: 0.04, 0.01,
0.00
Tasks: 74 total, 1 running, 73 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.4%us, 0.1%sy, 0.0%ni, 99.5%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 2074864k total, 1991960k used, 82904k free, 85852k buffers
Swap: 1951888k total, 84k used, 1951804k free, 1330932k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 2100 692 588 S 0 0.0 0:04.48 init [2]
2 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kthreadd]
3 root RT -5 0 0 0 S 0 0.0 0:00.38 [migration/0]
4 root 15 -5 0 0 0 S 0 0.0 0:01.98 [ksoftirqd/0]
5 root RT -5 0 0 0 S 0 0.0 0:00.00 [watchdog/0]
6 root RT -5 0 0 0 S 0 0.0 0:00.00 [migration/1]
7 root 15 -5 0 0 0 S 0 0.0 0:01.82 [ksoftirqd/1]
8 root RT -5 0 0 0 S 0 0.0 0:00.00 [watchdog/1]
9 root 15 -5 0 0 0 S 0 0.0 0:15.72 [events/0]
10 root 15 -5 0 0 0 S 0 0.0 0:10.56 [events/1]
11 root 15 -5 0 0 0 S 0 0.0 0:00.00 [khelper]
44 root 15 -5 0 0 0 S 0 0.0 0:00.48 [kblockd/0]
45 root 15 -5 0 0 0 S 0 0.0 0:00.32 [kblockd/1]
47 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kacpid]
48 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kacpi_notify]
121 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kseriod]
160 root 20 0 0 0 0 S 0 0.0 0:00.22 [pdflush]
161 root 20 0 0 0 0 S 0 0.0 0:04.02 [pdflush]
162 root 15 -5 0 0 0 S 0 0.0 0:01.16 [kswapd0]
163 root 15 -5 0 0 0 S 0 0.0 0:00.00 [aio/0]
164 root 15 -5 0 0 0 S 0 0.0 0:00.00 [aio/1]
624 root 15 -5 0 0 0 S 0 0.0 0:00.38 [ksuspend_usbd]
625 root 15 -5 0 0 0 S 0 0.0 0:00.06 [khubd]
700 root 15 -5 0 0 0 S 0 0.0 0:00.00 [ata/0]
706 root 15 -5 0 0 0 S 0 0.0 0:00.00 [ata/1]
708 root 15 -5 0 0 0 S 0 0.0 0:00.00 [ata_aux]
788 root 15 -5 0 0 0 S 0 0.0 0:00.00 [scsi_eh_0]
789 root 15 -5 0 0 0 S 0 0.0 0:00.00 [scsi_eh_1]
932 root 15 -5 0 0 0 S 0 0.0 0:06.66 [kjournald]
1008 root 16 -4 2936 1532 488 S 0 0.1 0:00.54 udevd --daemon
1544 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kpsmoused]
1776 root 15 -5 0 0 0 S 0 0.0 0:00.54 [kjournald]
1838 daemon 20 0 1892 512 416 S 0 0.0 0:00.00 /sbin/portmap
1849 statd 20 0 1956 724 624 S 0 0.0 0:00.00 /sbin/rpc.statd
2112 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kondemand/0]
2114 root 15 -5 0 0 0 S 0 0.0 0:00.00 [kondemand/1]
2155 root 20 0 28164 1516 948 S 0 0.1 0:00.47 /usr/sbin/rsyslogd -c3
2166 root 20 0 1764 648 548 S 0 0.0 0:00.00 /usr/sbin/acpid
2176 messageb 20 0 2620 888 704 S 0 0.0 0:00.00 /usr/bin/dbus-daemon --system
2223 root 20 0 6544 2948 2168 S 0 0.1 0:01.02 /usr/sbin/cupsd
2490 Debian-e 20 0 6296 936 616 S 0 0.0 0:00.14 /usr/sbin/exim4 -bd -q30m
2526 daemon 20 0 2048 440 316 S 0 0.0 0:00.00 /usr/sbin/atd
2546 root 20 0 3456 988 804 S 0 0.0 0:02.42 /usr/sbin/cron
2569 root 20 0 5432 920 612 S 0 0.0 0:00.00 /usr/bin/xdm
2575 root 20 0 1764 504 436 S 0 0.0 0:00.00 /sbin/getty 38400 tty1
2578 root 20 0 1764 504 436 S 0 0.0 0:00.00 /sbin/getty 38400 tty2
2593 root 20 0 477m 37m 11m S 0 1.8 28:49.27 /usr/bin/X :0 vt7 -nolisten tcp -auth /var/lib/xdm/authdir/authfiles/A:0-PoAm0q
2598 root 20 0 6740 3752 2908 S 0 0.2 0:00.48 -:0
2619 rmoen 20 0 6816 1208 976 S 0 0.1 0:00.12 /usr/lib/WindowMaker/WindowMaker
2640 rmoen 20 0 4756 604 304 S 0 0.0 0:00.78 /usr/bin/ssh-agent /usr/bin/dbus-launch --exit-with-session x-window-manager
2643 rmoen 20 0 3100 672 456 S 0 0.0 0:00.00 /usr/bin/dbus-launch --exit-with-session x-window-manager
2644 rmoen 20 0 2620 860 696 S 0 0.0 0:00.00 /usr/bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
2662 rmoen 20 0 9856 5204 3276 S 0 0.3 4:30.69 /usr/lib/WindowMaker/WindowMaker --for-real
2669 rmoen 20 0 5308 2228 1856 S 0 0.1 0:01.12 /usr/lib/libgconf2-4/gconfd-2 11
2698 root 18 -2 2180 612 488 S 0 0.0 0:00.00 dhclient3 -pf /var/run/dhclient.eth0.pid -lf /var/lib/dhcp3/dhclient.eth0.leases eth0
2760 rmoen 20 0 3868 1256 1084 S 0 0.1 0:00.00 /bin/sh /usr/bin/icedove
2772 rmoen 20 0 3908 1276 1064 S 0 0.1 0:00.00 /bin/sh /usr/lib/icedove/run-mozilla.sh /usr/lib/icedove/icedove-bin
2777 rmoen 20 0 225m 124m 25m S 0 6.1 26:35.85 /usr/lib/icedove/icedove-bin
3526 rmoen 20 0 10656 4372 2404 S 0 0.2 0:04.34 xterm
3527 rmoen 20 0 5668 2944 1404 S 0 0.1 0:00.12 bash
3538 rmoen 20 0 4364 1412 996 S 0 0.1 0:00.00 /bin/sh /usr/local/bin/irc
3539 rmoen 20 0 9692 4652 3368 S 0 0.2 1:19.65 irssi --port=7000 --connect=irc.example.com --nick=rmoen --password=hahyouwish
3931 rmoen 20 0 430m 313m 23m S 0 15.5 132:53.25 /usr/lib/iceweasel/firefox-bin
5843 rmoen 20 0 10648 4284 2400 S 0 0.2 1:05.20 xterm
5844 rmoen 20 0 5672 2976 1432 S 0 0.1 0:00.20 bash
10055 rmoen 20 0 5260 2236 1808 S 0 0.1 0:02.74 ssh rick at 198.144.195.186
10102 rmoen 20 0 12036 5752 2400 S 0 0.3 0:04.02 xterm
10103 rmoen 20 0 5680 2988 1436 S 0 0.1 0:00.40 bash
10601 root 20 0 2388 1016 800 R 0 0.0 0:00.04 top -b -n 1 -c
$
That's "-c" for the command-line display, "-b" for batch, and "-n 1" for
display once.
Notice that batch mode makes "top" default to Process ID order.
The other main thing to note is that (aside from %MEM) there are now _three_
memory figures: VIRT RES SHR
VIRT is VSZ again, just as RES is RSS. The new one is SHR. Manpage says:
"Shared Mem size (kb). The amount of shared memory used by a task. It
simply reflects memory that could be potentially shared with other
processes." This concept is important for applications like
OpenOffice.org or Mozilla-family applications that rely on core libs
that will be shared among multiple instances of the app and its
relatives. For example, the two icedove-bin (Mozilla Thunderbird)
processes shown have RSS ("RES") of 1.2MB and 124MB, respectively, but
also SHR figures of 1MB and 25MB, respectively. A third process,
"icedove", has figures almost identical to the first icedove-bin.
The point? About a meg of code is almost certainly near-100% common
among all three processes, reducing the real memory consumption by that
amount for two of the three. That's not a huge effect in Thunderbird's
case, but OpenOffice.org shows it in spades.
I hope the above is handy in better assessing where your RAM is _really_
going.
More information about the conspire
mailing list