Thanks for the memories

Need to know where your ram has gone?

Of course you could use vmstat to get a quick view of your memory status

enterprise:~# vmstat 1 5
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr f0 s2 s6 s1 in sy cs us sy id
0 0 0 1771952 452488 3 12 2 0 0 0 0 0 1 0 0 542 187 291 1 2 97
0 0 0 1358416 82120 2 15 0 0 0 0 0 0 0 0 0 521 304 339 1 2 97
0 0 0 1358416 82120 0 0 0 0 0 0 0 0 0 0 0 498 172 288 1 2 97
0 0 0 1358416 82120 0 1 0 0 0 0 0 0 0 0 0 508 485 358 2 4 93
0 0 0 1358416 82120 0 0 0 0 0 0 0 0 0 0 0 514 236 309 0 2 98
enterprise:~#


sure gives you an idea of what your current memory is. If you wish to see how each application is using RAM you can use prstat.


PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
10924 root 2016K 1696K sleep 59 0 0:00:02 0.3% prstat/1
141 root 2872K 1464K sleep 59 0 1:49:07 0.2% in.routed/1
10955 root 2016K 1704K cpu1 59 0 0:00:00 0.2% prstat/1
1753 mysql 37M 13M sleep 59 0 2:33:30 0.1% mysqld/9
1604 root 69M 15M sleep 44 5 2:20:39 0.1% java/20
2372 noaccess 189M 83M sleep 59 0 2:28:22 0.1% java/25


But how do you figure out how the kernel is using its memory?

This is where you have to dig down to the bowels of the kernel and use kdb, Here is the command.

enterprise:~# echo "::memstat" | mdb -k
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 177557 1387 68%
Anon 62233 486 24%
Exec and libs 2388 18 1%
Page cache 4614 36 2%
Free (cachelist) 5321 41 2%
Free (freelist) 7225 56 3%

Total 259338 2026
Physical 255493 1996

As you can see ZFS loves caching everything that it touches, that explains about 1GB of the Kernel ram usage.

You can also look at output from sar –k


#sar -k 1 5

SunOS enterprise 5.11 snv_39 sun4u 11/24/2006

11:50:13 sml_mem alloc fail lg_mem alloc fail ovsz_alloc fail
11:50:14 320430528 285628992 0 2231025664 1945390720 0 78077952 0
11:50:15 320430528 285628992 0 2231025664 1945390720 0 78077952 0
11:50:16 320430528 285628992 0 2231025664 1945396352 0 78077952 0
11:50:17 320430528 285628992 0 2231025664 1945396352 0 78077952 0
11:50:18 320430528 285629184 0 2231025664 1945396608 0 78077952 0

Average 320430528 285629024 0 2231025664 1945394176 0 78077952 0
enterprise:~#


Honestly I don’t knw what ½ of that means without resorting to reading Solaris(TM) Performance and Tools: DTrace and MDB Techniques for Solaris 10 and OpenSolaris (Solaris Series). [ A book I highly recomend, and should be on every Solaris sysadmim's book shelf.

You can also see statistics on memory being paged in and out by the kernel using sar –p

enterprise:~# sar -p 1 5

SunOS enterprise 5.11 snv_39 sun4u 11/24/2006

11:58:19 atch/s pgin/s ppgin/s pflt/s vflt/s slock/s
11:58:20 0.00 0.00 0.00 2.91 14.56 0.00
11:58:21 0.00 0.00 0.00 0.00 1.96 0.00
11:58:23 0.00 0.00 0.00 0.00 0.00 0.00
11:58:24 0.00 0.00 0.00 0.00 0.00 0.00
11:58:25 0.00 0.00 0.00 0.00 0.00 0.00

Average 0.00 0.00 0.00 0.58 3.31 0.00

I guess I should include a look at one command that is not standard on Solaris installs but still lots of people use it anyway. top

last pid: 11057; load averages: 0.07, 0.10, 0.11 11:48:26
140 processes: 139 sleeping, 1 on cpu
CPU states: 97.0% idle, 0.6% user, 2.4% kernel, 0.0% iowait, 0.0% swap
Memory: 2048M real, 80M free, 614M swap in use, 1326M swap free