It is often convenient to know just how much memory is being used by your application. Often, you may want to predict the memory requirements for a new, larger data set. Knowing the trend in memory usage (linear, quadratic, etc) can be helpful in assessing whether a new simulation is possible on your current hardware, or even if it is possible at all.
Linear vs quadratic, why run things at 1,2,4,8x sizes
One of the simplest methods to understand the memory used by your program is to run it within the /usr/bin/time program. The syntax is just:
jbp@head1 [ 32 ] % /usr/bin/time my_prog bido0.txt
...
... output from my_prog program will show up here
...
0.02user 0.00system 0:00.01elapsed 133%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (407major+104minor)pagefaults 0swaps
The last two lines are output from the time program and indicate the amount of user/system/elapsed (or wall-clock) time, average percent CPU utilization, and then the amount of memory used by the 'text' and 'data' segments, and the 'maxresident' or maximum amount of memory that was used at any point in time by the program.
top Program:If you have ssh access to the machine where your program is running, another quick and easy option is to wait for the program to go through its initialization phase (reading data from disk, etc.) and then run the top command. You can just type 'top' and run it interactively (type 'q' to quit), or you can run it non-interactively with 'top -bn1'. The output will look like:
11:00:23 up 24 days, 20:20, 1 user, load average: 0.97, 0.51, 0.20
54 processes: 52 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 0.0% 99.0% 0.0% 0.0% 0.0% 0.0% 100.8%
cpu00 0.0% 47.5% 0.0% 0.0% 0.0% 0.0% 52.4%
cpu01 0.0% 51.4% 0.0% 0.0% 0.0% 0.0% 48.5%
Mem: 2055464k av, 1447644k used, 607820k free, 0k shrd, 153840k buff
552388k actv, 500224k in_d, 30612k in_c
Swap: 4192880k av, 44k used, 4192836k free 1102160k cached
PID USER PRI NI SIZE RSS SHARE STAT MEM TIME CPU COMMAND
5470 jbp1 39 19 6276 6276 512 R N 99.8 0.3 3:33 1 Run
1 root 18 0 516 516 456 S 0.0 0.0 0:23 1 init
2 root RT 0 0 0 0 SW 0.0 0.0 0:00 0 migration/0
3 root RT 0 0 0 0 SW 0.0 0.0 0:00 1 migration/1
4 root 15 0 0 0 0 SW 0.0 0.0 0:00 1 keventd
5 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
6 root 34 19 0 0 0 SWN 0.0 0.0 0:00 1 ksoftirqd/1
9 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 bdflush
7 root 15 0 0 0 0 SW 0.0 0.0 0:11 1 kswapd
As with the previous example, we are running the 'Run' program which shows up in the second block of output, starting under the 'PID USER PRI ...' line. In this run, the program is the only one running (as indicated by the '99.8' under the '%CPU' column), it is also using 6276 KB (as indicated by the 'RSS' column).
/proc File System:Linux provides a "pseudo-filesystem" located in /proc which looks like just-another directory, but actually contains up-to-the-minute information on running processes on the machine. Each of the numbered directories is a Process-ID or PID, underneath each directory will be a number of files including one called "status". That will appear to be a text file like:
machine [ 5 ] % cat /proc/23870/status
Name: progname
State: S (sleeping)
Tgid: 23870
Pid: 23870
PPid:1
TracerPid:0
Uid: 500 500 500 500
Gid: 500 500 500 500
FDSize: 32
Groups: 500 10 48
VmSize?: 19668 kB
VmLck?: 0 kB
VmRSS?: 10728 kB
VmData?: 4316 kB
VmStk?: 36 kB
VmExe?: 284 kB
VmLib?: 14104 kB
SigPnd?: 0000000000000000
SigBlk?: 0000000100000000
SigIgn?: 8000000000001000
SigCgt?: 00000000800104f0
CapInh?: 0000000000000000
CapPrm?: 0000000000000000
CapEff?: 0000000000000000
The VmRSS and VmRSS lines are the most important. VmRSS is the total amount of memory required by this program, and VmRSS is the "Resident Set Size" (the amount actually in memory right now). In many cases, some of this memory can be swapped out to disk so your program can actually be "bigger" than the physical memory of the machine and still not have any problems.
While your program is running, you can open this file and read it's contents, even from the program itself! In C, you can find the process ID number with the getpid system call, see "man getpid" for more info. You can then use that PID number to generate a filename (with sprintf and others), and then open and read the file:
int pid;
char buffer[1024];
FILE* fp;
pid = getpid();
sprintf( buffer, "/proc/%i/status", pid );
fp = fopen( buffer, "r" );
/* read buffer and look for VmRSS etc. */
/* .. don't forget to convert kb/mb/gb */
fread( ..., fp );
fclose( fp );