virtual memory on compute nodes in a grid

When I run a job on a node, using PBS, and I get finally in the job report:

resources_used.mem开发者_开发问答=1616024kb
resources_used.vmem=2350176kb
resources_used.walltime=00:06:32

What does the virtual memory really means? I don't think there is a hard drive connected to each node.

Which kind of memory should I take into account when I try to increase the size of the problem, such that I don't hit the 16GB capacity of the node memory, the normal memory (mem) or the virtual memory (vmem) ? Thanks

The vmem indicates how much memory your job was using in total. It used all of the available physical memory (see the mem value), and more. An operating system allows programs to allocate more memory than there is physical memory available.

If you're actively using more memory than there is physical memory available, you'll start seeing swap activity (data that was swapped out to disk being brought back into memory, and other stuff being put to disk). That's bad, it will basically kill your performance if it happens a lot.

So, as long as you're not actively using more than 16GB, you're fine. But the mem or vmem values won't tell you this, it depends on what the application is actually doing.