With Render Multiple Frames Simultaneously, After Effects can start separate background processes of the After Effects application to render multiple frames at the same time. This can really speed up renders, both for final output and for RAM previews.
2GB per process is the limit on 32-bit Windows without special (and some say somewhat risky) configuration. And, of course, the OS can only see 4GB, so that's severely limiting.
Nearly 4GB per process is the limit on 64-bit Windows, with no special configuration required. ~3.5 GB per process is the limit on Mac OS (though the foreground application gets a little less because of the Mac OS UI libraries). Because these numbers are per-process, and because a 64-bit operating system can see a lot of RAM, you can make use of around 30GB of RAM in an 8-core computer.