Java has a couple of settings that help control how much memory it uses:
For a server with a 'small' amount of memory, then we recommend that -Xms is kept as small as possible. e.g. -Xms 16m. Some customers set this higher, but that can lead to issues. e.g. the command that restarts tomcat runs a java process. That Java process picks up the same -Xms setting as the actual Tomcat process. So you will effectively be using two times -Xms when doing a restart. If you set -Xms too high, then you may run out of memory.
When setting the -Xmx setting you should consider a few things... -Xmx has to be enough for you to run your app. If it is set too low then you may get Java OutOfMemory exceptions (even when there is sufficient spare memory on the server).
If you have 'spare' memory, then increasing the -Xmx setting is often a good idea. Just note that the more you allocate to Java the less will be available to your database server or other applications and less for Linux to cache your disk reads.
Whatever you do, you should ensure that the total of the -Xmx settings does not exceed the 'spare' memory on your server. You can figure your 'spare' memory by stopping your Java processes and starting up everything else that is normally running on your server. When you run free -m it will tell you how much memory you are using. If you subtract that value from your total memory and then subtract a bit more (for safety) that will be the maximum you should allocate with -Xmx. e.g. if you have 96MB 'spare' then you could allocate two -Xmx48m Java processes. Or one -Xmx96m process.
We put our -Xmx/-Xms settings for Tomcat in /usr/local/tomcat/bin/setenv.sh in the JAVA_OPTS variable. We find that -Xmx48 is sufficient to run a minimal Tomcat with the sample webapps installed.
Note that Java can end up using (a lot) more than the -Xmx value worth of memory, since it allocates extra/separate memory for the Java classes it uses. So the more classes are involved in your application the more memory that Java process will require.
If you get an error like:
java.lang.OutOfMemoryError: PermGen
The PermGen space is used for things that do not change (or change often). e.g. Java classes. So often large, complex apps will need lots of PermGen space. Similarly if you are doing frequent war/ear/jar deployments to running servers like Tomcat or JBoss you may need to issue a server restart after a few deploys or increase your PermGen space.
To increase the PermGen space use something like: -XX:MaxPermSize=128m
The default is 64MB.
(Note that Xmx is separate from the PermGen space, so increasing Xmx will not help with the PermGen errors).
The PermGen memory in addition to the Xmx memory. e.g. -Xmx128m and MaxPermSize=128m would use up to 256MB.