The correspondence between the java.lang.Runtime memory figures and actual system memory usage is not close. Like most native applications, the Java interpreter never gives back memory it's taken from the system - so you shouldn't expect to see rising and falling numbers here matching what the Java GC is doing.

It is theoretically possible for an application to give back memory with the low-level sbrk() kernel call, but there are practical difficulties - the memory given back must be a contiguous chunk at the high end of the address space. That's virtually impossible without garbage collection (i.e., in most native applications) and also impossible with current implementations of libc's standard allocator (malloc()).

So why doesn't the Java interpreter, which does have GC, give back memory? My conjecture is that this would be vastly more trouble to implement (if even possible) than it's worth. Not to fear: garbage-collected memory is not lost, it's endlessly reused. It's just not returned to the system, where it would be reflected in falling memory usage values.

So, on to the questions:

1) The 37M includes the heap and stack sizes managed by the JVM, plus the overhead of managing those memory pools, plus allocations of non-objects made by native code - in other words, a lot of overhead. It might also (depending on top's behavior) include the shared memory occupied by the JVM's native code itself.

2) There's not much you can do about it. That overhead is the cost of running the Java interpreter above and beyond the heap and stack space for objects and the Java thread stacks.

3) The explanation at the start of this answer should address this question.

1. You mentioned overhead due to the interpreter. Is the Hotspot server (-server option) considered an interpreter?
2. The app started with 17M in top and grew to 37M. If java.lang.Runtime's totalMemory=4M can be believed, does that mean the 20M growth in top is due to stack and other things after the program was initialized? The memory grew (but never decreased) as the server accepted HTTP connections.
3. Would compiling the app into native code decrease the amount of memory used? I may try GNU's GCJ on Solaris.
4. Does the SIZE column in top mean 37M of virtual memory is being used by the process? I want to run as many instances of the server app as possible on a Solaris box with 1 GB of RAM.
   Is the max number of instances (JVMs) equal to (total virtual memory - other processes's memory) / 37M) where 37M is the top size for an instance of the server app?
5. Even though java.lang.Runtime.totalMemory says 4M, can top's 37M be decreased by decreasing the number of object creations in the app? For example, the JVM's profiler showed an excessive number of live java.util.Vector$Enumerator objects. Those objects were returned by myVector.elements that was called inside a run method that can execute for hours. Apparently, they aren't gc'd as quickly as if the loop ended earlier. An alternative, which would perform better as well as reduce obj. creation, is to access the Vector directly, e.g., elementAt.

Before I address these questions, I've got one overriding question: what's the problem? Servers are typically big machines because server processes are typically big. And Java is a demanding environment. I wouldn't lose a lot of sleep over a server process consuming 37M of memory. That said...

1) Java is an interpreted language - the java executable itself is an interpreter. HotSpot is a performance engine that improves Java's performance, and uses certain launch options (such as -server) to optimize its behavior around the sort of code typically run in certain environments. HotSpot is good stuff - don't disable it unless you find failures attributable to HotSpot bugs.

2) There's nothing shocking about these numbers. As explained earlier, you'll never see the number go down. Assuming the server reaches a steady state - that is, new connections aren't accepted faster than old ones are closed - you should see the number stabilize as resources from old connections are GC'd and reused for new connections. If the number keeps going up after you reach steady state, something is wrong.

3) I can only make a conjecture, but I suspect a natively compiled version of your app will use less memory. It should also run faster. GCJ and its class libraries do not deliver a full Java 2 implementation, but if you're able to build your app with what it does deliver, it's certainly worth a try.

4) That 37M is the virtual memory used for the process, but don't ignore the RSS (resident set size) column - that's the actual memory in RAM, the rest is on disk (either swap space or sharable code space). Once the process is in steady state, this number will probably stabilize - and you'll also notice that a lot of processes hang around with very little of their address space resident in RAM. These are the numbers that can't add up to more than your total memory.

That said, adding up the numbers and subtracting from your total memory may put you in the ballpark, but there is undoubtedly memory used that doesn't show up in top - for example, it's likely that memory used by the kernel itself isn't accounted for in top's output. The best way to determine your machine's load limits are to load it down and see what happens. Tools are available that let you monitor memory paging activity (sorry, I don't know what they're called in Solaris) so you can see when the system is starting to thrash from the memory load.

5) There are always ways to tune apps for runtime performance and/or memory use, and tools that can help you do it. I think you'll find some good stuff at Sun's Java Developer's Connection site to get you started.