How can an applet detect whether it's running behind a firewall or not?

There is no sure way, a problem compounded by the fact that there is no single definition of what a "firewall" is.

In many places there is no direct IP connectivity through the firewall - in those cases only an HTTP (and maybe other protocols) proxy exists. A lookup of a public machine (e.g. the one from which the applet was downloaded) using InetAddress will probably fail - if this happens then your applet will probably only be able to perform outgoing HTTPConnections.