Is it possible to use the addShutdownHook or runFinalizersOnExit (Class Runtime) to trigger the shutdown of IE or the plugin.

Not a definitive answer, but I would say no. The addShutdownHook() method requires the shutdownHooks permission, which untrusted applets don't have. There are also issues with relying on the running of finalizers.