How can I get a JWindow to show up on the Windows Taskbar?

You can't. Only JFrames (Frames really !) are shown. That is because of the choice of AWT engineers to use the kind of MSWindow's object as a peer.