How can I display a component instead of a label in my TitledBorder?

You'll need to subclass TitledBorder and override the paintBorder() method to draw the appropriate component. This is demonstrated at http://www2.gol.com/users/tame/swing/examples/BorderExamples1.html.