How can I create a JTable with a "row header". The first column should look similar to the column header (that is it looks "button-like", it mustn't be editable, ...)

You will need to do two things -

1. Implement the TableCellRenderer to return JButton with proper image and label alignment. You could subclass DefaultTableCellRenderer to do this. Mind you - the JButton you return will be used as a ruberstamp. It is not actually 'there' i.e. you cannot click on it and get actionPerformed(..) called back.
2. Return false from the -

   public boolean isCellEditable(int row, int col)
   

   method for the zeroth column.