Even if the department is casual, you should dress in professional business attire for an interview. Dressing professionally will always make a good first impression, anything else is a risk.
Be sure to choose clothes and shoes that are both comfortable and make you feel good. Uncomfortable clothing may be distracting and affect your interview performance.