Certainly! Some studies suggest that it has to do with genetic variability. If I recall correctly, young women in a blind t-shirt sniffing study rated their brothers’ t-shirts as the least sexy, or something like that. It can be very conscious, too. It was for me.