When you interact with the people around you, the way you dress matters especially if we’re talking about men. Actually it matters a lot.
When you interact with a woman she will analyze whether you’re following the latest trends or not, if your outfit is well matched and if you are wearing it on the correct occasion. But men won’t do it at all. They won’t be jealous on you, won’t judge you (unless you run over a fashion stylist).
For men it matters to be decent, don’t be ridiculous and according to the occasion, to be of course hot.