It's interesting - I was listening to NPR today on the way into work, and they were interviewing W. Kamau Bell about his show's upcoming season on CNN. Anyway, they asked him about whether he considers himself a feminist, and he gave this answer (paraphrased from memory):
"I don't think any man can simply call himself a feminist because I think that's something that has to be awarded to you by a woman due to your actions. You can't just decide you're a feminist and call it a day."
There's some compelling reasoning there. Often, privileged classes describe themselves as allies, when at best they're only superficially supportive, or at worst, actually hurtful to the movements they claim to be a part of. Going down that line of thought, I can see why acknowledgement by the oppressed group should be the signifier of a true ally, and not simply a Twitter bio mention or what have you.
On the other hand, I think that many men distance themselves from the "feminism" label because of dumb stereotypes about girls on Tumblr with blue hair, contempt for men, and annoying attitudes. Maybe those people exist, but like 99% of the feminists I've met in life are just regular women who want the same fair shake that I get. So why wouldn't I want to align myself with them? I'm proud to say that I want equality and respect for all genders, so giving a cagey "Oh, I dunno, labels and such" answer to "Are you a feminist" feels... dismissive.
I suppose the most honest answer I can give is that I certainly hope I'm a feminist. I aspire to live my life as one. I think it's a noble pursuit that deserves our attention and respect.
Does that make sense?