Point is that this is yet another case of taking a word with a well-defined meaning (from the Greek phobos, meaning “fear”) and diluting it to turn it into a vague term which, in this case, for the purpose of socially disparaging the person which is targeting.
Simply put: It is quite obvious that (to take another example) opposing gay marriage can be done without disliking anyone. Obviously. Because religion.
So, with that fact under consideration, what term would you apply to a person who opposes gay marriage, but has no “fear” of them, and also believes in equal treatment for homosexual people in any other sphere of human existence?