I can totally imagine being mono. I've had enough people tell me about how when they love someone, it "turns off" their ability to be attracted to new people in a romantic way. I get that part.
What people are talking about finding weirder is those who say they can be in love with more than one person, but because it's not often, they identify as mono.
I think a better analogy would be sexual orientation. (In my opinion polyamory is a relationship orientation, while polygamy is a relationship practice. So you can be polyamorous in a monogamous relationship or the other way around).
Anyway, my analogy would be, say, a female who can be attracted to another female physically and romantically, but very rarely. They're more likely to be attracted to males. They might identify as straight despite having some same-sex attraction, because it's so rare, she might feel like lying if she said "I'm bi" but then never met a woman she's attracted to in 10 years, and her friends could accuse her of pretending to be bi just for the attention.
Or someone who rarely experiences sexual attraction at all could identify as asexual even though they're actually demisexual or grey-a.
So I find it useful to think of it in terms of "rounding up". They might have some small poly tendencies, but they're unlikely to ever affect their lifestyle, and so it might feel counterproductive to identify as poly, since it reduces their prospects with mono partners, and probably will never come up (and if it does they might prefer not to act on it anyways).