Honestly, the idea that a man would ever say to a woman who he loves and wants a romantic relationship with that he wants to date other women because he wants "more attractive" lovers rubs me completely the wrong way. I am not an underwear model. Neither is my husband. Neither, for that matter, is the guy I hope to form a secondary romantic relationship with. But they are both gorgeous men to me, partly because of my own "shallow" physical preferences (love me some blue eyes and dimples), and partly because they are wonderful men deserving of affection and respect. And they both treat me with affection and respect as well as demonstrating their physical attraction to me.
If physical attraction is not a part of your relationship, that's fine. It doesn't devalue the relationship in any way. But I get a bad vibe from the idea of a guy who is WILLING to sleep with you, even though he really wants someone he finds "more attractive." I don't even need to see you to know that you are worth more than being someone's back up plan. I might be way off base with this assessment, and if I am I apologize. But I've not always been so lucky to know such great men as I do now, and I've known both men and women to do worse.