Good question! I think that your man sounds very monogamous and is under the understanding, because he lives it, that there is only room for one in ones heart. Perhaps he cares for you and likes having casual sex with you, but doesn't love you and is reserving that special love for someone else. Now he realizes that it has been fun, but that special love is not getting it's needs met.
My boyfriend is also Mono and this is how he describes it... he can chime in whenever he wants if he wants and say more or dispute what I am saying, but he has a special love for just one. Not that anyone elses love for more than one is any less special, this is just the words he uses to describe it for himself. That love he has belongs to me and only me. He doesn't have a need to find another to give that too. I am his forever. We all are now, husband, child, my girlfriend, my extended family. He has chosen me regardless.
Similar to what Quath said, I think that your boyfriend needs to decide if he is going to chose you regardless of what others say and regardless of your husband and life beyond him. I would wonder if he is having second thoughts about his involvement with you. Sounds like he wants someone of his own, not to share. That is fair. Sad for you, but fair enough.
Anyone want to be friends on Facebook?
Send me your name via PM