Firstly how do you know you're mono and can't love more than one person?
Secondly I think it's important to know how old you are? If you're young and wanting to have a family I agree that this may all be in the "too hard" basket.
But if that isn't an issue I don't really see what your problem is, except that you seem to have decided you're mono, need to have a mono primary and that all their problems are your fault.
If his wife was still really anti I could understand, but she's come around. Poly relationships are always going to be more complex than mono ones simply because there are more people involved. I think the break-up of his wife's other relationship is what caused her upset, not you and she had to work through that which is perfectly understandable. It sounds as if you're projecting the old pain from your previous relationship with a married man on to this one.
In situations like this I think that it's best to take a look at your basic long-term wants and needs. I wanted a deep and committed relationship with someone who would partner me through life. That didn't mean he couldn't be poly and it didn't mean that I couldn't choose to be mono.
It sounds as if you need to have a discussion with this couple about primary and secondary labels. These labels are helpful on a superficial level but in reality as the relationships deepen they are made redundant. Discuss the concept of being co-primaries. I love the saying "If you have to move a primary says when do we leave, a secondary says when can I visit and a tertiary says it was nice while it lasted. It sounds like you and this guy have a primary kind of love and maybe the reality of that has also come up for his wife. It doesn't sound like she couldn't handle it, just that it requires work.