The way I see your situation is that you are monogamous and that you want the typical white picket fence dream. He's been very clear that he's unable (or unwilling) to give that to you. So you have a decision to make, either stay with him and find it within yourself to truly accept that he has a real and meaningful relationship with his wife (even if it is very different than your relationship with him). Or you can move on and find a partner who is willing and able to give you their all. Neither of these decisions are easy and neither give you exactly what you want but that's the reality of life.
Everything will be ok in the end. If it's not ok it's not the end.