From your BF's perspective, it sounds like understandable fear to me. He found himself in love with a polyamorous person - not just any polyamorous person, but the wife of his best friend.
When fear kicks in, what do we, as humans, do? Often, we seek a safe place. Fear is so often seen as a negative thing - but I think it is actually a developmental, human, red flag - survival instinct. What stops us from running out blindly across the road to go and get food from our favourite restaurant when we are starving? Fear of being hit by an oncoming car. So we stop, look both ways, consider our options, and cross when it's safe.
Emotions work in the same way - yet, often, we do not realise it. (Anecdote): throughout her young adult life, my girlfriend turned to copious male attention when she is feeling a personal void. It became her crutch; something that she learned to do, subconsciously. Recently, I went home to the UK after a three-month stay with her. She felt a void. What was the first thing she did, literally within hours of me getting on the plane? Go online and talk to a guy. The rest of the week? Filled her time talking to various guys. Her safe place. What is the first thing someone who has always believed in and followed a monogamous path will do when they fall in love with a person who cannot meet this future goal? They will feel fear. Fear will cause upset. Upset leads to reverting back to our crutches - reverting back to what we know, to find comfort. For him, this is monogamy.
Seeing that and understanding that might help to soften your (understandable) feelings of frustration, confusion, hurt, anger and resentment. It is much more difficult to feel angry towards a person when you pick apart their motivations and see their fear. Seeing it helps you to feel compassion.
Now, he has come back to you. That means something. Yes, it could mean that he's been playing the field and you are the next best option, or something to fill the gap whilst he searches for monogamy. More likely, he is genuinely in love with you and whilst his head and his crutches are telling him to run, his heart keeps pulling him back.
There is an idea in poly that 'love is just love'. It works in whatever way it works. But... that only works... WHEN it works! (does that make sense?!) Yes, the two of you feel love. Yes, you could have a long-term relationship, with those feelings of love, where both of you are happy. But if you want one thing and he wants another... it's a love that is just love... not a love that works. To me, love that works (practical love) is the kind of love to pursue in life.
So, let's say he is monoamorous by nature. Let's say he is incapable of loving more than one person; or simply prefers to love only one. He is not strictly monogamous by nature, since he is happy to date you whilst you are married and also seeing other people.
The next step is to ask - "what do you want?" There may be a middle ground that he would be happy with. There might not be. If, in the long term, he wants marriage, children, one home, one love, then... unfortunately... as GG said... you are shopping in the wrong store.
(Anecdote #2!) A few years ago, I would never have thought I'd be happy in a relationship with someone who was married. I never entertained the idea - it just wasn't a thought. I was vehemently against cheating and thought monogamy was the only option available. Yet, I met a woman who was married and poly, fell in love, entered a V relationship... and can honestly say... yes, I'd be very happy in this relationship for the rest of my life, even though I previously only thought I'd be monogamous. I actually also sometimes consider myself monoamorous - I have felt different levels of love for more than one person at a time; but I am very happy with the thought of just loving one person and sometimes prefer it, regardless of my girlfriend's activities with others. So, I can relate to your boyfriend's feelings on that one.
So... have you asked him about that? What if he would actually be happy with some sort of V between himself, yourself and your husband? What if he would love that, but doesn't see it happening because you also have other relationships?
And what about you? What do you want? Do you want to be open poly, closed poly? Would you like him to be a serious boyfriend, even if that meant cutting out your other relationships?
If your relationship has been called FWB... he is surely only going to act in a way that fits that label - regardless of what he says about his emotions. I see it so often that 'we don't need to use labels' etc etc... but I don't see the problem with it. I think it actually helps to define appropriate behaviour and get needs met.
I don't think you should let your feelings lie - I think you should talk to him and see what he has to say.
Me: (30f) open poly
Serious long-distance relationship with GF (40f)
Casual FWB with Descartes (27f)
“Peace comes from within. Do not seek it without." ~ Buddha