I'm sorry, but I don't see an answer to your predicament...
If your husband does not see the good that FL did for you then and actively hates him, then, even if he would be okay with a poly arrangement, it sounds like he would not be likely to be okay with a poly arrangement with this particular guy.
Someone can be part of "who you are", and important part of your history, someone that you love and STILL not be the right person for you to be in a relationship with.
If FL's marriage is not rock solid (i.e. he is married to a woman he doesn't love) then he is in an even worse position to be considering a poly relationship.
There is nothing wrong, in my opinion, on loving both. What would be wrong, again in my opinion, would be acting on it without the knowledge and consent of EVERYONE involved (you, your husband, FL, FL's wife). We cannot control our feelings, we CAN control our actions.
Just my take. Sorry I don't have a brilliant solution for you.
PS. Whether or not FL would be comfortable with having another girl in the mix seems to be an irrelevant fact...maybe he is more open to being poly than your husband but that doesn't seem to be the issue.
Me: poly bi female, in an "open-but-not-looking" Vee-plus with -
MrS: hetero polyflexible male, live-in husband (23+ yrs)
Dude: hetero poly male, live-in boyfriend (4+ yrs) and MrS's best friend
Lotus: poly bi married female, "it's complicated" relationships with Dude/JaneQ/MrS (1+ years)
+ "others" = FBs, FWBs, lover-friends, platonic G/BFs, boytoys, etc.
My poly blogs here:
The Journey of JaneQSmythe
The Notebook of JaneQSmythe