What does your husband think? You said you've spoken to him, but didn't tell us what his opinions are.
I am also married and have a boyfriend. My husband is thirteen and a half years younger than me, and my boyfriend is ten years younger. My husband and I have a six year old daughter.
The relationship with my boyfriend became intimate about three and a half months ago, and has only gotten better over time. We are all great friends and it's been a fantastic experience so far.
So, as far as I can tell, this sort of thing can work out great so long as everyone is open and honest and caring.
Read read read this forum. There are a lot of great stories here and links that will help you get your head around what's going on.