You can only make it work if you want to. It sounds like you have some serious soul searching to do to decide if this is something you can live with or not. The fact that he is being patient and trying to help you work through it speak well of him.
I would suggest reading up on the topic of polyamory both on this website where you can get personal stories and situations and in books/other online resources. Book/website recommendations from members can be found in this thread
. I would also encourage your husband to read up so the two of you can continue discussing openly.
Another thing to do is just examine yourself and your relationship with your husband to see if there is ANY cause for your fear or if it is based purely in insecurity. If it is just insecurity on your part (and there is nothing wrong with that), then your husband and yourself can probably figure out some way to set some boundaries on what is and is not acceptable on a short term basis to kind of "test the waters." If he is interested in BDSM and you are incapable of doing that, maybe let him attend a BDSM event or seek out a play partner (and by play partner, I mean strictly that - someone he meets just for BDSM things) to see how that works out. Then you are not toying with anyone else's emotions if it DOESN'T work.
You two definitely need to talk about you being able to seek out other relationships as well. Does he really feel comfortable with this or was he just saying to to be "fair?" Also, hopefully after reading a bit about poly you won't think that it's a sign that he cares so little. I would take it a sign of his trust in you that you will uphold and boundaries you two set, his faith in your love lasting regardless of what else is going on, and his desire to see you happy while he is also seeking his happiness. And on that note, honey... Men like all shapes and sizes... I am NOT a small woman, and I have never had issues finding men (and women) who want me. If you choose not to pursue this, that's fine. It's not for everyone, but please don't just dismiss it because of your physical appearance.
If you decide you are okay with testing it a bit, make sure you two discuss what is and isn't okay (boundaries - there are also threads out there about that) and set a specific time that you are next going to discuss it (I would say about a month or the day after he first acts upon anything or whatever length of time you think is fair to let him experience a bit) so that you can readdress how you are feeling, whether your boundaries still feel right for both of you, etc.