I know where you are coming from. When my wife and I first opened up our marriage I only agreed because I felt it was my only choice if I wanted to stay with her because she wanted to sleep with other people and was afraid she would cheat if we weren't in an open relationship. During that time I dealt with a lot of jealousy and resentment because I felt like she didn't love me and was willing to sacrifice me so she could be with other men. But my wife and I talked it over and she addressed some of my concerns and made me feel better. I still feel jealousy sometimes but I have mostly gotten over it.
You really just need to talk things out with him and tell him how you're feeling and find out how he's feeling. If he really loves you he won't force you into something you don't want. Just don't pretend to be okay with it when you really aren't because you just end up being hurt worse. Trust me I've been there.
ME: 23 year old slightly bi-curious male. B: 19 year old bi female and my wife