Don't forget, the question "Is this person even worth it?" is a question you can ask yourself, too.
If you don't want to be in a relationship with a poly person, you don't have to be. You can leave him and eventually find a mono person who might be more compatible with you.
If you truly feel you can't trust your partner not to sleep with 100 people while pretending you don't exist, it's certainly not worth being with him.
However, it sounds more like these are your own irrational fears. You know he's not going to sleep with 100 people.
You also know he's not going to tell people he's not in a relationship. What would be the point of that? He's going to a convention EXPRESSLY so he can tell people that he's in a committed relationship AND that he wants to date other people. (Since that's not something you can say out in the "real world.")
I think it's a good idea if you go to the convention as well, so you can see what it's like, maybe meet some poly people and see that they are not awful people.
Single, straight, female, solo, non-monogamous.