Actually, I think its a reasonable concern. You don't want to give it three months, six months, a year, and then find that you're in so deep that you couldn't end it without wrecking your heart, and yet you feel you can't possibly be happy like this because you can't have the sort of relationship with him that you want.
So, if you're not comfortable with the idea of getting too involved with a person who already has a primary partner, then you can choose to keep the relationship "secondary" in your lives -- have fun together, take it as seriously as you both want to, but don't start building your lives as a unit, dont blend finances or move in. A relationship can be serious and committed without taking on all the trappings of a traditional two-person partnership. And, that way, assuming you continue dating other people, you'll be more likely to have the space/time/logistical-capability to form a primary partnership with someone else in the future. Someone else who doesn't mind you having a boyfriend of course, but obviously such people exist or else none of this would be possible.
Alternately, perhaps there will be the possibility, at some point down the line, of you and his wife being co-primary with him. You could all share a home, share chores and bills, have kids with him, even have a ceremony and write up a contact that mimics a marriage contract -- people have certainly done that. Obviously, that would require you and his wife to get along well, and for both of them to also see that as a future possibility.
Either way, the first step is a conversation. Ask him where he sees this going, and what he considers possible/preferable, with the understanding that you're just feeling out the far-future possibilities, since this is all new territory for you.