The grass is always greener on the other side. My wife and I have had many arguments because one of us wants to be involved with someone and the other gets envious and wants to say no but knows that it's not right.
If he's truly polyamorous, I believe he would be alright with you seeing other men. Being worried about what gender you date because he might not get to have sex with one of the genders sounds more like a swinger problem. Poly people don't always share partners and it's naive and insulting to the third person to think that would always be the case.
It sounds like you're both in need of a long conversation with ground rules and boundaries. Don't pursue the singer until you've worked out what the two of you are willing to do. Does he get veto? If so, this singer is out. Do you get veto? Can you see men? Are you going to want a V, a triad, an N, or some other mish mash of partner relationships?
Unicorns (girls that are single and will be with the both of you) are as rare as the mythical beast they are named after. So keep that in mind.
"Regret for the things we did can be tempered by time; it is the regret for the things we did not do that is inconsolable." - Sydney Smith
Kyle: 27 year old male
Katie (rymmare): 25 year old female
Kids: girl: 5 years old, boy: 3 years old