You might be poly--the distinction between "poly" and "swinger" isn't black and white. It's more of a sliding scale or a spectrum.
For example, some poly people do start out as swingers, but eventually realize they want more of an emotional connection to their play partners, so they move more toward polyamory and seek to develop loving relationships with their partners.
You could identify as "non-monogamous" until you figure out exactly what you want. The key, I think, is to be totally honest with women you meet.
It does sound like you are looking for one primary girlfriend who would be happy to have some type of open relationship with you. Don't give up! Such women exist (and are unhappy with men who don't want an open relationship!)
I would also recommend that you think about what you are looking for in a primary girlfriend. Do you want a woman who is bi, and you and she will be seeking other women together? How would you feel if your girlfriend wants to date other men besides you? (If you have a problem with that--why, exactly?)
These are useful questions to ask yourself. You have to know what you're looking for in order to find it, I think.
I think it's perfectly reasonable to feel that your sex drive is so high that you need multiple partners. I think there are a number of dimensions to understand about one's sexuality. A lot people think that your "sexuality" is just where you fit on the gay-bi-straight spectrum, but there are a lot of other dimensions: how often you want to have sex (whether alone or with others), how much variety you need in terms of one or multiple partners, how frequently or strongly you experience attraction to other people, how kinky or vanilla you are in terms of what sex acts you want to do...oh, and also, how much emotional intimacy you need or don't need from your sex partner(s).
Anyway, all that stuff is what every person needs to understand about themselves.
You should seek out communities (whether swinger, or poly, or both) where you can learn more about these issues and become comfortable with being who you are.