Many people talk with me about their relationships. (I suspect they assume a happy poly that's not in a straight-jacket must know how relationships work.). Of the monos, almost every one expressing a sex drive disparity has a female with a higher drive. This makes me wonder if societally we've programmed women to think their default is much lower than it really is.
In any case, your drive is what it is. You'll have to manage it regardless if what your relationship(s) look like. Polyamory isn't necessarily a good answer to a high sex drive. Heck, poly takes a lot if work at times, and some of thise times you might just really be wanting a good and thorough fucking.
You might want to start by asking what it is you're really after. Are you interested in one or more additional relationships? Then poly might be your thing. Are you interested in one or more additional sexual partners? Then swinging (or a open relationship) might be your thing. The right answer is what works for you, but in what you wrote, it seemed to me like the emphasis was on attraction and physical desires and less on connection and emotional togetherness. Poly and swinging are basically two areas on the long spectrum of relationships, but each has its own quirks.
Realistically speaking, I see a lot of theory and little practice. The poly approach tends to emphasize communication. Thus, think about how you feel and what you want, and then talk it over with your partner. If things seem like they could work, take small steps until you are comfortable.
Best of luck.