PaperGrace, I do hear where you're coming from.. One of the biggest concerns I had when I was a "professional secondary" (for lack of a better term -- I wasn't, however, getting paid for it
) was that my loneliness would affect my perfectly good relationships with my partners. My solution (one that they in fact helped me figure out) was to talk to them a LOT about my feelings and ask them for suggestions and support while I worked through my issues. They were fantastic about it, too -- so fantastic that when I did find a partner who is now more of a primary to me in terms of time and energy, they welcomed him warmly and gave me the extra time I needed to nurture that relationship.
It's true that time is limited, but partners are partners for a reason, and one of the biggest is their commitment to supporting and reassuring us when we're dealing with tough emotions and decisions, no matter how much facetime we actually have with them. I can't urge you strongly enough to be open with your sweetie (and maybe even his other partners, if you feel close to them). Intimacy is born of openness, and so long as you're not expecting your partner to solve your problem but rather expecting him to listen, love and support you while you find your way, this should not affect a good poly relationship in a harmful way at all.
Good luck! It sounds like you're really working on things in the best way possible.