You have two partners and you want to hold him to an empty promise for "one at a time" made when he had none? You seem to want to decide a hell of a lot for him, instead of supporting him in the process of him deciding what he wants and what he will do.
When speaking of various forms of non-monogamy...it ain't poly if you're just fucking around.
While polyamory, open relationships, and swinging are all distinctly different approaches to non-monogamy, they are not mutually exlusive. Folks can, and some do, engage in more than one of them at a time--and it's all good.