I just talked with him for a bit on the phone. Apparently he is confused as well. Neither of us have ever gone down this road, and we have our own insecurities and fears. He is worried that I will change my mind and decide not to explore polyamory, I was worried he would try to rush me.
We didn't have much time, and will get together after she leaves to talk some more. Although not everything was brought out of under the rug, we both felt a little better knowing that our fears were not justified.
I will update this once I talk with him. Thanks for your help. I cannot tell you how much the people on here have helped me process this stuff.