If I recall correctly, your counselor is poly-friendly, right? I ask because I think this re-examination and question your counselor wants you to ask yourself is really based on the fact that poly doesn't work if a marriage isn't stable and in good shape. She (is it a woman?) would be remiss, I believe, if she didn't do all she could to make sure you two are on solid ground before encouraging any sort of exploration of polyamory. And it has only been one session alone, and one together, correct? Your counselor is also just beginning to know you. Find a way to be a little more patient, though I know you feel as though you've been chomping at the bit for far too long now. The process of getting to know yourselves better can only benefit you both.
A few other points hit me when reading your post.
One is about developing friendships with women and the possibility of falling in love. I say, what's wrong with that? I think it would be wonderful to have a beautiful deep friendship with a man, even if it never became sexual. I met a male-female couple last year who are pretty much best friends and love each other very much but they're not romantically involved. For some, that would still fall under the poly umbrella, I think, because it is about love. Not every love needs to be consummated with the physical act of sex. It could be a great way to enter into poly if you do find a friendship like that. And you never know where your wife's feelings about it will be if some day in the future you have a close woman friend and wish to take it further into the physical. In other words, having the go-ahead to develop close friendships with women could be a blessing, not a booby prize!
The other thing that stood out for me is a pattern I'm seeing emerge from your posts. It seems that when you do talk about what you want, either with your wife or the counselor, you apparently acquiesce or agree to something, and then rethink it and rail against it, pretty much feeling like you've agreed to getting the short end of the stick. I wonder if this is a behavior you have adopted around issues that keeps you feeling like you will never be satisfied, or that you are being forced to compromise yourself more than you really want to. Or it could just be that you personally need to take some time to let things sink in for a bit before you make choices, so you can gain more clarity.
I don't know how you could do it differently, other than either giving yourself permission to fully voice all of your concerns, desires, etc., in the moment, rather than later after ruminating about it... OR... ask for time to think about things before you make any agreements. Perhaps even asking for time would prompt others to ask you "why?" and you would be able to speak up for yourself a little more.
All the things you brought up as concerns are valid and good stuff to bring up in your nest counseling sessions. It is quite evident that you are a thoughtful man, though feeling frustrated right now. I knew someone who used to say that if human beings weren't meant to learn how to be patient, we would have been born as mosquitoes or some other insect.
It is a huge step you want to take with your wife, and it requires all the patience and "due diligence" you can give it. It is obvious to anyone reading your story that the love is there. So, you're in good shape. And remember, you are just starting out! There are so-o-o many stories I've read here of couples who say they jumped into poly too soon, and then they had to deal with monumental problems with jealousy, betrayals, esteem issues, and all sorts of dramas.
So, take it easy, go slow, keep breathing, and know you will have more chances to examine, confront, and express your needs.