You may be right
If it's another employee I would tread really carefully. That's a dynamic that cAn go sideways very quickly, I would also agree with the meeting requirement. He needs to devise if this is going to be a quick fling or more. If its just a quick fling (translation fu*k) that would not define my definition of poly.
Then it's up to you. The other option is to keep him busy for the rest of the week, plan some things to see/do. Then this will be all over. Just thinking out loud.