It's a big leap from 'she's struggling with jealousy' to 'she might hate me or already hates me'.
Meetings like this are nerve wrecking, sure, but is it really so nerve wrecking that you are considering to give up on what otherwise seems like a promising new relationship, not to mention the possibility of meeting someone interesting?
If it were me, I'd take the chance. But I guess a lot depends on how much you like him, and how invested you are in making things work with him - invested enough to do something that feels like a challenge, or not?
early forties, straight.