I am very private, too, and haven't had anything like this with my family, but something much less drastic happened with Gray's family. Some years ago now, she left for a long weekend to visit a friend in another state. A male friend. Meanwhile, Gray's sibling was visiting my in-laws, who live a short drive away, so I drove over to see them (and get a free meal
We are all chatting pleasantly when the subject of Gray visiting her friend came up. There's this pause, then one one of them says seriously, "and you are all right with this?"
"Um, yeah. Surprise, surprise, I talk with my wife about our relationship."
Ok, that last part might not have been out loud, but I think it was evident in my tone of voice. There hasn't been another "concerned talk" since.