You are worrying way too much. You're putting a lot of pressure on yourself and making the situation unbearable. You need to just talk to him. Make sure he knows that his opinion is respected and if he can't deal with it, he has a right to feel that way.
Your anxiety will only continue to build until you do something. Out with it!
"Regret for the things we did can be tempered by time; it is the regret for the things we did not do that is inconsolable." - Sydney Smith
Kyle: 27 year old male
Katie (rymmare): 25 year old female
Kids: girl: 5 years old, boy: 3 years old