Sometimes it's just like that
I can't speak from experience as I don't have kids, but I've heard that sometimes married couples fall into the role of domestic partners. They see each other more in the role of parent, reliable provider, etc, and over time grow apart a bit sexually.
As long as you both enjoy your marriage, and get the sexual satisfaction you need with no one's feelings being hurt, it makes perfect sense to me.
As for not being upset, that makes sense to me too. My fiancee considers all of my random sexcapades to be hot, and doesn't seem to have a jealous bone in his body.