"I'm not saying he doesn't want to be with me. He does make an effort with me. But, deep down, I know that many times, he yearns to be with her and I don't blame him, I see how happy she makes him."
So he doesn't *always*
want to be with her, but you suspect that he doesn't spend *as much*
time with her as he feels like spending.
It's possible there's a certain amount of NRE (New Relationship Energy) involved between him and her. NRE takes awhile to wear off, and it can be rather blinding in the beginning. He may be trying to resist the effects of NRE, in which case it's not entirely inappropriate for him to "push himself" when it comes to spending time with you. Later on, the logistics may still be similar, but they may "flow" a little more naturally.
I kind of get the impression that *control*
is an important thing for you to have within the relationship. Letting go of a little bit of that control may seem a little scary. But there may be rewards if you can do it.
You have a complex situation and will need to study things from every angle as you go along.