There is a different balance that's right for every relationship, whether it's primary or secondary. I agree with the other posters, if this balance is not working for you, you should speak up. His excuses seem pretty lame and his behavior seems perry thoughtless. On the other hand, he's far from the only guy to act this way. If he won't reform, the only other option is for you to try to give yourself a little distance, invest less, find other things/people to focus on, etc.
Me, 30ish bi female, been doing solo poly for roughly 5 years. Gia, Clay, and Pike, my partners. Davis, ex/friend/"it's complicated." Eric, Gia's husband. Bee, Gia and Eric's toddler.