If I'm understanding right: You don't see Bob often, which you're okay with because you have a lot of independence; however, you feel that since your opportunities to see Bob are so limited, you should get just a little more privacy (i.e., have it be "just you-and-Bob time"), and, you should get a little more opportunity to see Bob when you can.
I think those are reasonable things to want, even if Bob and Ann have a different arrangement that is incompatible with what you want. Perhaps it just comes down to a question of whether to keep the relationship with Bob going, if he and Ann are unwilling to compromise a little more, but I would at least discuss your feelings with Bob first and find out if there is a chance of some compromise.
That's how I see it anyway.
Love means never having to say, "Put down that meat cleaver!"