I think it depends on the situation....
I spend almost every free moment with my husband. We check other people out all the time in front of each other. No big deal. HOWEVER, when we go out on a date, I don't and he tries not to (or at least does it more subtly so I don't notice as much).
It also depends on the people. He and I don't care. We know we both check people out and we enjoy talking about the people that catch our attention. It also doesn't offend us or make us uncomfortable because we feel very confident about our relationship and our dedication towards one another. Now, there are times that I will get after him when he is ogling a bit too openly and could seem like a creepy staring guy by strangers. That's more so people don't judge him though.
There are also times when I'm not in the mood to check people out (usually when I've chosen to be frumpy and am wearing comfy, unattractive clothes and didn't really do anything with my hair and he starts pointing out these women in short skirts and heels and nice hair) so he tries to keep it in check at those times, too.
My ex girlfriend, however, got PISSED whenever I or her husband would check people out. It also didn't help that he and I have the same taste in women and she really doesn't fit our stereotypical "type" (we generally like tall, blonde, booby... she was average height, brunette, small-ish boobs) so it really bothered her to catch us both staring at the same chick.
Pretty sure it's all in the other person's head. When I'm feeling kind of frumpy or unattractive and he points out someone that I perceive to be more attractive than I am, then I feel like he is comparing us and that I am losing. Not the case, he's just enjoying the view, but I let it bother me.
Being rude about it and letting attractive people walking by distract you from talking to or paying attention to the person you're with, not good, but it's natural for people to look at others (I think) so.. yeah. As long as you're somewhat subtle and don't let it negatively impact the time spent, why not?