I like to give it at least a couple dates. It takes me a while sometimes to know how I really feel about someone, and I don't always think first dates are a great way to judge attraction. After a 2 or 3 dates, though, I think people usually know what they want, and at that point it should be communicated.
First date with Fly, there was zero chemistry or attraction. He was too short, too old, kind of dorky. I actually had a friend call during the date, and used it as an excuse to cut it short. I started dating someone else, but kept chatting online with Fly over the following month, and he really grew on me. Our next date, about 5 weeks later, I let him relieve me of my virginity and we've been together for over 7 years.
I'm glad I gave him another chance.