I agree - what really matters is how he behaves with you. The words have little meaning in and of themselves.
I am a woman, and I feel that I need to be ready to accept the responsibility associated with the words "I love you". I do feel the emotion towards my bf, and I want to be sure that I can uphold what I think comes with "admitting" that. - It seems like it is a guy thing, but not so.