In my own mind I don't have too much trouble distinguishing who's a friend and who's a lover. Although I tend to invest a lot into the friendships I have as well to the point of appearing to be a threat to partners of certain friends. It baffles me a bit. If I was interested in pursuing a romantic relationship with these friends I would have attempted to do so when I had the opportunity.
I have also been known to have deep friendships in the past that have turned into lover relationships. The relationship with my husband being a prime example. I haven't ever experienced going from lovers to friends though.
I wouldn't cross the line from casual friends to sexual friends. For me it's a trust thing. If I don't know someone well I don't feel comfortable being vulnerable with them. For me sex is a vulnerable experience which is a really good thing with the right person/people because it brings you closer together. But it's really not something I want to share with most of the world.
Everything will be ok in the end. If it's not ok it's not the end.