Talk. Talk. Talk. Make absolutely sure you two are on the same page; don't assume.
Where does your sense of guilt originate?
Be aware that sometimes casual flings turn into some thing more intense. What are you going to do if you get attached to your new sexual partners? Some people can maintain casual; some people can't. While you probably don't know into which camp you fall since this is new, it might be a good topic for discussion in case either of you end up feeling intensely about another person.