Minxxa, thank you sooo much for your reply. I felt so bad because I love them both and have been trying so hard to not make any "rules" but it just seemed like this one was a safety thing for the family as well. I don't have to see these people often, but when I do I dont want it to be uncomfortable. The others in the family say they don't care what others think, and f that is the case that is fine for them, but out of respect they should care enough what I think.
I don't think it is going t be a issue anymore as I have made my feelings known to both and both said they understood, I just more so wanted to make sure I wasn't being selfish or anything.