I don't think you're selfish in the least. What you are doing is taking nothing away from him. You are not depriving him of anything he had.
Love is that condition in which the happiness of another person is essential to your own...
Robert A. Heinlein
Me: female, bi, (formerly hinge of a vee)
with FirstBoyFriend (FBF)(moderately long-distance)
and no longer with CurrentBoyFriend (CBF)(who now lives in a house far away-with stairs I can't climb)