You seem to have a clearer view than most of the situation. That's good. However, it's not so much if you feel taken advantage of or used. If you don't feel that's the case, that's a positive.
However, think of it this way when I suggest again you move on from this sexual relationship. Even for poly folks, there is only so much time and space and energy in a given day, a given week, a lifetime. This man is not going to be a partner, long or short-term. I realize you are not looking for something long-term or committed right now. However, he's not truly available to you. He's tied up in some very disordered relationships right now. He doesn't love you. Yes, both of you may get a lot out of the relationship - I don't doubt that. You are emotionally involved with him and love him. There is nothing wrong with that; it's freely given and you've been clear you don't expect anything along those lines back from him. But he is clearly taking up quite a bit of your emotional, mental and physical energy as well as some of your time. After all, you wrote a post about him to strangers on a forum! He's (the mental version of him in your head anyway) has claimed a certain amount of your time, energy, space and love. Long story short, consider cutting him loose so you take back your energy, time and emotions. This action may open up other opportunities, other possiblities for relationships.
A friend came out of a very unpleasant divorce which was the result of a poly quad exploding. Poly drama everywhere. She did not want to marry again and so dated casually for quite a bit. However, when she reached the point where, maybe, she might want someone for a more committed relationship (if not marriage which was still scary), she realized she also had to end her more casual relationships, her FWBs. Not because there was anything wrong with them, or with her. She did not want a committed relationship with either of them. Possibly she could have continued as FWBs until she found a more committed partner or partners. However, staying in the casual situation took up time, energy and space that she could not use to go find what she really wanted in life. A few months after ending her FWB relationships, she met the man who she would eventually marry. I'm not saying you are in this position now but someday, you might be.
And if the answer is no, I want to continue - which is cool, you're an adult
who can certainly direct your own life - ask yourself why. Why stay in this setup with this particular man in this particular situation? The answers, whatever they turn out to be, will tell you a great deal about where you are mentally and emotionally in your life in general, not just specific to this relationsihp.
I wish you the best of luck and I really, really wish the situation doesn't develop into a clusterfuck. (Forgive me my cynicism. We read about a lot of stupid situations on this board that originate in one or more partners cheating. I sincerely hope you avoid that.)