You are very strong... and that's a good thing. I agree that for now, at the very least, poly needs to wait.
If he has issues with not addressing your needs when there are friends around, I am sure it would be a major challenge for him to address the needs of two women he was romantically and sexually involved with, so I would say not getting into that situation is the best solution.
I am so sorry for what you had to go through... I know what it is like to feel like you have things to work through and you can't talk to the two people closest to you, but nothing I've had to work through has been half as major as what you've been dealing with - it doesn't compare.
I admire your courage, honesty and strength!