Welcome to the forum!
I think that there is a significant number of poly folks that came into poly through a similar path, and the road to repair has had to be slow. I envy those who knew enough about themselves and who weren't concerned with what society thought enough to go down their path, and not have to try to hide how they felt from others (and themselves).
You do recognise that what you have in place right now are "training wheels" - that is hugely positive - the trust needs to be built up, and that is a slow process after it being betrayed in such a fundamental way.
A hard journey, but one it seems you are approaching well.
"Listen, or your tongue will make you deaf." - Native American Proverb