One thought occurred to me that may be worth mentioning.
It's important to not confuse TRUST (and respect) with poly, sex or anything else. Although they can be connected they are not the same and have to be looked at and dealt with separately.
We all get our trust violated at some point, some more seriously than others. From little things like what money was spend on to something more substantial like the affair you note. Once that trust is violated there's going to remain a certain amount of suspicion and skepticism for a LONG time - maybe forever. You have to build it back through a string of things where your judgement is tested and proves solid.
Now, that being said, living a poly life has it's own set of complications and the trust is only a part. In my experience only, I've found that once something like this finally comes into the light (affairs etc) and you survive that initial shock and all proceed down the road together, I feel that there will be a much higher degree of openness and therefore potential for trust. And validation of that trust and better judgement. So it can actually help rebuild and reinforce more honesty and openness.