"He has seen the good, the bad, and the ugly."
Well, once one has seen all three of those sides, I guess little doubt remains in one's mind as to whether one wants it.
I think he is trying hard to see that this is your decision not his, even though his very presence in the marriage depends on that decision. He's probably convinced that in any poly situation, he just won't fit in.
Yep, one cannot be too careful about a decision such as this. It looks so innocent at first glance ...