Can I clarify: Matt - did you have these feelings of wanting Si around less before before becoming romantically involved with her? If not, I think it will be inordinately difficult to separate your strong feelings of guilt from your feelings about Si in general. Even though you didn't actually do anything wrong, you feel guilt about what transpired, and those feelings are what counts. It is extremely common for people in such a situation to feel adversely to the person they "cheated" with*. Seeing Si reminds you of those feelings, and makes you uncomfortable.
That may not be the entirety of the issue, or even the majority of it, but it is likely a contributing factor. If Si is important to your wife, it may be wise to just take one step at a time. Have the break-up, deal with the fallout, and see where life lands. It sounds like you have let all these emotions bottle up inside until now you just want to throw the whole thing away and not have to deal with it. That may be the easiest solution for you, but it also might cause unnecessary pain to your family. Not just Si and your wife, but if your kids have grown accustomed to having her around it will be painful to them as well if she is totally cut out of the arrangement.
* I know you didn't actually cheat, but it sounds like you are experiencing similar feelings to those circumstances.