Sorry to hear about the setback. It seems to me that Matt is determined in his mind that your relationship with Si should not appear on his radar. That only works if your time with Si is limited to when Matt is at work, essentially. It seems to me like that is a difficult proposition, but Matt is indifferent if it is, and if it means you and Si never get to see each other well that's fine with Matt. I don't know if any of that will change in the future, but probably not for a long time if ever.
Love means never having to say, "Put down that meat cleaver!"