hi ray. sorry that you're going thru this frustration. i don't think that ur SO really knows what is right for you. does he feel guilty? i think maybe that's the case with the "drag you into it" comment.
you said that you had an honest open long discussion. he knows how you feel. what is he going to do to help you feel better? i think he needs to do something. you seem to be sensitive to his needs, like giving him time to figure things out.
i hope that you can find peace soon.