Opalescent put it better than I could. I completely agree.
Since you say you've lost self-respect, now if the time to start re-building your self-respect by choosing self-respecting behaviors. Do not go along with his crazy plans and schemes. Do not give in to his manipulations and gaslighting. YOU ARE NOT CRAZY.
Leave the situation and start taking care of YOURSELF. I understand that you love him so it's going to be really, really, REALLY difficult, but he is not showing you love at all in any of this. You WILL be okay on your own (and it's okay to fall back on friends and family as needed also).
Pan Female, Hinge in a V between my mono (straight) husband, Monochrome and my poly (pan) partner, ThatGuyInBlack