i m sorry for your situation Invi, you are a good mother but being a stay at home mum while you are not married is really dangerous. Your "bf" can kick you at anytime and leaving you penniless. Yeah i guess you had done the right thing to leave...... how the heck can you live in a place where you had no friends while your bf controlling the finances i mean he is making you more a prisoner rather than a wife or mum............
I hope the best you can leave from him (even though it is a poly forum but his attitude in having outside relationships without ANY communication is a really big red flag). I may even suspect that he may be cheating on you under the name of poly (or swinging since he is more sex-oriented).
Best of luck, dear member and may you deserve a better treatment from a more humane person than your "soon to be ex bf"
p.s really disturbing at your current situation