Were your husband's needs being met? Did you all talk and talk and talk before transition from swingers to polyamorous. There is a difference and the approaches are different. Is it possible that with you spending 3-4 days with this man that you were not giving your marriage the attention it needed?
This sounds like a case of the grass is greener on the other side. Have you and your husband sought marital counselling? Are you making your husband feel loved and special or are you all wrapped up and consumed with this other man? You can be poly and have a mono spouse.
I, too, am not inclined to tell you what to do, but I do think you should talk to your husband and not tell him what you are going to do. Saying I am going to spend most of my nights with this chap is not the right approach. You need to ask him what he needs to be comfortable and what he needs from you. If you know the marriage is over, end it now and prevent more heartache.
Ry - Me. Panromantic demisexual with a history of polyamorist tendencies. Married to...
Matt (Hubby) - The once distant stranger that I complement beautifully. DH of 13 years and father of our four children.