It's a hard call because it's apparent you and hubby do have some serious self-work ahead of you. Regardless of his reasoning, he has a potential legitimate reason for concern about whether you can have the time and focus (maybe most important) necessary to work on yourselves and your marriage.
It's also only fair to share this conundrum with your BF. Ideally, he may be able to help you both in some way. But on the other hand, it may be more than he's capable of or willing, depending on the nature of your relationship with him.
I guess you can try to juggle it, but I feel you need to be prepared for some rough days ahead. Hopefully I'll be wrong !
If your BF and husband were closer, it would be much easier to form a team. You don't mention anything about that so we have to assume a widely spread V ?
You're going to have to set some priorities - and stick to them.
Good luck !