Damn, sounds like you have hit one of those bumpy periods in life.
When things kind of suck - it's natural for the fantasies to abound !
We all need and want something positive and exciting in our life when things are tough.
But I'd suggest not putting much thought into 3rd (or more) parties right at this point. You and your S O have got to pull together for each other and your family. That's kind of priority one.
Living a poly lovestyle can be a lot of work - especially when it's new. Taking on that extra wouldn't be fair to either of you - taking away your focus from things more important. And it wouldn't be fair to any other parties either because you have too much on your mind to focus on the relationship.
Go out - buy yourself some little treat to get a little positive going and set the new relationships aside for now. At least that's what I would do !