The weekend ended up not being as easy or cheerful as I'd hoped, but it was productive in getting the three of us talking more, and left me with more hope for things. The main issue for me is that of Ginko and I having enough time/focus for each other even when J is around. All of us are working on making it possible for J to be able to be at our place and us at his while we go about our day more freely, closer to how we would if he weren't there, with him occasionally joining us.
Right now I'm looking at how to feel more at home while at J's house, looking at what I can do in the area, what friends I can see more easily since they live closer, and so on, basically looking at how life can be more rewarding with a 2nd place to call home.
If the issue of time/focus could be settled, everything else seems manageable.