It seems to me like, if you two are planning to stay married, and if he and you are both ok with it emotionally, there's absolutely nothing wrong with having a mostly-sexless-open-marriage versus a sexless-open-marriage. Love is love, and it sounds like you love him, even if you don't desire him. That is a valid way to feel, and it's up to you two to sort out where to go from here. There are many possibilities, and none are right or wrong as long as everyone is being listened to and respected.
Me, 30ish bi female, been doing solo poly for roughly 5 years. Gia, Clay, and Pike, my partners. Davis, ex/friend/"it's complicated." Eric, Gia's husband. Bee, Gia and Eric's toddler.