Our relationship definitely developed out of a good friendship with our partners. My husband and I have been poly in some form every since we started dating. He had a serious girlfriend and he and I were casual. Obviously the situation changed radically and now four years later we are happily married. We met our partners several years ago and liked and were attracted to them then, but decided that being friends was more important at the time. Eventually we found out that they were trying opening up their relationship and when that person didn't pan out, we asked if they were interested in us. They were.
I think that what I keep hearing is that when you date a couple, or anyone, actually, it's important that there isn't an expectation that it turn out a particular way. If you end up being more into one partner than your primary is with the other partner, that could cause friction if theres an expectation that everyone be on the same track. Relationships have to take different paces depending on the people and you don't want to force it. That said, I think it's awesome and definitely possible.
Oh, and good luck