Yeah, polyamory means several loves, and I interpret it to mean several loving relationships, but I don't see why friendship wouldn't apply. It's a form of love too.
Sure, some people use "FWB" to mean "some stranger I have sex with but FWB sounds better" but there are also people who use "my wife and I are polyamorous" when they mean "I'm cheating on my wife and there is no way she would be fine with this, but poly sounds better", so...
I'm not into casual sex but I can imagine having sex with a close friend. It's obvious to me the two are different.