Definitely need to speak to a lawyer and research laws for where YOU live.
A friend in Washington spent 2 years in a NASTY custody battle where there was no abuse. But the state removed her rights "temporarily" to "assess the risk". That lasted a year. By which time there was a case for "maintaining him in his current living situation. Meaning she got visitation only. In a dynamic where there was NO abuse. Just that the poly gf and dh decided to settle down together and end being poly. They got her parents to testify against her STRICTLY due to her being bi and poly.
The risk is there.
Where i live, in Alaska, if there is any history of abuse (spousal or child) the abuser cant get custody.
Each state is VERY different.
"Love As Thou Wilt"