Your experience on Polyamory.com may give you occasion to examine those fears. Note that not all fears are irrational; some are legitimate concerns about real inequities (or crossed boundaries) in a situation. But do a thorough self-analysis first, before figuring in the external triggers (and a fear/concern may be a mix of the rational and irrational).
Also remember that communication can always be improved: for more honesty, more clarity, and more consideration. It's always a good idea to work on/improve communication (both quality and quantity).
Can you describe your feelings, and the hard time you're having of it? Are there any triggers you can identify?
Love means never having to say, "Put down that meat cleaver!"