If you want to have a relationship with this person too, what right has your partner to stop that? He has no right to forbid you from contact, unless he is being selfish and trying to deny you something you obviously want.
In my opinion, poly relationships work FAR better when there is an open line of communication between all involved. When there are blocks to it, it opens it up to lots of miscommunication, either accidental or deliberate.
"Listen, or your tongue will make you deaf." - Native American Proverb