Past abuse or no, I'm sure you are quite capable of having feelings for reasons that are not only due to having experienced abuse. It sends up a big red flag to me that he would write off your feelings as being only due to your past without examining if he is doing anything to contribute to them in the present.
Careful careful. While having experienced abuse as a child can alter your responses as an adult, it can also alter who you naturally gravitate to for relationships. It can have you gravitating to people who feed into your vulnerabilities and keep you from healing and moving on to experiencing relationships that are healthier for you. And a person who is healthier for you won't blame every feeling you experience on your past. Blaming it all on your past can be a way to not slow their roll on to what they want to happen or take responsibility for what they might be contributing to the emotions you are dealing with TODAY.