I'm seeing red flags about. He sounds a lot like my douchey ex-boyfriend (before I married). Anytime he thought I was flirting (thought, when I never did) with other guys, he would accuse me of cheating on him, right off the bat, and I would chase friends off to appease him.
The fact that you were his first girl friend tells me a lot about his jealousy. Also the fact that he said it's YOUR fault he married so quickly is another huge red flag. Another key to his jealousy as well.
If he's saying things like, "Well, early on in our relationship, you did X, and I didn't get to, so I want to, but you can't now" is yet another red flag. We call this behavior "retribution" or "revenge". Is this how loving spouses treat their past issues?
His jealousy (even if it has gotten "a little better"), to the point where he has chased off friends of yours in the past, does speak loudly about him, but the fact that you encouraged this (with good intentions of course) also says something about you. It says that you may be willing to do more than your fair share for his sake and his feelings, when really those things are HIS problems, and his alone to deal with, at the cost of your own emotional well being.
I'm basing this off my own very similar experience, and in hind sight, I was quite a bit to blame for my unhappiness in that relationship because I didn't stand up for myself and walk away sooner.
Would I encourage counseling? Certainly, there's no harm in trying, but you need to sit down with yourself and get a clear, REALISTIC, expectation. How do you want your marriage to look? Is that reasonable for the average person to achieve? How "happy" do you want to be? What will that require you to do for the marriage, and what will that require your spouse to do? Most importantly, if your marriage stays near the level of where it is, your daughter is involved. Do you want to teach her that it's okay for a man's jealousy to run a woman's life to the point of sacrificing her friends and loved ones? Picture you being your daughter; is that the life you want her to live?
Best of luck, and I hope my cloudy, gloomy response was helpful in some way