And another " I agree".
3 weeks is NOTHING!
You say that you are spending lots of time with him - maybe taper that off a bit too. That should help you gain perspective about the relationship and the time you have been in it. Not that this is what is happening, but I get an immediate feeling that codependence is blossoming for you here, and that isn't healthy.
Take more time to yourself, maybe date others?
I am sure if you do that, the fog will clear and you will feel less stressed out about this.