I can't say I've been in your situation, though I've sort of been on the other side of it. A couple comments:
1) I tend to agree that a legal marriage isn't really necessary (unless maybe you want to have children and desire something more legal/official involved).
2) If he loves you and desires to stay with you (which I'd assume he does from what you said) and you feel the same, then I think forgiving him would be good and also be something that would show he doesn't need to hide things from you.
It would definitely seem good to discourage a repeat of the same, but if you think he's truly sorry and it was just fear that you might not accept it that kept him from telling you earlier, then showing him you still care for him and that you're simply upset that he didn't say something earlier would seem like enough.
In any case, I hope things work out ok for the two of you and again, welcome to the site (I'm rather new here myself).