Well, a lot of fear surfacing here eh ?
Don't worry - seems to be pretty human reaction to the unknown
All I'll offer is this. And I hope it doesn't sound trite because it's not. Just realistic - based on how the world truly is.
If we start out building a house and we have little or no experience in that, we are naturally afraid that in the end it may all come tumbling down. Natural. What you have to be afraid of though, is that your distraction from the fear actually causes you to do shoddy work along the way ! Then it becomes a self fulfilling prophecy !
Focus on the quality as you go. Be loving, honest, build a QUALITY relationship(s). Build a quality YOU ! The house - and the relationship(s) - will stand the test of time. It's really the only way.