Neither I nor my partner, we're secondaries wear our wedding rings. We didn't wear them before we got involved with each other either. I wouldn't have a problem if she wore her wedding ring when me and her were out together or if she wore it with her husband and took it off when she went out with me.
I guess my point is that the wedding ring doesn't matter in our case. If someone were to see us out together they would think we were a couple and that really all that matters to me.