Sometimes what goes around comes around, and there are consequences that we pay for our previous indiscretions. No matter, you are in this relationship now, and it's probably worth working these things out rather than chalking it up. As the previous poster pointed out, if you don't face your fears now, they will always be with you and manage to sabotage every other union that you will engage in.
I wish you the very best and hope you can overcome these challenges without sacrificing the relationship, unless of course, there are other issues involved besides the fidelity/trust.