I think the big point here is that if neither of you really wanted your second child why did you close off to your husband. In general we close off to people we love because they have the power to hurt us. We know they have the power to hurt us because they have let us know this by deeply hurting us, there fore we close off to them to protect ourselves. My guess here is… and I don’t know because I don’t know you or your husband I am just going on what you have said… but my guess is that you did really want this child and your husband didn’t and the pain of seeing that child being brought up by someone close to you must be eructating… I don’t know and I could be wrong and if I am I apologise.
I think that maybe you have not only closed off to him but you have pushed him away and that you are both in a very lonely place, with walls in between you, this is not to excuse his behaviour because I don’t think that it is excusable! It sounds like you both still love each other and if that’s the case I think you have both got a lot of hurt and pain and work to go through to survive….you may come out the other end stronger and better people with a depth of love for each other and your daughter's that did not exist before. You may not.
I wish you all that you deserve happiness joy and health … Best Thoughts Moonie