You're right. A vegan/raw diet just doesn't contain adequate nutrients to support conception and healthy babies, namely fat-soluble vitamins such as vitamin A and vitamin D. I strongly urge you to check out the Weston A. Price foundation. Go to their website and do a search or look under "women's health".
They are a non-profit organization and don't accept any money from meat, dairy, soy or any food industries. All of their information is well-referenced and scientifically accurate; you can check on the sources as they are listed at the end of each article. Just read the information with an open mind. I've been following their dietary principles for about two years now and have never felt better. My skin is clear, my weight has normalized, my mood is much improved (no more PMS!) and I just plain feel GOOD.
I wish you much luck as I know how heartbreaking infertility can be---but I also know that in most cases it is reversible. Your body will not carry a baby that it doesn't have the nutrients and resources to support.