3 suggestions

In my experience, i have three suggestions:

1) just let it go. anything you pay attention to will be activated.  In fact, almost all of us will come across such things. take it easy.

2) make your visualization vivid.  the more clear and vivid your visualization, the easier you can focus on it. before visualization, write down detailed information about your visualization will be very helpful. you can speak it aloud before visualization too.

3) you can also practice meditation more. meditation is an effective way to get peaceful and focus. the more focus and peaceful you are, the less unwanted things you will see in your visualization.

you can also check: my experience on creative visualization to get more information.