The low carb diets for candida are supposed to be short term, about 4 weeks, and then other foods are introduced gradually.
As far as the difference between sugary foods and white potatoes, as I said, Sugar doesn't cause blood glucose to spike as much as boiled white potatoes. 150 grams of boiled white potatoes have a GI of 96, while a 60g serving of banana cake made with Sugar has a GI of only 47.
150 grams of boiled sweet potato has a GI of 44, half that of white potatoes.
So it appears to me that High GI foods may be problematic for you. They cause a spike in blood glucose which requires your pancreas to react with insulin to convert the glucose to glycogen.
You can still get plenty of fiber by sticking with low GI vegetables & fruits.