So you are mostly whole plant based foods diet? you are avoiding all fruits? how long are planning on doing it for? how long have you been on this current diet? the first weeks of stopping Sugar are HARD!
Quitting refined sugars was huge for my health, but I would get cravings from time to time, especially the first month or two but then it was smooth sailing. Plus if you are experiencing die off symptoms, you might be feeling tired, cranky, flu-like, etc, and the Sugar seems even more tempting because it can give that quick energy boost.
Sorry I can't give you much info about the die off and Sugar cravings, but glad to hear you are making major improvements!