It is the vitamins and amino acids that form the enzymes to extract the calories. People get plenty of calories from starches. Even amino acids and vitamins have calories in them.
The issue is that as we age, our ability to produce enzymes weakens. If you do not have the necessary enzymes, even 10 baked potatoes or 5 pounds of wheat is not going to keep you functioning. Two of the biggest issues with proper absorption of nutrients is clogged intestines and parasites in the intestines that take the most important nutrition first and only leave you the empty calories.
Cleansing, flushing, and elimination of parasites is important to the proper absorption of nutrients.