Could be considered "hard on the liver" - as its really the liver that does the work of recombining the 'wrong food' into things usable by the body.
Nuts and Seeds are hard in their natural state - thats why soaking/sprouting is beneficial as it activates the enzymes AND converts the starches into polysaccharides and Aminos more usable by the body.
Citrus is hard physically on the body - so generally should be consumed earlier in the day to give the body time to process it {this is where the Rainbow Diet really comes into play}.
Bananas - well I would imagine due to its "fatiness" as its specifically the liver that would process it.
In essence the liver is a muscle - and its job is to work, its the kind of 'work' you put it through that matters.