You should try to work on your memory because memory needs to be trained in order to develop ... for instance, http://www.memoryimprovementbooks.com/
might be a good starting point. Don't let your problem persist because in the future this might evolve into dementia.