Brain morphology and cognitive ability change with age. Gray and white matter volumes decrease markedly by the 7th decade of life when cognitive decreases first become readily detectable. As a consequence, the shape complexity of the cortical mantle may also change. The purposes of this study are to examine changes over a five year period in brain structural complexity in late life, and to investigate cognitive correlates of any changes. Brain magnetic resonance images at 1.5 Tesla were acquired from the Aberdeen 1936 Birth Cohort at about ages 68 years (243 participants) and 73 years (148 participants returned). Measures of brain complexity were extracted using Fractal Dimension (FD) and calculated using the box-counting method. White matter complexity, brain volumes and cognitive performance were measured at both 68 and 73 years. Childhood ability was measured at age 11 using the Moray House Test. FD and brain volume decrease significantly from age 68 to 73 years. Using a multilevel linear modeling approach, we conclude that individual decreases in late life white matter complexity are not associated with differences in executive function but are linked to information processing speed, auditory-verbal learning, and reasoning in specific models-with adjustment for childhood mental ability. A significant association was found after adjustment for age, brain volume and childhood mental ability. Complexity of white matter is associated with higher fluid cognitive ability and, in a longitudinal study, predicts retention of cognitive ability within late life.