Integrating Convolutional Neural Networks and Multi-task Dictionary Learning for Cognitive Decline Prediction with Longitudinal Images

Qunxi Dong*, Jie Zhang*, Qingyang Li, Junwen Wang, Natasha Leporé, Paul M. Thompson, Richard J. Caselli, Jieping Ye, Yalin Wang (*Equal Contributions)

Disease progression prediction based on neuroimaging biomarkers is vital in Alzheimer’s disease (AD) research. Convolutional neural networks (CNN) have been proved to be powerful for various computer vision research by refining reliable and high-level feature maps from image patches. However, a key challenge in applying CNN to neuroimaging research is the limited labeled samples with high dimensional features. Another challenge regards how to improve the prediction accuracy by joint analysis of multiple data sources (i.e., multiple time points or multiple biomarkers). To address these two challenges, we propose a novel multi-task learning framework based on CNN. First, we pre-trained CNN on the ImageNet dataset and transferred the knowledge from the pre-trained model to neuroimaging representation. We used this deep model as feature extractor to generate high-level feature maps of different tasks. Then a novel unsupervised learning method, termed Multi-task Stochastic Coordinate Coding (MSCC), was proposed for learning sparse features of multi-task feature maps by using shared and individual dictionaries. Finally, Lasso regression was performed on these multi-task sparse features to predict AD progression measured by the Mini Mental State Examination (MMSE) and the Alzheimer’s Disease Assessment Scale cognitive subscale (ADAS-Cog). We applied this novel CNN-MSCC system on the Alzheimer’s disease Neuroimaging Initiative (ADNI) dataset to predict future MMSE/ADAS-Cog scales. We also compared the prediction performances with seven other similar methods and found that our method achieved superior results. Our work may add new insights into data augmentation and multi-task deep model research and facilitate the adoption of deep models in neuroimaging research.

Figures (click on each for a larger version):