BACKGROUND: Memory performance in older persons can reflect genetic influences on cognitive function and dementing processes. We aimed to identify genetic contributions to verbal declarative memory in a community setting. METHODS: We conducted genome-wide association studies for paragraph or word list delayed recall in 19 cohorts from the Cohorts for Heart and Aging Research in Genomic Epidemiology consortium, comprising 29,076 dementia- and stroke-free individuals of European descent, aged ≥45 years. Replication of suggestive associations (p < 5 × 10-6) was sought in 10,617 participants of European descent, 3811 African-Americans, and 1561 young adults. RESULTS: rs4420638, near APOE, was associated with poorer delayed recall performance in discovery (p = 5.57 × 10-10) and replication cohorts (p = 5.65 × 10-8). This association was stronger for paragraph than word list delayed recall and in the oldest persons. Two associations with specific tests, in subsets of the total sample, reached genome-wide significance in combined analyses of discovery and replication (rs11074779 [HS3ST4], p = 3.11 × 10-8, and rs6813517 [SPOCK3], p = 2.58 × 10-8) near genes involved in immune response. A genetic score combining 58 independent suggestive memory risk variants was associated with increasing Alzheimer disease pathology in 725 autopsy samples. Association of memory risk loci with gene expression in 138 human hippocampus samples showed cis-associations with WDR48 and CLDN5, both related to ubiquitin metabolism. CONCLUSIONS: This largest study to date exploring the genetics of memory function in ∼40,000 older individuals revealed genome-wide associations and suggested an involvement of immune and ubiquitin pathways.