We examined the feasibility of recruiting US adults ≥45 years old with Fabry disease (FD) for telephone assessments of cognitive functioning. A case–control design matched each FD participant on age, sex, race, and education to four participants from a population-based study. Fifty-four participants with FD age 46–72 years were matched to 216 controls. Standardized cognitive assessments, quality of life (QOL), and medical histories were obtained by phone, supplemented by objective indices of comorbidities. Normalized scores on six cognitive tasks were calculated. On the individual tasks, scores on list recall and semantic fluency were significantly lower among FD participants (p-values < 0.05), while scores on the other four tasks did not differ. After averaging each participant’s normalized scores to form a cognitive composite, we examined group differences in composite scores, before and after adjusting for multiple covariates using generalized estimating equations. The composite scores of FD cases were marginally lower than controls before covariate adjustments (p = 0.08). QOL and mental health variables substantially attenuated this finding (p = 0.75), highlighting the influence of these factors on cognition in FD. Additional adjustment for cardiovascular comorbidities, kidney function, and stroke had negligible impact, despite higher prevalence in the FD sample. Telephone-based cognitive assessment methods are feasible among adults with FD, affording access to a geographically dispersed sample. Although decrements in discrete cognitive domains were observed, the overall cognitive function of older adults with FD was equivalent to that of well-matched controls before and after accounting for multiple confounding variables.