Background: Social determinants of health (SDH) are strong predictors of morbidity and mortality but health care systems struggle to integrate documentation of SDH into health records in ways that can be used for health services research. Given the impact of social factors on health, it is important to examine the relationship with emergency department (ED) utilization. Objective: To examine the association between seven indicators of SDH and ED utilization using electronic health record (EHR) data from the Veterans Health Administration (VHA). Methods: This cross-sectional analysis included data from all patients who had at least one health care visit in the Veterans Integrated Service Network region 4 from October 1, 2015 through September 30, 2016 (n = 293,872). Seven categories of adverse SDH included violence, housing instability, employment or financial problems, legal problems, social or family problems, lack of access to care or transportation, and non-specific psychosocial needs identified through structured coding in EHR. Negative binomial regression was used to examine the association of the count of adverse SDH (0–7) with the count of ED visits, adjusting for socio-demographic and health-related factors. Results: Approximately 18% of patients visited the ED during the observation period. After adjusting for covariates, adverse SDH were positively associated with VHA ED utilization. Each of the SDH indicators, other than legal issues, was positively associated with increased ED utilization. Conclusion: Even after accounting for several demographic and health-related factors, adverse SDH demonstrated strong positive associations with VHA ED utilization.