Social factors account more for health outcomes than medical care, yet health services research in this area is limited due to the lack of social factors data contained within electronic health records (EHR) systems. Few investigations have examined how cumulative burdens of co-occurring adverse social factors impact health outcomes. From 293,872 patients in one region of the Veterans Health Administration (VHA), we examined how increasing numbers of adverse social factors extracted from the EHR were associated with mortality across a one-year period for male and female patients. Adverse social factors were identified using four sources in the EHR: responses to universal VHA screens, International Classification of Disease (ICD) diagnostic codes that indicate social factors, receipt of VHA services related to social factors, and templated social work referrals. Seven types of adverse social factors were coded: violence, housing instability, employment or financial problems, legal issues, social or familial problems, lack of access to care or transportation, and nonspecific psychosocial needs. Overall, each increase in an adverse social factor was associated with 27% increased odds of mortality, after accounting for demographics, medical comorbidity, and military service-related disability. Non-specific psychosocial factors were most strongly associated with mortality, followed by social or familial problems. Although women were more likely than men to have multiple adverse social factors, social factors were not associated with mortality among women as they were among men. By incorporating social factors data, health care systems can better understand patient all-cause mortality and identify potential prevention efforts built around social determinants.