Background: Advances in breast cancer research are making treatment options increasingly effective and reducing mortality. Body composition is an example of a prognostic tool that can help personalize breast cancer treatments and further increase their effectiveness. In this study, we examine the association of several body composition measures with comorbidities, physical function, and quality of life. Methods: This study is a cross-sectional analysis of 99 women with early breast cancer scheduled for chemotherapy. Univariate regression models were used to identify significant associations of body composition metrics with patient demographics, clinical characteristics, measures of physical function, and patient-reported outcomes (PRO)s. Multivariable modeling was used to evaluate associations adjusted for age. Results: Median age was 58 (range 24–83), 27% were non-white, and, 47% were obese (≥ 30 kg/m2). Increasing age was associated with lower Skeletal Muscle Density (SMD) (p = 0.0001), lower Skeletal Muscle Gauge (SMG) (p = 0.0005), and higher Visceral Adipose Tissue (VAT) (p < 0.0001). In patients with a prolonged Timed Up and Go tests (> 14 s), mean VAT was 57.87 higher (p = 0.004), SMD 5.70 lower (p = 0.04), and SMG 325.4 lower (p = 0.02). For each point of higher performance on the Short Physical Performance Battery (SPPB), VAT decreased 12.24 (p = 0.002) and SMD rose 1.22 (p = 0.02). In multivariable analysis adjusting for age, the association of TUG > 14 with higher VAT remained significant (p = 0.02). Conclusions: Suboptimal body composition prior to treatment is associated poor physical function and may be an indicator of clinical importance.