Background Hospital readmission rates after surgery can represent an overall hospital effect or a combination of specialty and patient effects. We hypothesized that hospital readmission rates for procedures within specialties were more strongly correlated than rates across specialties within the same hospital. Study Design For general, orthopaedic, and vascular specialties at Veterans Affairs hospitals during 2008 to 2014, 30-day risk-adjusted readmission rates were estimated for 6 high-volume procedures and each specialty. Relationships were assessed using the Pearson correlation coefficient. Results At 84 hospitals, 64,724 orthopaedic, 24,963 general, and 10,399 vascular inpatient procedures were performed; mean readmission rates were 6.3%, 13.6%, and 16.4%, respectively. There was no correlation between specialty-specific adjusted hospital readmission rates: general and orthopaedic (r = 0.21; p = 0.06), general and vascular (r = 0.15; p = 0.19), and vascular and orthopaedic surgery (r = 0.07; p = 0.55). Within specialties, we found modest correlations between knee and hip arthroplasty readmission rates (r = 0.39; p < 0.01) and colectomy and ventral hernia repair (r = 0.24; p = 0.03), but not between lower-extremity bypass and endovascular aortic repair (r = 0.13; p = 0.26). Overall, controlling for patient-level factors, 1.9% of the variation in readmissions was attributable to specialty-level factors; only 0.6% was attributable to hospital-level factors. Conclusions Hospital readmission rates for orthopaedic, vascular, and general surgery were not correlated between specialties; within each of the 3 specialties, modest correlations were found between 2 procedures within 2 of these specialties. These findings suggest that hospital surgical readmission rates are primarily explained by patient- and procedure-specific factors and less by broader specialty and/or hospital effects.