Objective: Knee cartilage damage is often linked to mechanical overloading. However, cartilage requires mechanical load to remain healthy, suggesting that underloading may be detrimental. This study was undertaken to examine knee overloading and underloading by defining cumulative load as the joint effects of body mass index (BMI) and daily walking, and examine the relationship between cumulative load and worsening cartilage damage over 2 years. Methods: We used data from the Multicenter Osteoarthritis Study. Steps/day, measured by accelerometry, and BMI were calculated at the 60-month visit. Cartilage damage on magnetic resonance imaging was semiquantitatively scored using the Whole-Organ Magnetic Resonance Imaging Score (WORMS) at the 60-month and 84-month visits; worsening damage was defined as increased WORMS between visits. Risk ratios (RRs) and 95% confidence intervals (95% CIs) were calculated using binomial regression, with adjustment for potential confounders. Results: Our study included 964 participants, 62% of whom were female, with a mean ± SD age of 66.9 ± 7.5 years. Participants had a mean ± SD BMI of 29.7 ± 4.8 kg/m2 and walked a mean ± SD of 7,153 ± 2,591 steps/day. Participants who walked a moderate number of steps/day (6,000–7,900) or a high number of steps/day (>7,900) and had a high BMI (>31 kg/m2) had a greater risk of worsening medial tibiofemoral (TF) damage (RR 2.83 [95% CI 1.46–5.48] and RR 2.61 [95% CI 1.50–4.54], respectively) compared with those who walked similar steps/day and had a low BMI (18–27 kg/m2). Participants with a low number of steps/day (<6,000) and a low BMI had a greater risk of worsening medial TF and lateral patellofemoral (PF) damage (RR 2.03 [95% CI 1.06–3.92] and RR 2.28 [95% CI 1.06–4.85], respectively) compared with those who walked a high number of steps/day and had a low BMI. Effect estimates for other compartments of the knee did not reach statistical significance. Conclusion: This study provides preliminary evidence that both overloading and underloading may be detrimental to medial TF cartilage, and underloading may be detrimental to lateral PF cartilage.