Background: Although higher sedentary behavior (SB) with low light intensity (LPA) and moderate-to-vigorous intensity physical activity (MVPA) are thought to increase risk for obesity, other data suggest excess weight may precede these behaviors in the causal pathway. We aimed to investigate 10-year bidirectional associations between SB and activity with weight. Methods: Analysis included 886 CARDIA participants (aged 38–50 years, 62% female, 38% black) with weight and accelerometry (≥ 4 days with ≥ 10 h/day) collected in 2005–6 (ActiGraph 7164) and 2015–6 (ActiGraph wGT3X-BT). Accelerometer data were calibrated, harmonized, and expressed as counts per minute (cpm) and time-dependent intensity categories (min/day of SB, LPA, and MVPA; SB and MVPA were also separated into long-bout and short-bout categories). Linear regression models were constructed to estimate adjusted associations of baseline activity with 10-year change in weight and vice versa. When activity categories were the independent variables, standardized regression coefficients (βstd.) estimated associations of replacing SB with a one SD increase in other categories, adjusted for accelerometer wear time. Results: Over 10-years, weight increased by a mean 2.55 ± 8.05 kg and mean total activity decreased by 50 ± 153 cpm. In adjusted models, one SD higher baseline mean total activity (βstd. = −1.4 kg, p < 0.001), LPA (βstd. = −0.80 kg, p = 0.013), total MVPA (βstd. = −1.07 kg, p = 0.001), and long-bout MVPA (βstd. = −1.20 kg, p < 0.001) were associated with attenuated 10-year weight gain. Conversely, a one SD higher baseline weight was associated with unfavorable 10-year changes in daily activity profile including increases in SB (βstd. = 12.0 min, p < 0.001) and decreases in mean total activity (βstd. = 14.9 cpm, p = 0.004), LPA (βstd. = 8.9, p = 0.002), and MVPA (βstd. = 3.5 min, p = 0.001). Associations varied by race and gender. Conclusions: Higher SB with lower activity and body weight were bidirectionally related. Interventions that work simultaneously to replace SB with LPA and long-bout MVPA while also using other methods to address excess weight may be optimal.