Damage to the white matter underlying the left posterior temporal lobe leads to deficits in multiple language functions. The posterior temporal white matter may correspond to a bottleneck where both dorsal and ventral language pathways are vulnerable to simultaneous damage. Damage to a second putative white matter bottleneck in the left deep prefrontal white matter involving projections associated with ventral language pathways and thalamo-cortical projections has recently been proposed as a source of semantic deficits after stroke. Here, we first used white matter atlases to identify the previously described white matter bottlenecks in the posterior temporal and deep prefrontal white matter. We then assessed the effects of damage to each region on measures of verbal fluency, picture naming, and auditory semantic decision-making in 43 chronic left hemispheric stroke patients. Damage to the posterior temporal bottleneck predicted deficits on all tasks, while damage to the anterior bottleneck only significantly predicted deficits in verbal fluency. Importantly, the effects of damage to the bottleneck regions were not attributable to lesion volume, lesion loads on the tracts traversing the bottlenecks, or damage to nearby cortical language areas. Multivariate lesion-symptom mapping revealed additional lesion predictors of deficits. Post-hoc fiber tracking of the peak white matter lesion predictors using a publicly available tractography atlas revealed evidence consistent with the results of the bottleneck analyses. Together, our results provide support for the proposal that spatially specific white matter damage affecting bottleneck regions, particularly in the posterior temporal lobe, contributes to chronic language deficits after left hemispheric stroke. This may reflect the simultaneous disruption of signaling in dorsal and ventral language processing streams.