The social communication impairments defining autism spectrum disorders (ASD) may be built upon core deficits in perspective-taking, language processing, and self-other representation. Self-referential processing entails the ability to incorporate self-awareness, self-judgment, and self-memory in information processing. Very few studies have examined the neural bases of integrating self-other representation and semantic processing in individuals with ASD. The main objective of this functional MRI study is to examine the role of language and social brain networks in self-other processing in young adults with ASD. Nineteen high-functioning male adults with ASD and 19 age-sex-and-IQ-matched typically developing (TD) control participants made “yes” or “no” judgments of whether an adjective, presented visually, described them (self) or their favorite teacher (other). Both ASD and TD participants showed significantly increased activity in the medial prefrontal cortex (MPFC) during self and other processing relative to letter search. Analyses of group differences revealed significantly reduced activity in left inferior frontal gyrus (LIFG), and left inferior parietal lobule (LIPL) in ASD participants, relative to TD controls. ASD participants also showed significantly weaker functional connectivity of the anterior cingulate cortex (ACC) with several brain areas while processing self-related words. The LIFG and IPL are important regions functionally at the intersection of language and social roles; reduced recruitment of these regions in ASD participants may suggest poor level of semantic and social processing. In addition, poor connectivity of the ACC may suggest the difficulty in meeting the linguistic and social demands of this task in ASD. Overall, this study provides new evidence of the altered recruitment of the neural networks underlying language and social cognition in ASD.