Human language requires constant learning of new words, leading to the acquisition of an average vocabulary of more than 30,000 words in adult life. The ability to learn new words is highly variable and may rely on the integration between auditory and motor information. Here, we combined diffusion imaging tractography and functional MRI to study whether the strength of anatomical and functional connectivity between auditory and motor language networks is associated with word learning ability. Our results showed that performance in word learning correlates with microstructural properties and strength of functional connectivity of the direct connections between Broca’s and Wernicke’s territories in the left hemisphere. This study suggests that our ability to learn new words relies on an efficient and fast communication between temporal and frontal areas. The absence of these connections in other animals may explain the unique ability of learning words in humans.