Baidu's AI Lab Director on Advancing Speech Recognition and Simulation

Coates says Baidu Research connects cutting-edge AI research with business and product needs. The lab’s mission is to create AI technologies that can significantly affect at least 100 million people. It sometimes takes on basic research problems, then works through the final steps needed to bring the results into products.

Baidu’s Deep Speech began as research into making voice interfaces work across everyday settings, accents, and speaking distances. Coates says the team scaled neural networks and training data; its English system uses 10,000 to 20,000 hours of audio, while Mandarin systems use more for top-end products. The system learns to map audio directly to text instead of relying on multiple hand-engineered stages.

Coates describes two research directions for reducing the data burden. Text-to-speech systems could learn common properties from many voices, then adapt to a new voice with less audio. For speech recognition, unsupervised learning could let a system learn how speech works from raw audio before it is taught a language.

Coates says better recognition could support voice-first products, such as Baidu’s TalkType keyboard, rather than adding speech as a secondary feature. He emphasizes latency: even the difference between 50 or 100 milliseconds and 200 milliseconds can be noticeable. The team adapted its research model to respond with less future audio context, while continuing to work on noisy rooms, overlapping speakers, and long-form transcription.

Coates says synthetic media raises social questions that call for critical thinking, while speech interfaces may help people who cannot easily type, including his mother, who has muscular dystrophy. He also describes a need for flexible engineers who can connect research with GPUs, production systems, product teams, and user experience. He values self-directed people who keep learning and take responsibility for getting research into users’ hands.