“We work with social media apps to enhance some of their AI filters because our camera knows where your eyes, your nose, your ears, and where your hair are, and correctly pastes those effects on your face,” Asghar says.
Artificial intelligence (AI) is hard at work capturing photo and video thanks to the Snapdragon mobile platform which streamlines and enhances the ability to create professional-level shots.
The AI at work is delving beyond filters, too, allowing users to focus the cameras on a specific subject and follow them as a cinematographer would. There’s much technological gymnastics at play here as the engine jumps from camera to camera to keep the subject in focus. The seamlessness of this process can be attributed to the powerful processing technology on the Snapdragon platform.
Clear communication—in any language
Taking a professional quality photo or video isn’t the only way AI capabilities on the Snapdragon get involved in today’s smartphones, though. On video calls, there are multiple neural networks running simultaneously to make the call as sharp as possible. One neural network polishes up the speaker’s voice to crystal clarity, while another runs background noise suppression. After all, who wants the barking dog in the upstairs apartment to interrupt an important conversation with your boss?
And because the noise suppression algorithm is running on an AI-powered neural network, it can be trained over time to understand different types of sounds and “de-noise” even better—successfully separating (and canceling) outside noises from the voices of the speakers having a phone conversation. That sort of sound quality is crucial on long-distance calls, especially when callers are speaking different languages. In these instances, usually a translator would be required: Previous technologies just couldn’t do real-time translation between languages. But this is another instance where the new platform sets the next generation of smartphones apart from their predecessors.
The Snapdragon processor, for instance, begins translating speech as soon as it enters the microphone. Inside the device, neural networks knock out three important tasks: automatic speech recognition; language translation (Transformer network or BERT); and text-to-speech. Say two people are talking, but one speaks Mandarin and the other speaks English. Without ever knowing, a person’s English speech is transcribed into text, and then translated at the construct level to capture different meanings of different words across languages, ensuring the original context is kept intact. Lastly, that transcribed text is then converted to Mandarin speech. Voila: cross-border communication with ease.
Snapdragon real-time translation is cutting-edge. It’s complex but follows three general steps that are enabled by AI—and it all happens on device.
Source: https://www.wired.com/sponsored/story/meet-the-ai-powering-todays-smartest-smartphones/