On the recent Google I/O 2019, Google showed a new feature called Live Caption. It is built into Android Q, which can convert the audio played on the mobile phone into subtitles in real time, suitable for podcasts, video websites, voice messages, local shooting videos and other scenes.
What's more, the Live Caption process is all done locally on the phone, and it doesn't need to be uploaded to the cloud, which guarantees speed and security. What are the strengths of Live Caption? This article is a complete introduction to you.
Can also be used without the network, processing locally
Subtitles may be more important than many people think. In the eyes of the general audience, it is an optional option, and for people with hearing impairment, subtitles are an important tool for them to understand the content., Live Caption is designed for them.
Can you add real-time subtitles to all the audio on the phone? This is the next goal of Google. After the efforts of many teams, the plan was finally realized.
At first, however, the system needed to upload audio to the cloud and then run a 2GB machine learning model, which was time-consuming and unsafe.
Later, they implemented the same processing effect on the mobile terminal, and only need to run an 80M model, which can be automatically updated when the mobile phone is connected to the network, and then run completely offline without the network.
There are three main advantages of Live Caption: first, it runs completely locally and is suitable for all scenarios. Second, data is not uploaded to the cloud to ensure data security. Third, content creators and application developers do not need to do any work. , you can let users experience real-time subtitles.
System-level real-time subtitles for all applications
In Google's demo, Live Caption can be used in live broadcasts, video sites, podcasts, local video shots, social media, voice messages, and more. The way to open it is to click the volume button and click the live caption button.
As the audio plays, dynamic real-time subtitles are displayed in the center of the screen. You can adjust the display size of the subtitles and drag them to any position. Even more amazing, Live Caption works fine even when muted.
Although Google mainly emphasizes the role of Live Caption in hearing impaired people, it is also very useful for ordinary users.
You can better understand the content by turning on subtitles while watching the video. Google makes this feature at the bottom of the system, which means you can turn on subtitles in any video. In addition, in special scenes such as meetings and subways, you can understand video content without turning on audio.
Launched later this year.
A few days ago we recommended several "automatic subtitles" tools that video creators can use to lighten the burden of work.
But Google's thinking is obviously more thorough, they make real-time subtitles a system-level feature, content creators and application developers don't need to do any work to allow users to experience real-time subtitles at any time.
This function is integrated in Android Q, once popularized, it can be convenient for a large number of users.
Currently Live Caption is not yet able to experience in the Android Q system. Brian Kemler of the Android team revealed that it will be available later this year.
Because of memory and space requirements, Live Caption will first appear on high-end Android Q devices, and then more devices will be supported.