— Mobile

Apple’s Live Text lets you interact with text in your photos – TechCrunch

Apple has introduced a new feature to its camera system that automatically recognizes and transcribes Text in your photos, from a phone number on a business card to a whiteboard full of notes. As the feature is called, Live Text doesn’t need any prompting or exceptional work from the user — just tap the icon, and you’re good to go. Announced by Craig Federighi on the virtual stage of WWDC, Live Text will be arriving on iPhones with iOS 15. He demonstrated it with a couple pictures, one of a whiteboard after a meeting, and a couple snapshots that included restaurant signs in the background.

Screenshot of a phone selecting text in an image.

Tapping the Live Text button in the lower right gave detected Text a slight underline, and then a swipe allowed it to be selected and copied. In the case of the whiteboard, it collected several sentences of notes, including bullet points, and with one of the restaurant signs, it grabbed the phone number, which could be called or saved.

Certain types of text strings can be recognized, as well: a tracking code will be seen as such, and a link to the tracking URL will be made immediately available. Translation can be done quickly too, to or from any language supported by Apple’s other translation tools. The feature is reminiscent of many found in Google’s long-developed Lens app, and the Pixel 4 added more robust scanning capability in 2019. The difference is that the texts are captured more or less passively in every photo taken by an iPhone running the new system — you don’t have to enter scanner mode or launch a separate app.

This is a nice thing for anyone, but it could be beneficial for people with visual impairments. A snapshot or two makes any text, otherwise difficult to read, able to be dictated or saved. The process takes entirely on the phone, so don’t worry that this info is being sent somewhere to a data center. That also means it’s pretty quick, though until we test it for ourselves, we can’t say whether it’s instantaneous or, like some other machine learning features, something that happens over the next few seconds or minutes after you take a shot. Your back catalog of photos will be Live Text-ified in your phone’s idle moments, though.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button