Mark this one down as super-experimental but insanely cool, too. Chrome (and by extension, Chrome OS and Chromebooks) is always in development, always moving forward, and always up to new tricks that we never really thought possible with the web platform. With the nature of the open-source Chromium project, however, all it takes is a bit of an itch from a talented developer and we get a new feature in Chrome.
Debuting in Chrome 76, Chrome for Windows and Android (for now) now have the ability to scrape text directly from images using a combo of the same tech that drives the bar code detector, face detector, and text detector in Chrome. Using these technologies, Chrome is able to actually scan and detect text on an image, pull it out, and then feed it back in a plain text format.
We’ve talked a bit about Google Lens’ ability to do this same thing in the app, but we’ve not had this available as a core part of the Chrome browser up until now. The addition of this tech could lead to better accessibility features and searchable images in the future, never mind the ease of pulling text right off of a photo you come across online.
If you want to give it a go, keep in mind that this entire test is just that: a testing bed for this API. It works and the functionality is baked into Chrome 76 (you can try this test on Stable channel and the API won’t work in the following demo, for instance), but the execution of this feature isn’t really implemented just yet. Paul Kinlan, the developer responsible for this addition, has created a testing site to show you the API in action, so you’ll need to test things there for now.
Here’s how to try it out:
- Download & install Chrome Canary for Android or Windows
- Navigate to chrome://flags
- Search for ‘Experimental Web Platform Features’
- Enable it and restart Chrome
- Go to http://copy-image-text.glitch.me/
- Upload an image that has text on it
- Submit and view the resulting scraped text
That’s really it. As I said, the execution is raw at this point, but this demo shows what this new feature is capable of. With this info being able to be passed from a standard image over to a website or web app, it is now up to developers to get creative on how and where they will implement this. I think the clearest example would be situations where text is trapped in an image and you simply need to copy it. We’ve all been there and been frustrated by it. Soon, it won’t be quite as big of a deal.
