Now, visually impaired people can enjoy memes: Here's how
While the internet is a place for everyone, some find it hard to use, like those suffering from visual impairment. Visually impaired people have to use assistive technologies to convert content into speech - a system that works but often fails to convey nuances, say the humor behind a meme. Now, the researchers at Carnegie Mellon University have developed a way to tackle this.
Modern-day assistive technologies, like screen-readers, are text-to-speech programs that recognize text on a page and convey it in the form of audio. However, when it comes to image content, like memes, these systems struggle to recognize what most of the photos say. This is mainly because they rely on the inclusion of 'Alt-text' to convert images into speech.
Alt-text, as many already know, is an HTML attribute that an uploader can add to an image to describe its content. For instance, what a particular meme says. Assistive technologies pick the alt-text and then convey the same through speech.
Platforms like Twitter provide the option to add Alt-text to photos, but not many people use the same, as it's difficult to locate on apps and websites. In fact, the researchers at Carnegie found that nearly one million tweets out of nine million had photos, but only 0.1% of that million had Alt-text describing the content of the image.
Now, to make all social media photos, including memes, accessible to the visually impaired, the researchers have developed a system that automatically adds the Alt-text needed for a photo through pre-written templates. It identifies the content of an image through optical character recognition and then uses the template to add Alt-text, giving the visually impaired a chance to perceive it using visually assistive tech.
While the tech automatically adds Alt-text describing an image, it struggles to capture some nuances expressed in memes. This is why the team involved in the work developed a sound library that could be used to better express the sentiment conveyed in a meme in the template. The tech, if evolved, could one day make memes, even the complicated ones, easy-to-access for visually impaired.
"Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention," project member Jeff Bigham said while explaining their work. "Many people use memes, and so they should be made accessible."
