Page Loader
Now, visually impaired people can enjoy memes: Here's how

Now, visually impaired people can enjoy memes: Here's how

Dec 01, 2019
12:08 am

What's the story

While the internet is a place for everyone, some find it hard to use, like those suffering from visual impairment. Visually impaired people have to use assistive technologies to convert content into speech - a system that works but often fails to convey nuances, say the humor behind a meme. Now, the researchers at Carnegie Mellon University have developed a way to tackle this.

Issue

Accessing memes is hard with assistive technologies

Modern-day assistive technologies, like screen-readers, are text-to-speech programs that recognize text on a page and convey it in the form of audio. However, when it comes to image content, like memes, these systems struggle to recognize what most of the photos say. This is mainly because they rely on the inclusion of 'Alt-text' to convert images into speech.

Information

Addition Alt-text HTML attribute

Alt-text, as many already know, is an HTML attribute that an uploader can add to an image to describe its content. For instance, what a particular meme says. Assistive technologies pick the alt-text and then convey the same through speech.

Details

But, Alt-text is mostly missing from photos, memes

Platforms like Twitter provide the option to add Alt-text to photos, but not many people use the same, as it's difficult to locate on apps and websites. In fact, the researchers at Carnegie found that nearly one million tweets out of nine million had photos, but only 0.1% of that million had Alt-text describing the content of the image.

Solution

Automated solution to add Alt-text

Now, to make all social media photos, including memes, accessible to the visually impaired, the researchers have developed a system that automatically adds the Alt-text needed for a photo through pre-written templates. It identifies the content of an image through optical character recognition and then uses the template to add Alt-text, giving the visually impaired a chance to perceive it using visually assistive tech.

Sentiments

Even sentiments can be expressed

While the tech automatically adds Alt-text describing an image, it struggles to capture some nuances expressed in memes. This is why the team involved in the work developed a sound library that could be used to better express the sentiment conveyed in a meme in the template. The tech, if evolved, could one day make memes, even the complicated ones, easy-to-access for visually impaired.

Quote

Memes are widely used and deserve people's attention

"Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention," project member Jeff Bigham said while explaining their work. "Many people use memes, and so they should be made accessible."