Research, Collaboration, Impact! Making digital content accessible for all through customisable technology

An ever-increasing number of people today present with complex and diverse needs, which makes the issue of accessing digital content a pressing one. With many of the established solutions such as subtitles and audio description being decades old, Dr Timothy Neate and his project team are giving accessibility technologies a complete overhaul – digital content is to become configurable and customisable for all. 

Background

The past twenty years have witnessed a radical change in how we consume media. In the UK, more than 50 percent of people (and more than 70 percent of 18-25s) now prefer accessing news content online over traditional print newspapers. Demand for entertainment follows a similar pattern. ‘Rapid change continues to characterise the TV and online video sectors and the radio and audio sectors’, a recent Ofcom report finds. The ubiquity of smartphones, apps and streaming platforms have changed dramatically how we access content, where and when we watch broadcasts, and how long we stay tuned in.

Accessibility technology, however, has not kept up with this pace of development. Oftentimes users with complex needs must content themselves with decades-old solutions such as subtitles and audio description. While their paramount importance is not to be dismissed, there is no one-size-fits-all solution to accessibility at a time when an increasing number of people present with complex needs. ‘There's a large focus on visual and auditory impairment’, Tim explains. ‘While very important, the number of people with accessibility challenges is much larger and constitutes a complex range of needs'. Tim and his team aim to bring accessibility technology up to speed. ‘It’s really about accessibility beyond subtitles’.

The CA11y Project at King’s Informatics

Tim and his team aim for a change of perspective on digital content creation. Why not consider creation and accessibility in tandem and in parallel, not just as an afterthought? This means embedding diversity in access into the very process of content creation. ‘The idea of the project really is to explore the concept of a new kind of general accessibility solution which anybody can tweak to meet their needs’, Tim outlines.

To achieve this, Tim and his collaborator, Dr Madeline Cruice, Professor of Aphasia Rehabilitation and Recovery at City University London, have begun work on the CA11y Project, short for ‘Content Accessibility: Highly Individualised Digital Content for Supporting Diverse Needs’. Funding of £355,200 comes from the prestigious EPSRC New Investigator Award scheme. The chief aim of the project is to carry over into the realm of accessibility many of the highly configurable and customisable solutions that are currently being developed for the consumption of digital content across multiple devices.

With more than 20 published papers and several awards under his belt, Tim is certainly well placed to deliver on this two-year project. With a focus on aphasia, a language disorder typically caused by damage to the brain following a stroke, the team first needs to map out the typical requirements that users with aphasia have. ‘We want to understand the challenges that people have currently with accessing digital content – there's surprisingly little literature on this for people with aphasia’, Tim points out. When a person finds it difficult to follow a TV programme, the first task for the research team is to think conceptually ‘how could we debrief and essentially explain the missing bits that the person didn't understand?’, Tim explains.

The team has partnered with two charities, Dyscover and Aphasia Re-Connect, to gather empirically insightful responses to this fundamental question in a series of workshops. Participants are shown collections of clips, followed by discussions of what they had found easy or difficult to understand. The research process is creative and open-ended. ‘We try to think of this almost like a magic ball’, Tim explains. ‘If I had a magic ball and I could shake it and it would solve my problem, what would it do? What would the device do?’ In getting the workshops up and running, Professor Cruice’s involvement is indispensable, Tim notes. ‘A lot of the ideas around how we practically do this and the more medical side of things are really important to the success of project’, he says. Working with Professor Cruice, an expert in language therapy, helps getting most out of the sessions.

Fig 1: Dr Timothy Neate (second from right), doctoral researcher Alexandre Nevsky (second from left) and members of the Dyscover team. © Alexandre Nevsky.  

The sessions with people with aphasia are complemented by a series of workshops with content creators, the idea being to involve them early on in the process and communicate to them the requirements that people with diverse needs have. Creators can then respond and indicate the impact that different levels of adjustments would have on their work, and how comfortable they would be to accommodate them. ‘Let's imagine a producer of a nature documentary that the participants really liked and the producer had spent ages crafting’, Tim says. ‘How would they feel about somebody being able to just chop all the music out of it?’ The point of this process is to explore possible areas of tension and get the typically segregated communities of content creators and impaired consumers of digital content to engage with each other.

King's positive attitude towards interdisciplinary research and the support that the Department of Informatics offers for early career researchers has certainly helped to get the CA11y Project off the ground, Tim finds. ‘For instance, I took my grant proposal along to an internal review session’, he recollects. ‘The seniors immediately got the idea and articulated it back to me in such a way that I was able to completely change the way I structured the document and made my actual proposal much clearer’, he continues.

Next steps

In a series of sessions, the research process builds towards a shortlist of general ideas and learnings that are to be explored further in a co-design process. At the end of it the team is going to build actual technologies and prototypes. 

‘Once these ideas have solidified and we've built some prototypes we're then going to be testing them’, Tim outlines the final phase of the research project. ‘This is when we'll take three to four technological solutions, which are very likely to be apps that may run on tablet devices, smart speakers, or on a TV’, he continues. One example could be watching the same movie together with a partner who gets the director’s commentary laid over it on a separate audio channel. Further examples include the simplification of dialogues or the reduction of certain types of background sounds, depending on individual circumstances.

Fig 2: the vision of the CA11y Project is to make headway towards bespoke, individualised accessibility solutions that respond to individual needs. It builds on an access ontology that captures the rules for adaptation for each individual user. © Dr Timothy Neate.  

Closed captioning for analogue television works by embedding information in non-visible intervals of television signals that, once decoded, would typically be displayed as white letters over a black background (the CEA/EIA-608 standard). Its contemporary digital counterpart, CEA/EIA-708, allows for a wider range of available languages but is restricted to displaying Unicode text. Today, much more is possible. ‘If we were to stop and think again, with the current technological possibilities, what more could we achieve?’, Tim asks. Unlike the old days when displaying words and phrases to create subtitles would exhaust the affordances of broadcasting, communication over the internet allows for a much richer distribution of metadata, i.e. information about data. On the CA11y Project, metadata feeds into an ontology that specifies how digital content should be rendered and presented to meet a user’s specific needs.

The overall deliverable of the project is to provide a comprehensive open-source toolkit and a training programme that enables content creators to consider accessibility issues at the point of creation. The prototypes that the team develop should benefit people with aphasia and beyond – the UK being world-leading in innovation in the creative industries. ‘The ultimate success story would be that the types of ideas that people come up with in the sessions finally get integrated into, for example, the iPlayer or similar devices’, Tim sums up the overarching objective of his research. If the enthusiasm the team has met so far from the communities of users and creators is anything to go by, making digital content truly accessible for everyone is going to make some giant leaps in the years to come.

Written by Juljan Krause