I often play games with subtitles on, and noticed that in Marvel’s The Avengers, the game also has something it calls “closed captions.” With closed captions flipped on, the subtitles go far beyond purely dialogue, and begin to describe the action occurring on the screen, too.
- [Bruce begins to morph]
- KAMALA: No way! [She pulls out the hard drive that had turned on.]
- [Bruce closes his eyes briefly in frustration.]
Videos by VICE
Subtitles seem like a simple concept. It’s the words that people are saying onscreen, right? A win for accessibility, a win for parents playing video games with the sound down because their kids are asleep. But when subtitles are done right, it’s actually a lot more complicated, because words alone do not necessarily convey the entirety of the moment. A transcript won’t communicate tone of voice, where someone is speaking, or other sound cues that are being used by the developers to convey different emotions to the person with the controller.
Avengers attempts a form of this, although according to some accessibility experts, it’s a flawed approach.
“Watching a cinematic during a video game is no different than watching a film in a movie theater,” said designer Mariah Robinson, one of Crystal Dynamics’ point people for developing accessibility features. “Players who are hard of hearing or who are deaf miss out on tonality, the mood created by the music score, and many other important pieces of information when there is no dialogue to be subtitled. Story lines can become disjointed for anyone without context, so I decided to help join that cause through Marvel’s Avengers.”
These non-verbal cues can be taken for granted, but increasingly, games are taking notice, in ways big and small. You know how Gears of War signals the end of a combat encounter with a little slice of music? It’s not just a sound effect—it’s an important gameplay marker. And in Gears 5, the series finally acknowledged this point in the game’s subtitles. When a fight is over, “music settles” appears on screen. Now, everyone is on the same page.
“As a general rule of thumb for subtitle accessibility, give players options,” said game critic Laura Kate Dale, who is on the autism spectrum. “Do they want just dialogue? Sound effects too? Just non verbal cues? Let them be toggled on or off independently of overall dialogue subtitles.”
Dale was someone who also noticed what Avengers was doing with its subtitles, and found the information personally useful. During an early scene between Kamala Khan and Bruce Banner, for example, no words are exchanged between them. It’s a wholly silent sequence.
You can watch the scene with subtitles on in the video below, which includes lines like “only ice is heard as she shakes the cup” and “Bruce changes the radio back to static.”
“Things like Kamala deliberately slurping her drink in Bruce’s direction as a sign she was annoyed,” said Dale, “or Bruce’s blink length being slightly longer meaning frustration are great examples of small wordless moments that I often miss in media. I really appreciated having those subtitles there to help catch that.”
While writing the closed captions script, Robinson often played the game with the sound off, trying to figure out what the transcripted dialogue was not communicating to players.
“I played through the game multiple times with just the closed captions and subtitles on to make sure that I was landing each cinematic as if the script had been turned into a book,” said Robinson, who worked on the closed captions script. “This helped me decipher if there was something missing that could enhance the player’s experience for that cinematic.”
Avengers is not without its issues, however. The captions are, for example, inconsistent, choosing to explain some actions and emotions but not others.
But it goes beyond that. Accessibility specialist Ian Hamilton, who consults with studios building features like this, said players he’s talked to are unsatisfied with how Avengers approaches its captions.
Hamilton pointed at the gap between Avengers and something like this “audio description” trailer for the upcoming Assassin’s Creed: Valhalla. (You might need to flip the “CC” option on in the player.)
“That’s where this description of what is happening visually traditionally lives, in the form of an audio track, aimed at people with vision loss but also used by others for a wide range of reasons, including cognitive accessibility,” said Hamilton. “Could there be value in communicating it in text rather than audio? For some people perhaps. But just not in the form of bulking out captions with information that hinders the experience of other gamers.”
“Audio description,” where visual information is said aloud, is common in other media but rare in games.
“I don’t want to speculate as to the reasons why the captions in Avengers ended up the way they are,” said Hamilton. “But cramming all of that extra information that Deaf people already have access to through the visuals actually harms the captioning system. Captions can be hard to follow anyway, especially considering that you’re more likely to have a lower reading ability if you’re Deaf, due to written speech not being your first language.”
It’s also true that Avengers does not let you turn on subtitles without also turning on closed captions—it’s an all or nothing deal. Which can lead to a situation where the closed captions actually end up “spoiling” upcoming events because they drop on the screen too early.
It all comes back to something Dale pointed out earlier: giving players lots of customization.
It’s also probably explains how this showed up in the final product:
Crystal Dynamics did tell me there are “efforts to improve this feature” coming to Avengers, but did not elaborate on how it might change or when those improvements could arrive.
The big picture, Hamilton noticed, was that there was room to criticize at all. Being able to criticize Avengers reflects a world where a studio like Crystal Dynamics made this a priority.
“We’re now at the point of being able to have these conversations about how best to implement things,” said Hamilton, “rather than just hoping for something—anything—to be implemented.”
Follow Patrick on Twitter. His email is patrick.klepek@vice.com, and available privately on Signal (224-707-1561).
