“Working on Video Game Accessibility is Incredibly Fun” and Other Aha! Moments

Developers Matt Booth and Katie Scott talk about their experience developing a major new accessibility feature in Gears 5: Navigation Ping for Escape Mode.

By: Katie Scott

I can think of very few moments in my career that were more intense – the fear, the doubts, and then the highs when it actually worked – than the development of a new accessibility feature in Gears 5. “Navigation Ping” is a feature that helps players with low or no vision (or anyone else who wants to use it!) navigate the complex maze-like environments in Escape Mode with audio cues and complementary visual cues. Escape Mode was an area of our game that was nearly or completely inaccessible to a huge group of Gears fans, but Navigation Ping has opened up a whole new way for them to play Gears.

The fact that we released a major accessibility feature so long after we released the game is unusual, and the effects it had on the small group that developed it were profound.

I recently sat down, virtually, that is, with Matt Booth, the engineer who was the catalyst to this project, to talk about our experiences and our aha! moments working on a new accessibility feature for Gears 5 as members of Microsoft’s The Coalition Studio.

The catalyst for this project was a review, written by a huge Gears fan, SightlessKombat, describing his frustrations and the inventive solutions he used to navigate Escape. We later had SightlessKombat consult on this feature, and he was instrumental in its final design. The idea, to add a new accessibility feature to Gears 5, first came on my radar when Matt emailed me some snippets of the review with a video attachment of a rough prototype. The moment I saw it, I knew he was on to something that could be great.

Katie: Tell me about the moment that you read that review from SightlessKombat and thought to yourself, “Huh, I can actually fix this.”

Matt: Honestly, it was almost like “Here’s the problem, and here’s how to solve it.” So that’s more than a seed.

Katie: Talking to other people and understanding their perspectives and what they experience… the thing that I love about that is, it’s so freeing. You don’t have to know everything. You don’t have to have answers. You don’t have to try and envision what it would be like. You just have to ask them! Then this giant weight is lifted off your shoulders because they give you the answers.

Matt: But asking somebody what their experience feels intimidating. Especially the first time you actually ask somebody about how they feel and how they deal with certain things. In the back of your mind, you’re thinking, “What if they just say no? What if they say, ‘mind your own business- it’s nothing to do with you!’” … but nobody is like that, at all! Everybody wants you to know how it is so that everybody’s experience can improve.

Gears of War, as a franchise, is well-known for its themes of brotherhood, comradery, and loyalty. We see these themes in Escape – it is a social mode, designed to be played by three people in collaboration. It’s no secret that creating shared experiences is a core pillar of what we do at The Coalition and for many of us developers, it’s what attracted us to the studio. This project has allowed us to examine these themes through the lens of accessibility. With the pandemic and social distancing creating more barriers to shared experiences than ever, it’s become very clear how important features like this are.

Matt: I use games a lot now to hang out with family and friends and really have that connection. They’re all back in England and I haven’t seen them for over a year. It helps to sort of bridge that gap a little.

Katie: You know, I heard something the other day that stuck with me, about relationships – about what it means to have a relationship with someone. It was said that having someone who is a ‘witness to your lived experience’ is what relationships are about.

Matt: Ahh, yeah! That’s really interesting, and I think most of my very close relationships are born out of a sort of shared experience. Really, you’re just witnessing each other’s experience at the same time.

Katie: Yes! And that’s what’s so beautiful about it. Accessibility in video games is making all of that possible – these shared experiences, these adventures. That’s why this work – that you do, and that we do at The Coalition – is so important.

Matt: There are definitely people out there that can’t have this amazing positive influence on their life because the games don’t support it. People who would really benefit from having this amazing experience I’ve had my entire life… I used to live in a house with a bunch of other guys and we would play Halo all the time – for every decision that you make in the house, you played a round of Halo to determine who would do it. Who’s making those teas? You go play Halo. The loser’s gonna make the tea! And at some point, I realized how much of a positive impact that was having. I was like, man, I want other people to feel this way as well. Maybe I can make these things that will affect other people in the same way that affected me.

There were many, many times during this project that we doubted ourselves. We wanted to make something that would enable players to play this awesome mode, Escape, and failure wouldn’t just mean a sub-par experience, it would mean total exclusion. For a few of us, it was also our first time working on a major accessibility feature. The stakes were high, and the unknowns weighed on us.

Matt: I was so scared I’ll be honest. I never really tried anything like it before. The techie part of me wanted to get involved but it was really scary. The fact that it all came together and worked was like the cherry on top! I didn’t just learn something – I’d be happy with that as well – but actually being able to help somebody play was like a really cool moment for me. A massive highlight. I was so scared but that made the great feelings afterward all the better.

Katie: I really love that fear is often a predictor of an amazing thing just around the corner or just on the horizon. There were many times that I wondered what the results would be, and I’ll admit, I tempered excitement for the feature internally… gave a lot of caveats. But, like you, the moment we got the feature into the player’s hands, I felt so much satisfaction. It was a great day. I think that fact that it was scary made it even more special when it was effective.

We knew it was extraordinary to be releasing a feature like this nearly 18 months post-launch, but there was support was coming to us from so many directions. I am a lead metagame designer at The Coalition, but I am also the studio’s Accessibility Champion. The gist of this role is to help my fellow developers navigate all the challenges that inclusive design presents. The Champion program is just one of many support resources that make features like this possible.

Katie: Why do you think The Coalition and Microsoft were uniquely positioned to do something like this? Why was it possible, especially to do this so long after we released the game?

Matt: I think the Coalition and Microsoft are very well positioned to deliver this sort of thing because they have a huge focus on accessibility. So Gears of War 4 had a big accessibility push and then Gears 5 pushed it even harder. It’s really not hard to see that it’s at the heart of the studio and Microsoft’s priorities. We have an accessible, inclusive environment, we’ve got all the different controllers that people can use, and lots of different programs and resources in the Xbox Game Studios – you don’t really see that so much in a lot of other spaces. There are definitely other game studios that are kind of pushing accessibility, but the whole package – I feel we get that here. I really like the fact that Accessibility Champions are a thing in game studios. It gives people a point of contact to relay their fears to and a point of contact to spearhead all the stuff that you’re too scared to do yourself – it gives everybody a lot of confidence. So, thank you for taking the time to be that person. I started this little pebble rolling and pretty soon it was like this huge boulder!

Katie: Thank you and I agree. I feel like 90% of my job is to be the one person that people can ping when they don’t know what to do! Yeah, just to be there… just to be the one with the inbox!

“My game is not a good fit for accessibility” – this is one of the more common reasons I’ve heard for the resistance to adding assistive features to games. We’ve done a lot of work at The Coalition to examine this stance and uncover how it holds us back.

Katie: What would you say to people who think that accessibility isn’t for their game?

Matt: I think you need to consider the reasons why you think it isn’t for your game. If it’s because your vision is a very, very concrete vision with very concrete parameters and it’s so set in stone… I mean who is this game going to? Who are you making that vision for? Is it just for yourself? Is it for others? Because hopefully, it’s for others to understand your vision. And if that’s the case, then why wouldn’t you want more people able to experience their version of your vision?

Katie: This is very serendipitous. I was just listening to a podcast that Isaac Murdock (a member of The Coalition as well) passed me. Actually, he’s the host of it and he was talking to a professor, Dr. Julie Rak… she was talking about Harry Potter. She was talking about literary analysis and that it isn’t just trying to understand what the author “meant” because people wouldn’t read Harry Potter if that was all there was to it. If reading Harry Potter was just about knowing what JK Rowling “meant,” you could probably just read an essay about it. But people read it. They assign meaning to different things. They theorize, they speculate. They fill in the blanks where the written word isn’t. They make it personal.

Matt: Yeah, yeah, yeah!

Katie: I think it’s the same thing with video games. This idea that a creative vision is “adulterated” by accessibility features… it’s like, you know, your creative vision was never yours to begin with. We all go in, and we all take what you made, and we put it through our own lenses, and we apply our own agency to it, and we fill in the blanks with things that are important to us. So, when we’re talking about games and accessibility, it’s kind of like, “You know what? You never had the control in the first place.” This is true for all mediums, I think. Is there a medium that makes it impossible for people to interpret or personally shape what you made?

Matt: Absolutely, absolutely so… I kind of think that people’s opposition, a lot of the time, to accessibility features is based on a misconception around how valuable they’re going to be, or misconception about how much time it’s going to take. Or just fear – fear of the unknown. Fear of actually asking people what to do. But once you kind of take all those things away from it. To your question, which medium makes it impossible for someone to interpret…

Katie: Yeah, I was thinking maybe… cooking? If I make you a hamburger, is there anything for you to interpret? You can’t change a hamburger. Well, no, you can take the pickles off!

Matt: I find with cooking; I always have to make it a little more accessible anyway because I don’t have the ingredients that I need.

Katie: Analogies everywhere!

You may be familiar with the stat that most gamers play with subtitles on – as high as 80%. This is a clear indicator that features designed to reduce the mismatches between players and the game are benefiting us all. There are many examples of this outside of games: the typewriter, electric toothbrush, touchscreens… these were all pioneered by people who were trying to empower someone with a disability.

Matt: When I play games, I do often have subtitles on. I play a lot of games with my kids, and they are learning to read and so having constant reinforcement about what words are and where the words appear is really helpful. We’ve also tried… Xbox has a system where you can sort of take partial control away from a controller onto your own controller.

Katie: Co-pilot!

Matt: Thank you, yeah! My youngest daughter really wanted to play with the system, but she couldn’t quite do all the actions that she wanted to do, and so I was able to take control and move around. And then one of the things that I do take advantage of for myself is some audio stuff. I can’t have a big, amazing surround stereo set or anything like that at home, because the kids are going to be sleeping. So, anything that mixes the audio like a pop song, like low dynamic range, just so I can actually hear all the environmental cues, I use.

Katie: Yeah, totally, I’m with you. I’m actually surprised to see how much I’m using assistive features. I was playing The Last of Us Part 2 and I turned on so many! They have a navigation ping system as well!

Matt: Oh yeah!

Katie: …and I was using it constantly! I was just like ping, ping, ping, ping, ping! I wanted to play the game, but I also wanted to control my challenge level. So, an accessibility feature really enriched gaming for me.

Matt: Yeah, I find that there’s like this point – and I don’t know if difficulty is the right term for it – but there’s a certain challenge level I want, and on any given day that can kind of change, right? You might feel differently on any given day. You want to balance it just right.

Katie: That’s why I don’t appreciate some of the language in video games like “beginner” difficulty. I can play a game for two years straight – I’m not a beginner. There’s an assumption I’ll start at beginner and want to work my way up to higher and higher difficulties, which is simply untrue. This project has really opened my eyes to a lot of aspects of gaming like this that are centering on a worldview that is not shared by everyone. We can’t presume to know why anyone would want to set some set their game in a certain way.

Matt: Yeah, yeah, yeah. A lot of it is really old hangovers as well, from the Doom Days or whatever. A lot these difficulty levels… you don’t really consider it because that’s your base and we haven’t reconsidered that. So now we have the luxury of being able to communicate with so many people so easily across the world and start to understand their experience. We can start to understand what they’re living and question the base views.

Escape is a resource-light mode by design – the player starts low on weapons and ammo and must pick them up along the way. Our new accessibility feature is focused on enabling players to navigate with audio cues, but it does not include cues for these other important game resources. We wondered whether navigation assistance alone could help players enjoy the mode.

Katie: One of the things that we talked about was fear. The fear that I had came from knowing this feature was limited in the ways that it could assist players. There are areas of Escape that we’re not providing audio cues for, and I wondered “Is this going to be enough to help anybody?” How did you feel about some of the areas that weren’t able to cover in terms of audio cues?

Matt: Obviously, navigation is very important. Without navigation, you don’t stand a chance of anything, but it’s a very resource-light mode. You don’t get a lot of weapons. You don’t get a lot of ammo. So, helping people find these resources to fight is a challenge, and that is an area that we’re definitely lacking. And there will be several things that you might miss along the way. You’re might miss loot rooms that give you really good heavy weapons or help you with that resource game. We can recommend melee heavy classes, but it’s no substitution.

Katie: Yeah, and one of the things that the playtest really helped me to see was that disabilities are on a spectrum, and while this feature might not fully enable all players, it certainly is getting many, many players from total exclusion to being able to play. A big takeaway for me was just that we cannot let perfect be the enemy of good.

Matt: As we saw with some players, this was enough for them to have some quality of life in that game mode, which was fantastic.

Features like Navigation Ping take months to produce, and we were often tempted to test the feature ourselves. However, we found that our attempts at testing a feature intended for players with low or no vision felt short, as the small group creating this feature did not have any vision disabilities. We ultimately did a playtest with several players with low or no vision and found their skills digesting sounds were much more advanced than our own.

Matt: I was very surprised by how quickly and intuitively the playtesters were able to just use this feature. We tried in playtests at home, closing our eyes to try and navigate with mixed success, but they were able to really hone in on the soundscape and everything that made that useful.

Katie: I’m totally with you on that one. That was a huge takeaway for me. We cannot “recreate” what it’s like to have a disability. And when we were testing it ourselves, we were centering on our own worldviews, which is not the same thing! I haven’t relied on sound as much in my life, so I’m not as good at digesting sound in video games and a key contributor to my moment-to-moment decision-making. The playtesters had amazing hearing abilities. Of course, this feature doesn’t work perfectly for us because it’s not for us!

Matt: Exactly. And as you say, all the time, “Nothing about us without us.”

Gears 5 is available now on Xbox Game Pass (alongside the entire Gears console catalog), the Microsoft Games Store, Steam, and at major retailers.