Nowadays, playtesting seems as crucial to game development as the game design itself. There are so many moving parts in a game, and people perceive and experience games in so many different ways it would be crazy to publish your game without having it tested by your potential audience beforehand and evaluating their response to what you have designed. Playtesting is a great way to discover what works and what doesn’t before Steam reviewers scream it at you with torches and pitchforks.
However, testing can have a huge downside, making your game bland. In a way, it already has. If you have ever wondered why all the big AAA games look and feel the same, why are they so dumbed down and why they don’t take any risks, the answer is that’s the structure that gradually formed when game developers watched how people play their games and tried to optimize game design based on their findings.
How Playtesting Can Make You Lose Faith in Humanity
Between all of the game companies, perhaps Valve is the one most obsessed with playtesting. Mark Brown has made a video titled “ Valve’s “Secret Weapon” recently, talking about how Valve learned to regularly playtest their games after realizing Half-Life 1, their first game, sucks just months before publishing it and being forced to make it from scratch.
Valve’s testing has provided many benefits for them. One of the reasons their games are immediate successes is that they leave very little to chance. By the time their game is published, they have a very good idea of how it will be received. As a matter of fact, one of their most iconic creations – GLaDOS, the main villain of the Portal series – was created as a direct answer to player feedback; the playtesters loved the game, but they said they couldn’t wait to experience the real game. They WERE experiencing the real game, but playtesters felt like they were still in the tutorial section since there was no narrative arc. Enter GLaDOS to fill that void.
However, Valve has talked extensively about its experience testing games and published its findings as commentary tracks for some of its more famous games. In one of these commentary tracks for Half-Life 2: Episode 2, Greg Coomer says:
Before we simplified the path through the guardian’s lair, we allowed a right turn at this junction, leading the player right back to where they began. One of our playtesters continued to repeatedly turn right here for half an hour. That was a compelling argument for removing the mazelike elements.
In another instance, John Morello said this about Half-Life 2: Episode 1:
Late in development, we changed the art for these doors so that players understood that they were entering the core area. It turned out that having Alyx talk about the area wasn’t enough to give playtesters the sense that they’d finally arrived at the core.
When you think about it, what these people witnessed must have made them lose faith in humanity a little. To witness someone make the same mistake for 30 minutes or not paying attention to dialogue in a single-player story-based game would discourage developers from ever being subtle or sophisticated.
These examples only scratch the surface. If you delve deeper into player feedback within big studios, you’ll uncover numerous similar instances where people overlooked the most obvious things. Gradually, these reports of playtesters missing the mark profoundly impacted the entire game industry, compelling developers to simplify and reduce subtleties in their games. Suppose you’ve ever wondered why modern open-world games feature prominent arrows guiding your every move or why action games with puzzle elements, like God of War, have characters explicitly explaining how to solve already easy puzzles. In that case, the main reason lies in the results of testing. As testing expanded beyond the realm of hardcore gamers and devoted fans to encompass a broader audience, this shift became an inevitable course of action.
However, there’s one question that naturally comes to mind: why don’t developers brush off these instances as people acting dumb and stick to their vision?
Playtesting: Art Vs. Product
The answer to the question above is related to a fundamental question about video games: are video games art or product?
Yahtzee Crowshaw touches upon this subject in his video “For Everyone that Says I Hate Video Games”:
There’s a distinction between reviewing something as art and reviewing it as function. With art, you can only offer your own perspective, analyse the emotions that it brings out and the artist’s intention. Reviewing for function is a lot easier. You mow the lawn, and then you give the lawnmower a score based on how short the grass is now, so the reader knows what’s the best lawnmower to buy. And the sticking point of video games is that they have both an artistic and a functional aspect.
This is the issue here: when game developers are testing their game, they are essentially testing for a “lawn mower”, a product sold on the market with a practical purpose: providing entertainment for as many people as possible. If a company is testing its newest model of lawn mower and one of its testers is using it all wrong, they can’t say, “well, this guy is dumb. We are making this for intelligent people.” The point is that even the dumbest person should be able to use the lawn mower because if you are selling a product on an industrial scale, everyone should be able to use it. Intelligence doesn’t come into play.
But art doesn’t work that way. Art doesn’t exist because society asks for it for practical purposes. On the contrary, sometimes art exists to take society outside of its comfort zone. Sometimes great art is rough, coarse, uncomfortable, and even awkward because it pushes boundaries so that no one can judge it based on existing assumptions; great art wants to create new assumptions.
So how can we reconcile these two different points of view about video games during testing? When you think about it, playtesting is a huge existential threat to anyone who wants to approach video games creatively, discouraging them from attempting certain cool, unorthodox ideas because they know beforehand that the playtesters wouldn’t get it or mess it all up. If people have a hard time distinguishing between their left and right or listening to simple dialogue, that sets the bar pretty low, doesn’t it?
Playtesting: An AAA Problem
Playtesting wasn’t always so inclusive. Nowadays, AAA companies might even playtest games using children or non-gamers. Still, in the past, the games were playtested by the game developers themselves (referred to as “In-house playtesting”) or people who were interested pr proficient in the genre (like hardcore FPS players for an FPS game).
By and large, the increased diversity of testing has been good for the industry. It has helped game developers to expand their horizons about how their games are perceived. Sometimes the player feedback for hardcore gamers can be counterintuitive. For example, when Valve was testing Portal’s final level, hardcore FPS players said the final level required an action-oriented challenge-based boss fight. Still, Valve realized such a level didn’t feel right for everyone else who had gotten used to the game’s more cerebral pacing. So the final level is another puzzle; You don’t defeat GLaDOS by shooting her, but by outsmarting her.
However, there’s a problem here: you can’t keep testing games using people who are not its main audience, and not having their feedback influence your game negatively in one way or another.
Imagine if From Software playtested their games like your average AAA developer; They would turn into a mockery of what they are now! For example, in Elden Ring, one of the first enemies you see is Tree Sentinel, a big golden knight that would kick your ass if you fight it (but technically, it’s possible to defeat it). The purpose of the Tree Sentinel is to show you that you don’t need to engage with every enemy you see on the open field; sometimes, the best option is to avoid enemies altogether. In your average playtest setting, many testers would fight the Tree Sentinel, die to it, go back, fight it again, die again and then feel frustrated.
If From Software had the same mentality as Valve, they would remove the Tree Sentinel to save the players from this frustration. But the point is: the frustration is part of the experience. It’s the unique way the game teaches players an important lesson.
This idea that the frustration of players during testing is something bad that should be fixed results in bland games and bland experiences, and it is something that AAA games, in particular, suffer from, because AA and Indie games are usually playtested by people who are already interested in those games and played many other similar titles.
The people in your average AAA testing session might not be familiar with the conventions of the genre or the way video games are played. Still, the AAA developers might be compelled or even forced to change their game based on their in-game performance, which, judging by those Valve commentaries, is not a testament to human intelligence.
But this is not me being condescending to these playtesters. Even I might act dumb during a testing session because when you’re in the comfort of your own room playing a game you’re actually interested in, with full focus, your performance is different compared to being in a room full of strangers, having your every move surveyed, playing a half-finished game you have no emotional connection to. If I get put in that kind of environment, the feeling of being constantly watched might distract me and make my performance seem dumber than usual.
Why Dumbing Down Games Based on Playtesting Results Is Wrong
So if game developers pay attention to the kind of player feedback they receive at testing, they might make changes in the game that, in the grand scheme of things, make it more dumbed down. In other words, the data received at testing can be misleading. Because of this data, we have quest markers that always tell you exactly where you should go, even though there are all sorts of environmental and dialogue-related clues that point you in the right direction. When people are testing a game, it’s natural not to pick up on these clues because they are not immersed in it. But things are different when you are playing a game by yourself.
Game developers need to have faith in their players and don’t let playtest results to make them think they’re dealing with dumb people. They need to believe that whatever vision they have in mind, people are going to ease up into it in the comfort of their own home, playing the game on their own terms. Even if we are making judgments purely based on sales, many best-sellings games will not survive an average playtesting session: like From Software games and Cuphead. But when they were released just as the developers intended, they found their audience, and none of the seemingly frustrating and idiosyncratic qualities prevented people from liking them.
The fact of the matter is: no matter how much you dumb down your game, the majority of the people who bought your game are not going to finish it. You can easily confirm this by visiting a random game’s page on Steam and checking out the percentage of people who have finished it in the achievement section. Even for a highly popular, easy-to-finish game like Bioshock Infinite, the number is less than 50%.
People give up on games not just because they are confused by it; they abandon games for all sorts of reasons. Most of the time, it’s nothing personal. It’s just that there are so many distractions, so many things to do, so much stuff to deal with. The sad truth is that no matter how good your work is, it will only hook a very small percentage of the human population. Dumbing down your game for non-gamers cannot attract their attention because there was none to give in the first place, but it can make gamers who are the audience lose respect for it. AAA developers that change their games based on the player feedback by those who get frustrated easily are like insecure people who change their personality to date someone who’s not that interested in the first place and end up being dumped by them anyway. It’s a lose-lose situation.
So What’s the Message Here?
The point here is not that player feedback is bad. Testing is very useful and necessary. The point is that game designers should not let player feedback to dumb down their game. This has obviously happened because you can see the signs in your average AAA game nowadays.
The best way to use testing is to have an original vision in mind and then playtest it to see how much people are close to experiencing that. For example, when Nintendo was testing The Legend of Zelda: Breath of the Wild, they knew they wanted to provide the player a true sense of freedom, but they didn’t know how because that was their first open-world game.
Instead of asking the players what was wrong, they made a heat map of how they played the game. Initially, the heat map showed that 20% of the players wandered in the game world, while the other 80% just followed the trail of story-related Shekah towers.
Nintendo realized that this playstyle did not reflect the “freedom of exploration” they had in mind, so they made the game’s open-world environment more varied and added more points of interest. Thanks to this new change, players didn’t feel like they were guided by an invisible hand. In the new heat map, everyone was exploring the world randomly, constantly being swayed by the next shiny thing, but ultimately finding their way to the next Shekah tower on their terms. If you want to read more about that, watch this video by Mark Brown.
As we can see in this example, Nintendo had the right idea when it came to testing:
- They had a vision of how their game should be played
- They playtested to see if the players were playing the game how they intended it
- They realized that’s not the case
- They created a solution
- Playtested the game to see if the playstyle of the playtesters reflected the new change
- Didn’t dumb down the game, stuck to their vision to the end
So that’s pretty much the key to how testing should be done: the developer should already have a vision of what the game should be like and they should use playtesting to see how close they are to that vision, not to be discouraged and swayed by any misunderstanding or underperformance on the playtesters’ part.
