Watching Playthroughs as a Means of Evaluating Games
What Playthroughs Are and Types of Playthroughs
Comparisons to Other Media
We gave the example of live theatre earlier, which is a great example of fans using different means to evaluate a work, so we're expanding on it a bit here. A musical like Hadestown is widely regarded as a worthy SF work, but very few people would have been able to experience it fully as intended during its window of Hugo eligibility. Fans may live too far away and not have the means to travel, or live proximate but not be able to afford to go. The COVID-19 pandemic disrupted its release to the broader public. A fan may have sensory limitations or issues that make processing the experience of watching a live performance difficult, unpleasant, or impossible. For these and many other reasons, we broadly accept that reading the script, listening to the soundtrack, and watching clips or full recordings (official or illicit) are valid means towards saying you're a fan of a musical and for judging how good you think it is. These methods are obviously not equivalent to the experience of a live performance, but they are perfectly valid, whether this path is a choice or a necessity. Certainly we wouldn't tell a blind person they didn't have a sound basis for voting on a musical. The experience people have of a work is the experience people have.
Audiobooks provide another example. With an audiobook, you experience the prose at a speed and style not of your choosing. WSFS members agree that this is a valid way of experiencing a prose work. This can make prose more accessible to some readers, or accessible at all, e.g. if someone is low-vision or if their library only carries the audiobook version. Audiobook narrators vary in style and in quality, and different people will prefer different narrators. The same is true for players who are making Let's Plays: as someone watches more streams, they will learn whether people are good players, or if they do a style of playthrough that suits them.
We also evaluate movies watched on our laptops even though in many many ways it's not the intended experience, e.g. not getting to see as much detail or experience the fine points of the sound design. We even feel perfectly equipped to evaluate games without playing them when it comes to sports. Sports fans say, "this is a good sport," without playing it, whether they mean fun to watch, requiring skilled play, or any other metric. We even compare entire sports wholesale ("I prefer European football over American football") without playing them.
In the end, we accept many means of evaluating specific works or even entire genres of entertainment. More to the point, we accept people as fans based on the experience they are capable of or prefer to have.
Evaluating Playthroughs and Finding Good Playthroughs
A common issue even for those well-familiar with playthroughs is finding quality playthroughs and/or streamers that they like. It also must be acknowledged that the gaming community has suffered under virulent toxicity, some of which has even spilled over into the world of the Hugos via the Puppies. Navigating this landscape to find streamers compatible with your viewing and/or playing style is a known issue within the community, just as much as streaming is a wildly popular and accepted means of experiencing games. For purposes of Hugo voting, many voters will want to focus on evaluating the story without overlaid commentary, but as stated previously there are many types of playthrough that cater to different evaluation and entertainment preferences.
Towards this end, both major platforms for playthroughs — Twitch and YouTube — provide means for finding compatible players. Twitch categorizes by the game the streamer is playing, so viewers interested in a particular title can browse streamers who actively play it. Twitch also allows users to tag their streams in various genres and categories. Browsing by lower average concurrent viewership (AVC) will often lead to streams where you have a higher chance of influencing gameplay, and watching the chat in higher AVC streams (as well as listening to any commentary from the streamer) will generally quickly reveal the tenor of the stream. On YouTube, there are many channels devoted entirely to no-commentary playthroughs and game movies. Searching for a given game alongside the term "no commentary playthrough" will yield multiple results for most major or popular releases. The absence of commentary and chat obviates many compatibility factors with players and fellow viewers and allows for a focus on the game and the skill of the player. However, many viewers enjoy commentary, especially from "cold" or first-time playthroughs, in which case just a couple of minutes of watching actual gameplay (or even hearing the player's commentary on opening cutscenes) will generally signal player skill and compatibility.
It sounds like a bit more of an art than a science, and that's true! It's much like finding good audiobook narrators. If you find one you like, you may even follow them to other works that you may not have otherwise given a try, and the same is true for players doing playthroughs.
It's also important to point out that there is a strong community factor in play, especially in the context of Hugo voting. While Hugo administrators are very unlikely to include particular playthroughs or reviews in a Hugo Voter Packet in order to avoid bias (though please read our section on how a games Hugo Voter Packet would work — and has worked!), fans are welcome to recommend things to each other, and we expect this will become part of the community discussion about nominees for Best Game. In fact...
Example of Crowdsourced Videos on the 2020 Best Video Game Finalists
...The Games Hugo campaign has put together a list of some examples of videos that could have been used to evaluate the 2020 Best Video Game finalists without playing them. The biggest factor here is time, which is an aspect we've covered elsewhere in the FAQ, but watching videos such as those below is still generally faster than playing a game cold yourself. The videos below are a mix of playthroughs, game movies, recaps, and reviews that give some perspective on each game and what fans found valuable about them (or not). These are not meant to be recommendations of any particular channels or approaches and more a survey of the sort of easily accessible content available for various titles.
Animal Crossing: New Horizons
Final Fantasy VII Remake
The Last of Us: Part II
Would Playthroughs Themselves Qualify in this Category?
The Bottom Line: Accessibility and Democracy
A crucial aspect of playthroughs is that they democratize access to games in terms of finances, logistics, disability, skill, and any number of other factors.
Watching other people play often helps even those who play the game themselves, and often increases accessibility in other ways. Content that is difficult, gameplay-wise, to access can be shown off by skilled players so that a wider audience is exposed to it. Gamers also often use playthroughs to evaluate whether they want to play — or, more to the point, purchase — a game themselves. In the gaming community, it's a popular and well-accepted way to judge works, and it removes barriers such as ableism, classism, and other exclusionary factors in the same way that an audiobook makes prose accessible to a blind or low-vision person, or a script makes a dramatic presentation accessible to a D/deaf person.
It must be acknowledged that watching playthroughs for evaluating games does elide the characteristic that the Games Hugo campaign maintains sets games apart from all other media: the interactivity. Many games have such widely branching narratives or significantly incorporate haptic elements that watching playthroughs leaves out significant aspects of the intended experience. However, many other games, such as The Last of Us, are entirely linear, with the gameplay simply serving to gate and pace the linear plot, and for those unable to do or not interested in attempting the mechanics of its execution, watching the story play out and how a player navigates the challenges (which character attributes to focus on, what weapons to choose, aiming and shooting, executing stealth) conveys plenty of information about the quality and content of the game. That doesn't make The Last of Us better or worse, or less of a game, and that doesn't make watching playthroughs of such a game more or less valid than others. Games themselves are constantly evolving in their accessibility features. The Last of Us: Part II has robust accessibility features, but that doesn't mean that players who could play the second game but not the first game are less qualified to give opinions on the first game just because they couldn't play it themselves or had a very different gameplay experience. In fact, it gives such players an entirely different and rich aspect on which to evaluate both games qua storytelling experiences for them. It's not up to us to decide that watching playthroughs is not valid, whether it’s a choice made by the audience or a necessity. You might not ever evaluate a game this way for Hugo voting purposes. But it shouldn't stop others from doing it, and for having a category in which to do it.
To be clear, we are not saying that watching playthroughs is equivalent to playing a game. The entire basis of our proposal is that the interactive quality of games sets them apart from any other type of storytelling, and watching a playthrough is simply not the same experience as playing a game yourself. However, we firmly believe that watching playthroughs is a valid basis for voting in the Hugo for any individual voter so inclined. It does not mean everyone, even every gamer or Hugo voter interested in games, has to espouse this view vis-a-vis their own voting. But policing the means by which people experience a work is ableist, classist, and generally exclusionary. It's ableist to say that blind or low-vision people cannot evaluate a novel via its audiobook, or that D/deaf people cannot evaluate a podcast by its script and supporting materials. It's classist to say that those who can't afford to see a live theatrical production can't form opinions on it based on reading the script, listening to the soundtrack, watching clips, or watching recordings of the show. Players with mobility or processing issues (or any number of other conditions or even preferences) may not be able to play a given game themselves, but they can still decide if it's good in their opinion. None of these experiences, obviously, are equivalent, but they are valid.
In the end, it's antithetical to the democratic spirit of the Hugos to say that this well-established means of experiencing games is not a valid way for any given Hugo voter to evaluate games.
We believe a Best Game or Interactive Work Hugo category not just could, but needs to exist. And playthroughs are a valid way to participate in this category.