QA, or Quality Assurance, is an oft undervalued part of the game development process. Due to its nature, its greatest contributions are completely invisible while its greatest failures can render a game unplayable. Though it may not shine as brightly as art, music, or design, a good QA process can vastly improve the end user experience of a game.

To begin, there are 3 core values that define the objectives of QA throughout the development process:

1. Works as Intended
2. Good User Experience
3. Satisfies Requirements

The first goal of any QA process is to determine whether the game works as the designers and programmers intended. If a change to a game feature or system was made to affect A but ends up changing B, such a thing should be reported, as it may have unintended consequences (even if B ends up being kept as a better change). In practice, this goal is representative of the vast majority of the work done by QA.

The second core value of QA is to ensure a good user experience for the player. Throughout the development process, QA often serves either as a liaison between the players and developers or a surrogate for them. As a QA team often interacts with the game as a dedicated player would, they can detect frustrating or unfair elements more easily than most developers to determine what works in practice rather than theory. Likewise, they have more technical knowledge than the average player, allowing them to convey issues and suggest more reasonable changes than direct player feedback.

The third core value is making sure the final product satisfies any and all requirements placed upon it. These requirements can come from a variety of sources, from game directors and publishers to first parties such as Nintendo, Sony, Microsoft, and Steam. Regardless of their origin, it is QA’s responsibility to ensure that all required features are present and behave as expected. If a game features 4-player online multiplayer, it’s QA’s job to make sure that multiplayer is accessible and functional, and that it remains so in a variety of common and uncommon scenarios.

Before delving deeper, I will offer a single caveat to remind you that every development process is different, and the QA process offered below is merely a common expression of the above values. With this in mind, let’s look at how QA can pursue these goals at different stages of the development process.

1. Prototype

The first point in the dev process where QA would get involved is at the prototyping stage. At this point, game designers and programmers are trying to determine which ideas and mechanics are worth pursuing or investing in further. At this stage, QA is trying to determine:

• Does the prototype work well enough to evaluate?
• Are the ideas the designers are pursuing reflected in the prototype?
• What issues are present from a technical and phenomenological (play experience) standpoint?

2. Vertical Slice

After making decisions around game direction in the prototyping stage, the dev team then creates a vertical slice. This is a representation of what the dev team expects the final product to look and play like, showing off final mechanics, audio, and visual quality for a single mode of gameplay. Here, QA is primarily concerned with:

• The build’s visual and audio quality. Do they meet the standards of the product owner or game director?
• The gameplay loop. Is there a clear and playable gameplay loop present? Does it represent a gameplay mode that will be offered in the final product?
• Providing user experience (UX) feedback as surrogates for the players. Will people enjoy the visual style? Is the gameplay loop fun? What could be added or changed to create a more compelling gameplay experience?

3. Alpha

After the prototype and vertical slice, the game enters the alpha stage of development. At this stage, all of the core gameplay systems become implemented; in other words, the game becomes ‘feature complete.’ If a game is planning on launching with a single player campaign and online competitive multiplayer, for example, then a player must be able to access both of those modes in alpha and engage in gameplay in each. At this stage, QA tests for:

• High-risk tech that poses a risk to development. While these pieces of tech need not be of final quality, they do need to be functional from a technical standpoint.
  • Example: If your title is expected to support PlayStation Achievements, then the QA tester must be able to enter gameplay on the PlayStation platform and unlock an achievement, even if that achievement is titled ‘Placeholder 1’ with the condition ‘jump once.’
• Gameplay loops. Is the player able to engage in all of the different game types offered within the game and complete them?
  • Example: If your title offers online multiplayer, can two players connect via the internet and complete a match together? Can they complete a match from every planned mode (e.g. Capture the Flag, Team Deathmatch, etc.)?
• Secondary systems function. These are any systems that interact with the critical tech or gameplay loops to change player experience, such as matchmaking or lobby systems for online multiplayer.
  • Example: Your title expects to launch with online competitive multiplayer. Can you select ‘Ranked’ from the menus? Is there a matchmaking system implemented? Is there a rating system implemented? Do you have a system to handle disconnects?
• UX review. While iteration is more expensive at this stage in the development cycle than in prototype or vertical slice, it is still the role of QA to ensure the best possible end user experience. Whether or not the requested change is actioned upon, it is still important to make concerns known so that an informed decision can be made. This logic applies to every successive stage of development as well.

4. Beta

During the beta phase of development, the build becomes ‘content complete.’ In lay person’s terms, this term means that all content is at final quality and present in the build: every game mode, menu, model, and hair style. At this stage, the goal of QA becomes perfection, and the dev team becomes focused on fixing the myriad issues being reported by them.

• Visual target achieved. At beta, the quality of the art in the game should match that of the vertical slice for every asset (i.e. piece of content) in the game. Is a box clipping into the environment? Report it. Is a piece of text the wrong resolution? Report it.
• There is no inaccessible content in the game. Every piece of clothing can be worn, every weapon used, every quest completed.
• Testing uncommon user paths, edge cases. When releasing a game, thousands of people will be playing it in precisely the manner it is not meant to be played, and it’s QA’s job to figure out how.
  • Example: Your title launches with an online multiplayer system with a lobby. Have you tried inviting the same player multiple times? How about inviting someone already in the lobby? Or perhaps invite someone to the lobby and start the session before they load in?
• Does the game function as expected for the entire duration of gameplay? Does it crash under certain circumstances? Can you become stuck in certain areas or menus with no way out? Does it run at a consistent framerate?
• Games are often translated into other languages to allow players from different countries and backgrounds to play them. It is therefore QA’s job to ensure that all of those languages are present and accessible within the build, that the translations are correct, and that they are applied throughout. Every piece of text, from quests, to skill tooltips, to error messages, must be applied to each language.
• This is a requirement endemic to PC releases, but worth mentioning all the same. Unlike consoles, there are no unified standards, parts suppliers, or even generations to computers, which means that your game might not work as intended on an end user’s machine, or in some cases, at all. To minimize this risk, QA tests the build across a variety of hardware environments, determining the minimum specs required for systems to stably play the game and resolving as many issues with incompatible hardware as possible.

5. Release Candidate

The release candidate is the version of the game that you want to release to the world, and it is the role of QA to make sure the build meets the requirements of all the places that you want to release it. At this stage of the development process, QA is responsible for ensuring that the game is compliant.

• First Party: Nintendo, Microsoft, Sony, Steam, Epic, and other first party partners all have very long (and confidential) lists of what is required and what is not allowed in games on their platforms, to ensure the best experience for their users. If you plan on developing a game for one of these platforms, you will gain access to this list upon becoming a partner with them. Grab an energy drink and your best reading glasses, as you’ll need them to make sure that you get everything right.

6. Post-launch Support

Congratulations, you’ve done it! After looking through your compliance list and checking it twice, you’ve found no issues and launched your game across the platforms of your choice! While an amazing achievement, there’s still work for QA to be done.

One of the unfortunate truths of game development is that there are always more bugs to be found – they just occur under increasingly specific conditions. When a player encounters one of these issues, they will often exclude any information that would help identify and reproduce the issue in their report and include a range of creative expletives instead. During this stage of development, it is QA’s role to translate these vague and arcane bugs into easily reproducible issues that the developers can then fix. If there are enough updates to be implemented, or a high-profile bug that must be resolved as soon as possible, the devs can then release a patch to fix the outstanding issues.

Any content released after the launch of the game, whether it be additional DLC or a patch to fix problems with existing content, goes through the QA process from step 5, the release candidate. It is always dangerous to release a patch without thoroughly testing it first – you wouldn’t want to fix a lighting issue and completely break the game!

As mentioned previously in the article, this is not a prescriptive approach to the QA process: every game is different, and you might find your title has a vertical slice being delivered between alpha and beta, or perhaps another round of prototyping between alpha and beta. Whatever the specifics may be, as long as you abide by the 3 core values listed above, your QA rounds will be successful.

Happy Bug Hunting!

Kosmas Ulysses Stocking is the QA Lead at Modus Games with interests in game design and ludology. In addition to his current role, he served as a Narrative Designer for Override and Tyler: Model 005.