Ideal QA Process
November 3, 2017A good QA process can make or break a service, app, or product. On the surface it seems easy: check the product for flaws and make sure new features are done the way they should be.
Problems start to arise when you run into the question of “what should the feature be?” or “is that a flaw?” While a product owner, be that a cofounder, product manager, designer, or some combination of that, will typically know what they want, they may not have thought it all the way through or not communicated what they’re looking for. It helps when you’ve worked with someone for a while and you can anticipate desires, but that’s often not the case.
When a team is small, it’s easier to get away with not having a formalized process. It’s easier to communicate and it’s clearer who has what role. When a team and product gets larger, whether that’s lots of features or multiple platforms, then that informal process can start to break down.
Here’s how I see the ideal process once you start to have a larger, but not-yet-enterprise level team. These are the teams I’ve most often worked on and with. They’re later stage startups that can still mostly call themselves startups or have transitioned into being a larger company without truly being enterprise level.
Semi-Cross Functional
I’ve tried to find something better than “semi” for this, but it fits if we take it as what it really means: “half.”
Often QA engineers have to jump between different products or platforms like iOS, Android, and web, for example. That context switching takes effort mentally. What it often gets you is a QA engineer will have a moderate grasp of all of the platforms as opposed to an in-depth grasp of any. When possible, a QA engineer for each platform would be ideal. That means that when there are tickets ready to be tested for a particular platform, there’s no confusion about who’s responsibility that is and that QA engineer has the contextual knowledge around that platform.
To handle cases where a particular platform needs extra attention beyond an individual QA engineer or the engineer typically assigned to a project is out for some reason, the other QA engineers should have some knowledge about the other platforms because of the next point.
Bug Smashing
This can be regularly scheduled or based around large releases. In either case, QA engineers should switch roles and test on a platform they are not regularly assigned to. This allows someone to check against a platform they’re not used to. They might see something the regular QA engineer may have missed while also getting more insight on the platform they’re assigned to. I’ve found few things teach someone something new as well as moving onto something slightly different from what you’re used to.
For larger releases, it can be very helpful to have developers or managers from different platforms test the other’s app as well. Sometimes developers or managers can see missing features or corner cases they may have come across that haven’t been addressed on a different platform.
Alpha Dogfooding
Company-wide (or potentially just engineering-wide), every person should have the latest unreleased version of the app installed. This way larger bugs are caught before releasing to production. Additionally, this process provides fresh and potentially non-technical eyes on the product.
Automated Testing
This is where the benefits really start to become obvious. Ideally, QA engineers would be able to add and update automated tests themselves. This way they won’t have to spend as much time manually testing. When a feature changes, they can write a UI test for it while manually testing it the first time. Then, when regression comes around, they’ll be able to run the UI scripts and with some smoke testing, have a high degree of confidence that everything is working as expected. Additionally, by running test scripts on a variety of devices, you’ll avoid having unforeseen corner cases.
This does take a level of training and knowledge about the particular ecosystem. The payback after the initial work to get QA engineers writing automated testing will be worth it. It’ll make their lives easier and increase overall test coverage.
Finally, because a QA engineer now has a focus on one platform for most testing, they’ll be able to get more involved in automated testing. They won’t have to know all of the features of automated testing for all of the platforms, but instead can focus on one to get a deeper knowledge there.
Stakeholder Buy-In
There should be a stakeholder review process beyond just the typical QA testing. It gives the product person with the final say a chance to weigh in so that requirements are well-defined and clear. For any given feature or bug, it should be clear who the stakeholder is. Sometimes the stakeholder is the project or product manager, sometimes the designer (and there could be a few), or legal, etc. While the project manager will know who the appropriate stakeholder is, having one fewer step in assigning to the correct stakeholder is ideal while also lightening the workload on them.
Finally, it’s best to have the person who writes the ticket and who does the development be separate people. This ensures that the ticket and test steps are clear to more than just one person so they’ll more likely be clear to others.
