This is the first of a three-part writeup based on the talk "Five Years without QA: What I learned, What I missed, and What I wish I could change for a better software quality culture" that I presented at SOFTCON 2022: Metaverse Continuum held online on October 26, 2022
When I joined PageUp in early 2018 it was the first time that I joined a company without a dedicated Quality Assurance (QA) team. It was curious at first, but I realized the process didn't differ too much – just that developers were more conscious about possible bugs due to the increased responsibility over specific stories.
Over the years I have had my own fair share of support cases where there was no one else to blame but myself and failing to test specific scenarios. It was really just a different way of doing things at first. Sometimes though, there would be triggers that would you lead you to scrutinize the way things are done at work.
About a year ago I came across a meme that triggered those questions in my head, along with sour memories I have had with some QA testers I've worked with.
The meme led me to ask: what ever happened to the QA team at PageUp? It also just so happened that preparations for SOFTCON 2022 were underway; I decided to dig in to this whole issue and make a talk at the conference out of it.
What happened to PageUp's QA?
Talking to a colleague who had moved to our customer success team, instead of finding that there was a sweeping organizational decision to move away from having dedicated QA analysts. On the contrary, it just happened.
They were a small team of three people, and around 3 years before I joined the company their lead resigned. He then moved to the customer success team; the last QA analyst simply decided to follow suit and leave the company. The positions weren't backfilled, and so the devs took the cudgels of testing tasks.
To be honest though, to make it work, it's not as simple as that.
Post-QA Software Development
For any organization to exhibit a level of accountability that makes developer self-QA possible there are a lot of other practices in place that are part of a culture that fosters responsibility, a sense of ownership, and self-improvement. While this statement may sound preachy and idealistic, part of what makes it possible is a hiring process focused on looking for people who care about the code that they write and the product they come up with, ideally only onboarding coders with a passion for clean, maintainable code.
This is then supported with a leadership team that takes accountability for quality. In PageUp, a Product Trio takes care of each squad, composed of a Product Manager, an Experience Designer, and a Technical Lead. The Product Manager aligns large pieces of work – called an initiative – with company objectives and a product roadmap. The Experience Designer designs user experience and workflow elements, taking into account customer satisfaction as well as any existing feedback on the area being worked on. Finally, the Technical Lead is accountable for a technical solution for the initiative, drawn up in collaboration with different developers in the team.
Aside from the Product Trio being ultimately accountable for quality, the developers themselves need to have the right mindset and culture for things to work smoothly. A strong peer review culture, put in practice via pull requests, allows the team to be transparent in the work that they put in. Developers write automated tests, regardless of whether it's test-first before implementation or test-after. Strict linting rules and coding standards set the tone for these pull requests. A flat (i.e., non-heirarchical) organization culture amongst developers means that reviews and comments are sought from juniors by seniors, and not just the other way around, nor only from peers. Code ownership siloes are discouraged and cross-team peer reviews are sought when relevant. For stories written on more modern front-end stacks, automated screenshot-based tests are currently being put in place.
This in turn is empowered further by a strong customer support team, with highly technical 2nd-line support analysts already figuring out offending parts of the code or bad data. They also look for work arounds or resolutions before any support ticket is passed on to the developers. This means that there will be a clear picture of the problem, who's accountable, and developers can make better decisions around solutions to bugs that do reach clients.
So far, this set up has worked for us and hadn't prompted for the need to bring QA back in the fray. But why does this setup work in the first place?
The Power of Small Batches
Eric Ries, in the iconic book The Lean Startup, dedicates a chapter to the concept of The Power of Small Batches, a concept first described by James Womack and Daniel Jones in the book Lean Thinking. As told by Ries:
Womack and [Jones] recount a story of stuffing newsletters into envelopes with the assistance of one of the author's two young children. Every envelope had to be addressed, stamped, filled with a letter, and sealed. The daughters, age six and nine, knew how they should go about completing the project: "Daddy, first you should fold all of the newsletters. Then you should attach the seal. Then you should put on the stamps." Their father wanted to do it the counterintuitive way: complete each envelope one at a time. They – like most of us – thought that was backward, explaining to him "that wouldn't be efficient!" He and his daughters each took half the envelopes and competed to see who would finish first.
The father won the race, and not just because he is an adult. It happened because the one envelope at a time approach is a faster way of getting the job done even though it seems inefficient. [Eric Ries, Lean Startup, p. 184]
The crux of agile methodologies as envisioned in the Agile Manifesto and with the practices espoused by eXtreme Programming, had been, in fact, leveraging The Power of Small Batches. This had been the whole point of Iterative Releases, that is, to make complete units of work smaller, removing the need to work to be handed off to other people before declaring a release as complete.
In fact, I'll go out on a limb with an unpopular opinion and say that QA testers are a relic from non-agile practices like Waterfall, which treat quality assurance and verification of specification as a separate activity detached from the implementation of software.
In the next article, I'll talk more about thoughts of quality being intrinsic to the act of implementing software.