Developing the QA Mindset: Curiosity

Hey folks, I’m back again with the final installment of the QA mindset series. So far I’ve talked a lot about how and why we step into the QA mindset, but I haven’t really touched on what it actually is. If I were to sum it up in one word, I would say it’s curiosity. The trouble with that answer is it’s very hard to try to tell someone how to be curious. For today’s post, I’m going to try to do so by looking outside of our day-to-day work and exploring a few examples of others applying curiosity to what they do.

The first example comes from a job many years ago. I worked at a two-sided market place, so our users were both people buying things and people selling things. The sellers were a particularly lucrative target for scammers. The ploy was simple: a scammer would create an account with an avatar and name that appeared to be a site admin, send the seller a message that a sale had failed to process, and ask the seller to provide updated bank account information. I was on a project that was trying to prevent those scammers from contacting our sellers.

The messages started off easy to identify. The body was plain text, the username was overt, the avatar was simply our company logo. We built an AI model that classified these as spam, and the rate of sellers opening these messages dropped precipitously. In response, the scam messages steadily became more obfuscated. The avatars were just the first letter of our company name in our classic font, then generic clip art of people in tech support. The body became images instead of plain text. As we rolled out optical character recognition, the scammers began tweaking the resolution of the image and tinting the background to try to evade detection while still appearing legitimate enough to convince our sellers.

The most frustrating part of this work was that the scammers always seemed to have a new tactic that we would have to account for. The relentless evolution of their attacks brought us to a boiling point when one of my team members openly suggested we could have a mole in our company. That’s when the director overseeing the project brought us back down to Earth. “It’s simple,” he said, “these scammers are just using the QA mindset to find our weaknesses.”

While I’m not suggesting you start an illegal social engineering ring, the analogy is apt. These scammers were constantly asking, what if? What if I swap out the avatar? What if I darken the image background? What if I tweaked the language here? While their actions had a very specific goal of thwarting our defenses, the crux of their success was relentless curiosity. We can apply that same curiosity to our own testing practices to head off bad actors from doing it themselves.

My second example isn’t quite so bleak. Instead of using the QA mindset for nefarious purposes, this creator uses it for entertainment. Josh Knoles of the YouTube channel Let’s Game It Out regularly posts videos in which he gets an early-access video game and, for lack of a better description, intentionally plays it wrong. He forces his way out of bounds, pushes the limits on all features, and in general finds any way possible to complete his tasks while avoiding what the developer intended. As an example, here he is playing Parcel Simulator. I think this video is great. If you watch it, try to listen to Josh’s thought process as he goes through the game because it’s a fantastic model for how to approach an application with curiosity.

And that’s it! Four posts on the QA mindset. I hope you’ve gained some useful knowledge somewhere in all of this. I’ll be back again next time with a post on a new topic. In the meantime, feel free to keep the conversation going in the comments.