The importance of automatic testing: An interview with PRISMA’s Tom Köckeritz

Not all would be comfortable admitting it, but the truth is that many tech companies cut corners when it comes to product testing. But as Automated Software Testing has become an ever more effective alternative to traditional testing, the team at PRISMA are increasingly turning to this method to ensure our products and services keep improving.

 

It’s a common story told by developers: there they are, delivering an impassioned pitch to senior management on why testing is essential for improving your product, when the realisation dawns that it’s simply never going to get sign off.

“That all sounds great,” a manager might say, before the inevitable caveat arrives. “But I think our focus needs to be on the next feature, rather than testing the one we’ve already built.”

But the reality is that as you build more features, the more complex your product becomes. And the more complex your product, the harder it becomes to test. Better, therefore, to nip things in the bud by testing sooner rather than later, surely?

That’s certainly the prevailing view at PRISMA and one that’s shared throughout our organisation – including, we’re pleased to say, senior management. And one reason why there’s such a broad consensus on its benefits has been our decision to use automated testing, which has proven to be a significantly more reliable and time-efficient method than traditional, manual forms of testing.

On the frontline of PRISMA’s automated testing programme is Tom Köckeritz, our Software Quality Assurance Manager. We sat down with Tom to find out more about how automated testing has enhanced working practices at PRISMA, both for individual personnel and the company as a whole.

 

Thanks for speaking to us Tom – could you start by giving us a layperson's definition of what automated testing actually is?

“So essentially automated testing helps us to perform repetitive tasks that were previously performed manually by humans. For example, in times gone by, if you had to check that the login mechanism to your website works after every small change implemented it would need to be checked all over again. Before you knew it, a whole day or even week had gone by – and because it was so laborious small errors could easily creep in.

Not only is this method boring and time-consuming, it’s also financially inefficient for a company to get one person to do the same thing over and over again. Automated testing, in contrast, allows all of these checks to be performed automatically on a computer, and at any time of day, meaning you aren’t restricted to testing during business hours.”

 

Can you describe how it compares to the previous method of testing?

“The goal of testing is to cover the core ways in which your users interact with your platforms, and for us that’s auctioning, placing a bid, creating secondary trades, etc. In the past, our process involved testing functionality after implementation. So we’d create functionality and once it was transferred to the production system, we’d then start testing it to check whether it was working as expected, through writing and executing test cases without the help of any software testing tool.

Previously we’d provide step-by-step instructions for how a test should be performed by an – first login, then click menu on the left side, and so on.  We still have those step-by-step instructions, but the difference is that now they’re executed by a computer, rather than a human.”

 

When was the decision to switch to automated testing made and was it driven by a particular incident?

“It actually coincided with me joining PRISMA a couple of years ago. At that time we were half the size of today in terms of staff, and I quickly realised that to test everything in the old way we’d need more personnel. So I got thinking about our options and concluded that by using automated testing we could firstly make huge savings on human resources, and could secondly allocate resources more efficiently by freeing up individuals to focus on the stuff that really needs human engagement.”

 

How would you summarise the main benefits of automated testing in terms of its effect on the end product?

“We try to use automatic testing for every new feature that gets developed for our platforms. Our approach is to think about the test phase in parallel with the design phase of a feature, because we firmly believe that the earlier you think about test cases, the quicker you’ll discover what may be missing in your design. This is the best way of ensuring the end product is as good as it can be.”

 

Are there any negatives to automated testing?

“The main goal of automated testing should be to perform repetitive tasks because that’s the stuff a computer can understand. For more complicated and longer workflows that depend on each other, or require checking the content of emails for, example, this is when automated testing can become counter-productive. Another problem is that the more complicated an automated test, the more error-prone it’s likely to be, and you end up expending huge amounts of time on fixing things. In short, there’s a tipping point at which the effort required for automated testing negates its benefits.”

 

What would you say to a company that doesn’t bother with testing at all?

“I do understand that if you're starting to build a product, you need to get something out there as fast as possible to gain some attention. And it’s probably true that some smaller companies can probably get away without a comprehensive testing programme. But in general, the bigger a company gets and the more complex your product gets, the need to put at least some resources into acquiring staff with testing expertise becomes undeniable. There’s simply no substitute for a deep understanding of testing, and so you have to hire the right people with experience in this area.

I also think that as a company grow, the greater the imperative for automated testing becomes. Imagine there’s a testing resource that’s been implemented and overseen by a particular individual. What happens if they leave the company? You lose all of their knowledge too. A new tester then comes in who’ll find it impossible to replicate the same methodologies. Automated testing is a safety valve against this, as a computer never forgets.”

 

Finally, how would you say that automated testing has improved your own working practices and ability to be efficient with your time?

“The crucial thing is that when automatic tests are set up for the more simple stuff, this frees up your time to focus on the more complicated stuff. And it also gives you peace of mind that even if a test passes 100,000 times, if there happens to be an edge-case scenario, however unlikely it is, the automated test will have it covered. Without automated testing, the edge-cases are always vulnerable to slipping through, because as individuals we don’t have the time to endlessly conduct tests. So in this sense, automated testing is essential in keeping people like me fresh and motivated. And that can only be to PRISMA’s benefit.”