Get your business leaders to buy into test automation

What are some of the common misconceptions business leaders might have about test automation? Then how would you go about addressing those?

Test automation is often seen by business leaders as something that’s going to solve all of their problems. There’s quite often a lot of excitement about having something that is covered by test automation. But in reality, there are quite a few things that do need to be considered.

A common misconception from business leaders is that automated testing takes the place of manual testing. That’s not the case at all. Manual testing is a different skill set from automated testing. The automated testing side of things will allow you to check things that are straightforward, very common, and very frequent (run through several times) whereas the manual side of things is where you’re really making use of the tester’s eye – the creative side, looking at the edge cases, looking at what’s going wrong? The automated test will only check for what it’s been told to check for. Manual testing will cover a bit more so they’ve both got their benefits but the business leader has to understand that they’re not going to remove the manual test team. You may get some of the manual testers picking up the test automation and starting to do test automation, but it’s not one or the other. A blend of the two is quite frequently the approach.

The other thing that has to be explained is that it’s not a one-time effort. You’re not going to come in, set up an automation suite and go away. That automation suite takes a lot of time and effort to prepare. It needs to be updated as the product that you’re running the automation against matures so once you’ve got your automation in place, it needs to live and breathe with your product and to get the most value out of it.

Another thing that business leaders often expect is that it’s going to guarantee no defects! You come in, press the button and great, you find all the defects and everything goes live without a hitch. That’s not the case. It helps you reduce the amount of defects but it will never eliminate all of them. And one of the aspects of test automation is that it’s effectively running the same test a lot of the time. If you have an automated regression pack, it’s only going to be running the same tests. The software underneath will change but the test themselves may not change. If it’s not looking for something, it’s not going to find it so again [you] have to come back and complement that with other tests or just accept that’s the case.

How could you measure the ROI of test automation and then communicate that to the business leaders?

Business leaders often want to make use of test automation but may not know what it’s going to give them. The idea of running these tests quickly and getting the results quickly, it’s appealing, but what are you trying to achieve from that?

One of the main [benefits] of test automation is saving time. It will be quicker to run those tests when they’ve been created. It may take a little bit more time to set them up and get them to a good position but the benefit will come from being able to run those more frequently and more quickly. Reduced test cycle times, quicker time to market, and quicker time to find defects and resolve them are good metrics.

If you are putting in an automated regression pack, the start point of that will be your manual regression pack so in terms of a metric, you can demonstrate how quickly you are converting your manual, regression tests into automated ones. That will give you a view of progress in terms of what you’re moving towards with your automation, and then linked in with the time saving, that helps show benefit.

Then there’s defect detection – if these automated tests are going through and finding things quickly and frequently that’s the sort of thing you want to be capturing. Quite often, people don’t make the most of the automation and shout about it. If it’s the automation suite that’s found defects then quite often it’s passed straight to the development team and fixed before it’s even appeared on a defect dashboard so it gets undersung, sometimes, the benefit of doing it. Having an automated smoke test, for example, will run tests against the new version of the product. It will find a defect with that product before it’s got to the testers and then it can be resolved saving their time. Quite often people don’t capture those defects because they’re so easily found and fixed that they don’t show up in defect dashboards, but it really does make a difference to record those and then you can demonstrate how many defects have been found by the test automation.

In terms of communicating that to business leaders, what we always suggest is that you communicate in a simple, visual way. If we’re describing statistics then having a nice visual dashboard is always beneficial so that you can see the benefit that’s being achieved. You can see how many test cases are now converted to automated test cases. You can see the effects that have been raised by automation and you can see the time savings. And if that’s done in a nice dashboard with a bit of a story around it that you can explain as a result of test automation ‘we’ve now achieved this’, that goes a long way. Just putting stats on to a page doesn’t help tell the story so you want to have some commentary as to what’s been achieved and if those can be tied into the business goals and objectives as well then that’s even better, to be able to go back and explain the benefits of test automation.

How do you properly make a business case for test automation that will give the technology and the team a chance to succeed?

It tends to be that the further you go up to the decision-making chain, the less close people are to the details so you have to be very careful to not use jargon and bamboozle people with technology talk – it needs to be clear and descriptive, and tied into what the business wants to achieve. Rather than going in and talking about the ‘latest technologies’ and mechanisms and methods, it’s ‘what are we going to get out of this test automation?’

A lot of the time it’s to do with quality and risk mitigation. If we’re able to explain that bringing test automation in for this particular project is going to reduce the risk of introducing defects into production, that’s going to be of interest to a senior stakeholder. If it’s going to allow the product to be delivered quickly and at high quality, that’s going to be a benefit to a stakeholder.

Another aspect is efficiency. If we’re building test automation into our product delivery we’re shortening the feedback loop, we’re reducing the manual test effort, and we’re able to get things out to people’s hands a lot quicker, then that helps sell the benefits of automation.

When providing a business case, it’s often useful to be able to bring another business case, to be honest. If you’re trying to convince somebody that test automation is a good idea, a business case study from another industry or another similar client can help explain exactly what is needed to deploy the test automation and what benefit you’re going to get from it.

One thing to consider if you’re pitching to a business leader is their starting point. Are they new to test automation? Do you have to explain at least the high-level concepts of test automation to them? Or actually, have they been down that road already? Quite often you’ll find that a business leader will have tried test automation and it didn’t work out. In that case, you need to explain some of those pitfalls and say how you’re going to mitigate them. Sometimes it’s a single person that set up an automation framework [and] they didn’t think about designing it to be more maintainable or easy to add on to and they’ve had a bad experience from that point of view. Being able to come to them with some solutions to those bad experiences is very good and if you can get the approval to do a pilot, nothing beats seeing a small implementation of test automation on one of their projects and then you’re able to bring it to life and show them what they’ve been able to achieve.