Quality and Test Performance Health Check for investigative software provider

The Project

The client was looking to improve the testing aspects of their development process and sought independent professional advice from trusted industry experts. They initially engaged Ten10 to perform a strategic review of the testing phase of their SDLC in order to inform them of areas to prioritise for improvement. Ten10 carried functional testing for their new HUB release using the Ten10 Digital Testing approach, providing a valuable illustration of the feasibility of Automation Testing using ‘live’ development projects. This enabled the client to see tangible benefits of incorporating these elements into their SDLC and what ‘even better’ looks like.

What we did

Ten10 carried out a QA Health Check of the client’s testing function using the proven Tenology methodology, whilst at the same time, supporting the internal testing team to perform time critical functional testing. As a result of the QA Health Check, Ten10 worked with the client to deliver one of the key recommendations – to implement an Automation Factory, resulting in the internal testing team being able to increase the speed and quality of the testing function. Importantly, the software was delivered on time to the end client.

Define and Demonstrate

QA Health Check: Ten10 were initially engaged by the client to provide an expert strategic overview of their testing phase of the SDLC, using the Tenology methodology and framework. The QA Health Check, unique in the industry as it is both managed and delivered by seasoned industry professionals, examined the maturity and efficacy of the client’s testing team, processes and functions. It focused on areas such as requirements verification and automation across the development lifecycle and analysed both business and technical risk.

Deliver

Roadmap: Ten10 delivered a comprehensive analysis of what ‘even better’ looks like, coupled with metrics and advice from practitioners who have experience of implementing real change in the testing world. Using the Tenology methodology, Ten10 produced a roadmap containing an illustration of how to significantly improve their testing function over time – with recommendations clearly split into short, medium and long term, highlighting the benefits of each recommendation. Not only did Ten10 assess and validate the processes within the client’s current testing function, but we also gave them detailed guidance on how to implement measurable change.

Digital Test Lab: In parallel with the QA Health Check, Ten10 were also able to demonstrate the benefits of Digital Testing by working with the client’s staff to perform time critical functional testing. By working alongside their staff, Ten10 were able to mentor the team and help them improve their manual testing processes. Ten10 quickly identified key issues preventing test progress and worked with the client to implement best practice, resulting in a greater degree of confidence in the testing. By using more elements of the JIRA test tool, Ten10 illustrated the critical importance of workflow and what a good traceability matrix looks like and how it speeds up testing. The sharing of best practise across the team, comprising both companies’ staff, was a key factor in delivering the software to the end client on time.

Automation Factory: A key recommendation from the QA Health Check highlighted the benefits of implementing an Automation Factory. To prove exactly how the potential benefits could be achieved, Ten10 created a Proof-Of-Concept Automation Factory for UI and API testing using Cypress, and an associated release note process using JIRA workflow as the reference point. This Proof-Of-Concept achieved 100% of automation coverage in the client’s UI tests, extension tests and API tests within tight timescales.

This project demonstrated tangible results to the client – software defects were reduced to zero. The Ten10 team also realised the benefits the client would gain by adding in accessibility testing to the Automation Factory, using WCAG 2.1 criterion. Ten10 trained the client’s testing team by conducting regular workshops and knowledge transfer sessions in both UI and API automation. The subsequent successful transition of the Automation Factory means the testing team now have a valuable additional testing tool at their disposal, which will increase the speed and quality of the testing function. This is just one illustration from the QA Health Check recommendations, demonstrating how the impact of the change can be clearly measured.

In addition to the automation framework, Ten10 advised how to integrate automation framework with Jenkins/Circle CI, and the workflow has been successfully added as part of the UI and API framework.

Challenges

• Extremely tight timescales – with a fixed delivery date to the end client
• Reputational risk due to failed software delivery
• Multiple teams were involved in different aspects of the testing, requiring time critical information sharing and the need to include the right people at key stages of the project

Results

• The defects were reduced to zero by Ten10 introducing the Automation Factory
• The client’s entire SDLC is now benefitting from the introduction of the Automation Factory into their testing function
• The software delivery was made on time and budget to the end client
• The client received a comprehensive roadmap highlighting how to further improve the testing function, including benefits illustrated by metrics
• An improved release note process has been delivered

Benefits

• Strategic and operational changes to the testing function, described in detail in the roadmap, can now be efficiently planned – which will ultimately save time and money throughout the entire SDLC.
• By developing an ongoing partnership with Ten10, the client can call on us for assistance in the future should the need arise – we can quickly plug any shortfall in resource or expertise.
• The client can augment their technical teams in the future using specialist Ten10 Academy graduate resources. This immediate availability of experts, trained in all technologies, removes the anxiety and risk from recruitment.

Right from the start of our engagement, Ten10 adopted a flexible, collaborative approach and made a real difference to our testing function in a short amount of time. Ten10 clearly demonstrated they understood our needs – by delivering an insightful strategic review and partnering with our teams on a project with a tight deadline. I now regard them as our trusted testing partner, invested in our success. They know what we need and how best to deliver it.