Managed testing: Delivering increased confidence and consistency

The Project

The customer embarked on a digital transformation programme to enhance and develop existing, regulatory audit and control processes. The aim of the programme was to reduce the overhead of regulatory compliance while minimising the risk of audit and control errors and omissions. The programme was delivered over a series of six independent projects, coordinated by a programme management team. Some projects followed a waterfall methodology and others adopted an agile/ iterative approach.

What we did

Provided a proactive and fully accountable Managed Test Service:

Ten10’s Managed Test Service was led by dedicated Account (commercial) and Delivery (senior, strategic test specialist) Managers who worked with the Ten10 Programme Test Manager to deliver the service. The Ten10 Managed Test Service:

• Supplied a core team of a Programme Test Manager, Test Leads and Test Analysts to cover all in flight projects.
• Supplemented the core team with specialist test consultants to support project demands, including performance test capability and targeted, value-led test automation services.
• Provided flexible “burst” capacity to boost the core team in order to guarantee challenging timescales were achieved and to mitigate project slippages (e.g. due to late code delivery or higher than expected defect rates).
• Delivered regular service updates, keeping the customer programme and PMO teams informed of commercial status and service delivery highlights, risks and future plans.
• Initiated regular communication channels between the Ten10 Delivery Manager and customer Programme Manager to share pipeline and future plans, highlight Ten10 team successes, and provide an open and collaborative forum to raise risks to and issues with Ten10 or customer team delivery.
• Recommended improvements to the programme test strategy and individual project test approaches that increased test coverage, delivered test activities more efficiently, and increased business stakeholder confidence.
• Introduced agile working practices to manage and track Ten10’s test delivery.
• Provided technical test leadership across performance, automation and operational acceptance to recommend approaches that focused on the return on investment and long-term benefit to the project and future business as usual (BAU) support.
• Successfully supported the delivery of the first iterative/agile projects within the programme.
• Established strong working relationships within the programme team and with the many, varied and often challenging business stakeholders.
• Co-located the Ten10 test team to increase collaboration and share lessons learned across projects.
• Leveraged Ten10’s Digital Test Team to provide part-time test leadership for smaller projects and to support accessibility testing.
• Streamlined and industrialised the onboarding process by creating checklists and supporting documents to reduce lead times and mitigate
  the risk of delays.
• Responded rapidly to changing priorities and scope, adjusting course and proactively managing delivery risks and issues when required.
• Provided technical consultancy and delivery to support broader project objectives beyond traditional test deliverables.
• Tracked test budget and recommended the utilisation of test underspend to provide increased test coverage, de-risk project milestones and ensure the full benefits of test automation were realised; maximising the value of the agreed spend on test and quality.
• Initiated discussions on how to de-risk the transition of test collateral into BAU support teams, including the option ongoing use of Ten10’s flexible and cost-effective Digital and Technical Test Teams.

Challenges

Delivering consistently across a diverse programme with many stakeholders:

• Determining the common, programme-level drivers across the projects and stakeholders.
• Identifying additional, specific drivers and deliverables for individual projects/stakeholders.
• Coaching and supporting the Ten10 team to:
  • Understand the programme and project drivers.
  • Share lessons learned across projects.
  • Define templates, guidelines and ways of working that support effective and innovative test approaches while address programme and project priorities.
  • Recommend alternative and improved ways of working across all projects.
  • Challenge assumptions and suggest alternative solutions.

Providing meaningful and nuanced test reports for complex projects to non-test specialists:

• Effective, graphical reporting.
• Transparent and real time data available at all times within Confluence.
• Support and coaching for non-test specialists via software working groups.
• Seek regular feedback and adjust content and approach when needed.

Expanding project/programme management capacity by reducing their day-to-day involvement in test activities:

• Communicating Ten10’s delivery approach, supported by Tenology.
• Utilising existing tools (Jira/Zephyr and Confluence) to give transparent and real time access to all test assets.
• Hosting daily stand-up meetings and managing kanban boards to track test tasks.
• Implementing tailored and data-driven reporting.
• Instilling trust and confidence by highlighting and mitigating risks early.
• Providing Delivery Management oversight.

Benefits

The Ten10 test team has successfully completed 19 waterfall test phases, supported testing throughout 19 agile/iterative sprints, successfully went live
with 3 projects and delivered 2 projects to a state of go live readiness, all within the agreed timescales and budgets.

In addition, the Ten10 Managed Test Service has delivered the following benefits to the customer:

• Increased confidence and consistency in testing within the programme team and among the business stakeholders
• Full ownership of all aspects of test delivery across the programme, reducing the involvement in test activities from programme and project managers, freeing up their time to focus on other priorities
• Agile delivery expertise to support the programme team’s first steps into agile/iterative delivery
• Efficient and effective test cost and resource management
• Test process improvements that can be used on other programmes and projects