Test management in a multi-vendor environment for a global law firm
The client: Hogan Lovells
Technologies: Elite 3E Enterprise Business Management
Ten10 services: Test Management, Performance testing, UAT
Ten10 was engaged to provide testing services for an International Law firm, who were in the process of implementing a heavily customised and integrated Practice Management System.
Ten10 was joined by multiple providers on the programme including the client, the vendor, application experts, data analysts, training providers and other testing providers. Ten10’s responsibility included governing the overall testing effort, providing quality assurance of the other test providers, implementing test processes (defect, review, rework) and tools (TFS), providing regular reporting (passive and active) to various levels of stakeholders as well as executing the testing and defect management.
The development of customisations was to be delivered in a manner leaning heavily on Agile principals however the wider programme and integrations was following a more traditional Waterfall methodology.
The primary role of functional testing involved Ten10 working with a 3rd party test supplier who were application experts. The 3rd party was responsible for writing test scripts. Ten10 implemented a review process for these scripts using the 3rd party’s Leads to perform an initial peer review. This was subsequently passed to client Subject Matter Experts to review from a business perspective. Finally, Ten10 reviewed to ensure they were written to standards before execution and uploading into Microsoft’s Team Foundation Server.
The team was also responsible for planning and coordinating a UAT phase involving over 300 scenarios, executed by 150 business users, spread over 5 different global locations.
Ten10’s total team onsite flexed throughout the duration of the programme up to a maximum of 15 onsite concurrently. The team’s structure included; Test Director, Test Management, Test Leads and Test Analysts. The role of the Test Director was to set strategy, manage stakeholders, control budget and set resource models depending on demand and constraints.
By identifying forecast delivery from development of new code or fixes, Ten10 was able to identify any constraints to increasing productivity. When this constraint became test resource, Ten10 onboarded additional personnel to remove this constraint.
To help ensure efficient delivery of the testing, an onboarding process was set up that involved a mixture of application and process documentation, shadowing and mentoring as well as using logical functional areas of the system for initial execution. A rotation system was implemented to remove ‘key man’ dependencies.
“Partnering with Ten10 to develop a highly effective and professional testing approach has enabled us to pursue the implementation with pace and confidence.”Steve Capon, Head of Global Finance Projects, Hogan Lovells
An integration team was established to focus on the 30+ direct and indirect integrations in the overall landscape, that relied on the core product being implemented. This integration team had little existing architectural and infrastructure documentation to work with. A discovery phase started to document the entire application landscape with integration points and data flows. They worked with solution architects, enterprise architects, application owners and the client infrastructure team to put this in place. The resulting documentation was then used to create a test plan to cover; data migration testing, point-to-point integration testing, end-to-end system integration testing and user acceptance testing.
The plan utilised the priority of each integration, as well as the complexity to order the testing in a risk-based approach. It also provided a resource model and forecast. One critical component of the end-to-end solution involved a US-based 3rd party testing provider. Ten10 was asked to coordinate with them to set expectations on progress and ingest their metrics to provider wider reporting.
To support the onsite team and the client, Ten10 delivered through an account management service which involved regular account review meetings with the programme director and the client head of QA. A mid-point service review was also conducted. This involved a test expert not involved in the programme coming onsite reviewing processes and collateral, interviewing Ten10 resources and stakeholders, and delivering a health report to both the internal team and the client.
Finally, regular and ad-hoc meetings were organised for the programme director and various other stakeholders to meet Ten10’s Client Engagement Director to discuss tactical and strategic consulting needs, providing valuable expert guidance on best practice.
|Scripts Complete by Month||Issues Raised by Month|