Performance testing custom code in Microsoft Dynamics CRM Online
The client: A large housing association, based in the South of England
Industry: Public Sector
Technologies: Microsoft Dynamics CRM
Ten10 services: Performance testing
Ten10 was tasked with discovering and reporting on the reasons behind inconsistently poor performance of its custom CRM code in a variety of different end-user circumstances. We successfully tracked down the root causes of the performance issues, and achieved resolution for the client whilst building a good relationship.
The client is one of southern England’s largest housing associations, providing homes for over 50,000 people. From offices in a variety of locations, it offers a range of services across the south-east of the country, and works to provide activities, information, and advice that improve neighbourhoods and individual lives.
The housing association was in the process of upgrading its existing CRM system and replacing it with Microsoft Dynamics CRM Online. The CRM system manages the association’s customers, properties, and tenancies. It allows the handling of cases raised by customers, such as enquiries, complaints, or other issues about the properties.
The previous version of Dynamics CRM had demonstrated poor responsiveness across test environments with the same number of nodes as the system that would be released to production. For specific transactions, such as saving a case, response times were observed to reach values of up to a minute yet this was not consistent; response times varied throughout the day.
Ten10 proposed to carry out systematic performance testing across the association’s various locations. This would allow a sample of measurements to be acquired in order to identify the performance behaviour of the latest releases, whilst diagnosing and supporting the resolution of the observed performance issues.
We identified three key goals for the project. The first was to design and implement a solution for acquiring consistent performance metrics for the transactions of critical use-cases for Dynamics CRM. We also needed to understand the performance behaviour of Dynamics CRM Online when accessed by housing association users from within the intranet as well as via a Citrix virtualised environment from a selected number of key locations. And finally, we needed to analyse the results, diagnose potential performance issues, recommend actions following the performance tests, and investigate the effect of the solutions we had identified.
The key challenge related to the intermittent performance issues that appeared at seemingly random times. Monitoring random performance issues is never simple, especially in a complex environment such as the one used by the client. The issues were exacerbated by the fact that there were limitations on the extent to which the server side of the cloudbased CRM system could be monitored. This is due to it being provided as a sealed, turnkey service. Additionally, the service, which had been customised by the client, was being accessed from multiple locations, sometimes from within the virtualised environment.
An incremental approach to performance testing
We chose to take an incremental approach to performance testing, using a repeatable method that would be consistent for direct access to CRM, for access to CRM via Citrix from a location next to the client’s data centre, and from access to CRM via Citrix from remote locations.
We used visual automation with hundreds of samples to allow for the consistent gathering of metrics across the various access methods. Based on our analysis, we identified issues in both the application’s customisation and the infrastructure layer. We were able to identify the specific transactions which suffered very high transaction times, and provided the development team with this data for investigation. Very high overhead across all transactions was also identified when we ran tests from a virtual host servicing a Citrix session. This was possibly caused by the contention of multiple Citrix workers on the same physical host. An incremental testing approach was suggested to identify the level of contention at the virtual hosts that was detrimental to performance.
Ten10 used its testing expertise and proven methodologies to pin down performance problems – building a good relationship with the client in the process – and was able then to convey that information to the client for actioning.
Being an independent software testing partner enabled us to take a holistic view of the issues, which proved beneficial due to the number of separate organisations involved.To download the case study PDF please click here