Model-Based Testing for Dynamics 365: Keep pace with Rapid Microsoft Releases

Microsoft Dynamics 365 is a highly versatile and powerful tool for enterprise resource planning (ERP) and customer relationship management (CRM). A core tool residing in many organisations, it has radically transformed the way a business is ran. Dynamics leverages the Microsoft platform to provide specific modules for sales, customer services, operations, financials, marketing, and much more. Its overall aim is to make businesses faster, and more efficient in day-to-day operations.

No two dynamics implementations are the same. Businesses exist in different sectors and even when they do co-exist, they are usually operationally unique. Dynamics platforms are therefore highly configurable and customisable to meet specific business requirements and use-cases. This often also involves Integration with existing systems and infrastructure within a business.

The customisable aspect means dynamics applications must be treated as a unique piece of software that needs to be rigorously tested within the organisation. However, regular releases and custom, complex applications spell trouble for the traditional testing techniques often applied to Dynamics 365. This article considers the need for a new and automated approach, before setting out how test automation can be applied successfully to Dynamics 365 testing.

Dynamics 365 testing: The need for a new approach

Dynamics along with all other ERPs (like SAP) are the beating heart of any business and a vital component to keep the business operational and success. Defects can be expensive and costly to the organisation. Dynamics applications are mission critical systems that must be operational and work as expected for users to make informed decisions.

The way software is developed has changed widely over the past decade. The software industry has moved from waterfall, one-time deployments, to agile initiatives releasing on a frequent cadence. Microsoft is no different with the Dynamics platform. Dynamics upgrades are becoming more regular, and Microsoft has moved from annual major releases to bi-annual. There is even talk of increasing to quarterly releases on the horizon. Regular releases are great as they bring new features and innovation more regularly to users. However, they also presents a more demanding testing schedule for organisations who rely on the ERP and CRM applications.

ERP systems have traditionally been manually tested, primarily due to the complexity of the applications and difficulty in creating test automation for them. However, manual testing is too slow to keep up with the rate of change created by agile releases. The regular release cadence of Dynamics 365 in turn makes manual testing unviable, as there is no time to:

  1. Manually analyse the impact of each release on customised applications;
  2. Identify and update the associated tests;
  3. Manually execute every test required.

Reacting effectively to potentially unforeseen impacts on custom configurations makes reactive test automation an essential testing aide.

What is automated UI testing?

Automated testing is the process of automating the execution of test cases. That is, instead of having a team of testers executing each test scenario by hand, the computer automatically undertakes the interactions using automated test scripts. The automation mirrors a human clicking on various buttons, entering text, and more, all  without any human interception. The automated tests may be fired off on-demand, on a regular schedular, or as part of a build process to immediately get feedback on the quality of a new release. The many benefits of test automation mean that:

  1. Fewer resources are required. Every time a new manual test cycle is undertaken, manual testers must be allocated to execute each test. With test automation the script is executed without any human aide.
  2. Tests are extremely reproducible and can be re-executed at any time. Manual tests are open to interpretation from the tester. Think of how many ways you could write a test which clicks a button. Human language is inherently challenging.
  3. Feedback on the quality of a release can be obtained very quickly. Automated tests are much quicker than humans and can be parallelised to scale your tests infinitely with more compute power. This enables stakeholders to obtain feedback on the quality of a release within a couple of hours, rather than days or weeks required by manual testing.
  4. Early defect notification. Failures can automatically trigger the creation of defects and notifications for the relevant stakeholders to immediately notify them of a defect and the impacted components.

Challenges of Test Automation

While these benefits are empowering, successfully implementing and then maintaining test automation is challenging for many organisations. Curiosity Software Ireland has had the fortune of working with many of the fortune 500 companies over the years and has seen first-hand how automating test execution alone is not enough, especially when it comes to Dynamics 365 testing. It is a widely known fact in the testing industry that test maintenance can kill any return on invest from test automation. All those hours you would be saving executing those manual tests can very easily turn into more time maintaining test scripts to keep up with Dynamics 365 releases.

Test maintenance usually falls into two categories. Firstly, the UI of the application components might change (the look, appearance, or tags of a user-interface element). Secondly, the way the system flows can change (new dialogs, buttons, or inputs changing how we navigate through the application). Dynamics 365 is especially challenging for two reasons

  1. It often features complex business processes. An automation engineer must inspect every script to identify if it is impacted by a change and then update the corresponding script with the relevant changes.
  2. Automation scripts rely on extracting UI components from a web page. Finding a good identifier to extract an element is very difficult with no real science. A lot of time is spent crawling through the web page to find a good identifier to consume. With every dynamics upgrade, UI changes are prevalent between versions with identifiers changing regularly, which again involves searching through every script to fix.

A lot of organisations take the route of using Selenium or some other technology for automating dynamics applications. However, these tools are heavily code oriented. That means that you must either transform your manual testers into automation engineers or hire a new bread of staff who can script in this automation code.

This is an area where automation usually fails. Within an organisation we typically see islands of automation, where one or two of the top testers who have applied themselves to automation have had success. Yet automation does not in turn become widespread since the new skillsets required is unavailable to other teams. An alternative approach that “de-skills” test automation is required if automation can be adopted across the enterprise.

TestModeller.io Approach to Dynamics 365 Test Automation

Test Modeller has been purpose built to overcome these challenges of test automation. It builds on Curiosity’s 30+ years of experience in the field of creating automation frameworks, managing test data, and educating organisations on best of breed software testing methodologies.

Test Modeller revolves around an approach called Model Based Testing. MBT enables users to create visual models of how an application should operate, modelling its business process logic. Once this model is created, Test Modeller automatically creates the perfect set of test cases, test automation, and test data required within your test environments seamlessly.

The power of the model

A model is nothing more than a simple flowchart, or BPMN diagram to be precise. Most users are already familiar with flow charts from their basic school education. Testers who are already involved in Dynamics 365 projects will already also have an in-depth knowledge and mental model of how the various processes in their dynamics instance operates, and the associated business rules that are present. All they need to do is draw this model in Test Modeller. The model then becomes a living piece of documentation for how the application operates: a vital piece of information for anyone new onto the project, and for reference in the future.

Flowchart Modelling for Microsoft Dynamics 365
Figure 1 – Living documentation: a visual flowchart maps the logic of a Dynamics 365 form.

Reusable components

Every model in Test Modeller becomes a re-usable component, enabling easy test automation for Dynamics 365 testing. Once a model is created, it can be shared, distributed and consumed inside of other test cases. A lot of test scripting revolves around copying and pasting of common components. Now all you need to do is draw something once, then include it inside your scenarios.

Re-usability also helps overcome complexity. Dynamics processes can often be some of the most complex pieces of an application as they consist of large business. With Test Modeller, you can abstract away from this complexity at ease: just break the problem down into small components and build a model for each. Then link them together into an end-to-end flow covering the whole chain of processes.

Subflows for Microsoft Dynamics 365 Testing
Figure 2 – Re-usable subflows overcome the challenge of testing complex Dynamics 365 applications.

Optimal test cases

Test Modeller automatically generates tests from the model using sophisticated test generation algorithms. The automated test generation produces the minimum number of test cases to maximise test coverage across your application. This ensures that your dynamics processes are tested as rigorously as possible, and components impacted by a release are still operational after each upgrade. The ability to test rigorously within short iterations provides confidence that each Dynamics 365 upgrade has been tested as rigorously as possible.

Test Case Generation for Dynamics 365 Testing
Figure 3 – Automated test case generation rigorously tests fast-changing applications within short iterations.

Easy drag and drop test automation

drag-and-drop test automation for Dynamics 365
Figure 4 – Automating tests for Dynamics 365 is as quick and simple as selecting from re-usable components.

Automating tests with Test Modeller is as quick and simple as assembling visual flowcharts. Test Modeller comes with out-the-box actions for automating Dynamics 365 processes, including reusable components for both the new ‘unified interface’ and for legacy Dynamics applications. This gives you the flexibility to create automation rapidly, without having to write large amounts of code, or extract various complex identifiers.

Rapidly react to new releases

As discussed, test maintenance can easily kill any return on investment from test automation, and in fact could end up requiring more time and resources than manual test automation.

Model-Based Testing with Test Modeller removes the time and cost associated with test maintenance. Test automation scripts become a by-product of modelling. When a new release comes out, test teams only need to update the easy-to-maintain models, regenerating a set of rigorous automated tests each time:

Continuous Testing for Dynamics 365
Figure 5 – Continuous Testing for Dynamics 365

This means that testers only update models in Test Modeller, instead of crawling through hundreds or thousands of scripts to identify changes and update them accordingly. QA teams can therefore execute an up-to-date set of tests continuously for each and every release, even when faced with short iterations.

Continuous testing for Dynamics 365 allows organisations to enjoy all the value of new releases, but without the negative risk of damaging defects. Watch this short demo to view the approach in practice, and click here to book a live demo and discussion with a Curiosity expert.