5 Reasons to Model During QA, Part 3/5: Coverage Focuses QA
Welcome to part 3/5 of 5 Reasons to Model During QA! Part one of this series discussed how modelling enables “shift left” QA, eradicating potentially...
Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality!
Product Overview | Solutions |
Success Stories | Integrations |
Book a Demo | Release Notes |
Free Trial | Brochure |
Pricing |
Our innovative solutions help you deliver quality software earlier, and at less cost!
AI Accelerated Quality Scalable AI accelerated test creation for improved quality and faster software delivery.
Test Case Design Generate the smallest set of test cases needed to test complex systems.
Data Subsetting & Cloning Extract the smallest data sets needed for referential integrity and coverage.
API Test Automation Make complex API testing simple, using a visual approach to generate rigorous API tests.
Synthetic Data Generation Generate complete and compliant synthetic data on-demand for every scenario.
Data Allocation Automatically find and make data for every possible test, testing continuously and in parallel.
Requirements Modelling Model complex systems and requirements as complete flowcharts in-sprint.
Data Masking Identify and mask sensitive information across databases and files.
Legacy TDM Replacement Move to a modern test data solution with cutting-edge capabilities.
See how we empower customer success, watch our latest webinars, read our newest eBooks and more.
Events Join the Curiosity team in person or virtually at our upcoming events and conferences.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Help & Support Find a solution, request expert support and contact Curiosity.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Documentation Get started with the Curiosity Platform, discover our learning portal and find solutions.
Integrations Explore Modeller's wide range of connections and integrations.
Curiosity are your partners for designing and building complex systems in short sprints!
Meet Our Team Meet our team of world leading experts in software quality and test data.
Our History Explore Curiosity's long history of creating market-defining solutions and success.
Our Mission Discover how we aim to revolutionize the quality and speed of software delivery.
Our Partners Learn about our partners and how we can help you solve your software delivery challenges.
Careers Join our growing team of industry veterans, experts, innovators and specialists.
Press Releases Read the latest Curiosity news and company updates.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Contact Us Get in touch with a Curiosity expert or leave us a message.
4 min read
Thomas Pryce 27 June 2019 15:02:33 BST
Model-Based Testing (MBT) itself is not new, but Model-Based Test Automation is experiencing a resurgence in adoption. Model-Based Testing is the automation technique with the greatest current business interest according to the 2018 World Quality Report, with 61% of respondents stating that they can foresee their organisation adopting it in the coming year.[1]
Technologies like the Test Modeller have significantly reduced the time and technical knowledge needed to model complex systems. Organisations can now enjoy all the benefits of Model-Based techniques within short iterations, whereas previously modelling had been reserved for only the most high-stake projects, such as lengthy Waterfall projects in aerospace and defence.
This five-part blog series considers the benefits of introducing Model-Based Techniques to QA, discussing five reasons for adopting modelling. Today’s article begins with the very beginning of the delivery lifecycle, setting out how modelling can uproot defects earlier, where they require far less time and resources to fix.
Historic data from both industry and academic research has consistently shown that the errors introduced earliest in the software delivery lifecycle account for the greatest number of defects:
In 2001, research from The IT University, Denmark, found that 59% of defects are traceable to the design phase;[2]
In 2009, Bender RBT suggest that 59% stem from requirements;[3]
By 2012, the Hyderabad Business School found that 64% of bugs emerge in the requirements analysis and design phase.[4]
The same research has established that it is far more costly and time-consuming to remedy these defects after the design phase. In fact, requirements defects can account for as much as 82% of total defect remediation efforts,[5] while a bug identified during testing costs on average 15 times more to fix than one discovered during the design phase.[6]
This is the textbook argument for beginning QA earlier: it allows you to detect the majority of defects as they emerge, rather than after they have been perpetuated throughout the code. Detecting defects during the requirements analysis and design phase therefore gives you a greater chance of delivering quality software on time and within budget.
This raises an important question: what does it mean to perform QA during the design phase? To understand this, we must first consider why so many defects are introduced during requirements analysis and system design.
The quality of the requirements themselves remains the major cause of defects. The formats and methods used to capture software designs are ill-matched to reflect complex system logic. They tend to be ambiguous and incomplete, leading to miscommunications. These misunderstandings then require rectification through costly and time-consuming rework.
Organizations still frequently capture user and business needs in a range of disparate formats. These include written user stories, written documents and behaviour-driven specifications, as well as email or chat requests and business process diagrams. Such requirements are rarely connected formally and rely heavily on natural language.
Natural language is naturally ambiguous and is far removed from the precise system logic that needs to be converted into code. Written requirements therefore frequently allow for multiple interpretations of how the system should function, risking a drift in the vision held by BAs, developers, and testers.
A Gherkin Feature File: the natural language is ambiguous and the feature file focuses only on a handful of possible logical scenarios. A multitude of these feature files are needed to document a system. However, the files are rarely formally connected to one another.
The same requirements also tend to be incomplete. Unsystematically piling up unconnected requirements inevitably leave holes in the design of vastly complex systems, and requirements often focus near-exclusively on a limited number of expected scenarios. Developers must fill the remaining holes with their imagination, and defects arise when their vision deviates from the desired system.
Finally, the unconnected and informal nature of requirements formats prevents formal dependency mapping. Developers must then imagine how the mass of components in a complex system should integrate, again increasing the likelihood of bugs. Numerous high-profile outages that have followed supposedly routine updates point to the real danger this presents.
Requirements are a prime place to improve the speed and reliability of software delivery, given the prevalence of design defects. Formally modelling requirements offers one way to achieve this “shift left” QA. It is now furthermore now well-suited to short delivery lifecycles.
Flowchart modelling offers one such way to formally model a system’s logic. The blocks of a flowchart break a system down into its core cause-and-effect, with the process and decision blocks forming a directed series of “if this, then this” statements. These are consolidated clearly and graphically, mapping the logical journeys through a system that could be taken, and the data required to traverse that path:
A visual flowchart clearly depicts the validation rules of a new user registration page.
The act of modelling new or existing requirements works to eradicate ambiguity. If there are multiple ways of parsing a requirement, the singular and correct understanding must be established to build the model. Testers can thereby act as critical modellers, working with BAs from day one to crystallise the desired system logic. This “shift left” QA roots out ambiguity in the design phase.
Modelling further works to eradicate incompleteness. Any missing logic is far easier to spot in consolidated visual models when compared to unconnected disparate written requirements and diagrams.
Flowchart modelling offers the significant advantage that it is already frequently used by the BAs who gather requirements, many of whom already create business process. Flowcharts therefore enable testers to work collaboratively with BAs, to eradicate ambiguity and incompleteness from day one.
Test Modeller further provides a range of connectors for building flowcharts rapidly. This includes importers for existing requirements and test cases, as well as for recordings and scans of already built systems. This accelerates the modelling process, introducing all the advantages of modelling to short iterations. The choice between clear and complete specifications and rapid delivery is no necessary.
The Gherkin importer converts specification files automatically to unambiguous flowchart
models in Test Modeller.
Not only does explicitly modelling requirements to eradicate defects before they enter code, but the same requirements models can be used to generate the functional and performance tests needed to validate the code once it has been developed. This enables greater testing agility, and is the subject of the next article in this series.
[1] Capgemini, Micro Focus, Sogeti (2019), World Quality Report, 27-8. Retrieved from https://www.sogeti.com/globalassets/global/wqr-201819/wqr-2018-19_secured.pdf on 30-May-2019.
[2] Soren Lausen and Otto Vinter (2001), “Preventing Requirement Defects: An Experiment in ProcessImprovement”, in Requirements Engineering (2001, 6:37-50), 38. Retrieved from http://www.itu.dk/people/slauesen/Papers/PrevDefectsREJ.pdf on 30-May-2019.
[3] Bender RBT (2009), Requirements Based Testing Process Overview, 2, 16. Retrieved from http://benderrbt.com/Bender-Requirements%20Based%20Testing%20Process%20Overview.pdf on 30-May-2019.
[4] P Mohan, A Udaya Shankar, K JayaSriDevi (2012), “Quality Flaws: Issues and Challenges in Software Development”, in Computer Engineering and Intelligent Systems (2012, 3:12:40-48), 44. Retrieved from www.iiste.org/Journals/index.php/CEIS/article/viewFile/3533/3581 on 30-May-2019.
[5] James Martin, cited from Bender RBT, Requirements Based Testing Process Overview 2.
[6] P Mohan, A Udaya Shankar, K JayaSriDevi (2012), “Quality Flaws: Issues and Challenges in Software Development”, 44.
[Image: Pixabay]
Welcome to part 3/5 of 5 Reasons to Model During QA! Part one of this series discussed how modelling enables “shift left” QA, eradicating potentially...
Welcome to part 4/5 of 5 Reasons to Model During QA! If you have missed any previous instalments, use the following links to see how modelling can:
Yesterday, I had the immense pleasure of presenting on the second Cypress.io UK Community Meetup. My talk was called “Introducing model-based testing...
We rarely post ‘product’ articles here at Curiosity, preferring instead to draw on our team’s thought and expertise. This article is no different,...
Welcome to part 2/5 of 5 Reasons to Model During QA! Part one of this series discussed how formal modelling enables “shift left” QA. It discussed how...
In the dynamic, interconnected world of software development, clarity is key. Yet, requirements engineering - the process of defining, documenting,...
IT change remains a persistent struggle for most organisations today. Software teams are aware of the need to move faster and be more agile; yet,...
Welcome to the final instalment of 5 Reasons to Model During QA! If you have missed any of the previous four articles, jump back in to find out how...
Using Function Point Analysis and model-based testing to objectively measure services. A perpetual challenge in managing software testing projects is...