Key risk factors to mitigate during a data migration
Part one in this article series summarized the shockingly high failure rates for migration projects, identifying data migration as a key area of...
Design Complex Systems, Create Visual Models, Collaborate on Requirements, Eradicate Bugs and Deliver Quality!
Product Overview | Solutions |
Success Stories | Integrations |
Book a Demo | Release Notes |
Free Trial | Brochure |
Pricing |
Our innovative solutions help you deliver quality software earlier, and at less cost!
AI Accelerated Quality Scalable AI accelerated test creation for improved quality and faster software delivery.
Test Case Design Generate the smallest set of test cases needed to test complex systems.
Data Subsetting & Cloning Extract the smallest data sets needed for referential integrity and coverage.
API Test Automation Make complex API testing simple, using a visual approach to generate rigorous API tests.
Synthetic Data Generation Generate complete and compliant synthetic data on-demand for every scenario.
Data Allocation Automatically find and make data for every possible test, testing continuously and in parallel.
Requirements Modelling Model complex systems and requirements as complete flowcharts in-sprint.
Data Masking Identify and mask sensitive information across databases and files.
Legacy TDM Replacement Move to a modern test data solution with cutting-edge capabilities.
See how we empower customer success, watch our latest webinars, read our newest eBooks and more.
Events Join the Curiosity team in person or virtually at our upcoming events and conferences.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Help & Support Find a solution, request expert support and contact Curiosity.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Documentation Get started with the Curiosity Platform, discover our learning portal and find solutions.
Integrations Explore Modeller's wide range of connections and integrations.
Curiosity are your partners for designing and building complex systems in short sprints!
Meet Our Team Meet our team of world leading experts in software quality and test data.
Our History Explore Curiosity's long history of creating market-defining solutions and success.
Our Mission Discover how we aim to revolutionize the quality and speed of software delivery.
Our Partners Learn about our partners and how we can help you solve your software delivery challenges.
Careers Join our growing team of industry veterans, experts, innovators and specialists.
Press Releases Read the latest Curiosity news and company updates.
Success Stories Learn how our customers found success with Curiosity's Modeller and Enterprise Test Data.
Blog Discover software quality trends and thought leadership brought to you by the Curiosity team.
Contact Us Get in touch with a Curiosity expert or leave us a message.
3 min read
Thomas Pryce 06 May 2020 12:10:35 BST
Ever-tighter data privacy legislation like the EU General Data Protection Regulation and The California Consumer Privacy Act has made test data a pressing problem for organisations worldwide. It has put testing firmly on the radar of ‘the business’, who are now actively concerned with how data is being used in both testing and development.
This presents an opportunity for QA. The current business concern – and its associated budget – offers a chance to transform one of the most cumbersome and time-consuming testing processes: test data provisioning. However, this transformation requires a complete solution to the time and quality issues associated with current test data management.
Such a solution is set out on Curiosity’s webinar: Test Data Automation: Delivering Quality Data at Speed. Watch it on demand. This blog provides a flavour of the challenges and solutions that we discussed.
Using raw production data in less secure test environments is today riskier than ever.
Firstly, it raises questions around consent and the legitimate grounds for processing data in QA. Meanwhile, internal human error remains the number one cause of costly data breaches. Sharing sensitive information across test environments therefore increases the risk of damaging data breaches, for which the fines today can be in the hundreds of millions.
Organisations today furthermore often lack the infrastructure to locate one person’s data reliably across sprawling test environments. This makes it near-impossible to delete or share a copy of that person’s information on demand, potentially breaching both an EU Citizen’s Right to Erasure and to Data Portability.
The simplest way to avoid massive fines and brand damage is to limit access to sensitive information across the organisation, and masking production data before it moves to test environments is therefore a minimum for most organisations.
The challenge is that anonymising production data can be slow and complex. This in turn replaces compliance concerns with QA bottlenecks, swapping one set of testing challenge for another.
These bottlenecks in data provisioning arise because masking complex data from numerous sources is inherently complex. The relationships and complex trends that exist within and across data sources must be retained, even as sensitive information is removed.
Manually masking data therefore creates a bottleneck in test data provisioning. It leaves ‘parallel’ test teams and data-hungry automation frameworks waiting idly for a limited number of out-of-date copies of data, making it impossible to the latest system within an iteration.
Masking furthermore does nothing to improve the variety of production data, which lacks the outliers and unexpected results needed for rigorous testing. QA can in turn only execute a fraction of the tests needed for sufficient test coverage, exposing systems to costly and damaging bugs in production.
Figure 1 – Production Data can only execute a fraction of the tests needed for sufficient coverage.
Manually masking and moving production data to test environments accordingly creates a dependency and bottleneck. This undermines testing speed and quality, and conflicts with the principles of continuous testing, DevOps, and “Agile”. Testing at the speed of iterative delivery and automated test execution demands a new approach.
Rigorous testing at the speed of iterative delivery and automated testing requires constant access to comprehensive test data. That means data with which to execute every positive and negative test, available exactly when and where testers and automation frameworks need it.
A test data strategy today must therefore move beyond the logistics of a central team copying data slowly to test environments. Test data should no longer simply be “managed” in this way, but must instead be made available in a way that improves testing speed and quality.
Providing this “agile test data” is possible with the new paradigm in test data technology: “Test Data Automation”. TDA standardizes test data management processes and makes them re-usable on demand within automated testing and CI/CD pipelines. Testers can embed in the re-usable processes in both automation and manual testing, finding and making complete test data as tests are generated or executed:
Figure 2 – Automated “Find and Makes” allocate all the data needed to execute a test suite, prepared on-the-fly.
This approach to test data moves beyond the logistics of GDPR compliance. It enables a modern approach to test data that is:
Sound interesting? Come and see for yourself! Watch on demand: Test Data Automation: Delivering Quality Data at Speed!
Part one in this article series summarized the shockingly high failure rates for migration projects, identifying data migration as a key area of...
Today, any test data solution must be capable of fulfilling data requests of a greater volume, variety, and complexity, faster than ever before. This...
Last week, we published a blog making the case for the next generation in TDM “best practice”. We considered why the logistical approach of “mask,...
The previous article in this series set out how a successful data migration hinges on a range of criteria:
If you’re reading this, you’re probably already painfully familiar with the complaints that Curiosity hear from organisations seeking alternatives to...
Curiosity Software Ireland, creators of Test Data Automation, and Windocks, on demand database specialists, today announced a joint solution for...
Part two in this series identified some key risk factors associated with a data migration, which underpin the shocking migration failure rates...
Welcome to part 4/4 of this article series discussing the new paradigm in Test Data Management: “Test Data Automation”. The drawbacks of traditional...
Today, the majority of enterprises are engaged in ongoing system migrations – and most of those projects will either fail completely, overrun on...