Success Story

How EVERFI moved from manual testing to continuous test generation

Discover EVERFI's 9-month journey to test generation in minutes with the power of Curiosity's Quality Modeller.

The need

EVERFI required scalable course automation, capable of testing vast permutations at speed, across devices, and with visual testing built-in.

The solution

With model-based testing, EVERFI's CI/CD pipeline auto-generates targeted tests within minutes of a check-in.

The benefits

In just 9-months EVERFI reduced the time to create and run tests from weeks to minutes, with 100% permutation coverage.

The benefits at a glance

Close collaboration between Quality Engineering, development, product owners and designers.
85% automation of course testing, saving resources spent on outsourced manual testing.
100% course permutation coverage, with functional and visual testing of every page.
Up to ~30,000x faster test creation – from weeks to minutes.
Tests run in minutes, with concurrency of up to 90 machines.
Automated test maintenance following a new course check-in.

A challenge and opportunity for Quality Engineering

To match the pace and complexity of course development, QE embarked on a digital transformation project to remove manual testing from 100+ courses. The project sought to scale rigorous test automation across the whole organisation.

Moving from manual to automated testing

Before the introduction of a Quality Engineering (QE) mindset, course testing at EVERFI was made up wholly of manual testing. However, this was too time-consuming and costly. Suboptimal testing further risked hitting the same logic repeatedly, exposing EVERFI’s eLearning platform to bugs, while there was no way of communicating clearly what was being tested to developers or product owners.

The need for scalable automation

Course automation required a scalable approach to creating tests, capable of testing the vast permutations created by course creation. The Quality Engineering team decided that hand-scripting every test would be too slow to test the modularised system architecture. They sought a rapid and automated approach to reusing the high-quality Cypress automation, avoiding the manual effort of copying, pasting and editing boilerplate code.

From manual testing to generating tests in minutes

Generating optimised tests at speed

EVERFI used Quality Modeller's test generation to create course code as quickly as possible, while optimizing and clearly communicating test coverage. Model-based testing was used to break EVERFI’s courses down into reusable models, combining them to generate course automation quickly.

Accelerate Delivery Time - Curiosity Software

Visual and accessibility testing built-in

Quality Modeller generates visual test assertions to validate how users see and experience EVERFI’s courses. This is as simple as adding a blue automation waypoint into a reusable model. A step in the generated Cypress tests then makes an API call to Applitools Eyes, all built in Modeller's visual models.

Targeted test generation

With Quality Modeller, the Quality Engineering team introduced targeted test generation, further optimising and accelerating course test automation. The targeted test generation is based on GitHub check-ins, focusing testing on impacted pages or modules, instead of re-running end-to-end tests.

Discover EVERFI's complete 9-month journey

Continuous, collaborative test generation

Generate new tests within minutes of code check-ins
85% of systems automated in 9 months
100% permutation coverage

Read the full story

Visual models break our system down in reusable chunks that generate the functional and visual tests we need. We can further assemble these reusable building blocks automatically as new courses are checked in, generating rigorous tests.

Greg Sypolt

VP of Quality Engineering

Read the full story

How EVERFI moved from manual testing to continuous test generation

The benefits at a glance