Using machine intelligence for clearer regression testing insights

Regression testing helps development teams validate that when they fix bugs or add new features, existing functionality of their product still works. Typically, these tests are kicked off as part of a continuous integration (CI) process so that engineers know very quickly if anything broke based on their new commits. Hopefully not, but for sure it happens.

Given new application architectures including microservices, where individual services are decoupled into small, individual components, regression testing becomes even more important since unit testing would rarely catch code dependencies for a new feature that runs across one or more microservice.

However, regression testing is hard. Some of the biggest challenges are:

• Pass/Fail comparison is relatively simple, but comparing regressions for performance characteristics is hard
• Tracking, analyzing, and comparing historical results for tests that you want to run regressions against can take a lot of work
• Once regressions are identified, finding the cause takes a lot of time

These types of challenges are exactly what machine learning was built for and why mabl is using machine learning to help engineering teams with regression testing. As mabl is testing applications, she also collects lots of data which isn’t typically recorded by manual testers or other tools — test run time, page load times, screenshots showing visual changes in the app, etc. This data in turn trains several different types of machine learning models which are used to notify teams when something changes — either for the good or bad. Either something was working and now its not, or something was broken and now its fixed. Let’s take a look.

This is an example where mabl found a regression in test execution time for a specific journey. Specifically, you can see where mabl highlights that the run of this journey on January 20th was significantly faster than previous runs.

Another type of regression mabl can automatically detect is when there are broken links introduced in an application or site. mabl will identify and highlight which page and which link is broken, providing you with an Insight (what we call it when mabl deciphers a test result for you), and also let you know when they’re fixed.

These are just two of the many examples of the details you’ll see when mabl detects regressions. For a higher level view, mabl also offers an Insights feed on the right nav so that you can keep track of every new regression mabl detects. If you’re not hanging out in the mabl app, these Insights are sent to Slack as well.

mabl continues to add new automatic regressions frequently based on user feedback that we consistently gather. One of the newest regressions is visual differences. Since mabl captures screenshots of every step of every journey, machine learning models can automatically compare these screenshots to detect visual changes and she’ll let you know exactly where on the page the change was detected.

By using machine learning, mabl is trying to make regression testing easier for software teams, not forcing them to comb through test history, analyze test results, or manually test their applications.

Give mabl a try in your app in literally seconds — https://app.mabl.com

This is story is brought to you by Hacker Noon’s weekly sponsor, mabl. ML-driven test automation. mabl, uses machine intelligence to make testing easyOriginally published at www.mabl.com.

Using machine intelligence for clearer regression testing insights was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.