Avoid Object Recognition Frameworks in your Test Case Automation

Avoid Object Recognition Frameworks in your Test Case Automation

There are shiny tools that promise to automate functional cases using object recognition technology.

Sikuli and Eggplant are examples of tools that allow you to interact with UI elements using images. It’s easy to fall for the promises of using these tools to automate almost anything: from native apps to complex web sites.

There are several things to consider before you invest time using these kind of tools.

Test Case Creation

This task can quickly become overwhelming. These tools require a set of images to use as identifiers to click, fill or interact with any element in your app.

I dare you to checkin your source code to a control system and see the commit that added 100 new files to the repository just to automate 1 single test case.

Test stability

Although there are ways to customize image recognition sensitivity on all tools, you will soon discover that even small changes will break your tests. Forcing you to either capture images again (and again and again) or accept the fact that your test will be failing from time to time.

Test portability

If you are planning to collaborate and create or run Test Cases using computers with different OS (Win, Mac, Linux ) or even people that are using different resolution on their screens, this might easily become a source of stress and frustration. Image recognition is just a comparison between two images and as I already stated even small changes might cause a failure.

Imagine plugging these cases into a CI server. You will create the perfect job to fail every time somebody is pushing code to the environment.

Data driven capabilities

When doing data driven testing you sometimes want to randomize your data in order to interact with different elements on a website. For example, you might want to choose a randomly selected set of products on a search landing page; in this case you will need to create a new image for each of the products you might interact with. Maintaining all these images and the scripts that use them quickly becomes cumbersome and difficult to maintain.

Test Speed and Computing Resources

If you do not mind waiting 10–15 minutes to run (and see them fail) object recognition is your best choice. Also consider the computer power used to run these kind of tools. Image recognition is expensive when you talk about computer cycles.

In summary I would say that these kind of technologies might have an application ( Super stable UI, running only in 1 machine, etc.. ). But considering the complexity of the current apps you might want to take a look to other tools that offer you a maintainable and stable set of test cases.

Some people would say that they would use Sikuli to test how the site renders on a device, and I would agree if we did not have access to more suitable tools like PhantomCSS that allows you to create a set of images used as a baseline (using CSS locators ) in the same computer that you will run the comparisons (IOU a blog post about PhantomCSS).

Thanks for reading. Enjoy your testing!!!

About Tacit Knowledge

Tacit Knowledge develops, integrates, and supports enterprise software for household name brands and retailers. Together with our parent, Pitney Bowes, we offer solutions across all consumer touchpoints, from device to doorstep. We connect the entire ecosystem of our customers’ digital commerce business. We build and support the Ecommerce website and associated technologies through to providing fulfillment (Pick & Pack, Kitting) through to Delivery, Returns, and Post-purchase customer engagement tools.
Singles’ Day 11.11 Just Raked in $25B and Deserves Your Attention

Singles’ Day 11.11 Just Raked in $25B and Deserves Your Attention

Singles’ Day – the annual Chinese shopping fest that takes place annually on 11/11 for people to celebrate being single – just broke the...
Read More
How Important Really Is Shipping Speed?

How Important Really Is Shipping Speed?

Many of the world’s largest retailers have recently announced plans to launch same-day or next-day shipping programs for online shoppers – services which...
Read More
1 2 3 4 6