Avoid Object Recognition Frameworks in your Test Case Automation

Avoid Object Recognition Frameworks in your Test Case Automation

There are shiny tools that promise to automate functional cases using object recognition technology.

Sikuli and Eggplant are examples of tools that allow you to interact with UI elements using images. It’s easy to fall for the promises of using these tools to automate almost anything: from native apps to complex web sites.

There are several things to consider before you invest time using these kind of tools.

Test Case Creation

This task can quickly become overwhelming. These tools require a set of images to use as identifiers to click, fill or interact with any element in your app.

I dare you to checkin your source code to a control system and see the commit that added 100 new files to the repository just to automate 1 single test case.

Test stability

Although there are ways to customize image recognition sensitivity on all tools, you will soon discover that even small changes will break your tests. Forcing you to either capture images again (and again and again) or accept the fact that your test will be failing from time to time.

Test portability

If you are planning to collaborate and create or run Test Cases using computers with different OS (Win, Mac, Linux ) or even people that are using different resolution on their screens, this might easily become a source of stress and frustration. Image recognition is just a comparison between two images and as I already stated even small changes might cause a failure.

Imagine plugging these cases into a CI server. You will create the perfect job to fail every time somebody is pushing code to the environment.

Data driven capabilities

When doing data driven testing you sometimes want to randomize your data in order to interact with different elements on a website. For example, you might want to choose a randomly selected set of products on a search landing page; in this case you will need to create a new image for each of the products you might interact with. Maintaining all these images and the scripts that use them quickly becomes cumbersome and difficult to maintain.

Test Speed and Computing Resources

If you do not mind waiting 10–15 minutes to run (and see them fail) object recognition is your best choice. Also consider the computer power used to run these kind of tools. Image recognition is expensive when you talk about computer cycles.

In summary I would say that these kind of technologies might have an application ( Super stable UI, running only in 1 machine, etc.. ). But considering the complexity of the current apps you might want to take a look to other tools that offer you a maintainable and stable set of test cases.

Some people would say that they would use Sikuli to test how the site renders on a device, and I would agree if we did not have access to more suitable tools like PhantomCSS that allows you to create a set of images used as a baseline (using CSS locators ) in the same computer that you will run the comparisons (IOU a blog post about PhantomCSS).

Thanks for reading. Enjoy your testing!!!

About Tacit Knowledge

Tacit Knowledge develops, integrates, and supports enterprise software for household name brands and retailers. Together with our parent, Pitney Bowes, we offer solutions across all consumer touchpoints, from device to doorstep. We connect the entire ecosystem of our customers’ digital commerce business. We build and support the Ecommerce website and associated technologies through to providing fulfillment (Pick & Pack, Kitting) through to Delivery, Returns, and Post-purchase customer engagement tools.
Brands and retailers: are you ready for Progressive Web Applications (PWA)?

Brands and retailers: are you ready for Progressive Web Applications (PWA)?

Article provides a history of mobile technology, discusses why Progressive Web Application (PWA) is ideally placed to meet customer expectations.
Read More
SAP Hybris 6.3

SAP Hybris 6.3

Following the release of the SAP Hybris 6.3, the latest version of the leading commerce platform, we’ve had time to evaluate some of the key benefits for both B2C and B2B companies. Here’s what we think!
Read More
1 2 3 6