Apache JMeter is an application mainly designed for load testing and performance measuring yet with the increase of business distribution models, software delivery with SaaS and the continuous move towards cloud computing, the Quality Assurance process requires more and more verifications at multiple steps. One major advantage of JMeter is that it offers configurable logic that makes it easier to create functional testing user journeys without code or scripting. This article outlines a cross disciplinary approach for JMeter that allows users to get the most out of this powerful independent, open source tool and address the expanding Quality Assurance needs of thriving eCommerce businesses.
JMeter is broadly used in the Industry for Performance Testing, mainly because is open source, easy to use and platform independent, there are several options in the market but one of the main advantages of this tool is that it provides a configurable logic that makes it easier to create functional testing user journeys without code or scripting.
For functional testing JMeter provides the following advantages:
- Inherently parametric. Anyone that has created a test plan in JMeter knows that it is a tool based on parameters and scope. Variables that are used within the scope of a test plan can be set and overwritten dynamically, and columns from a CSV file are automatically set as variables.
- Set sequence of requests. A test plan describes a series of steps that will be executed when run. Using a Cookie Manager and or an Authorization Manager, with JMeter a session can be automatically held for test execution within the run context. Different from other tools, such as Postman, in which you will have to use the collection runner and pass data between requests via scripts.
- All values within the request-response can be saved and reused. Even text in the html response can be extracted and set as a variable.
- Assertions can be configured for any value in the response, header, code, data, URL.
- Logic elements available only to be configured. Drag and drop logic elements are enabled in JMeter, reducing the time spent in scripting.
- Validations of XML, HTML , JSON also XML Schema using configurable assertions.
Practical example: The case of the SiteMap Story
Typically, there will be a single story related to the construction of the SiteMap with the subsequent QA test cases.
There is often a lack of standard knowledge of how this feature should be tested. Testers may not understand why a SiteMap is required, they might apply a manual testing strategy without any regression planned.
The SiteMap is very important, it is a roadmap that leads Google to all relevant pages. SiteMaps are integral for SEO, ensuring that searchers can crawl to all the internal links included.
SiteMap is a very good candidate for automated testing for the following reasons:
- SiteMaps are automatically generated by a cronjob, meaning that their content might be updated periodically. So a single initial check would not be enough.
- They should have a specific schema. A schema is a pattern for the XML. To manually verify this pattern can be tedious and ineffective.
- It includes a predefined set of pages that might change in the project development. For example, content pages might be added and PDP pages might be disabled or created. A common issue is a leftover URL of a disabled page being included in the SiteMap; this bug would be difficult to find with manual regression testing.
How to verify a SiteMap with JMeter
The SiteMap should have a predefined URL per environment where it can be accessed. Depending on the client requirements, it may include Homepage, Category Pages, PDPs and other Content pages.
First, set the URL to get the SiteMap index or indexes. This categorization makes a site’s structure as clear as possible. Once you navigate to a SiteMap from its index, you’ll see all URLs in that particular SiteMap.
Second, make a request to the index. from the index, identify all URLs in the XML output and navigate to each one of them to verify that no 404 errors are presented(this happens when an inactive page is requested).
The first automatic verification will be the schema validation. An assertion element that verifies the schema of the XML retrieved by the request using a “xsd” file in our plan. An xsd file contains the specification of the schema to be verified. The path to this file can be set as relative or absolute depending on your structure.
Once you have verified the XML construction of the index, you need to get each one of the URLs contained in the file using an XPATH Extractor with an Xpath query. You will get each one of the SiteMaps and/or (if needed) separated by type. (Product, Category, Content, etc.)
Use the “siteProducts” variable to navigate to the PDP’s SiteMap and get each one of the URLS listed in this SiteMap with another XPATH query.
JMeter automatically creates an array with the list of matches. The name of the array is defined as a variable.
The SiteMap requirement includes xsd templates for the Index and also for the SiteMap itself. With JMeter you can use an XML schema assertion per request.
Using “For Each Controller”, all the URLs we get from an xpath query can be navigated.
It does not matter if the SiteMap is nested one or more times, using the same logic we can navigate until the Page URL level.
So, which validations are we running with this test?
- Sitemap Index Structure
- Sitemap Structure
- Sitemap categorization is set as expected
- All pages in the Sitemap are available
- All the URLs retrieved by the SiteMap are correct
- Number of pages retrieved by the SiteMap
You may add more validations, if required. For example, you can get the number of pages retrieved by each SiteMap and compare with an expected number (example: Active and/or with stock products). This value could be set as a test parameter or dynamically obtained via request or database call.
You may also add HTML validations to each page or use the full set of pages retrieved to verify other functionalities like the SEO tags that should be included on certain pages with a response assertion like this:
In conclusion. Testing tools are effective when they serve the main objective of finding relevant issues, and JMeter offers teams powerful capabilities to address the expanding Quality Assurance needs of thriving ecommerce businesses. The key offering is that JMeter provides configurable logic that makes it easier to create functional testing user journeys without code or scripting, allowing teams to do more with the tools they’re already using.