Wiggle Recommendations Engine

The Wiggle recommendations engine is an excellent example of applying Big Data technologies to solve a real-world problem in digital commerce.

In this case, the challenge was the ongoing capture, storage, and processing of billions of rows of user clicks to derive relevant product recommendations.

This kind of problem lends itself to a solution that trades consistency for high availability and partition tolerance. In addition, big data technologies store information in different kinds of structures: some store data as documents, some as name/value pairs, and others as tables. A quasi-tabular structure was selected as it best suited the type of information and associated processing required.