Amazon customers find it difficult to find and compare products that have many options and competitive offerings. For example, research conducted in Q4 2017 indicated that out of 100 people that start searching for a TV on Amazon 5 will actually make the purchase.
Myself as UX lead, several product managers, a technical product manager, a software development manager, a junior UX designer and stakeholders in the Amazon search team that included a design manager, front-end developer, and a design technologist.
The initiative got its start as a result of a study I secured funding for and contracted to Blink UX to conduct in Q4 of 2015.
Both desktop and mobile experiences of Amazon.com was tested, allowing participants to self direct their shopping journey. A few competitor websites were also tested in the study.
Blink UX created a thorough report of their findings and presented it to leadership within the Product & Service Discovery team. While some of the findings confirmed what we already knew to be challenges and aligned with roadmap objectives, others opened up completely new opportunities to improve the customer experience. The research made the team confident heading into 2016 that our roadmap priorities were addressing customers needs.
Mary
While the Product & Service Discovery teams roadmap scope didn’t include the retail search experience, that team reached out to us to partner on an initiative after I had shared our research with them a few months prior. Aside from attributable sales revenue, the effectiveness of product comparison features has historically been measured through reduction in click-thru-rates in search, with detail page comparison table features typically reducing that metric by 2-3%. New metrics would eventually be needed though to measure the effectiveness of product comparison features within the search experience itself.
At this stage of the project a recently hired Junior UX Designer joined the team who I challenged with driving the mobile experience (with my lead) while I focused on the desktop experience where most of our customer shopped.
While the wireframe explorations on mobile led to some novel interaction patterns that we really wanted to test, it was desktop that would eventually become the primary focus as most customers do the predominate amount of their product comparison shopping there. Technical constraints also dictate that desktop be the first experience to test.
Christopher
Working with Blink UX again, a second usability study was conducted on both a desktop and mobile prototype.
Jesse
Along with the feedback from the usability study the designs were refined leveraging data points such as average product title lengths across various languages. Sketch measure along with a limited UI specification was provided to the front-end developer within the Amazon search team.
After launching in an A/B test the experience was a success with some customers, but was a miss with many others. Hypothesis for failure included too broad of an experiment with too many product categories included along with a lack of data quality in those categories.
While myself and a number of other team members from the original project moved on to other teams in 2017, the original technical product manager kept the project alive and it returned in 2018 to be tested again after mechanisms were put in place to improve data quality and limit which product categories and search queries would result in the comparison experience being shown to customers.
