Sometimes a core truth about retail comes along that is so poignant, its core message far outlives the span of the news cycle that delivered it. One of those truths can be summed up as: “What customers say they want does not always align with their behaviors.” Retailers who fail to understand that can pay a hefty price. For a textbook example, look no further than Walmart’s very costly attempt to respond to customer feedback.

Walmart’s (very) costly misjudgment

Back in 2009, the retail giant surveyed their customers and asked a seemingly straightforward question: Would you like Walmart to be less cluttered? To which the answer came back: yes. In an effort to give customers what they wanted—and, in the process, attract higher income shoppers and directly challenge Target—Walmart conceived and began implementing what they called Project Impact. Project Impact was planned to be a five-year undertaking that improved the customer experience in all of Walmart’s stores by reducing clutter, making it easier to navigate the aisles, and improving aesthetics—essentially making the stores more Target-like in appearance and function. (It may have been a pure coincidence that Walmart’s Chief Merchandising Officer was, at the time, a former Target executive…but probably not.) The most noticeable change introduced by Project Impact was clearing away the merchandise that usually sat in the aisles between fixtures, accomplished by cutting back about 15% of the merchandise offered in the stores. Although initial reports of customer satisfaction were positive, all was not well for Walmart. By the time Project Impact reached 600 of the company’s thousands of locations, the impact (pun slightly intended) started to show: year over year same-store sales plummeted, and the rollout was eventually put on hiatus. The effort is estimated to have cost Walmart $1.85 billion in lost sales—a figure which does not include the hundreds of millions they spent renovating the stores. Ouch.

Where did Walmart go wrong?

Clearly, this was not the desired outcome for Project Impact. Walmart lost unfathomable amounts of money, their ex-Target CMO quickly joined the ranks of ex-Walmart executives, and the chain ended up reintroducing the SKUs they had just cut. So what went wrong? Some say that by “decluttering” and narrowing its range of available products, Walmart strayed from its core focus (offering a broad selection of low-priced merchandise) and paid the price. Others point back to troubles that began in 2005, when the company took out advertisements in Vogue in an attempt to appeal to higher-income shoppers—an incongruity that stood out to both ends of the socioeconomic spectrum. The simplest explanation, however, is that Walmart devised and executed Project Impact without a customer-focus mindset. True, the initiative was driven by input gathered from a survey—but, intentionally or not, the survey was designed with an answer already in place. It’s unlikely that customers would answer “no” to such a leading question, because most wouldn’t think through to what their answer would mean in the real world: reducing merchandise options and changing the basic nature of what it meant to shop at Walmart.

Better survey questions yield better survey results

Even a decade later, Project Impact serves as a sobering reminder that customer feedback is important, but it’s only useful when it’s gathered in a way that yields unbiased results. A more carefully-designed customer survey would have returned cleaner data to Walmart—instead of confirming what they thought they already knew—and helped guide decisions about how (or if) to rethink their approach to the customer experience. If your brand is interested in using customer feedback to drive business decisions, use the following guidelines to get unbiased and actionable survey results:

• Don’t ask leading questions. Questions that contain or imply the answer you expect to receive make it difficult for the respondent to answer outside of the phrasing of the question. Walmart’s survey question (Would you like Walmart to be less cluttered?) correlated shoppers’ level of satisfaction only with clutter, ignoring other areas in which the stores might have improved.
• Avoid questions that prompt simple “yes” or “no” answers. Those responses are narrow and rarely provoke deeper insight. Focus instead on open-ended questions that allow respondents to elaborate on their own unique thoughts. Walmart might have had better success with asking, “What kind of changes would improve your in-store shopping experience?”
• Don’t write questions that produce useless data. What if Walmart had used a true/false methodology and asked, “I don’t shop at Walmart because it’s too cluttered”? “True” answers may have provided some useful information, but what about those “False” answers? The answers would have been useless. Why? Well, a false answer could mean that a customer shops at Walmart despite the fact that it’s cluttered. Or, it could mean that a customer doesn’t shop at Walmart, but for some other reason. Maybe they don’t shop at Walmart because it’s crowded and has long lines. Or maybe they don’t care about low prices. There are countless reasons customers may shop  somewhere other than Walmart, so all of those “False” answers would have been uninterpretable.
• Don’t survey customers thinking you already know their answers, or you’re likely to run into confirmation bias. Hindsight indicates Walmart thought it knew what its customers wanted before they ever launched the survey that led to Project Impact; couple that with the leading question about clutter and an unbiased answer was never in the cards. Keep an open mind when writing your survey questions, and remember to really listen to the results.

As helpful as well-designed surveys can be, there’s an even better way retailers can get the information they need: by focusing on what customers actually do.

Focus on what customers do rather than what they say

It’s known among psychologists that we humans are rarely good at predicting our own behavior. Surveying customers about what they think they want often returns aspirational answers with a wide margin of variance, while real insights come from observation of behaviors over time. As far back as 2000, author and retail guru Paco Underhill built a business around observing customer behavior. His takeaways (some of which are a bit shocking) are summarized in Why We Buy, updated in 2008. Today, retailers get a big boost from technology. Heat maps can help decision-makers visualize how customers move through a store, which products they touch, and which areas of the store get little to no traffic. Other retailers enlist the help of customers by asking them to wear special eyeglasses that track the wearer’s eye movements as they move throughout the store. That helps retailers determine the effectiveness of their signage, identify products that attract attention (as well as whether customers buy them or just look at them), and whether customers can navigate the store easily or have to keep stopping and looking for the items they want. Ongoing customer satisfaction relies on delivering a consistently excellent customer experience—while making careful, informed decisions about where and how to evolve your approach. Walmart’s hard lessons learned are a valuable reminder to other retailers: the value of the data you collect is fundamentally tied to the way in which that data was collected. To put it simply: Garbage in, garbage out.

What’s Next?

No matter how solid your data is, your initiatives will only be successful if they’re properly executed at the store level. That means store employees need to understand what you want them to do and why you want them to do it, as well as the details of when and how to do it. (Should they move store displays while customers or shopping, or should they do it while the store is closed?) Whether you’re a Walmart or a Cartier, delivering the right experience in every store is critical to your success. That means the initiatives developed in headquarters have to be communicated consistently, so that everyone from multi-unit managers to part-time employees are on the same page. Retail Zipline’s store communications solution can make that happen. Learn more or set up a demo today!