Competing On Insight

A few years ago, one of this country’s leading car manufacturers assembled all of its marketers in one room for a first-ever look at a profile analysis of its owner base. These marketers were used to looking at consumer profiles from automotive research companies like J. D. Power and Maritz – never one based on actual vehicle purchases from their own database records. What could this analysis tell them that they didn’t already know?

Plenty, as it turned out. But as the presentation wore on, their interest waned. As far as they were concerned, their job was to drive sales by pumping up consumer demand, largely through advertising. They were used to obsessing over Gross Ratings Points, Share of Voice and Market Penetration. Now they were hearing alien terms like “customer lifetime value” and “churn” along with charts showing average purchase intervals and vehicle migration patterns. Numbed by the numerical onslaught of facts and figures, they stared in silence. The purpose of the meeting had been to provoke thought – instead it ended just as it had begun, with a mildly curious but disengaged audience anxious to get back to looking at sales reports.

It had taken nearly a year to build a consolidated database of vehicle owner records to conduct the analysis. Automotive data is notoriously dirty and unreliable, mainly due to the challenge of tracking owner vehicle records past the initial sale, as people sell or trade their cars in later years. A lot of effort had gone into matching and merging records to identify half a million “active” owners with their complete vehicle purchase and service history. The analysis was rigorous and thorough, covering everything from repurchase rates by cohort groups, to expected in-market buyers by year, to projected service revenue by owner segment. And there were enough provocative insights to spur discussion – for instance, the actual owner retention rate was far below what they thought – and a majority of owners had bought their next car within the surprisingly short span of 4 years. Yet all of these meaningful statistics sailed over the heads of the audience. The analysis should have convinced them to redirect more of their marketing budget toward retention of existing owners. Instead, the financially struggling car maker continued to spend heavily on conquest marketing to attract new buyers.

This story is not an unusual one. In fact, it is commonplace in most companies: the struggle to convert data-driven insight into customer strategy.

Information Synthesis

According to a recent Gartner Research forecast, 60% of CMOs are expected to cut their marketing analytics investment by half in the next three years, a mystifying retreat considering the amount of data now available to mine for insight. The main reason, according to Gartner, is a “failure to realize promised improvements”. In other words, the time spent on analytics has not paid off in tangible gains for the business. Most marketers struggle to incorporate customer data into their strategic planning. They recognize its advantages for targeting purposes. For campaign planning. For web tracking. But what ultimately undermines their use of customer analytics is a lack of data fluency. They are content with superficial analysis that supports their main goal: to build brand awareness and drive sales. Which explains marketing’s fading influence on corporate strategy: they rarely have anything new to say.

The answer is not simply to hone the analytical skills of marketers. Or to expect data analysts to do their thinking for them. It is to elevate the role of marketing analytics in the business. Confined to their swim lane, marketers are fixated on their own narrow goals like customer acquisition. Who can blame them? The same is true of every business unit: the answers that interest them the most are the ones directly related to their accountabilities. Whereas deep insight – the kind that leads to new ideas and product innovation, to game-changing strategies, to accelerated growth – emerges out of information synthesis: distilling everything known about customers from all sources into a holistic understanding of their needs, actions and motivations.

The siloed structure of most companies is not conducive to an omnibus analysis. The analytical work is always divvyed up according to the planning needs of the different business units. Customer Care is mainly concerned with service satisfaction and call volume levels – Sales Support only cares about pipeline velocity and lead conversion – Product Management keeps a watchful eye on distribution channel shipments – the Digital Marketing specialists just want to see web traffic and social media data – the Demand Creation group is mostly interested in digital advertising results. Usually, each business unit is assigned its own purpose-built data mart, reporting system and analytical workbench, pulling the source data from a data lake, enterprise warehouse or directly from operational systems of record.

Independent of these analytical nodes is the market research group whose knowledge source is primarily survey-based, drawn from consumer panels or direct customer feedback. Their work typically includes usage and attitude studies, market segmentation profiling, brand health tracking, concept testing and consumer trend analysis. Researchers live in the world of representative population samples. They are less comfortable with transactional analysis, web analytics and data mining. Attitudinal research remains their sole field of expertise, amounting to just another knowledge fiefdom, disconnected from other sources of insight.

A New Analytical Framework

Without a broader mandate, marketing analytics is doomed to remain just another parochial function, serving a single constituency. Deprived of access to all of the information, marketers fall back on what they believe to be true instead of what they know to be true. But simply building a single view of the customer, or forming an analytical “centre of excellence”, or finding some “unicorn” data scientist to serve as a “business translator” is not going to make marketers any wiser about what customers want. The solution is not more data or more reports or more algorithms – businesses are awash in enough of that already. Instead, it lies in training marketers to be insight-driven. That means basing decisions on facts, not intuition; on knowing the right questions to ask; and on expanding their field of vision beyond the point-of-sale. As Forrester Research notes, “To be customer-obsessed, marketers need to embrace analytics across the entire customer lifecycle rather than myopically focus on phases that cover awareness and purchase.”¹

Deprived of access to all of the information, marketers fall back on what they believe to be true instead of what they know to be true.

Yet most businesses today are stuck in a time warp when data was frugally shared with marketers. The business intelligence systems that grew out of the transition to data warehousing technology in the 1990s were intended to support basic financial and operational analysis. They allowed decision-makers to closely monitor and streamline the internal workings of the company. Marketers were begrudgingly given access to sales data from internal CRM and ERP systems through batch reporting and dashboards – otherwise they had to seek help from external analytics providers or adopt marketing automation technology on their own.

Figure 1 – To have a greater influence on corporate strategy, marketers must become “customer-obsessed”, focused on moving customers up the loyalty pyramid.

That all began to change in the mid-2000s when the era of Big Data dawned. A vast new universe of online data exploded into view. Database technology was forced to evolve, as NoSQL database management systems and data lakes were developed to store the torrent of web data.

Today the leading insight-driven businesses are mainly the direct-to-consumer giants – Amazon, eBay, Netflix, Etsy, Uber and so on. All of them have thrived by embedding analytics into every facet of their business, every decision they make, every action they take. As Thomas Davenport, the author of Competing on Analytics, has observed, “The common thread in these examples is the resolve by a company’s management to compete on analytics not only in the traditional sense (by improving internal business decisions) but also by creating more valuable products and services.”²

Amazon is a textbook example, using its massive transactional database to deliver the right products to customers in the shortest period of time at the lowest cost. They boost repeat purchasing through recommendation algorithms; calculate how much of each product SKU should be available at a local fulfillment centre; calibrate merchandise selection and pricing; plan the best delivery routes, and match packages and destinations to fleet availability.³ No wonder Amazon accounts for almost 40% of U.S. online sales.

Etsy, on the other hand, has developed a sophisticated in-house research capability fully integrated with e-commerce analytics to help product and user experience teams understand the habits and needs of millions of sellers and buyers. No product decisions get made without a factual base of evidence, based on exhaustive user research and customer analysis.⁴

Figuring out ways to create new value for customers is the highest order of analysis, where the purpose is to flag everyday problems that need to be solved. For marketers, this is an opportunity to steer the business forward, not by chasing after brand awareness, or worrying about media efficiency, or fussing over fluctuations in web traffic, but by devising novel ways of pleasing customers.

The starting point to become more insight-driven is to come up with a new marketing analytics framework. That means linking patterns in how people behave to their motivations, beliefs, and drivers; using measurement data to continually improve marketing and sales performance; analyzing the lifecycle progression of customers in order to provide the right assistance, offers and treatment across touchpoints; and knowing which new products to offer, or adjacent markets to enter, in the quest for growth.

Figure 2 – Meaningful marketing insight connects how customers feel about the brand with how they actually behave.

By organizing all of the marketing analytical functions and decision-making processes around these four strategic pillars, marketers can start to have a greater impact on corporate strategy. The priority is certainly to move customers up the loyalty pyramid, turning them into brand advocates. But the way marketers will earn their keep – and be seen as heroes by executive management – is to increase the value of existing customers.

Hearts and Minds

Once the analytical model has been agreed upon, the next step on the path to becoming insight-driven is to put the right governance structure in place. An “Analytics Charter” is helpful, positioning the Customer Insight function as the bridge between data and innovation. The analytical group should be treated as business partners, not simply data jockeys, providing strategic counsel based on their reading of the findings.

More than anything, decision-makers are looking for the most sensible and least risky choice to make given all of the facts available. They are less interested in the path to get there – even less in the statistical jargon. The Insight team must be able to put their findings in context – tell a story that starts with a customer challenge and builds logically to a clear conclusion. Facts and figures left dangling out of context are just “nice to know” and easily forgotten. To have any lasting impression, they must have unmistakably positive or negative consequences. As Nancy Duarte, a communications expert and author of Data Story, advises, “Stories frame data so decisions can be made faster and inspire others to take action by changing their hearts and minds”.

Think about those automotive marketers who struggled to grasp the revelations that surfaced in the owner profile analysis. Was that lack of comprehension their fault – or due to the density of the information? Unaccustomed to seeing owner-level detail, they got lost in the maze of graphs and charts, failing to see what it all meant and why they should care. No wonder their collective reaction was: “So what?”

Figure 3 – The role of marketing analytics is to yield insights that lead to growth of the customer portfolio.

The Insight team must be able to put their findings in context – tell a story that starts with a customer challenge and builds logically to a clear conclusion.

When it comes to presenting the results of their analysis, most data scientists fare poorly, more at ease describing the statistical hoops they went through than creating a narrative arc. They simply lack the theatrical skills to bring a data story to life. And no wonder. Look at the full scope of their technical work: fetching the data – cleaning it – preparing it for analysis – running the machine learning algorithms – validating them – and then figuring out how to visualize the results. Expecting them to be equally gifted at storytelling is merely wishful thinking.

Data “wrangling”– pulling the raw data and refining it for analysis – soaks up much of their time. If the data lake has been hurriedly built without a lot of forethought, it can look more like a “swamp”, polluted by duplicate records, invalid or missing values, inconsistent formats, and much more, all of which have to be cleaned up before any of it is usable. Data scientists will typically spend 60% or more of their time on that stage alone. And then they have to explore the data for statistical patterns and anomalies; calculate new variables to enhance the explanatory power; and re-structure it to suit the purpose of the analysis before the data crunching can even begin.

To master this complexity, the CI function must be a team sport, staffed with data engineers, business analysts, data scientists, research specialists, and software developers, working together in a hub-and-spoke model. A central leadership team is charged with governance while field emissaries are embedded within the different lines of business, expected to become subject matter experts. This matrix model makes it easier to standardize best practices; exchange learning; pool resources; and collaborate on cross-divisional analytical projects. At the same time, it gives the business units a dedicated analytical resource conversant with their domain, their data aptitude, and their tolerance for complexity.

Figure 4 – An Analytical Centre of Excellence is structured to deliver the analytical expertise and data engineering support that marketers need

The role of research is to put a human face on the data and explain how customers feel as much as how they think and behave.

In setting up this “centre of excellence”, the market research function should be merged with customer analytics, eliminating the historical segregation of qualitative work and data analysis. The job of research remains essential, of course, responsible for inserting the “so what” into the story. The advantage of consolidation is that only one story needs to be told, backed by all of the information available, connecting what customers say with what they actually do.

The research specialists still own the voice of customer; they still have point on scouting the market for opportunities; they still monitor brand health. But on top of all that, they should be responsible for tracking customer health. And that requires them to work closely with the data analysts. If customer repurchase rates are off-target – if churn is higher – if NPS scores have fallen – if any of those vital signs are fluttering and alarm bells are ringing – they need to find out why. The role of research is to put a human face on the data and explain how customers feel as much as how they think and behave.

Systems of Insight

To support this expanded remit, the CI group is reliant on a systems architecture that not only makes ad hoc reporting and modelling easier but gives decision-makers the tools they need to integrate the right set of data into their planning process. This “system of insight” is made up of four tiers: the Source Tier encompasses all of the originating systems and external data sources; the Storage Tier is where the data is held in an intermediary state, directly accessible by all analysts; the Analytical Tier represents the workbench used to conduct the analysis; and the Access Tier gives the various business analysts and decision-makers a view of the data which conforms to their planning needs.

While a data lake (like Hadoop, Amazon S3 or Azure) ingests all of the originating data in its raw form through a data pipeline – everything from billing data to web log files to XML extracts from various 3rd party providers – a data warehouse (like Amazon Redshift, Oracle or Snowflake) stores it in a more familiar column and row format for analysts to query directly. Marketing analysts and data scientists only need to work with a sub-set of data that can be extracted using their preferred programming language (like Python or R). That way they can structure and format the data to suit the objectives of any bespoke analysis. Alternatively, the analyst might work with a data mart designed to serve the routine reporting needs of a specialized analytical function like product sales, product profitability analysis, e-commerce analytics, or market forecasting.

Marketing planners are usually given direct access to a data mart through a self-serve business intelligence tool of some kind (such as Power BI, Looker or Tableau) which allow them to view data visualizations and tabular reports, set up for them by a data analyst. For example, the data analyst might structure the data to show purchase activity by customer segment and then set up a standard report or dashboard in the BI tool with filters (e.g. data range, segment type, product category) for drill-down exploration by the planner.

Figure 5 – A Systems of Insight Architecture makes it easy for data analysts and data scientists to access all of the data they need on demand.

A complementary approach to a multi-tier analytical architecture is to opt for an all-in-one data management solution known as a Customer Data Platform (like Tealium or Adobe Experience Platform). CDPs offer out-of-the-box functionality to create a unified profile of customers with all of their browsing activity, purchase transactional data, channel usage, personal attributes, modelling scores, engagement history and much more. Some CDPs also come packaged with analytical reporting and model-building capabilities, saving the work involved in shunting the data to a different platform.

The other big challenge is for marketers to connect all of the disparate sources of customer knowledge. A Knowledge Library is required (like Dovetail) where marketing planners can find everything they need to know about customers – one central, searchable repository of past research studies, customer profiles, personas, performance reports and more, with direct links to the underlying summary data, all tagged in ways that make the information easily discoverable and sharable.

A Vivid Portrait

By plugging into these “systems of insight”, while relying on the CI group to serve up the insights they need, marketing can make customers the central focus of their strategic planning versus the brand. They will be able to ask the strategic questions about customers that really matter, starting with how to earn more of their business. Instead of obsessing over buyers’ path to purchase, they can educate themselves as much as possible about the people they serve: Who are our best customers? How can we make the brand indispensable in their lives? What can be done to enrich their experience and seal their loyalty? How do we encourage them to stay continuously connected?

Making the brand a more integral part of people’s lives calls for a deeper understanding of customer needs.

The combination of behavioural and needs-based segmentation is critical to knowing who customers are and their expected lifetime value. It helps to figure out how much future growth is likely to come from existing customers compared to new buyers – to calculate share of wallet and the growth potential of customers – to estimate the degree of flight risk amongst high-value customers – to know the size and potential value of adjacent markets based on the estimated population of “lookalikes”.

Making the brand a more integral part of people’s lives calls for a deeper understanding of customer needs. It means using direct feedback to learn more about potential product and service gaps. But it is also where predictive analytics comes into play, combing the data for product affinities and purchase propensities, leading to more relevant cross-sell and upsell offers.

Figure 6 – For marketers to get the insights they need, a New Analytical Framework is required which places greater emphasis on the health of the customer relationship.

Marketers succeed when customers choose to stick with their brand even in the face of a tempting competitive offer. But that means knowing the specific factors that drive customer loyalty. It also means using journey mapping to minimize pain points and identify ways the current experience can be improved; contacting recent defectors to determine what caused them to abandon the brand; and monitoring the level of customer loyalty, whether through NPS or some form of composite score that incorporates behavioural proxies (such as tenure, spending velocity, and relationship depth).

Finally, a sure sign of a committed customer is a willingness to hear from the brand regularly and be an active participant in the user community. Here is where social listening can reveal the level of brand enthusiasm – or where engagement analysis can track content popularity and degree of involvement in brand-sponsored events and activities.

Together these various methods of insight add up to a vivid portrait of customers which should become a catalyst for innovation and the foundation of corporate strategy. By “transforming numbers into narratives”, marketers can envision alternate paths, exert greater influence over the direction of the business, and reclaim their status as strategic leaders. But that can only happen if marketers first learn to ask the right questions.

1. “Seven Steps to Kick Off a Customer-Obsessed Insights Program”, Forrester Research, February 2018.
2. “Analytics 3.0”, Thomas H. Davenport, Harvard Business Review, December 2013.
3. “Amazon is a logistics beast – A detailed teardown”, Sangeet Paul Choudary, Platforms, AI and The Economics of Big Tech
4. “Case Study: Etsy Invests in Customer Understanding to Steer Growth”, Forrester Research, June 21, 2016

Stephen Shaw is the chief strategy officer of Kenna, a marketing solutions provider specializing in customer experience management. He is also the host of a regular podcast called Customer First Thinking. Stephen can be reached via e-mail at sshaw@kenna.ca