You only know how well your campaign is working if you can ask the data to prove its impact. For when a typical control sale method doesn’t work, we’ve developed a way to measure the real-world success of your campaigns using causal impact analysis combined with Microsoft Power BI.
And it works. Having run many B2B campaigns for a leading energy provider operating in global markets stretching from the UK to the Far East, this causal impact analysis showed where and how much value these campaigns added. Vital knowledge to help our client plan and target campaigns better for the future. Here’s how we do it. It could help your marketing campaigns too.
What is causal impact analysis?
Causal impact analysis is a data science tool that uncovers the real-world value of your marketing by considering the effects that outside factors can have. Originally developed by Google largely to test the impact of website changes, our data scientist Andy Aldersley took the methodology and adapted it as a means of analysing how sales are affected by email campaigns for this leading energy provider.
When you’re running several campaigns in tandem and cannot use methods such as A/B testing, causal impact analysis gives you an overview of how much of an impact your campaigns have had. This was the case for our client. As a B2B provider, where account managers maintain close links to their customers, they could not afford to withhold campaigns from certain customers as would be required with an A/B test.
Also, there are many external factors that can affect sales for a complex, global organisation like our client’s. To truly measure the impact of campaigns for a client like this, you need a solution that can account for all these factors.
That’s what causal impact analysis can do. It allowed us to measure the true impact of campaigns while still sending them to the relevant customers and still giving us an overview of various external factors within the market.
How we analyse the data
So, how does causal impact analysis work? Essentially, we’re trying to find out if a marketing campaign has worked based on an external piece of data that reveals outside factors in the market – for example, if a pandemic or changes in the political landscape have affected customers’ spending.
To do this, we build a model of the past behaviour of what we’d like to measure, namely sales, up until the point a campaign went out. This would be based on a mixture of data sources that wouldn’t have been targeted by the campaign. In our client’s case, the data tends to come from other companies from the same or other markets.
From there, we can predict what would have happened if we hadn’t sent the campaign – this is called the counterfactual model. When we compare the results of the counterfactual model against what we observed from the campaign, we can work out the campaign’s true impact.
To put all this into practice and present our findings back to clients in an understandable way, we turn to Microsoft Power BI.
Why use Power BI?
When monitoring the outputs of multiple counterfactual models, Power BI gives you an efficient way to report on their results and test what other aspects should influence your reporting. When running this kind of model, we run a very granular segment based on the profile of the customer.
This means we can easily say which campaigns work for certain types of customers and which work better for others. What that means for you is that we can quickly and easily find out which attributes are important for your campaigns and how to improve your targeting moving forward.
In our client’s case, we sit over the top of all the transactional data going through one of their refuelling solutions, including where the programme’s cards are being used, how much fuel is being bought and what service products are going through particular accounts. So, we could see how campaigns around products were being targeted.
By building our models, we could see exactly how much influence a campaign was having on product sales. With so much data to work through, putting it through Power BI reduces the time spent on this analysis by several hours. Power BI also produces the visual analytics to help us understand what causes customers to respond to a campaign and drive revenue, as well as which markets the campaign worked best in.
How will this benefit your business?
Using causal impact analysis and Power BI together reveals the real-world effect of your campaigns, finding out the true cause of effects to your KPIs and, ultimately, sales. With this information, you can determine how to change your campaigns and who to target better to improve your sales.
In our client’s case, one of their 2021 goals in working with us is to take the lessons learned from our analysis and use it to tailor their marketing activity. Thanks to our work, they were able to see the relevancy of different campaigns and how to reduce the number of sends – this could eventually lower the costs of their marketing and increase return on investment.
Like this leading energy provider, this approach is best applied to clients working with a high volume of transactional activity – for example, in the retail or hospitality industries. With this large amount of data, as well as any third-party datasets or groups of controls that you’ve got, we can model the expected behaviour of your clients and determine the effectiveness of your campaigns and their targeting.
However, while our approach to causal impact analysis has been designed to be easily adapted to many different clients, this combined with Power BI isn’t for everyone. We can find what’s best for you. Our data science team works with clients to uncover what solution will suit their needs and help them make the best marketing decisions.
Contact us to see what we can do for you.
