Machine Learning Implementation is just like Good Strategy
I often find that people treat machine learning as a black box – they put something in and they expect to get a wondrous result out. The more I’ve learnt about machine learning over the years, the more the parallels between any machine learning implementation and good marketing strategy have become more striking, so hopefully this rather long comparison will help.
Controversies in Machine Learning
You may be familiar with the various controversies machine learning has had over recent years relating to facial recognition and recruitment.
In 2015, a software engineer called Jacky Alciné pointed out that Google Photos were classifying his black friends as gorillas, because the initial test set was performed on Google Engineers, who were primarily white or of asian/eastern descent.
In 2018, a study from the National Institute of Standards and Technology in the USA indicated that there is a 10-fold difference in error rate for black and white women. White women were falsely matched at a rate of 1/10,000 whereas black women were given a false match rate of 1/1,000.
Similarly, Amazon realised by 2015 that its machine learning models used for recruitment were favouring men, penalising CVs which mentioned the word ‘women’s’ and favouring those which used typical masculine language like ‘executed’. This is because they’re based on the CVs of existing employees, which brings with it their own historical organisational biases.
These may have been innocent mistakes early on, but we can learn from those errors in setting up machine learning, especially as they turned into quite significant and well-publicised cases. The machine learning we implement may or may not have such significant impact as these, but they serve as very useful reminds of how far we can get things wrong and may help in the argument internally of why it’s worth spending the budget to get it right.
Fundamentally, this all relates to them not being implemented with some clear strategy going into it. Like any execution, if we don’t have a strategy behind it we will trip up.
Basics of Strategy & Machine Learning
I come from more of an operational background within marketing, leading into a strategic one, but the more I’ve learnt about machine learning over time, the more the parallels have become quite striking to me between traditional marketing strategy and machine learning. The same core threads exist between strategy and any machine learning or other data analysis exercise.
If you aren’t sure of the input and the why of the input, then you will likely end up with the wrong output.
To do that let’s talk about the basics of strategy to make sure we’re on the same page.
First, think of strategy as having core aspects of a diagnosis – the identification of the challenge, scope, capabilities, constraints, underlying objective, the guiding policy or direction and how we wish to solve the key thrusts or challenges from our diagnosis and the plan of how we actually implement the direction and answer the problem in detail. Bearing this in mind will help with my comparisons.
Similarly, to make sure we’re on the same page about machine learning, it’s important to bear in mind that at its core, it’s just a way of doing a very manual process – which a person could do – at speed. For lots of really basic automation and machine learning tasks, we could hire a few dozen people to look at a lot of data and spot patterns over the course of a few weeks or months. Until you get to much more sophisticated pieces of work, that’s all it is: a massive saving in time and effort or cost to get some of this analysis done.
Put in that way, this should immediately demystify it, as what we’re talking about is not a magic black box, but a significant efficiency driver – so efficient that it allows us to do things we’d normally never bother with without lots of money, time and floorspace. It doesn’t allow us to do anything we couldn’t already theoretically do.
Just like if you hired a bunch of people to do this laborious task, though, we need to train the machine learning models to understand the data set and understand what a good result looks like.
For instance, if anyone here is familiar with LinkRisk/Link Detox, these were tools which were used in the days of manual penalties and early Penguin updates to identify spammy links which could harm organic rankings. At a basic level, all these did were take the input from what links people labelled as bad or good links manually, learnt from this and automatically categorised new links which people submitted. They didn’t do anything transcendent except in virtue of taking out a lot of the effort of pouring over tens of thousands or millions of links and working out if they should be removed or disavowed (if one trusted the underlying methodology and inputs).
So, just like a strategy, in order to implement this correctly we need to work out that diagnosis, guiding policy and plan. We need to work out where we are, where we want to get to in what conditions and how we get there.
So in a marketing strategy we might want an understanding of the market, what competitors are doing, how good our product or service is in the market and so on. Then we need to work out what position we want to be in the market and what revenue we want, and then the plan of what changes we need to make to our ads, to our branding, to our website to do that.
What data do we have?
With data, it’s more like looking at what the situation with your data is, what kind of outcome or result you want to get and then what process you’d need to go through to get there.
So the first step is to diagnose the data situation we are in: what are the data sources we have access to, are they inter-relatable and share some identifying fields or properties so we can connect e.g. customers together, how clean is the data, which bits can we use it from a GDPR perspective, is there any data in there which could confuse or mislead machine learning, etc.
Again, as machine learning isn’t that black box, ask what a person would or could do with the data: if you and I can’t understand it, how will the machine learning?
This is just like if you asked for a report on your performance in the market place across organic search, paid search, display etc., and they all came back in different languages and with entirely different and incomparable metrics, objectives and competitor comparisons.
An example of this in terms of data cleanliness from several years ago is from when I worked with a travel client. Shortly after I started working with them I asked what I thought was an innocuous question, of how many travel destinations they had, across flights, cruises, hotels, etc. They couldn’t tell me. Then I asked it a different way, as what places do they go to and they couldn’t tell me that either.
Finally, I received a raw data dump and I could understand why: they had brought together several different databases, some of which used different kind of location data (e.g. lat/long, postcodes, address names). Even within data sets, it was inconsistent, so that, for instance, Cologne in Germany had 11 or 13 different spellings of it, let alone encoding issues. No wonder they couldn’t tell me anything about their numbers! Similarly, machine learning, in this case, would get very stuck (unless it had some data cleansing incorporated) and find it difficult to tell anything useful about these destinations, e.g. which ones were more profitable or what kind of customer went to different destinations.
What do we want to do with the data?
Next, we need to work out what we ultimately want to do with this. The Amazon/LinkedIn situation reflects this well: they weren’t clear enough on the outcome they wanted. More specifically, what does good look like? They assumed that good looks like the people they already have, without realising that this just perpetuates existing organisational bias. When we do this, we need to figure out, say, what a good customer actually looks like and not just what a current good customer looks like. Similarly, if we want to know what a current typical customer journey looks like, we should bear in mind that this includes the constraints of our customer journey and it might require them to do stupid things to be able to make a conversion.
We should bear in mind that there may be existing problems when we’re trying to look at what’s going on with machine learning.
I’m not saying that I’m more clever than these people at Amazon or LinkedIn or Google, but let’s learn from their mistakes – and this may mean including some wider questions or thinking in your decision making process behind machine learning.
This is just like with a marketing strategy where we don’t want to get pigeonholed in thinking that the best approach is the one we’ve always done. Old Spice substantially changed their presentation when they came out with their newer range of ads almost a decade ago, we need to make sure we’re not just perpetuating the same old, same old with our machine learning. (If you’ve still not seen the Old Spice range of “Smell like a man, man”, ads, there’s a link in my References.)
Do we need to do anything else?
Finally, work out what you need to get this working. To start with, you’ll need to work out what data model to use. There are loads of data models out there – just a quick Google shows you different hit lists people recommend – so you’ll need to work on the right one for the situation. At Edit we have 40 models we can wheel out to try in use at the moment, so you need to work out which is the right one for the situation. Even once that’s done, it’ll still need training, tuning and optimising, so don’t think it’s always as easy as pulling something off the shelf and the work is done.
Hopefully, the days of buying a generic SEO plan or having a standardised PPC account structure across every sector from a handle-turning digital agency are behind us and this should be no different.
Also, based on the diagnosis, you should work out what things you need to put in place to allow you to do that machine learning properly. This might be as basic as data clean-up, but it might mean changing your processes and data gathering quite significantly. We might need to gather entirely new data to allow us to connect data sets, or change our CRM platform or CMS to make things work in the right way.
Don’t underestimate this either, as there’s no point developing a machine learning approach which misses the essential strategic element of feasibility/capability. If you can’t do it, it doesn’t matter how great the idea is – just like a marketing campaign which hinges on just getting something to go viral all on its own without any ad spend. It’s probably a bit unlikely.
Once you’ve done all that and let the model run, you likely won’t have something which learns from itself (which led to the previous scandals talked about) and will need to monitor it and make tweaks it yourself, but that’s obviously a talk in itself!
To sum up, lots of it is just about getting the basics right, as with any good strategy and you need to work out:
- What place your data is in
- What you want the outcome to be
- What you need to do to make it make sense and if you can even do that
The video of this talk can be found here
References / Further Reading
- Facial recognition – https://www.wired.com/story/best-algorithms-struggle-recognize-black-faces-equally/
- Human recognition – https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai
- Amazon/LinkedIn recruitment – https://www.theguardian.com/technology/2018/oct/10/amazon-hiring-ai-gender-bias-recruiting-engine
- Old Spice ads – https://www.youtube.com/watch?v=owGykVbfgUE
- Good Strategy Bad Strategy – http://goodbadstrategy.com/
- The Bilbao Effect – https://www.theguardian.com/artanddesign/2017/oct/01/bilbao-effect-frank-gehry-guggenheim-global-craze