modeling is the pinnacle of analytics value. It doesn’t focus on what happened, or even what will happen – it takes analytics further by telling us what we should do to change what will happen. To harness this extra prescriptive power, however, we must take on an additional assumption…a causal assumption. The naive practitioner may not be aware that moving from predictive to prescriptive comes with the baggage of this lurking assumption. I Googled ‘prescriptive analytics’ and searched the first ten articles for the word ‘causal.’ Not to my surprise (but to my disappointment), I didn’t get a single hit. I loosened the specificity of my word search by trying ‘assumption’ – this one did surprise me, not a single hit either! It is clear to me that this is an under-taught component of prescriptive modeling. Let’s fix that!
When you use prescriptive modeling, you are making causal bets, whether you know it or not. And from what I’ve seen this is a terribly under-emphasized point on the topic given its importance.
By the end of this article, you will have a clear understanding of why prescriptive modeling has causal assumptions and how you can identify if your model/approach meets them. We’ll get there by covering the topics below:
- Brief overview of prescriptive modeling
- Why does prescriptive modeling have a causal assumption?
- How do we know if we have met the causal assumption?
What is Prescriptive Modeling?
Before we get too far, I want to say that this is not an article on prescriptive analytics – there is plenty of information about that in other places. This portion will be a quick overview to serve as a refresher for readers who are already at least somewhat familiar with the topic.
There is a widely known hierarchy of three analytics types: (1) descriptive analytics, (2) predictive analytics, and (3) prescriptive analytics.
Descriptive analytics looks at attributes and qualities in the data. It calculates trends, averages, medians, standard deviations, etc. Descriptive analytics doesn’t attempt to say anything more about the data than is empirically observable. Often, descriptive analytics are found in dashboards and reports. The value it provides is in informing the user of the key statistics in the data.
Predictive analytics goes a step beyond descriptive analytics. Instead of summarizing data, predictive analytics finds relationships inside of the data. It attempts to separate the noise from the signal in these relationships to find underlying, generalizable patterns. From those patterns, it can make predictions on unseen data. It goes further than descriptive analytics because it provides insights on unseen data, rather than just the data that are immediately observed.
Prescriptive analytics goes an additional step beyond predictive analytics. Prescriptive analytics uses models created through predictive analytics to recommend smart or optimal actions. Often, prescriptive analytics will run simulations through predictive models and recommend the strategy with the most desirable outcome.
Let’s consider an example to better illustrate the difference between predictive and prescriptive analytics. Imagine you are a data scientist at a company that sells subscriptions to online publications. You have developed a model that predicts that probability that a customer will cancel their subscription in a given month. The model has multiple inputs, including promotions sent to the customer. So far, you’ve only engaged in predictive modeling. One day, you get the bright idea that you should input different discounts into your predictive model, observe the impact of the discounts on customer churn, and recommend the discounts that best balance the cost of the discount with the benefit of increased customer retention. With your shift in focus from prediction to intervention, you have graduated to prescriptive analytics!
Below are examples of possible analyses for the customer churn model for each level of analytics:
Now that we’ve been refreshed on the three types of analytics, let’s get into the causal assumption that is unique to prescriptive analytics.
The Causal Assumption in Prescriptive Analytics
Moving from predictive to prescriptive analytics feels intuitive and natural. You have a model that predicts an important outcome using features, some of which are in your control. It makes sense to then simulate manipulating those features to drive towards a desired outcome. What does not feel intuitive (at least to a junior modeler) is that doing so moves you into a dangerous space if your model hasn’t captured the causal relationships between the target variable and the features you intend to change.
We’ll first show the dangers with a simple example involving a rubber duck, leaves and a pool. We’ll then move on to real-world failures that have come from making causal bets when they were not warranted.
Leaves, a pool and a rubber duck
You enjoy spending time outside near your pool. As an astute observer of your environment, you notice that your favorite pool toy – a rubber duck – is typically in the same part of the pool as the leaves that fall from a nearby tree.
Eventually, you decide that it is time to clean the leaves out of the pool. There is a specific corner of the pool that is easiest to access, and you want all of the leaves to be in that area so you can more easily collect and discard them. Given the model you have created – the rubber duck is in the same area as the leaves – you decide that it would be very clever to move the toy to the corner and watch in delight as the leaves follow the duck. Then you will easily scoop them up and continue with the rest of the day, enjoying your newly cleaned pool.
You make the change and feel like a fool as you stand in the corner of the pool, right over the rubber duck, net in hand, while the leaves stubbornly stay in place. You have made the terrible mistake of using prescriptive analytics when your model doesn’t pass the causal assumption!
Perplexed, you look into the pool again. You notice a slight disturbance in the water coming from the pool jets. You then decide to rethink your predictive modeling approach using the angle of the jets to predict the location of the leaves instead of the rubber duck. With this new model, you estimate how you need to configure the jets to get the leaves to your favorite corner. You move the jets and this time you are successful! The leaves drift to the corner, you remove them and go on with your day a smarter data scientist!
This is a quirky example, but it does illustrate a few points well. Let me call them out.
- The rubber duck is a classic ‘confounding’ variable. It is also affected by the pool jets and has no impact on the location of the leaves.
- Both the rubber duck and the pool jet models made accurate predictions – if we simply wanted to know where the leaves were, they could be equivalently good.
- What breaks the rubber duck model has nothing to do with the model itself and everything to do with how you used the model. The causal assumption wasn’t warranted but you moved forward anyway!
I hope you enjoyed the whimsical example – let’s transition to talking about real-world examples.
Shark Tank Pitch
In case you haven’t seen it, Shark Tank is a show where entrepreneurs pitch their business idea to wealthy investors (called ‘sharks’) with the hopes of securing investment money.
I was recently watching a Shark Tank re-run (as one does) – one of the pitches in the episode (Season 10, Episode 15) was for a company called GoalSetter. GoalSetter is a company that allows parents to open ‘mini’ bank accounts in their child’s name that family and friends can make deposits into. The idea is that instead of giving toys or gift cards to children as presents, people can give deposit certificates and children can save up for things (‘goals’) they want to purchase.
I have no qualms with the business idea, but in the presentation, the entrepreneur made this claim:
…kids who have savings accounts in their name are six times more likely to go to college and four times more likely to own stocks by the time they are young adults…
Assuming this statistic is true, this statement, by itself, is all fine and well. We can look at the data and see that there is a relationship between a child having a bank account in their name and going to college and/or investing (descriptive). We could even develop a model that predicts if a child will go to college or own stocks using bank account in their name as a predictor (predictive). But this doesn’t tell us anything about causation! The investment pitch has this subtle prescriptive message – “give your kid a GoalSetting account and they will be more likely to go to college and own stocks.” While semantically similar to the quote above, these two statements are worlds apart! One is a statement of statistical fact that relies on no assumptions, and the other is a prescriptive statement that has a huge causal assumption! I hope that confounding variable alarms are ringing in your head right now. It seems much more likely that things like household income, financial literacy of parents and cultural influences would have a relationship with both the probability of opening a bank account in a child’s name and that child going to college. It doesn’t seem likely that giving a random kid a bank account in their name will increase their chances of going to college. This is like moving the duck in the pool and expecting the leaves to follow!
Reading Is Fundamental Program
In the 1960s, there was a government-funded program called ‘Reading is Fundamental (RIF).’ Part of this program focused on putting books in the homes of low-income children. The goal was to increase literacy in those households. The strategy was partially based on the idea that homes with more books in them had more literate children. You might know where I’m going with this one based on the Shark Tank example we just discussed. Observing that homes with lots of books have more literate children is descriptive. There is nothing wrong with that. But, when you start making recommendations, you step out of descriptive space and leap into the prescriptive world – and as we’ve established, that comes with the causal assumption. Putting books in homes assumes that the books cause the literacy! Research by Susan Neuman found that putting books in homes was not sufficient in increasing literacy without additional resources1.
Of course, giving books to children who can’t afford them is a good thing – you don’t need a causal assumption to do good things 😊. But, if you have the specific goal of increasing literacy, you would be well-advised to assess the validity of the causal assumption behind your actions to realize your desired results!
How do we know if we satisfy the causality assumption?
We’ve established that prescriptive modeling requires a causal assumption (so much that you are probably exhausted!). But how can we know if the assumption is met by our model? When thinking about causality and data, I find it helpful to separate my thoughts between experimental and observational data. Let’s go through how we can feel good (or maybe at least ‘ok’) about causal assumptions with these two types of data.
Experimental Data
If you have access to good experimental data for your prescriptive modeling, you are very lucky! Experimental data is the gold standard for establishing causal relationships. The details of why this is the case are out of scope of this article, but I will say that the randomized assignment of treatments in a well-designed experiment deals with confounders, so you don’t have to worry about them ruining your casual assumptions.
We can train predictive models on the output of a good experiment – i.e., good experimental data. In this case, the data-generating process meets causal identification conditions between the target variables and variables that were randomly assigned treatments. I want to emphasize that only variables that are randomly assigned in the experiment will qualify for the causal claim on the basis of the experiment alone. The causal effect of other variables (called covariates) may or may not be correctly captured. For example, imagine that we ran an experiment that randomly provided multiple plants with various levels of nitrogen, phosphorus and potassium and we measured the plant growth. From this experimental data, we created the model below:
Because nitrogen, phosphorus and potassium were treatments that were randomly assigned in the experiment, we can conclude that betas 1 through 3 estimate a causal relationship on plant growth. Sun exposure was not randomly assigned which prevents us from claiming a causal relationship through the power of experimental data. This is not to say that a causal claim may not be justified for covariates, but the claim will require additional assumptions that we will cover in the observational data section coming up.
I’ve used the qualifier good when talking about experimental data multiple times now. What is a good experiment? I’ll go over two common issues I’ve seen that prevent an experiment from creating good data, but there is a lot more that can go wrong. You should read up on experimental design if you would like to go deeper.
Execution mistakes: This is one of the most common issues with experiments. I was once assigned to a project a few years ago where an experiment was run, but some data were mixed up regarding which subjects got which treatments – the data was not usable! If there were significant execution mistakes you may not be able to draw valid causal conclusions from the experimental data.
Underpowered experiments: This can happen for multiple reasons – for example, there may not be enough signal coming from the treatment, or there may have been too few experimental units. Even with perfect execution, an underpowered study may fail to uncover real effects which could prevent you from meeting the causal conclusion required for prescriptive modeling.
Observational Data
Satisfying the causal assumption with observational data is much more difficult, risky and controversial than with experimental data. The randomization that is a key part in creating experimental data is powerful because it removes the problems caused by all confounding variables – known and unknown, observed and unobserved. With observational data, we don’t have access to this extremely useful power.
Theoretically, if we can correctly control for all confounding variables, we can still make causal claims with observational data. While some may disagree with this statement, it is widely accepted in principle. The real challenge lies in the application.
To correctly control for a confounding variable, we need to (1) have high-quality data for the variable and (2) correctly model the relationship between the confounder and our target variable. Doing this for each known confounder is difficult, but it isn’t the worst part. The worst part is that you can never know with certainty that you have accounted for all confounders. Even with strong domain knowledge, the possibility that there is an unknown confounder “out there” remains. The best we can do is include every confounder we can think of and then rely on what is called the ‘no unmeasured confounder’ assumption to estimate causal relationships.
Modeling with observational data can still add a lot of value in prescriptive analytics, even though we can never know with certainty that we accounted for all confounding variables. With observational data, I think of the causal assumption as being met in degrees instead of in a binary fashion. As we account for more confounders, we capture the causal effect better and better. Even if we miss a few confounders, the model may still add value. As long as the confounders don’t have too large of an impact on the estimated causal relationships, we may be able to add more value making decisions with a slightly biased causal model than using the process we had before we used prescriptive modeling (e.g., rules or intuition-based decisions).
Having a pragmatic mindset with observational data can be important since (1) observational data is cheaper and much more common than experimental data and (2) if we rely on airtight causal conclusions (which we can’t get with observational data), we may be leaving value on the table by ruling out causal models that are ‘good enough’, though not perfect. You and your business partners have to decide the level of leniency to have with meeting the causal assumption, a model built on observational data could still add major value!
Wrapping it up
While prescriptive analytics is powerful and has the potential to add a lot of value, it relies on causal assumptions while descriptive and predictive analytics do not. It is important to understand and to meet the causal assumption as well as possible.
Experimental data is the gold standard of estimating causal relationships. A model built on good experimental data is in a strong position to meet the causal assumptions required by prescriptive modeling.
Establishing causal relationships with observational data can be more difficult because of the potential of unknown or unobserved confounding variables. We should balance rigor and pragmatism when using observational data for prescriptive modeling – rigor to think of and attempt to control for every confounder possible and pragmatism to understand that while the causal effects may not be perfectly captured, the model may add more value than the current decision-making process.
I hope that this article has helped you gain a better understanding of why prescriptive modeling relies on causal assumptions and how you can address meeting those assumptions. Happy modeling!
- Neuman, S. B. (2017). Principled Adversaries: Literacy Research for Political Action. Teachers College Record, 119(6), 1–32.
