Analysis of Sales Shift in Retail with Causal Impact: A Case Study at Carrefour

Disclosure: I work at Carrefour. The views expressed in this article are my own. The data and examples presented are published with my employer’s permission and do not contain any confidential information.

A store’s assortment is a complete and varied range of products sold to customers. It is subject to evolve based on various factors such as: economic conditions, consumer trends, profitability, quality or compliance issues, renewal of some product ranges, stock levels, seasonal changes, etc.

When a product is no longer available on the store shelves, some of its sales may shift to other products. For a major food retailer like Carrefour, it is crucial to estimate this sales shift accurately to manage the risk of loss due to product unavailability and approximate the loss due to it.

This measurement serves as an indicator of the consequences of the unavailability of a product. Additionally, it gradually builds a valuable history of sales shift impact estimates.

Yet, estimating sales shifts is complex. Customer behavior — influenced by hard-to-predict emotional factors — seasonality of certain products, or introduction of new products can all affect sales shifts. In addition, many products become unavailable across all stores simultaneously, making it impossible to establish a control population.

The Causal Impact synthetic control approach, developed by a Google team, fits the particularities of our analysis framework. It enables us to isolate the effect of product unavailability on sales from influencing factors, and is suitable for both quasi-experimental and observational studies. Based on Bayesian structural time-series models, Causal Impact performs a counterfactual analysis, calculating the effect on sales as the difference between the sales observed after a product becomes unavailable and, through a synthetic control, the sales that would have been observed had the product remained available.

This article presents our Causal Impact approach for estimating the sales shift effect following product unavailability, as well as a heuristic for selecting control group time series.

Due to confidentiality concerns, the quantitative values on the graphs have been redacted. Note that each block represents one month along the x-axis, and the y-axis represents a variable quantity, which can be quite large.

I) Specifying the Use Case

Product unavailability occurs in two main forms:

• Complete unavailability: the product is no longer available in the national assortment, affecting all stores.
• Partial unavailability: the product is no longer available from some — but not all — stores. It remains available in others.

We consider that a reliable sales shift impact estimate should accurately assess both lost sales and portion of sales transferred to other products. Yet, knowing the exact value of these quantities is impossible, making this challenge complex.

Our study analyzes cases of complete product unavailability as these cases are the most significant in terms of sales impact.

Please also note that causal inference is not a predictive framework for future events: it identifies causal links in the past rather than forecasting future events.

II) Why did we choose Google’s Causal Impact model?

Causal approaches aim to understand causal relationships between variables, explaining how one affects another by isolating the effect we are trying to analyze from all other existing effects.

Among these tools, Causal Impact is a user-friendly library, and it operates within a fully Bayesian framework, allowing prior information integration while providing inherent credibility intervals in its results. Its predictions represent expected outcomes had the intervention not occurred, expressed as distribution functions rather than single values.

Causal Impact generates predictions by combining endogenous components, such as seasonality and local level, with user-chosen external time series (covariates). These covariates must be unaffected by the intervention and should capture trends or factors that could influence the main time series. We will discuss covariate selection later.

Fig. 1: A simplified example of Causal Impact in action. The top graph shows two time series: the orange line represents actual observed data, while the blue line is the model’s prediction, created using covariates and endogenous components. Each block represents a month. This prediction estimates what would have happened if the event of interest (marked by the vertical dashed line) had not occurred. The blue shaded area indicates the prediction’s uncertainty. The second graph displays the point-by-point difference between the prediction and the observed data, and the bottom graph shows the cumulative impact.

III) Managing Outliers and Anomalies in data

To ensure accurate analysis, we addressed sales data anomalies by following two key steps:

• We excluded time series with negative sales or a large number of zero sales from the analysis.
• For time series with occasional zero sales, we replaced these values with the average of the preceding and following weeks’ sales.

IV) Model Design

The choice of covariates significantly influences counterfactual prediction accuracy. These time series must capture trends or external factors likely to influence the target time series without being affected by the intervention.

In addition, it is crucial to consider the size of the estimated sales shift effect relative to the time series being studied: if the intervention is expected to affect the target series by only a few percent, the series may not be appropriate, as small effects are difficult to distinguish from random noise (especially as the library designers have shown that effects less than 1% are difficult to prove as being linked to the intervention). Therefore, we analyzed sales shift only when the theoretical maximum sales shift rate exceeds 5% of sales in its sub-family. We calculated this as S/(1-S), where S represents the percentage of turnover the product generated in its sub-family before becoming unavailable.

Given those preliminary considerations, we designed our Causal Impact model as follows:

Target

As the target time series, we selected the sum of sales for the product’s sub-family, excluding the product that became unavailable.

Covariates

We first excluded the following types of time series:

• Products from the same sub-family as the discontinued product, to prevent any influence from its unavailability.
• Products from different families than the discontinued product, since covariates should remain business-relevant.
• Time series that showed correlation but not co-integration with the target series, to avoid spurious relationships.

Using these filters, we selected 60 covariates:

• 20 covariates were chosen based on their highest co-integration with the target series during the year before intervention.
• 40 additional covariates were selected from the top 200 co-integrated series, based on their strongest correlation with the target series during the year before intervention.

Note that these numbers (20, 40, and 60) are rules of thumb derived from our previous model fits.

This empirical approach combines time series that capture both long-term trends (through co-integration) and short-term variations (through correlation). We deliberately chose a large number of covariates because Causal Impact employs a “spike and slab” method, which automatically reduces the influence of less significant series by assigning them near-zero regression coefficients, while giving greater weight to important ones.

V) Model Validation

To validate our covariate selection strategy, we drew heavily on the approach used by the Causal Impact designers. We conducted a study of partial product unavailability as follows:

1. We examined cases where products became partially unavailable and performed an initial conventional statistical analysis using difference-in-differences.
2. We applied Causal Impact using, as covariates, a control population that consisted of the product’s sub-family sales (excluding the unavailable product) in stores where the product remained available. These covariates provided the best available counterfactual since these stores were unaffected by the intervention.
3. Finally, we applied Causal Impact without a control population, instead using our selection process based on co-integration and correlation as outlined in the Model Design section.

Consistent estimates across multiple reports (spanning different products, quantities, and categories) would demonstrate that we can reliably apply this approach on a broader scale.

Additionally we developed two metrics to evaluate the synthetic control’s quality: a fitness measure and a predictive capability measure.

• The fitness measure, scored between 0 and 1, assesses how well the synthetic control models the target over the pre-intervention period.
• The predictive capability measure is a form of backtesting that evaluates the synthetic control’s quality during a simulated false intervention in the past.

A Practical Validation Example

To validate the process described above with a practical example, we analyzed a case where a yogurt pack became unavailable in certain stores. We established treatment and control groups by matching each store where the product became unavailable with a similar store that still had the product, based on criteria such as sales performance, customer characteristics, and geographic location.

The theoretical maximum sales shift rate for this product was 9.5%, and our previous analyses showed very high sales shift rates in the dairy product family. Consequently, we anticipated to obtain an estimate close to the theoretical maximum rate.

Following our three-step validation method, we obtained these results:

1. The difference-in-differences analysis estimated the causal effect at 8.7% with 98.7% probability.
2. As shown in Figure 2 (below), the Causal Impact analysis using a control population estimated a causal effect of 9.0%, with a confidence interval of [3.7%, 14.4%] and 99.9% probability. We can also see that while the model effectively tracks the time series fluctuations, it does show some minor deviations.

 Fig. 2: Causal effect estimation for the dairy product brand after product unavailability, using a control population to construct the synthetic control.

In addition, when using covariates selected based on co-integration and correlation instead of a control population, the Causal Impact analysis estimated a causal effect of 8.5%, with a confidence interval of [2.4%, 15.1%] and 99.9% probability as shown in Figure 3 (below). Again, the model effectively tracks the time series fluctuations, yet showing some minor deviations.

 Fig. 3: Causal effect estimation for the dairy product brand after product unavailability, using proxies (only data from stores in the treatment population to constitute the synthetic control).

Here is a summary of the estimates obtained across the three different analysis methods:

It shows that the estimates remain consistent in magnitude, whether using a control population or not, thus validating our selection process for covariates when no control population is available.

VI) Complete unavailability: A rice pack no longer available

We examined a nationwide case where a pack of rice brand became unavailable. We restrained our analysis to the following few months after the product became unavailable to avoid capturing unrelated effects that might emerge over a longer period. The theoretical maximum sales shift rate for the product was 31.2%. We applied the covariate selection methodology described earlier to estimate the potential sales shift effect.

Fig. 4: Causal effect estimation after the pack of rice brand became unavailable, using proxies (only data from stores in the treatment population to constitute the synthetic control).

As shown in Figure 4, the synthetic control models the target very well over the period before the intervention. The prediction accurately captures seasonal trends after the intervention. The credibility interval is very narrow around the estimate.

We obtained a statistically significant estimate at 22% increase in turnover caused by the product unavailability over the following months, with over 99.9% probability. This quantity represents approximately 70% of the pack of rice total sales before the product became unavailable, implying that 30% of the pack of rice sales did not shift.

VII) Usage recommendations and experience report

Causal Impact is a robust and user-friendly tool for causal inferences. Yet after significant time spent specifying the model and improving its accuracy, we encountered challenges in fine-tuning it to obtain an industrializable solution.

• The first point we want to highlight is the importance of the “garbage in, garbage out” principle, which is particularly relevant when using Causal Impact. Regardless of the covariates used, Causal Impact will always produce a result, sometimes with very high probability, even in cases where results are unrealistic, or impossible.
• Time series selected solely based on the co-integration criterion sometimes overshadow others in model feature importance, which can drastically reduce the estimation accuracy when adjustment is not well-controlled.
• The selection of 20 series for co-integration and 40 for correlation is an empirical rule of thumb. While effective in most cases we encountered, it could benefit from further refinement.

Conclusion

In this article we proposed a causal approach to estimate the sales shift effect when a product becomes unavailable, using Causal Impact. We outlined a methodology for selecting analyzable products, and covariates.

Although this approach is functional and robust in most cases, it has limitations and areas for improvement. Some are structural, while others require spending more time on model adjustment.

• We tested the methodology on different products with promising results, but it is not exhaustive. Some very seasonal products or ones with little historical data pose challenges. Additionally, products that became unavailable in only a few stores are rare, limiting our ability to validate the method on a large number of diverse cases.
• Another structural limitation is the model’s requirement for post-hoc analysis: the tool does not allow sales shift effect prediction before a product becomes unavailable. Being able to do so would greatly benefit business teams. Work is underway to approach sales shift prediction using bayesian structural time series forecasting.
• The sales shift effect analysis ignores margin impacts: the product that became unavailable may have a higher unit margin than the products to which its sales shifted. The commercial conclusions to be drawn could then differ, but analysis at a sub-family level precludes this level of detail.
• Finally we could explore alternative synthetic controls, such as Augmented SC, Robust SC, Penalized SC, or even other causal approaches such as the two-way fixed effect model.