Building A Successful Relationship With Stakeholders

As a data scientist your job is to leverage data to solve business problems and bring value, normally by building models. This typically involves running a series of experiments where lots of ideas are iterated through until the best solution is selected as part of the business proposal. Evaluating the best model is typically done by minimizing or maximising some performance metric, such as the mean squared error for regression models or the F1 score for binary classification models.

However, the creation of a model is just one part in the overall process. Surrounding your model are two significant questions, namely does your solution answer the original problem, and how much benefit does it bring to the business. These questions can only be answered by the stakeholders of your project as they set the requirements and the success criteria. In an ideal world these would be clearly defined but this can often not be the case. It may be that the requirements are quite vague in nature and broad, sometimes being as simple as trying to prevent customer churn or protecting customers against fraud. In this case it will be up to the data scientists and stakeholders to work together to better refine these questions and define what success means. To do so they must be on the same page so to speak, as a failure to do so can lead to miscommunication and friction that will inevitably end up with a project not succeeding.

Throughout my career I have seen stakeholders and data scientists speak different languages to each other, one looking outward to the business and the other facing inward to the data. The consequence of this is that good projects fail to hit the mark and not gather the enthusiasm they deserve, leading to them not reaching deployment. I believe that to succeed as a great data scientist you must be able to bridge this gap between the business and the technical. Illustrating your solutions impact through business outcomes and showing what can be gained from it is the key to getting stakeholder buy in to your solution. In this article I want to outline some philosophies that have helped me to improve my communication when engaging with the wider business.

Translating Requirements And Reporting Performance

The start of a new project is a hectic time with lots of kick-off meetings, bringing together of team members and getting access requirements set up just to name a few. However, one aspect that you may not have been a part of as a data scientist is the one that decided the need for the project in the first place. This is done by members like stakeholders and product owners, typically the management layer of an organisation. This means that a project’s high level goals are decided before a data scientist ever joins.

Due to requirements already being decided, there will be a tendency that the data scientist may go straight into the experimentation process without giving due attention to the goals of the project. They know its overall aim and think that is enough for them to progress. However, it is critical that time is taken at this point to refine the business question into a set of very clear requirements. This ensures that:

• There is no ambiguity between data scientists and the wider business
• There is a clear understanding of what is to be solved
• There are clear metrics that define whether the objective has been achieved

As an example, let’s return to the earlier ask of a stakeholder wanting to protect customers against fraud. There are many possible avenues such an ask can take, and refining this requirement is key in ensuring that your project hits the mark. It is therefore critical that meetings are put in place to allow follow up questions to be asked. Some examples are:

• Do we want to prevent fraud as it is occurring or inform customers if they are at risk?
• Do we want a yes / no answer or something more nuanced?
• Do we want something more autonomous in decision making or something that augments current processes?
• How often will the solution be executed? It it offline batch or online real time?
• Are there any operational constraints we need to be aware of?

So for instance requiring a real time fraud defence solution is very different from predicting that a customer may become at risk of fraud in the next 30 days. Asking these questions will help steer you towards solutions that you will want to investigate further.

The end of project experimentation can be just as hectic as the beginning. At this point you need to choose your best solution and present it to the business. This is crucial as there is no guarantee that your solution will be accepted and will progress onward to become a new product. Putting in any new process such as a model into a live state comes with costs that must be weighed against the benefit. There are considerations about who is responsible for its deployment and monitoring, as well as maintenance if its performance no longer meets requirements. You need to consider how often adverse outcomes can occur, their potential severity, and any repercussions from them. You may need to consider any additional operational impact your new process introduces. Consider a fraud detection platform, you need to think about:

• How often will your detector miss fraudulent transactions?
• How often will your detector wrongly classify genuine transactions as fraud and impact the customer?
• What is the total volume of transactions that will be flagged as fraud and is there operational capacity to investigate all these events?

To overcome any apprehensions or misgivings you need to be able to sell your solution, just building it is not enough. When showcasing your solution you should:

Start With A Problem, Not A Technology

It is tempting to focus on the technical acumen of your solution, such as the model used or the data processing pipeline. This is where you have the spent the past months of your life, and you want to show that you have worked very hard to solve this problem. Therefore when you present to stakeholders you will be tempted to talk about things like how you used one hot encoding, performed mean imputation and used the Optuna library for hyperparameter tuning a LightGBM model.

The problem with this is that the stakeholders priority is not how the model works, but what it can do. They care about how the business question is being answered and what benefit can be derived. In this case we need to reframe how we present our results to be business oriented and focus on what our solution solved rather than how it is solved. We should therefore say less sentences like:

We developed a LightGBM binary classification for fraud detection

And more sentences like

Our proposed solution improves the ability of our current systems to detect fraud

Business vs Model Performance

Related to the above point, it is all too common to focus on reporting the model performance. Metrics such as F1, AUC etc. give an objective way to decide what is the best model and you want to pass that information on to the stakeholders. To a data scientist it is clear what the difference between a recall of 0.8 and 0.9 means.

However to a stakeholder, the model performance doesn’t tell them what value the solution brings to the business. They need to know the impact that it will have on current processes and procedures. Data scientists should therefore frame the performance of the model in terms of business level KPI’s. A good idea is to always consider:

Does it generate money, save money or save time? If so, how much?

Clearly quantifying what you solutions brings will help to drive engagement and greatly increase the chance of it being adopted. We should therefore says less of:

Our LightGBM model achieved a recall of 0.9

and more of:

Our solution can detect £10m worth of fraud annually

Never Neglect Explainability

Being able to understand and justify why your solution made its decisions is key in building trust with stakeholders. If you are implementing a solution around accepting mortgage applications for example, being able to justify why applications are declined is vital if customers challenge this decision. It also ensures the model has not picked up any biases or prejudices that could put you at risk of legal or regulatory issues.

Explainability can also provide sense checks or even challenge preconceived notions about what information is useful. All of this means that embedding explainability throughout the process can give assurances to stakeholders that care and consideration has been taken. Key points to adhere to are:

• Be able to say which features the model relies on
• Be able to explain a decision in terms of its features

This means either sticking to a model that has good explainability (regression, decision trees etc) or rely on 3^rd party explainability libraries (SHAP, LIME, etc).

Presenting Results to Maximize Engagement

After experimentation has finished and you have chosen your solution, the next step is to share your results with stakeholders for them to give the go-ahead. This is normally done in the form of a presentation deck, where you will need to motivate the problem and show why your solution is the right choice. This is a critical point where you must be able to communicate clearly with your stakeholders. I have seen good proposals fall flat due to presentations that either did not engage the audience or even worse put them off. Designing an engaging presentation is a mixture of art and skill, and is something that you need to actively work on.

Some general tips that should serve as guidelines are:

Know Your Audience And Objective

When first starting to write a presentation you need to ask yourself:

What am I trying to sell and who am I selling it to?

While having a presentation just to capture your work has merit, if you are trying to secure buy in for your project then you should be laser focussed on the point you are trying to convey. Trying to cover too much within a single presentation will lead to confusion and may lead to your overall message being diluted. You should ask yourself “what is the one thing I want my audience to know about” and then structure your presentation around that.

Knowing the technical and project knowledge level of your audience can impact how you decide to convey your message. If your stakeholder is more intimately familiar with the subject matter then there is background knowledge that can be assumed. But if they are not, then you will need to really think on what can and can’t be assumed to ensure everyone involved can follow your message. If your stakeholder has a more technical skillset then there is some scope to give a bit more details on the methods you have used but I would keep this to a minimum. As previously discussed we want to emphasise the business benefit of a project.

Style Matters

Being able to follow a presentation relies a lot on things. Your audience has to both listen to you and look at what is on the screen at the same time, so the styling of your presentation will have a huge impact on their ability to do so. When designing a presentation these tips have helped me to maximise its impact:

• Use a theme: Either provided by your business or from a stock website, having a pre-set colour scheme, font sizing’s etc make a big difference
• Use partitions to draw the eye: Encasing important points in coloured boxes help to guide the audience through your slide
• Don’t go overboard on text and visuals: Don’t write paragraphs the audience can’t read and keep visuals such as graphs big and simplified

All Killer No Filler

Your time is limited when engaging with stakeholders. You need to make an impact and hold their attention while you sell them on your solution. You therefore need to find a balance between background, theory, solution and impact. So you need to make sure that each slide brings something useful to the table. Some ways of doing this are:

• Start with the results: This isn’t a mystery novel leading up to a big reveal, put your best foot forward and say exactly what you are selling
• Use headings to make an impact: Heading are a summary of what the slide contains and should give the most important information
• Lead by example: If you are trying to explain how things work, use data to make your point. Don’t live in the abstract

Conclusion

In this article I have discussed the importance of engaging with stakeholders to help showcase the value of proposed data science solutions. Refining requirements and being business impact driven in your work can ensure that your results are easily interpretable and can be acted upon. All of this is embodied in creating an engaging and knowledgeable presentation deck as a means of showing stakeholders you can translate requirements into actionable outcomes.