Why, if we look at the bigger picture, black-box models are not more accurate
When I started as a data scientist, I was expecting to use state-of-the-art models. XGBoost, Neural Networks. These things are complex and interesting and surely they would drive improvements. Little did I know, the models faced a hurdle — explaining them to other people.
Who’d have thought you need to understand the decisions your automated systems make?
To my joy, I stumbled down the rabbit hole of model agnostic methods. With these, I could have the best of both worlds. I could train black box models and then explain them using methods like SHAP, LIME, PDPs, ALEs and Friedman’s H-stat. We no longer need to trade accuracy for interpretability!
Not so fast. That thinking is flawed.
In our pursuit of best performance, we often miss the point of machine learning: that is, to make accurate predictions on new unseen data. Let’s discuss why complex models are not always the best way of achieving this. Even if we can explain them using other methods.
Source link
#Accuracy #Interpretability #Tradeoff #Lie #Conor #OSullivan #Oct
Unlock the potential of cutting-edge AI solutions with our comprehensive offerings. As a leading provider in the AI landscape, we harness the power of artificial intelligence to revolutionize industries. From machine learning and data analytics to natural language processing and computer vision, our AI solutions are designed to enhance efficiency and drive innovation. Explore the limitless possibilities of AI-driven insights and automation that propel your business forward. With a commitment to staying at the forefront of the rapidly evolving AI market, we deliver tailored solutions that meet your specific needs. Join us on the forefront of technological advancement, and let AI redefine the way you operate and succeed in a competitive landscape. Embrace the future with AI excellence, where possibilities are limitless, and competition is surpassed.
_Brain_light_Alamy.jpg?disable=upscale&width=1200&height=630&fit=crop&w=150&resize=150,150&ssl=1 "Breaches Don’t Have to Be Disasters")
/cdn.vox-cdn.com/uploads/chorus_asset/file/25746645/22832_2026_Sportage_Family_Lineup.jpg?w=150&resize=150,150&ssl=1 "Kia announces high-performance EV9 GT with virtual shifting and native Tesla charging")
