are racing to use LLMs, but often for tasks they aren’t well-suited to. In fact, according to recent research by MIT, 95% of GenAI pilots fail — they’re getting zero return.
An area that has been overlooked in the GenAI storm is that of structured data, not only from an adoption standpoint, but also from a technological front. In reality, there is a goldmine of potential value that can be extracted from structured data, particularly in the form of predictions.
In this piece, I will go over what LLMs can and can’t do, what value you can get from AI running over your structured data, specifically for predictive modeling, and industry approaches used today — including one that I developed with my team.
Why LLMs aren’t optimized for business data and workflows
While large language models have completely transformed text and communication, they fall short in making predictions from the structured, relational data that moves the needle, driving real business outcomes — customer lifecycle management, sales optimization, ads and marketing, recommendations, fraud detection, and supply chain optimization.
Business data, the data enterprises are grounded in, is inherently structured. It often resides in tables, databases, and workflows, where meaning is derived from relationships across entities such as customers, transactions, and supply chains. In other words, this is all relational data.
LLMs took the world by storm and played a key role in advancing AI. That said, they were designed to work with unstructured data and aren’t naturally suited to reason over rows, columns, or joins. As a result, they struggle to capture the depth and complexity within relational data. Another challenge is that relational data changes in real time, while LLMs are typically trained on static snapshots of text. They also treat numbers and quantities as tokens in a sequence, rather than “understanding” them mathematically. In practice, this means an LLM is optimized to predict the next most likely token, which it does incredibly well, but not to verify whether a calculation is correct. So, whether the model outputs 3 or 200 when the true answer is 2, the penalty the model receives is the same.
LLMs are capable of multi-step reasoning through chain-of-thought-based inferencing, but they can face reliability challenges in certain cases. Because they can hallucinate, and do so confidently, might I add, even a small probability of error in a multi-step workflow can compound across steps. This lowers the overall likelihood of a correct outcome, and in business processes such as approving a loan or predicting supply shortages, just one small mistake can be catastrophic.
Because of all this, enterprises today rely on traditional machine learning pipelines that take months to build and maintain, limiting the measurable impact of AI on revenue. When you want to apply AI to this kind of tabular data, you are essentially teleported back thirty years and need humans to painstakingly engineer features and build bespoke models from scratch. For each single task separately! This approach is slow, expensive, does not scale, and maintaining such models is a nightmare.
How we built our Relational Foundation Model
My career has revolved around AI and machine learning over graph-structured data. Early on, I recognized that data points don’t exist in isolation. Rather, they are part of a graph connected to other pieces of knowledge. I applied this view to my work on online social networks and information virality, working with data from Facebook, Twitter, LinkedIn, Reddit, and others.
This insight led me to help pioneer Graph Neural Networks at Stanford, a framework that enables machines to learn from the relationships between entities rather than just the entities themselves. I applied this while serving as Chief Scientist at Pinterest, where an algorithm known as PinSage transformed how users experience Pinterest. That work later evolved into Graph Transformers, which bring Transformer architecture capabilities to graph-structured data. This allows models to capture both local connections and long-range dependencies within complex networks.
As my research advanced, I saw computer vision transformed by convolutional networks and language reshaped by LLMs. But, I realized the predictions businesses depend on from structured relational data were still waiting for their breakthrough, limited by machine learning techniques that hadn’t changed in over twenty years! Decades!
The culmination of this research and foresight led my team and me to create the first Relational Foundation Model (RFM) for business data. Its purpose is to enable machines to reason directly over structured data, to understand how entities, such as customers, transactions, and products, connect. By knowing the relationships between these entities, we then enable users to make accurate predictions from those specific relationships and patterns.
Unlike LLMs, RFMs have been designed for structured relational data. RFMs are pretrained on a number of (synthetic) datasets as well as on a number of tasks over structured business data. Like LLMs, RFMs can be simply prompted to produce instant responses to a wide variety of predictive tasks over a given database, all without task-specific or database-specific training.
We wanted a system that could learn directly from how real databases are structured, and without all the usual manual setup. To make that possible, we treated each database like a graph: tables became node types, rows turned into nodes, and foreign keys linked everything together. This way, the model could actually “see” how things like customers, transactions, and products connect and change over time.
At the heart of it, the model combines a column encoder with a relational graph transformer. Every cell in a table is turned into a small numerical embedding based on what kind of data it holds, whether it’s a number, category, or a timestamp. The Transformer then looks across the graph to pull context from related tables, which helps the model adapt to new database schemas and data types.
For users to input which predictions they’d like to make, we built a simple interface called Predictive Query Language (PQL). It lets users describe what they want to predict, and the model takes care of the rest. The model pulls the right data, learns from past examples, and reasons through an answer. Because it uses in-context learning, it doesn’t have to be retrained for every task, either! We do have an option for fine-tuning, but this is for very specialized tasks.
But this is just one approach. Across the industry, several other strategies are being explored:
Industry approaches
1. Internal foundation models
Companies like Netflix are building their own large-scale foundation models for recommendations. As described in their blog, the goal is to move away from dozens of specialized models toward a single centralized model that learns member preferences across the platform. Analogy to LLMs is clear: like a sentence is represented as a sequence of words, a user is represented as a sequence of movies the user interacted with. This allows innovations to support long-term personalization by processing massive interaction histories.
The benefits of owning such a model include control, differentiation, and the ability to tailor architectures to domain-specific needs (e.g., sparse attention for latency, metadata-driven embeddings for cold start). On the flip side, these models are extremely costly to train and maintain, requiring vast amounts of data, compute, and engineering resources. Furthermore, they are trained on a single dataset (e.g., Netflix user behavior) for a single task (e.g., recommendations).
2. Automating model development with AutoML or Data Science agents
Platforms like DataRobot and SageMaker Autopilot have pushed forward the idea of automating parts of the machine learning pipeline. They help teams move faster by handling pieces like feature engineering, model selection, and training. This makes it easier to experiment, reduce repetitive work, and expand access to machine learning beyond just highly specialized teams. In a similar vein, Data Scientist agents are emerging, where the idea is that the Data Scientist agent will perform all the classical steps and iterate over them: data cleaning, feature engineering, model building, model evaluation, and finally model development. While a true innovative feat, the jury is still out on whether this approach will be effective in the long term.
3. Using graph databases for connected data
Companies like Neo4j and TigerGraph have advanced the use of graph databases to better capture how data points are connected. This has been especially impactful in areas like fraud detection, cybersecurity, and supply chain management, places where the relationships between entities often matter more than the entities themselves. By modeling data as networks rather than isolated rows in a table, graph systems have opened up new ways of reasoning about complex, real-world problems.
Lessons learned
When we set out to build our technology, our goal was simple: develop neural network architectures that could learn directly from raw data. This approach mirrors the current AI (literal) revolution, which is fueled by neural networks that learn directly from pixels in an image or words in a document.
Practically speaking, our vision for the product also entailed a person simply connecting to the data and making a prediction. That led us to the ambitious target of creating a pretrained foundation model designed for business data from the ground up (as explained above), removing the need to manually create features, training datasets, and custom task-specific models. An ambitious task indeed.
When building our Relational Foundation Model, we developed new transformer architectures that attend over a set of interconnected tables, a database schema. This required extending the classical LLM attention mechanism, which attends over a linear sequence of tokens, to an attention mechanism that attends over a graph of data. Critically, the attention mechanism had to generalize across different database structures as well as across different types of tables, wide or narrow, with varied column types and meanings.
Another challenge was inventing a new training scheme, because predicting the next token is not the right objective. Instead, we generated many synthetic databases and predictive tasks mimicking challenges like fraud detection, time series forecasting, supply chain optimization, risk profiling, credit scoring, personalized recommendations, customer churn prediction, and sales lead scoring.
In the end, this resulted in a pretrained Relational Foundation Model that can be prompted to solve business tasks, whether it is financial versus insurance fraud or medical versus credit risk scoring.
Conclusion
Machine learning is here to stay, and as the field evolves, it’s our responsibility as data scientists to spark more thoughtful and candid discourse about the true capabilities of our technology — what it’s good at, and where it falls short.
We all know how transformative LLMs were, and continue to be, but too often, they are implemented hastily before considering internal goals or needs. As technologists, we should encourage executives to take a closer look at their proprietary data, which anchors their company’s uniqueness, and take the time to thoughtfully identify which technologies will best capitalize on that data to advance their business objectives.
In this piece, we went over LLM capabilities, the value that lies within the (often) overlooked side of structured data, and industry solutions for applying AI over structured data — including my own solution and the lessons learned from building that.
Thank you for reading.
References:
[1] R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton and J. Leskovec, Graph Convolutional Neural Networks for Web-Scale Recommender Systems (2018), KDD 2018.
Author bio:
Dr. Jure Leskovec is the Chief Scientist and Co-Founder of Kumo, a leading predictive AI company. He is a Computer Science professor at Stanford, where he has been teaching for more than 15 years. Jure co-created Graph Neural Networks and has dedicated his career to advancing how AI learns from connected information. He previously served as Chief Scientist at Pinterest and conducted award-winning research at Yahoo and Microsoft.
Source link
#LLMs #Arent #OneSizeFitsAll #Solution #Enterprises
