Building a Unified Intent Recognition Engine

systems, understanding user intent is fundamental especially in the customer service domain where I operate. Yet across enterprise teams, intent recognition often happens in silos, each team building bespoke pipelines for different products, from troubleshooting assistants to chatbots and issue triage tools. This redundancy slows innovation and makes scaling a challenge.

Spotting a Pattern in a Tangle of Systems

Across AI workflows, we observed a pattern — a lot of projects, although serving different purposes, involved understanding of the user input and classifying them in labels. Each project was tackling it independently with some variations. One system might pair FAISS with MiniLM embeddings and LLM summarization for trending topics, while another blended keyword search with semantic models. Though effective individually, these pipelines shared underlying components and challenges, which was a prime opportunity for consolidation.

We mapped them out and realized they all boiled down to the same essential pattern — clean the input, turn it into embeddings, search for similar examples, score the similarity, and assign a label. Once you see that, it feels obvious: why rebuild the same plumbing over and over? Wouldn’t it be better to create a modular system that different teams could configure for their own needs without starting from scratch? That question set us on the path to what we now call the Unified Intent Recognition Engine (UIRE).

Recognizing that, we saw an opportunity. Rather than letting every team build a one-off solution, we could standardize the core components, things like preprocessing, embedding, and similarity scoring, while leaving enough flexibility for each product team to plug in their own label sets, business logic, and risk thresholds. That idea became the foundation for the UIRE framework.

A Modular Framework Designed for Reuse

At its core, UIRE is a configurable pipeline made up of reusable parts and project-specific plug-ins. The reusable components stay consistent — text preprocessing, embedding models, vector search, and scoring logic. Then, each team can add their own label sets, routing rules, and risk parameters on top of that.

Here is what the flow typically looks like:

Input → Preprocessing → Summarization → Embedding → Vector Search → Similarity Scoring → Label Matching → Routing

We organized components this way:

• Repeatable Components: Preprocessing steps, summarization (if required), embedding and vector search tools (like MiniLM, SBERT, FAISS, Pinecone), similarity scoring logic, threshold tuning frameworks,.
• Project-Specific Elements: Custom intent labels, training data, business-specific routing rules, confidence thresholds adjusted to risk, and optional LLM summarization choices.

Here is a visual to represent this:

The value of this setup became clear almost immediately. In one case, we repurposed an existing pipeline for a new classification problem and got it up and running in two days. That typically used to take us almost two weeks when building from scratch. Having that head start meant we could spend more time improving accuracy, identifying edge cases and experimenting with configurations instead of wiring up infrastructure.

Even better, this kind of design is naturally future proof. If a new project requires multilingual support, we can drop in a model like Jina-Embeddings-v3. If another product team wants to classify images or audio, the same vector search flow works there too by swapping out the embedding model. The backbone stays the same.

Turning a Framework into a Living Repository for Continuous Growth

Another advantage of a unified engine is the potential to build a shared, living repository. As different teams adopt the framework, their customizations including new embedding models, threshold configurations, or preprocessing techniques, can be contributed back to a common library. Over time, this collective intelligence would produce a comprehensive, enterprise-grade toolkit of best practices, accelerating adoption and innovation.

This eliminates a common struggle of “siloed systems” that prevails in many enterprises. Good ideas stay trapped in individual projects. But with shared infrastructure, it becomes far easier to experiment, learn from each other, and steadily improve the overall system.

Why This Approach Matters

For large organizations with multiple ongoing AI initiatives, this kind of modular system offers a lot of advantages:

• Avoid duplicated engineering work and reduce maintenance overhead
• Speed up prototyping and scaling since teams can mix and match pre-built components
• Let teams focus on what actually matters — improving accuracy, refining edge cases, and fine-tuning experiences, not rebuilding infrastructure
• Make it simpler to extend into new languages, business domains, or even data types like images and audio

This modular architecture aligns well with where AI system design is heading. Research from Sung et al. (2023), Puig (2024), and Tang et al. (2023) highlights the value of embedding-based, reusable pipelines for intent classification. Their work shows that systems built on vector-based workflows are more scalable, adaptable, and easier to maintain than traditional one-off classifiers.

Advanced Features for handling the real-world scenarios

Of course, real-world conversations rarely follow clean, single-intent patterns. People ask messy, layered, sometimes ambiguous questions. That is where this modular approach really shines, because it makes it easier to layer in advanced handling strategies. You can build these features once, and they can be reused in other projects. 

• Multi-intent detection when a query asks several things at once
• Out-of-scope detection to flag unfamiliar inputs and route them to a human or fallback answer
• Lightweight explainability by retrieving examples of the nearest neighbors in the vector space to explain how a decision was made

Features like these help AI systems stay reliable and reduce friction for end-users, even as products expand into increasingly unpredictable, high-variance environments.

Closing Thoughts

The Unified Intent Recognition Engine is less a packaged product and more a practical strategy for scaling AI intelligently. When developing the concept, we recognized that the projects are unique, are deployed in different environments, and need different levels of customization. By offering pre-built components with tons of flexibility, teams can move faster, avoid redundant work, and deliver smarter, more reliable systems.

In our experience, applications of this setup delivered meaningful results — faster deployment times, less time wasted on redundant infrastructure, and more opportunity to focus on accuracy and edge cases with a lot of potential for future advancements. As AI-powered products continue to multiply across industries, frameworks like this could become essential tools for building scalable, reliable, and flexible systems.

About the Authors

Shruti Tiwari is an AI product manager at Dell Technologies, where she leads AI initiatives to enhance enterprise customer support using generative AI, agentic frameworks, and traditional AI. Her work has been featured in VentureBeat, CMSWire, and Product Led Alliance, and she mentors professionals on building scalable and responsible AI products.

Vadiraj Kulkarni is a data scientist at Dell Technologies, focused on building and deploying multimodal AI solutions for enterprise customer service. His work spans generative AI, agentic AI and traditional AI to improve support outcomes. His work was published on VentureBeat on applying agentic frameworks in multimodal applications.

References :

1. Sung, M., Gung, J., Mansimov, E., Pappas, N., Shu, R., Romeo, S., Zhang, Y., & Castelli, V. (2023). Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification. arXiv preprint arXiv:2305.14827. https://arxiv.org/abs/2305.14827
   
2. Puig, M. (2024). Mastering Intent Classification with Embeddings: Centroids, Neural Networks, and Random Forests. Medium. https://medium.com/@marc.puig/mastering-intent-classification-with-embeddings-34a4f92b63fb
3. Tang, Y.-C., Wang, W.-Y., Yen, A.-Z., & Peng, W.-C. (2023). RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training. arXiv preprint arXiv:2310.09773. https://arxiv.org/abs/2310.09773
4. Jina AI GmbH. (2024). Jina-Embeddings-v3 Released: A Multilingual Multi-Task Text Embedding Model. arXiv preprint arXiv:2409.10173. https://arxiv.org/abs/2409.10173