5 Things To Look For When Evaluating AI Startups

By Bob Ma

According to a report by McKinsey, generative AI could have an economic impact of $2.6 trillion to $4.4 trillion annually. For context, the United Kingdom’s entire GDP in 2021 was $3.1 trillion. Roughly 75% of that value will emanate from productivity gains across customer operations, sales and marketing, software engineering, and R&D.

AI’s eye-popping potential has given rise to numerous enterprise generative AI startups focused on applying large language model technology to the enterprise context. Standard products include employee copilots, content generation for marketing, back-office automation and enterprise knowledge search.

These AI-driven solutions are beginning to realize the projected global economic impact by significantly reducing operating expenses, generating sales and boosting employee productivity.

With the enormous opportunity, enterprise generative AI startups have multiplied quickly over the past two years. A search for AI customer service chatbots alone returns hundreds of startups. Two factors help explain the significant number of applied AI startups. First, LLM technology is readily accessible via APIs from large AI research companies such as OpenAI. Second, while most generative AI startups are new, many startups established pre-2022 are also integrating generative AI into their product suites.

Given the noise in the market and the large field of enterprise generative AI startups, it can be challenging for enterprise customers and investors to evaluate and differentiate among the options.

Below are five things to look for.

1. LLM customization

Is the startup using a mostly off-the-shelf LLM — e.g., OpenAI’s ChatGPT — or a meaningfully customized LLM? Different ways to customize an LLM include fine-tuning an off-the-shelf model or building a custom one using an open-source LLM like Meta’s Llama. Greater customization generally increases AI accuracy and reflects the startup’s technical expertise in AI.

2. Vertical-specific training data

Does the startup have access to a large volume of proprietary, vertical-specific data to train its LLMs? For example, an AI copilot for customer service call centers will be enhanced if the AI model is trained on large amounts of existing customer interaction data. The more industry-specific the training data, the better.

3. Strength of speech-to-text/text-to-speech

Many enterprise generative AI products support voice audio for comprehension or generation. This technology is distinct from LLMs and is typically offered by large technology companies such as Google, Amazon and Microsoft.

Understanding and testing the STT/TTS offerings, such as support for industry jargon or dialects, is important if the AI solution has voice applications.

4. Breadth and depth of API integrations

A distinguishing feature of enterprise generative AI is that the AI system can interact directly with enterprise systems, e.g., Salesforce 1 , SAP, etc. This is primarily done through APIs, so broader and deeper API integrations will allow the AI system to provide more process automation. Ease of integration is also something to look for.

5. Business-user friendly

A generative AI solution will see better enterprise adoption if it offers extensive features that are accessible to nontechnical and nondata science users.

Look for solutions that offer a low-/no-code environment for development and operation, as well as robust analytical tools and A/B testing capabilities accessible to business users.

The UI/UX should be intuitive enough to require no training for business users, and even for implementation, developer training should take no more than a few days.

Successful AI will be versatile

The world continues to be captivated by the potential of generative AI to revolutionize how knowledge work is conducted throughout the economy. AI is poised to transform tasks and boost performance across key enterprise functions such as customer operations and sales and marketing.

Looking ahead to the next few years, enterprise generative AI technology will be increasingly multimodal, with new solutions that can simultaneously handle multiple types of inputs and output such as images, audio, video and text. This will unlock more sophisticated and versatile enterprise AI applications, particularly outside of traditional use cases, and will help drive the next leg of growth in this exciting space.

---

 Bob Ma is an investor at Copec Wind Ventures, where he invests in retail, fintech, consumer and energy technologies. He joined Wind from Soma Capital, where he invested globally in both consumer and enterprise companies. Prior to that, Ma was the head of strategy for Verizon’s go90 mobile television app, where he led the development of go90’s overall business plan and product strategy. He joined Verizon by way of its acquisition of Intel Media, an independent consumer technology startup within Intel. He began his career on eBay’s global corporate strategy team, and has also worked as a technical product manager at Amazon. He is a graduate of Stanford and Wharton business schools.

Illustration: Dom Guzman