How Can You Prepare Your Cloud to Safely Implement AI?

The Risk of Sharing Sensitive Data with Public AI Platforms

As employees and students embrace generative AI platforms, IT professionals must find ways to ensure that sensitive data isn’t being shared publicly. Users need ways to explore large language models (LLMs) without disclosing any of their data.

“First, we do a data governance check. What kind of data are you going to be using? What are the controls around that data? Then we can design a solution that allows you to keep your data in-house and not expose any data,” says Roger Haney, chief architect for software-defined infrastructure at CDW.

Data governance is key for organizations looking to prepare their infrastructure and users for AI and LLMs.

“We have a workshop called Mastering Operational AI Transformation, or MOAT,” Haney says. “You’re drawing a circle around the data that we don’t want to get out. We want it to be internally useful, but we don’t want it to get out.”

WATCH NOW: See the risks and rewards of artificial intelligence in higher education.

To ensure data security, partners such as CDW can help organizations set up or build cloud solutions that don’t rely on public LLMs. This gives them the benefits of generative AI without the risk.

“We can set up your cloud in a way where we’re able to use a prompt to a make copy of an LLM,” Haney explains. “We build private enclaves containing a chat resource to an LLM that people can use without a public LLM learning the data they’re putting in.”

When to Host AI Databases in the Cloud

Higher education’s plans for generative AI will determine how it should prepare its infrastructure for the future of this technology. Haney says most users want to communicate with their data for retrieval or analysis purposes.

“Chatting with your data doesn’t require a new data store. You don’t have to build a huge data lake or warehouse,” he says. “If you have student data, then we add another model that can create the query in SQL, do the query and pull the data back. Then you can ask it questions using that data as part of your prompt, and you can talk with your data.”

Partners such as CDW can give colleges and universities this functionality quickly and inexpensively by creating a retrieval-augmented generation database. When asked a simple question, it can return two or three top answers. Often, these solutions don’t require the cloud and, in fact, some higher education institutions are investing in on-premises infrastructure to support the research and development of new AI tools.

“Higher education institutions are the ones creating most of these solutions today,” Haney says. “They’re building customized models. They’re building a lot of the machine learning routines for industry today. So, it’s a great place for a lot of experimentation.”

That said, higher education institutions shouldn’t eschew the cloud entirely, either. Hybrid cloud solutions give colleges and universities better control over data usage, and a partner can help institutions better understand which workflows should be moved to the cloud and which should stay put.

“They must understand their data governance and security and the usage too,” says Haney. “They need to understand the controls around the processes. Otherwise, people will click, click, click and $10,000 later, they’ve used a lot of resources and, oops, they’ve run out of funding for that.”

So, when it comes to determining how you’ll prepare your cloud infrastructure for AI, think first about how you want to use AI, how you want to use your data and what that will require at your institution. Working with an experienced partner can help you answer these questions and more to prepare your college or university’s digital infrastructure for whatever comes next.