AlexNet, the AI model that started it all, released in source code form – for all to download

There are many stories of how artificial intelligence came to take over the world, but one of the most important developments is the emergence in 2012 of AlexNet, a neural network that, for the first time, demonstrated a huge jump in a computer’s ability to recognize images.

Thursday, the Computer History Museum (CHM), in collaboration with Google, released for the first time the AlexNet source code written by University of Toronto graduate student Alex Krizhevsky, placing it on GitHub for all to peruse and download.

“CHM is proud to present the source code to the 2012 version of Alex Krizhevsky, Ilya Sutskever, and Geoffery Hinton’s AlexNet, which transformed the field of artificial intelligence,” write the Museum organizers in the readme file on GitHub.

Also: Is OpenAI doomed? Open-source models may crush it, warns expert

Krizhevsky’s creation would lead to a flood of innovation in the ensuing years, and tons of capital, based on proof that with sufficient data and computing, neural networks could achieve breakthroughs previously viewed as mainly theoretical. 

The code, which weighs in at a scant 200KB in the source folder, combines Nvidia CUDA code, Python script, and a little bit of C++ to describe how to make a convolutional neural network parse and categorize image files.

The Museum’s software historian, Hansen Hsu, spent five years negotiating with Google, which owns the rights to the source, to release the code, as he describes in his essay about the legacy of AI and how AlexNet came to be.

Krizhevsky was a graduate student under Nobel Prize-winning AI scientist Geoffrey Hinton at the time. A second grad student, Ilya Sutskever, who later co-founded OpenAI, urged Krizhevsky to pursue the project. As Hsu quotes Hinton, “Ilya thought we should do it, Alex made it work, and I got the Nobel Prize.”

Google owns the AlexNet intellectual property because it acquired Hinton, Krizhevsky, and Sutskever’s startup company, DNNResearch.

Until AlexNet, Hinton and others had toiled for years to prove that “deep learning” collections of artificial neurons could learn patterns in data.

As Hsu notes, AI had become a backwater because it failed to demonstrate meaningful results. The convolutional neural network (CNN) had shown promising starts in performing tasks such as recognizing hand-written digits, but it had not transformed any industries until then. 

Also: AI scholars win Turing Prize for technique that made possible AlphaGo’s chess triumph

Hinton and other true believers kept working, refining the design of neural networks, including CNNs, and figuring out in small experiments on Nvidia GPU chips how increasing the number of layers of artificial neurons could theoretically lead to better results.

According to Hsu, Sutskever had the insight that the theoretical work could be scaled up to a much larger neural network given enough horsepower and training data.

As Sutskever told Nvidia co-founder and CEO Jensen Huang during a fireside chat in 2023, he knew that making neural networks big would work, even if it went against conventional wisdom.

“People weren’t looking at large neural networks” in 2012, Sutskever told Huang. “People were just training on neural networks with 50, 100 neurons,” rather than the millions and billions that later became standard. Sutskever knew they were wrong.

Also: What if AI ran ER triage? Here’s how it sped up patient care in real-world tests

“It wasn’t just an intuition; it was, I would argue, an irrefutable argument, which went like this: If your neural network is deep and large, then it could be configured to solve a hard task.”

The trio found the training data they needed in ImageNet, which was a new creation by Stanford University professor Fei Fei Li at the time. Li had herself bucked conventional wisdom in enlisting Amazon Mechanical Turk workers to hand-label 14 million images of every kind of object, a data set much larger than any computer vision data set at the time.

“It seemed like this unbelievably difficult dataset, but it was clear that if we were to train a large convolutional neural network on this dataset, it must succeed if we just can have the compute,” Sutskever told Huang in 2023. 

The fast computing they needed turned out to be a dual-GPU desktop computer that Krizhevsky worked on in his bedroom at his parents’ house. 

Also: What is DeepSeek AI? Is it safe? Here’s everything you need to know

When the work was presented at the ImageNet annual competition in September of 2012, AlexNet scored almost 11 points better than the closest competitor, a 15.3% error rate. They described the work in a formal paper.

Yann LeCun, chief AI scientist at Meta Platforms, who had earlier studied under Hinton and had pioneered CNN engineering in the 1990s, proclaimed AlexNet at the time to be a turning point. 

“He was right,” writes Hsu. “Before AlexNet, almost none of the leading computer vision papers used neural nets. After it, almost all of them would.”

What the trio had done was to make good on all the theoretical work on making “deep” neural networks out of many more layers of neurons, to prove that they could really learn patterns.

“AlexNet was just the beginning,” writes Hsu. “In the next decade, neural networks would advance to synthesize believable human voices, beat champion Go players, model human language, and generate artwork, culminating with the release of ChatGPT in 2022 by OpenAI, a company co-founded by Sutskever.”

Sutskever would later prove once again that making neural networks bigger could lead to surprising breakthroughs. The arrival of ChatGPT in the fall of 2022, another shot heard around the world, was the result of all the GPT 1, 2, and 3 models before it. Those models were all a result of Sutskever’s faith in scaling neural networks to unprecedented size.

Also: The CTO vs. CMO AI power struggle – who should really be in charge?

“I had a very strong belief that bigger is better and that one of the goals that we had at OpenAI is to figure out how to use the scale correctly,” he told Huang in 2023.

Huang credited the trio during his keynote speech at the Consumer Electronics Show in January. “In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton discovered CUDA,” said Huang, “used it to process AlexNet, and the rest is history.”

The release of AlexNet in source code form has interesting timing. It arrives just as the AI field and the entire world economy are enthralled with another open-source model, DeepSeek AI’s R1. 

Want more stories about AI? Sign up for Innovation, our weekly newsletter.