Whereas Qualcomm’s Cloud AI 100 accelerator household has lengthy been out there from a number of tier-one expertise suppliers similar to Lenovo, Hewlett Packard Enterprise (HPE), Inventec, Foxconn, Gigabyte, and Asus, it’s beginning to see deployment within the public cloud.
Amazon Internet Companies (AWS) recently introduced its first Qualcomm-based accelerated occasion sort, the DL2q, that includes the Qualcomm Cloud AI 100. Whereas the brand new occasion sort can be utilized for normal inference purposes, the businesses spotlight the accelerator’s particular applicability in creating automotive ADAS and associated purposes – an space through which Qualcomm is quickly increasing its presence.
Qualcomm’s Cloud AI 100
Qualcomm first launched its Cloud AI 100 accelerator in 2020, delivering a tool particularly engineered to spice up the capabilities of cloud computing environments by environment friendly, high-speed AI inference processing.
The Cloud AI 100 is tailor-made for inference, which is the appliance part of AI the place a skilled mannequin is used to interpret new information. It is a crucial perform in AI deployments that require rapid outcomes, similar to recognizing speech, translating languages, analyzing photos, or processing real-time information from IoT units.
The accelerator gives a pleasant steadiness of efficiency and effectivity. Qualcomm constructed a tool that tells a demonstrably substantial complete price of possession (TCO) story whereas delivering the efficiency required by demanding AI inference workloads.
MLPerf 3.1 Outcomes
In September 2023, MLCommons released its MLPerf Inference 3.1 benchmark outcomes, through which Qualcomm demonstrated vital developments with its Cloud AI 100 inference accelerators.
The outcomes present notable enhancements in efficiency, energy effectivity, and decrease latencies, notably for Pure Language Processing (NLP) and computer-vision networks for the Qualcomm Cloud AI 100.
Qualcomm’s MLPerf Inference v3.1 benchmarks surpassed its earlier data. In a number of classes, the Cloud AI 100 confirmed developments in peak offline efficiency, energy effectivity, and latency discount.
As an illustration, a 2U datacenter server platform geared up with 16 Qualcomm Cloud AI 100 PCIe Professional (75W TDP) accelerators displayed a 15-20% enchancment in energy effectivity throughout NLP and pc imaginative and prescient networks.
On the similar time, Qualcomm’s efficiency on the RetinaNet Community on platforms using the Cloud AI 100 noticed enhancements of round 12%. This optimization signifies Qualcomm’s continued efforts to reinforce AI fashions’ processing effectivity and velocity.
The MLPerf Inference v3.1 outcomes clearly exhibit the effectiveness of the Qualcomm Cloud AI 100 throughout a broad vary of purposes, together with each edge and information middle classes, highlighting its efficiency in key metrics like inference-per-second and inference-per-second-per-watt (I/S/W).
Introducing the Cloud AI 100 Extremely
In November 2023, Qualcomm added to its Cloud AI 100 lineup with the introduction of its new Qualcomm Cloud AI 100 Extremely. The brand new accelerator is tailor-made explicitly to serve the wants of generative AI and enormous language fashions (LLMs).
The brand new accelerator gives 4 occasions the efficiency of earlier Cloud AI 100 variants. The AI 100 Extremely can assist extraordinarily giant AI fashions, dealing with fashions with as much as 100 billion parameters on a single 150-watt card.
The Extremely can scale as much as assist 175 billion parameter fashions with two playing cards. A number of AI 100 Extremely playing cards might be mixed to deal with even bigger fashions.
Regardless of its excessive efficiency, the Cloud AI 100 Extremely maintains the power effectivity inherent in the remainder of the household, essential for decreasing operational prices in information facilities and helps sustainability objectives in AI operations.
AI inference is turning into a crucial performance, particularly with giant language fashions. Bringing AI to the sting, particularly the cellular edge, is the subsequent frontier of accelerated computing. Qualcomm places a major stake within the floor with its Cloud AI 100 accelerators, properly complementing its present edge-targeted compute and communication expertise.
Whereas Qualcomm entered this market with its Cloud AI 100, the brand new Extremely providing takes these capabilities additional, explicitly focusing on the calls for of generative AI and enormous language fashions. This superior model stands out for its skill to assist extraordinarily giant AI fashions.
Its enhanced efficiency and power effectivity make the Qualcomm Cloud AI 100 Extremely a compelling answer for advanced AI duties whereas retaining operational prices in examine.
Past its technical capabilities, the brand new accelerator gives one other waypoint as Qualcomm continues its growth into the AI-enabled edge market. Qualcomm leverages the expertise throughout the Cloud AI 100 household to service the wants of assorted markets, now together with the general public cloud.
Qualcomm isn’t alone on this market. Past the choices of trade stalwart NVIDIA, we’ve seen AWS, Google, and Microsoft all introduce inference-specific accelerators. AMD’s MI300-series of accelerators play on this area, as does Intel’s Gaudi.
Qualcomm’s differentiates with its skill to mix the high-performance, energy-efficient inference typified by its Cloud AI 100 choices with an IP portfolio that may service the broader wants of the sting market. That is uncommon amongst present expertise suppliers.
Qualcomm’s Cloud AI 100 product line underscores the corporate’s strategic transfer into high-end AI inference markets, showcasing its potential to reshape AI processing in numerous industries, from healthcare to automotive and past. It’s a compelling story that Qualcomm’s rivals wrestle to beat.
Disclosure: Steve McDowell is an trade analyst, and NAND Analysis an trade analyst agency, that engages in, or has engaged in, analysis, evaluation, and advisory companies with many expertise firms, which can embrace these talked about on this article. Mr. McDowell doesn’t maintain any fairness positions with any firm talked about on this article.
#Qualcomm #Builds #Momentum #Inference