SambaNova Launches Second-Gen DataScale System

By Oliver Peckham

September 14, 2022

SambaNova Systems is announcing – and shipping – its second-generation DataScale system, the DataScale SN30. Powered by the eponymous Cardinal SN30 RDU (Reconfigurable Data Unit), SambaNova claims that the DataScale SN30 is capable of a 6x speedup (on certain AI workloads) relative to a comparable system powered by Nvidia A100 GPUs. The new hardware is shipping today. Ahead of the announcement, HPCwire spoke to Marshall Choy, SambaNova’s senior vice president for product, about the launch.

The Cardinal SN30 RDU powering the new DataScale system is manufactured on TSMC’s 7nm process, contains 86 billion transistors and is capable of 688 teraflops at bfloat16 precision. Choy said they weren’t going into “Hot Chips-level detail” on the chip because they don’t actually sell the chip itself – they only use them for SambaNova-sold systems and service offerings. Each SN30 system is equipped with eight SN30 RDUs, along with other improvements. “Beyond the chip, we’ve also thought a lot about memory,” Choy said. “Massive models require massive amounts of memory.” To that end, the SN30 reportedly offers “12.8x more memory capacity than [Nvidia’s] DGX A100” – “a full terabyte versus 80 gigabytes,” Choy said. “We basically doubled all the compute and memory on the chip from the previous generation.”

Marshall Cloy with the new SambaNova SN30. Image courtesy of SambaNova.

“Put that all together with our SambaFlow software stack, with enhancements around enterprise integration, with things like native Kubernetes support for the orchestration of containerized and virtualized models and applications,” Choy continued, “and you get our new DataScale SN30 hardware platform.”

Choy said that the new DataScale system is comparable to a DGX system, but completely rack-integrated for ease-of-use (“roll them in, plug it in and it works”). Customers can access the SN30 by purchasing systems or via SambaNova’s “Dataflow-as-a-Service” model, which can be run either through a cloud service provider or on-premises through a private cloud (which Choy said that some customers prefer due to firewalls and privacy). Choy said that while the full list of supported public cloud providers had yet to be finalized, some providers – like Aicadium, Cirrascale and ORock – were already on board.

SambaNova bills its offering as “a fully integrated AI platform innovating in every level of the stack,” and the company is positioning this offering against Nvidia’s suite in its comparisons. According to SambaNova, when training a 13-billion parameter GPT-3 model, the new DataScale SN30 system outperformed an eight-socket DGX A100 system sixfold. Compared to SambaNova’s previous system, the DataScale SN10, Choy said that “we’re saying generally 2-6x” better performance.

Customers

So far, SambaNova is disclosing just a few of the initial customers for the DataScale SN30 system, including Argonne National Laboratory and Lawrence Livermore National Laboratory (LLNL) – both of which have reputations as early adopters of new HPC technologies, particularly with respect to AI-focused systems and novel accelerators.

“We look forward to deploying a larger, multirack system of the next generation of SambaNova’s DataScale systems,” said Bronis de Supinski, CTO for Livermore Computing at LLNL. “Integration of this solution with traditional clusters throughout our center will enable the technology to have a deeper programmatic impact. We anticipate a 2 to 6x performance increase, as the new DataScale system promises to significantly improve overall speed, performance, and productivity.”

More customers, Choy said, would be announced by the end of the year.

“What we’re also seeing is as end users run into these deep learning type models, foundation models, they’re now getting a little bit more experimental in their approach,” commented Chirag Dekate, vice president, analyst at Gartner. “They’re actively exploring the technology landscape, so to speak.” Dekate added that by “dramatically improving the accelerator performance” and “exposing the dataflow ecosystem as a service,” SambaNova was creating a “unique technology offering for end users exploring large, more complex models.”

“For end users that want to experiment and explore diverse innovation options, they’ve just lowered the threshold for entry dramatically,” Dekate said.

A new foundation

SambaNova views its offerings as well-suited to a perceived boom in foundation models – large AI models that are able to handle wide ranges of specialized tasks. “We’re certainly seeing an evolution in artificial intelligence,” Choy said. “We’ve come from the world of predictive analytics, the machine learning, the deep learning – and the next frontier is all about foundation models. We think that foundation models are going to be as necessary for enterprises and organizations from now and into the future as a high-speed internet connection and a mobile application are to business today.”

By way of explanation, Choy cited a customer case – a large bank that used to have thousands of different deployments of the BERT language model, all fine-tuned for specific tasks. With foundation models like GPT-3, Choy said, that has changed.

“The old thinking was you needed a smaller model that was specifically tuned to a specific task with a large dataset to get the highest levels of accuracy,” he said. “We’ve proven that you can achieve the same or better levels of accuracy with a GPT-based foundation model.”

“The big challenge, of course, is there’s not a lot of people out in the world who can actually build and deliver a foundation model deployment. … You look at GPT-3, for example – your choices are basically OpenAI, AWS, Meta and SambaNova.”

To that end, of course, SambaNova’s Dataflow-as-a-Service offering has a GPT model – a model that Choy said would soon see further enhancements, and which would soon be complemented by a vision-oriented foundation model, as well.

“Gartner agrees that a lot of the focus going forward will include a very heavy emphasis on foundational models,” Dekate commented. “What is starting to happen in end-user ecosystems is they are working very aggressively to not just productionalize a lot of the deep learning-based innovations in their ecosystems – some homegrown, others borrowed from elsewhere – but in many cases they’re also turning towards leading-edge technologies like foundation models or multimodal AI where you can combine different types of AI techniques together.”

SambaNova has raised $1.1 billion at a $5 billion valuation and employs more than 500 people. The company will have a presence at SC22 this November in Dallas, and its speakers are featured several times at this year’s AI Hardware Summit, which began yesterday and continues through Thursday.

Header image: LLNL’s existing first-gen SambaNova deployment.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

TACC Supercomputing Powers Climate Modeling for Fisheries

January 28, 2023

A tremendous portion of the world depends on the output of the oceans’ major fisheries, which have, in recent decades, found themselves under near-constant threat from mismanagement (e.g. overfishing). Climate change, Read more…

PFAS Regulations, 3M Exit to Impact Two-Phase Cooling in HPC

January 27, 2023

Per- and polyfluoroalkyl substances (PFAS), known as “forever chemicals,” pose a number of health risks to humans, with more suspected but not yet confirmed – and, as a result, PFAS are coming under increasing regu Read more…

Sweden Plans Expansion for Nvidia-Powered Berzelius Supercomputer

January 26, 2023

The Atos-built, Nvidia SuperPod-based Berzelius supercomputer – housed in and operated by Sweden’s Linköping-based National Supercomputer Centre (NSC) – is already no slouch. But now, Nvidia and NSC have announced Read more…

Multiverse, Pasqal, and Crédit Agricole Tout Progress Using Quantum Computing in FS

January 26, 2023

Europe-based quantum computing pioneers Multiverse Computing and Pasqal, and global bank Crédit Agricole CIB today announced successful conclusion of a 1.5-year POC study “to evaluate the contribution of an algorithmi Read more…

Critics Don’t Want Politicians Deciding the Future of Semiconductors

January 26, 2023

The future of the semiconductor industry was partially being decided last week by a mix of politicians, policy hawks and chip industry executives jockeying for influence at the World Economic Forum. Intel CEO Pat Gels Read more…

AWS Solution Channel

Shutterstock_1687123447

Numerix Scales HPC Workloads for Price and Risk Modeling Using AWS Batch

  • 180x improvement in analytics performance
  • Enhanced risk management
  • Decreased bottlenecks in analytics
  • Unlocked near-real-time analytics
  • Scaled financial analytics

Overview

Numerix, a financial technology company, needed to find a way to scale its high performance computing (HPC) solution as client portfolios ballooned in size. Read more…

Microsoft/NVIDIA Solution Channel

Shutterstock 1453953692

Microsoft and NVIDIA Experts Talk AI Infrastructure

As AI emerges as a crucial tool in so many sectors, it’s clear that the need for optimized AI infrastructure is growing. Going beyond just GPU-based clusters, cloud infrastructure that provides low-latency, high-bandwidth interconnects and high-performance storage can help organizations handle AI workloads more efficiently and produce faster results. Read more…

Riken Plans ‘Virtual Fugaku’ on AWS

January 26, 2023

The development of a national flagship supercomputer aimed at exascale computing continues to be a heated competition, especially in the United States, the European Union, China, and Japan. What is the value to be gained Read more…

PFAS Regulations, 3M Exit to Impact Two-Phase Cooling in HPC

January 27, 2023

Per- and polyfluoroalkyl substances (PFAS), known as “forever chemicals,” pose a number of health risks to humans, with more suspected but not yet confirmed Read more…

Critics Don’t Want Politicians Deciding the Future of Semiconductors

January 26, 2023

The future of the semiconductor industry was partially being decided last week by a mix of politicians, policy hawks and chip industry executives jockeying for Read more…

Riken Plans ‘Virtual Fugaku’ on AWS

January 26, 2023

The development of a national flagship supercomputer aimed at exascale computing continues to be a heated competition, especially in the United States, the Euro Read more…

Shutterstock 1134313550

Semiconductor Companies Create Building Block for Chiplet Design

January 24, 2023

Intel's CEO Pat Gelsinger last week made a grand proclamation that chips will be for the next few decades what oil and gas was to the world over the last 50 years. While that remains to be seen, two technology associations are joining hands to develop building blocks to stabilize the development of future chip designs. The goal of the standard is to set the stage for a thriving marketplace that fuels... Read more…

Royalty-free stock photo ID: 1572060865

Fujitsu Study Says Quantum Decryption Threat Still Distant

January 23, 2023

Global computer and chip manufacturer Fujitsu today reported that a new study performed on its 39-qubit quantum simulator suggests it will remain difficult for Read more…

At ORNL, Jeff Smith Becomes Interim Director, as Search for Permanent Lab Chief Continues

January 20, 2023

UT-Battelle, which manages Oak Ridge National Laboratory (ORNL) for the U.S. Department of Energy, has appointed Jeff Smith as interim director for the lab as t Read more…

Top HPC Players Creating New Security Architecture Amid Neglect

January 20, 2023

Security of high-performance computers is being neglected in the pursuit of horsepower, and there are concerns that the ignorance may be costly if safeguards ar Read more…

Ohio Supercomputer Center Debuts ‘Ascend’ GPU Cluster

January 19, 2023

Less than 10 months after it was announced, the Columbus-based Ohio Supercomputer Center (OSC) has debuted its Dell-built GPU cluster, “Ascend.” Designed to Read more…

Leading Solution Providers

Contributors

SC22 Booth Videos

AMD @ SC22
Altair @ SC22
AWS @ SC22
Ayar Labs @ SC22
CoolIT @ SC22
Cornelis Networks @ SC22
DDN @ SC22
Dell Technologies @ SC22
HPE @ SC22
Intel @ SC22
Intelligent Light @ SC22
Lancium @ SC22
Lenovo @ SC22
Microsoft and NVIDIA @ SC22
One Stop Systems @ SC22
Penguin Solutions @ SC22
QCT @ SC22
Supermicro @ SC22
Tuxera @ SC22
Tyan Computer @ SC22
  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire