Artificial intelligence (AI) applications need a highly scalable hardware assist to execute fast enough for their output to be used in decision-making. Help is on the way. Graphcore’s IPU-POD systems offer massive scalable potential to thousands of IPU-M2000 machines while allowing users to ‘dial-up’ IPU resources and their regular compute resources in set increments to match their needs. Such capabilities satisfy the compute requirements of the most demanding AI applications with the additional benefit of being economically easier to manage and more ‘tuned for purpose’. IPU-PODs are configured in a range of scale-out options and capacity can be adjusted to accommodate the power budgets available in your racks.

An AI acceleration solution for production environments

At the heart of the solution is the Colossus GC200 MK2 IPU, a massively parallel 59.4 billion transistor processor. It delivers 250 Trillion Operations per Second (TOPS) across 1,472 independent cores with 900MB of In-Processor Memory and an internal IPU memory bandwidth at a blazingly fast 47TBs. The IPU-M2000s are interconnected across a 2.8Tbps low-latency fabric.

The IPU-M2000 integrates four of these processors in a 1U shelf, offering one petaflop of mixed-precision compute. This is the core building block for a family of IPU-POD systems that are designed to fit different AI workloads and datacenter power provisioning. Four IPU-M2000’s and a direct attach server are offered as an IPU-POD_16, eight IPU-M2000’s with server and switches make up the IPU-POD₃₂, while sixteen IPU-M2000’s make up an IPU-POD₆₄. Such a solution provides a core building block to further scale-out the solution into the exascale domain.

The scale-out capacity is complemented with Graphcore’s unique IPU-Fabric™ interconnect technology. IPU-Fabric is a collection of technologies that ultimately accelerate the speed at which the AI workload can be processed. It covers communication between the IPUs in an IPU-M2000, intra-rack IPU-M2000 to IPU-M2000 interconnect, and inter-rack IPU-M2000 to IPU-M2000 communication. IPU-Fabric is ultra-low latency, deterministic, and jitter-free.

Comfortable fit into existing environments

Getting workloads to run on new acceleration processors is often an arduous task. Graphcore addresses this issue with its Poplar® SDK, which is a complete software stack co-designed with the IPU. Poplar is fully integrated with standard machine learning frameworks, like TensorFlow and PyTorch so developers can work in a familiar environment. Additionally, Poplar minimises complexities when scaling to more systems. These features help developers easily use the IPU system’s processing power for any existing or new compute-intensive AI application.

As well as building AI graphs in the Poplar compiler, all the communication events that will happen during the job are calculated and allotted during the compilation process. This, together with the guaranteed very tight timing of packet delivery in IPU-Fabric, means you get a super-efficient and performant system without contention, collisions, or losses.

Ola Tørudbakken, SVP at Graphcore, states “The IPU-POD architecture is designed specifically to accommodate scale-out at a massive level. A combination of innovations has been introduced to make this possible. Three that are of particular note are the deterministic compiled in communications that happen in the Poplar software stack, our very fast, low-latency IPU-Fabric that allows all-to-all IPU communication, and our comprehensive communications library to support the massive number of collective operations that happen in deep learning workloads. All of these work in concert to achieve ground-breaking performance at scale”.

From an IT management perspective, Graphcore uses the most popular industry standard tools such as OpenBMC, RedFish, and IPMI over LAN. The management software is deliberately a flexible modular solution with a rich collection of open APIs for integration into existing systems.

Once an application can run on the system, Graphcore offers help deploying and maintaining the apps into a production environment. The IPU-POD uses common orchestration solutions like Kubernetes and Slurm. As a result, DevOps teams have the tools to ensure applications operate efficiently and deliver the expected performance and reliability needed in a production environment.

The IPU Systems are designed for virtualized datacenters. They offer virtualized hardware resource allocation and provisioning with Virtual IPU and containerized Poplar AI graph workloads using industry-standard tools such as Docker and Kubernetes.

The 1U IPU-M2000 is accessed over 100GbE RoCEv2 (RDMA over Converged Ethernet) for low-latency access. Using Ethernet avoids the bottlenecks and costs of PCIe connectors and enables a flexible CPU to accelerator ratio.

Additionally, the IPU-M2000 includes integrated scale-out networking, enabling the user to easily scale from a small system for development to massive rack deployments, all networked over standard networking at a lower cost, and with larger scale-out possibilities than using InfiniBand. The IPU-Fabric connects  IPUs by tunneling over Ethernet, maintaining the same programming model,  regardless of the size of the deployment.

“We use a standard 100-gigabit ethernet fabric for rack-to-rack connectivity. The integrated networking lets an organization start very small and go very big”, continues Tørudbakken.

Takeaways

IPU-POD systems are architected to seamlessly fit into an existing data center network and offer multiple ways of scaling up. A company can start with a single IPU-M2000 attached directly to a server providing one petaflop of compute, although many companies choose an IPU-POD₁₆  made of four IPU-M2000’s directly attached to a server, for their experimentation and pilot work. From there they can scale out to IPU-POD₆₄ IPU-POD₁₂₈ and larger systems, connected to host servers through a standard ethernet-switched fabric.

Simply put, the Graphcore IPU-POD delivers the performance at scale for AI workloads with minimal training or adjustments and without disrupting production environments and operations.

Graphcloud is available now for evaluating the unique advantages of the IPU and IPU-POD Systems: https://www.graphcore.ai/graphcloud

To learn more about scalable machine intelligence with IPU-POD Systems in the datacenter watch our webinar: //play.vidyard.com/aEe1y5aGDBSHcfUEBLB3Pk.jpg?