Data-centric Organizations need both Performance and Flexible Data Management

January 27, 2020

Converged AI and HPC have arrived in 2020 and commercial IT organizations are putting data at the core of their business model. Just in the HPC market alone, Intersect360[1] found that around 90% of HPC organizations were either running machine learning (56%) or investigating/planning its introduction. Gartner[2] claims that 60% of organizations are adapting their business model to AI and a 2019 ESG[3] survey showed that 49% of IT organizations state that “data is their business”, with another 31% expecting to offer datacentric products in the next 2 years.  Yet, how to reliably accomplish that goal often remains elusive.  The same Gartner survey reported a 50% failure rate for these projects.

Performance, as ever, is important and DDN’s EXAScaler systems continue to outperform forthcoming competitor offerings with over 63 Million IOPs per Rack with DDN’s AI400X and over 140GB/s per rack in HDD-only sequential performance. But delivering this performance potential to today’s workloads is the real target. Datacentric organizations are investing heavily in more powerful compute platforms, often relying on GPUs. Keeping systems operating at 100% utilization is not an easy proposition without at-scale experience and concerted engineering effort in optimizing for containerized workloads, AI frameworks, GPU platforms, extreme-scale CPU and fast networks. Underutilized infrastructure is a recipe for failure in a world where competitive stakes are high, investments are large and the amount of data to be analyzed is vast. A data management vendor must deliver performance no matter how large the requirement nor how diverse the applications.

Modern organizations shifting to a data-centric strategy need very large scale data-in-place analytics environments that allow data to be accessible by many different methods for ingestion, labeling, processing, etc. In this rapidly changing and converging ecosystem of AI, Big Data and HPC, fast yet flexible access to all data is a differentiator. Accessing and manipulating that data from workstations and cloud applications require protocols like NFS, SMB, and S3 as well as super-fast Native access. With the demands for data mobility (getting the right data in the right place at the right time), the ability to copy, move or sync data between systems and to third party cloud providers is equally important.

There is a clear priority for organizations to build the right infrastructure for data-centric services and selecting the right vendor partnerships will improve their path to success. Some suppliers struggle with the ability to manage data easily at petabyte scale. Many vendors have neither the focus on scalable data processing nor familiarity with those types of environments. Other vendors can’t deliver consistent performance or reliable large scale solutions. Whilst Data Lakes and pure Big Data approaches are declining due to inflexibility and inefficiency, there is a surge in implementations of true parallel filesystems to address AI, Big Data and HPC challenges. DDN’s EXAScaler is regularly implemented at scales of 100s of Petabytes without the downsides of data duplication and with far higher performance delivered to analytics and AI applications than any object store. EXAScaler brings a simpler experience and a powerful in-built data management engine to ensure businesses can stay agile with their data as new analytics models evolve and their data sets broaden.

The IT infrastructure market is transforming when it comes to data-centric requirements. No longer consigned to a small research group or the high performance computing team, the ability to acquire, analyze, manipulate and distribute data is core to IT operations. Similarly, the landscape of vendors to consider is undergoing significant change, whether through mergers and acquisitions or other market realities. Selecting a proven, stable and trusted partner with a track record of innovation to meet at-scale needs is an important first step to becoming a data-driven organization or informing the next market-changing AI application.

About DDN

DDN ranks #1 for HPC Storage deployments[4] at scale and holds a leadership position in AI and HPC with over 10 EXAbytes of shipped capacity and 150+ patents in data management. Recently awarded “Top 5 vendors to watch”[5] for our A3I product line and capability advances in the EXA5 platforms DDN continues to invest in innovation focussed on our customer’s forward path. As stated by IDC[6], “Increasingly, parallelization is the preferred approach, with AI infrastructure starting to resemble HPC infrastructure”. With a market leading 16% of revenues invested back into our R&D, DDN’s accelerated cadence of innovation in our parallel filesystems for AI and HPC is second to none.

Why is DDN EXA5 your undisputed filesystem of choice for the new converged world of AI and HPC?

#1 DDN Delivers the best customer experience. DDN is a long-term dependable partner with the technical expertise that improves time to value and accelerates your productivity.

#2 EXA5 Performance and Scale Leadership – DDN platforms attain the highest performance efficiency.

#3 EXA5 Capability – EXA5 brings the capabilities needed for converged AI and HPC – MultiCloud, Data Aware Intelligence, Global Collaboration and Strong Security including Multi Tenancy support

The DDN Customer Experience

DDN is different because our focus is different. That focus is wholly on our 7000 customers and their data challenges. Our customers benefit by partnering with DDN for storage alongside compute vendors, bringing DDN’s unique expertise in storage. With a DDN storage strategy, businesses can escape from monolithic proprietary systems, mitigating against spiralling license costs and retaining their control in getting the best return on investment.

“With the goal of expanding the boundaries of science as we know it today, we are excited about the arrival of advanced new technology that can dramatically increase the performance at scale of our systems, and specifically of our new Top 10 supercomputer, Frontera. DDN’s new EXA5 has the power to provide the best I/O performance our users have ever experienced, and greatly reduce the I/O bottlenecks in large scale computation. We believe that EXA5 will play a role in many of the ground breaking discoveries scientists will make with Frontera.” – Dan Stanzione, executive director, Texas Advanced Computing Center.

DDN brings the skills of more than 1000 technical staff, all focussed on storage and data management. With filesystem engineering teams that contribute tens of thousands of lines of code each year[7] to our open filesystems, field teams that deliver the largest tiered storage into top 10 supercomputers[8][9]and global operations that support billions of dollars of infrastructure, DDN is the largest privately held storage company in the world. Now with DDN’s rapid expansion into Enterprise[10] we have dramatically added our ability to accelerate your entire infrastructure.


[1] Intersect360, HPC User Site Census: Storage Suppliers, May 2019

[2] https://www.idc.com/getdoc.jsp?containerId=prUS45344519

[3] ESG Master Survey Results, 2019 Data Storage Trends, November 2019

[4] Intersect360 Research, ibid.

[5] https://www.ddn.com/press-releases/ddn-honored-top-five-vendor-watch-annual-datanami-awards/

[6] https://www.idc.com/getdoc.jsp?containerId=DR2019_T2_PR

[7] http://cdn.opensfs.org/wp-content/uploads/2019/07/LUG2019-Community-Release-Update-Jones.pdf

[8] https://www.ddn.com/blog/tacc-frontera-helping-make-impossible-possible/

[9] http://lustrefs.cn/wp-content/uploads/2018/04/Lustre-Persistent-Cache-on-Client-for-AI-Machine-Learning-and-Bigdata-Processing.pdf

[10] https://www.nextplatform.com/2019/10/09/ddn-uses-acquisitions-to-grow-in-the-enterprise/

0Shares
0
found that around 90% of HPC organizations were either running machine learning (56%) or investigating/planning its introduction. Read more…

" share_counter=""]
Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

GTC21: Dell Building Cloud Native Supercomputers at U Cambridge and Durham

April 14, 2021

In conjunction with GTC21, Dell Technologies today announced new supercomputers at universities across DiRAC (Distributed Research utilizing Advanced Computing) in the UK with plans to explore use of Nvidia BlueField DPU Read more…

The Role and Potential of CPUs in Deep Learning

April 14, 2021

Deep learning (DL) applications have unique architectural characteristics and efficiency requirements. Hence, the choice of computing system has a profound impact on how large a piece of the DL pie a user can finally enj Read more…

GTC21: Nvidia Launches cuQuantum; Dips a Toe in Quantum Computing

April 13, 2021

Yesterday Nvidia officially dipped a toe into quantum computing with the launch of cuQuantum SDK, a development platform for simulating quantum circuits on GPU-accelerated systems. As Nvidia CEO Jensen Huang emphasized i Read more…

Nvidia Aims Clara Healthcare at Drug Discovery, Imaging via DGX

April 12, 2021

Nvidia Corp. continues to expand its Clara healthcare platform with the addition of computational drug discovery and medical imaging tools based on its DGX A100 platform, related InfiniBand networking and its AGX develop Read more…

Nvidia Serves Up Its First Arm Datacenter CPU ‘Grace’ During Kitchen Keynote

April 12, 2021

Today at Nvidia’s annual spring GPU technology conference, held virtually once more due to the ongoing pandemic, the company announced its first ever Arm-based CPU, called Grace in honor of the famous American programmer Grace Hopper. Read more…

AWS Solution Channel

Volkswagen Passenger Cars Uses NICE DCV for High-Performance 3D Remote Visualization

 

Volkswagen Passenger Cars has been one of the world’s largest car manufacturers for over 70 years. The company delivers more than 6 million automobiles to global customers every year, from 50 production locations on five continents. Read more…

Nvidia Debuts BlueField-3 – Its Next DPU with Big Plans for an Expanded Role

April 12, 2021

Nvidia today announced its next generation data processing unit (DPU) – BlueField-3 – adding more substance to its evolving concept of the DPU as a full-fledged partner to CPUs and GPUs in delivering advanced computi Read more…

The Role and Potential of CPUs in Deep Learning

April 14, 2021

Deep learning (DL) applications have unique architectural characteristics and efficiency requirements. Hence, the choice of computing system has a profound impa Read more…

Nvidia Serves Up Its First Arm Datacenter CPU ‘Grace’ During Kitchen Keynote

April 12, 2021

Today at Nvidia’s annual spring GPU technology conference, held virtually once more due to the ongoing pandemic, the company announced its first ever Arm-based CPU, called Grace in honor of the famous American programmer Grace Hopper. Read more…

Nvidia Debuts BlueField-3 – Its Next DPU with Big Plans for an Expanded Role

April 12, 2021

Nvidia today announced its next generation data processing unit (DPU) – BlueField-3 – adding more substance to its evolving concept of the DPU as a full-fle Read more…

Nvidia’s Newly DPU-Enabled SuperPod Is a Multi-Tenant, Cloud-Native Supercomputer

April 12, 2021

At GTC 2021, Nvidia has announced an upgraded iteration of its DGX SuperPods, calling the new offering “the first cloud-native, multi-tenant supercomputer.” Read more…

Tune in to Watch Nvidia’s GTC21 Keynote with Jensen Huang – Recording Now Available

April 12, 2021

Join HPCwire right here on Monday, April 12, at 8:30 am PT to see the Nvidia GTC21 keynote from Nvidia’s CEO, Jensen Huang, livestreamed in its entirety. Hosted by HPCwire, you can click to join the Huang keynote on our livestream to hear Nvidia’s expected news and... Read more…

The US Places Seven Additional Chinese Supercomputing Entities on Blacklist

April 8, 2021

As tensions between the U.S. and China continue to simmer, the U.S. government today added seven Chinese supercomputing entities to an economic blacklist. The U Read more…

Habana’s AI Silicon Comes to San Diego Supercomputer Center

April 8, 2021

Habana Labs, an Intel-owned AI company, has partnered with server maker Supermicro to provide high-performance, high-efficiency AI computing in the form of new Read more…

Intel Partners Debut Latest Servers Based on the New Intel Gen 3 ‘Ice Lake’ Xeons

April 7, 2021

Fresh from Intel’s launch of the company’s latest third-generation Xeon Scalable “Ice Lake” processors on April 6 (Tuesday), Intel server partners Cisco, Dell EMC, HPE and Lenovo simultaneously unveiled their first server models built around the latest chips. And though arch-rival AMD may... Read more…

Julia Update: Adoption Keeps Climbing; Is It a Python Challenger?

January 13, 2021

The rapid adoption of Julia, the open source, high level programing language with roots at MIT, shows no sign of slowing according to data from Julialang.org. I Read more…

Intel Launches 10nm ‘Ice Lake’ Datacenter CPU with Up to 40 Cores

April 6, 2021

The wait is over. Today Intel officially launched its 10nm datacenter CPU, the third-generation Intel Xeon Scalable processor, codenamed Ice Lake. With up to 40 Read more…

CERN Is Betting Big on Exascale

April 1, 2021

The European Organization for Nuclear Research (CERN) involves 23 countries, 15,000 researchers, billions of dollars a year, and the biggest machine in the worl Read more…

Programming the Soon-to-Be World’s Fastest Supercomputer, Frontier

January 5, 2021

What’s it like designing an app for the world’s fastest supercomputer, set to come online in the United States in 2021? The University of Delaware’s Sunita Chandrasekaran is leading an elite international team in just that task. Chandrasekaran, assistant professor of computer and information sciences, recently was named... Read more…

HPE Launches Storage Line Loaded with IBM’s Spectrum Scale File System

April 6, 2021

HPE today launched a new family of storage solutions bundled with IBM’s Spectrum Scale Erasure Code Edition parallel file system (description below) and featu Read more…

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

Saudi Aramco Unveils Dammam 7, Its New Top Ten Supercomputer

January 21, 2021

By revenue, oil and gas giant Saudi Aramco is one of the largest companies in the world, and it has historically employed commensurate amounts of supercomputing Read more…

Quantum Computer Start-up IonQ Plans IPO via SPAC

March 8, 2021

IonQ, a Maryland-based quantum computing start-up working with ion trap technology, plans to go public via a Special Purpose Acquisition Company (SPAC) merger a Read more…

Leading Solution Providers

Contributors

Can Deep Learning Replace Numerical Weather Prediction?

March 3, 2021

Numerical weather prediction (NWP) is a mainstay of supercomputing. Some of the first applications of the first supercomputers dealt with climate modeling, and Read more…

Livermore’s El Capitan Supercomputer to Debut HPE ‘Rabbit’ Near Node Local Storage

February 18, 2021

A near node local storage innovation called Rabbit factored heavily into Lawrence Livermore National Laboratory’s decision to select Cray’s proposal for its CORAL-2 machine, the lab’s first exascale-class supercomputer, El Capitan. Details of this new storage technology were revealed... Read more…

New Deep Learning Algorithm Solves Rubik’s Cube

July 25, 2018

Solving (and attempting to solve) Rubik’s Cube has delighted millions of puzzle lovers since 1974 when the cube was invented by Hungarian sculptor and archite Read more…

African Supercomputing Center Inaugurates ‘Toubkal,’ Most Powerful Supercomputer on the Continent

February 25, 2021

Historically, Africa hasn’t exactly been synonymous with supercomputing. There are only a handful of supercomputers on the continent, with few ranking on the Read more…

The History of Supercomputing vs. COVID-19

March 9, 2021

The COVID-19 pandemic poses a greater challenge to the high-performance computing community than any before. HPCwire's coverage of the supercomputing response t Read more…

AMD Launches Epyc ‘Milan’ with 19 SKUs for HPC, Enterprise and Hyperscale

March 15, 2021

At a virtual launch event held today (Monday), AMD revealed its third-generation Epyc “Milan” CPU lineup: a set of 19 SKUs -- including the flagship 64-core, 280-watt 7763 part --  aimed at HPC, enterprise and cloud workloads. Notably, the third-gen Epyc Milan chips achieve 19 percent... Read more…

HPE Names Justin Hotard New HPC Chief as Pete Ungaro Departs

March 2, 2021

HPE CEO Antonio Neri announced today (March 2, 2021) the appointment of Justin Hotard as general manager of HPC, mission critical solutions and labs, effective Read more…

Microsoft, HPE Bringing AI, Edge, Cloud to Earth Orbit in Preparation for Mars Missions

February 12, 2021

The International Space Station will soon get a delivery of powerful AI, edge and cloud computing tools from HPE and Microsoft Azure to expand technology experi Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire