New Moab HPC Suite Turns Up Knob on Scalability, Ease of Use

By Michael Feldman

March 13, 2012

Adaptive Computing has released Moab HPC Suite 7.0, a major revision that scales the popular workload management suite to be able to handle system with more than 100 thousand nodes. The new release also adds a host of new features aimed at commercial HPC, including a new web services interface, more flexible accounting support, and a new admin dashboard.

The new HPC suite will be the first major Moab release since Robert Clyde took over as CEO of Adaptive Computing back in July 2011. In his previous tenure as CTO at Symantec, the company grew its revenues from under $1 billion to over $5 billion, which suggests what Adaptive’s board of directors had in mind when they brought Clyde aboard.

The HPC workload management business has been good to the company, in large part thanks Adaptive’s popularity at the big national labs and universities. Moab is currently running on 4 of the top 10 supercomputers in the world and 14 of the top 50. But according to Chad Harrington, Adaptive’s VP of Marketing, they see the big growth happening in commercial HPC.

“Our academic and research institutions have always been our bread and butter and they still will be,” says Harrington. “But it’s a fixed size market.”

Clyde agrees and sees the most recent gyrations in the HPC industry favoring the commercial side of the business. According to him, demand is flat or down in the academic sphere of HPC, while the government space is basically treading water. But for commercial HPC systems, they’ve been installing a lot of new software.

Mirroring their counterparts in research and academia, businesses are moving to larger HPC clusters as they ride the ever-improving price-performance curve. And although the price of flops continues to drop, the cost of running these big machines is heading in the opposite direction. So companies are looking to squeeze as many job cycles out of the hardware as possible.

That’s where Adaptive can press its advantage. The company is infused with $20 million worth of new funding and is looking to go after its expanding base of commercial HPC users. “We are in a high-growth mode.” says Clyde. In particular, the company has its sights set on broadening its footprint in manufacturing and oil & gas, where high performance computing is already well established.

“It’s not like we’re trying to figure out how to get Mom-and-Pop shops to do HPC,” Clyde told HPCwire. “That would be a big stretch.”

Turning Moab into a more enterprise-friendly offering drove much of the feature development of version 7.0, and led to a seamless integration between TORQUE (the open source resource manager) and Moab proper (Adaptive’s workload manager). Prior to version 7.0, the two components were on separate release schedules and were treated, more or less, as independent products.

Although TORQUE will still be maintained as an open-source project, bringing the resource manager under the Moab fold for the purpose of productization was just a logical move if they were going to drive deeper into the commercial realm. Whereas academics and national labs can throw grad students at tweaking TORQUE for their own purposes, businesses do not have that luxury. They expect shrink-wrapped software and a high level of usability.

To meet some of those particular needs, 7.0 added a dashboard to simplify the admin tasks like tracking running jobs and node status. For example, an administrator is now able to filter on categories like user name, job run-time, or node utilization to get particular snapshots of the system. Although all of this information was accessible before, a lot of it had to be dug out via command lines or custom-built scripts. In conjunction with the new dashboard, Moab has updated its user portal to simplify job submission and tracking.

The new suite also provides a single universal Web Services interface (in this case, the RESTful APIs) to integrate user portals, plug-ins, and scripts, which replaces the various low-level C, Java, and Perl APIs supported in the past. Now essentially any script or external package can be plugged into Moab, without regard to programming environment.

Accounting management has been spruced up too. System usage can be tracked (and controlled) with arbitrarily complex department hierarchies. This is most important for businesses that need to budget system time down to the penny, but also for research labs and universities that increasingly have to account for HPC resource allocations across their user base.

Moab 7.0 also adds a nifty job cancellation feature, whereby an array of jobs can be terminated once an answer is found. In this scenario, a bunch of jobs are submitted to ferret out a particular result, like a facial recognition match or a drug molecule match on a protein. Whichever job finds the answer first terminates with a special exit code that Moab recognizes as a signal to kill all associated jobs. The idea is to save time and resources that could be spent on other work waiting in the queue.

Despite the focus on commercial HPC, the new suite continues to serve the high end of the market, and in fact now has the capability to scale beyond any current supercomputer deployed today. Thanks to some rearchitecting in the latest TORQUE software (version 4.0), Moab is able to support systems with over 100 thousand nodes. Today, the number one ranked K system, at 10 petaflops, has 80,000 nodes, but 100K-plus-node configurations will almost certainly become commonplace at the top end over the next several years as double-digit, and then triple-digit, petaflop systems start to roll out.

Moab 7.0 can also manage over 10 thousand users and more than a million jobs — something apparently Adaptive’s customers have already been clamoring for. According to Harrington, Moab’s competitors, in many cases, can handle a large number of jobs, a large number of users, or a large number of nodes, but not all three.

Not everyone is going to be able to take advantage of those capabilities, but Clyde expects that even mainstream commercial clusters will eventually scale to the dimensions that Adaptive is targeting. As the demand for HPC continues to pump up system sizes, user numbers and job counts, Adaptive wants Moab to be ready. As Clyde puts it: “We want to skate to where the puck is going.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

PRACEdays Reflects Europe’s HPC Commitment

May 25, 2017

More than 250 attendees and participants came together for PRACEdays17 in Barcelona last week, part of the European HPC Summit Week 2017, held May 15-19 at t Read more…

By Tiffany Trader

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurr Read more…

By Doug Black

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

Nvidia CEO Predicts AI ‘Cambrian Explosion’

May 25, 2017

The processing power and cloud access to developer tools used to train machine-learning models are making artificial intelligence ubiquitous across computing pl Read more…

By George Leopold

HPE Extreme Performance Solutions

Exploring the Three Models of Remote Visualization

The explosion of data and advancement of digital technologies are dramatically changing the way many companies do business. With the help of high performance computing (HPC) solutions and data analytics platforms, manufacturers are developing products faster, healthcare providers are improving patient care, and energy companies are improving planning, exploration, and production. Read more…

PGAS Use will Rise on New H/W Trends, Says Reinders

May 25, 2017

If you have not already tried using PGAS, it is time to consider adding PGAS to the programming techniques you know. Partitioned Global Array Space, commonly kn Read more…

By James Reinders

Exascale Escapes 2018 Budget Axe; Rest of Science Suffers

May 23, 2017

President Trump's proposed $4.1 trillion FY 2018 budget is good for U.S. exascale computing development, but grim for the rest of science and technology spend Read more…

By Tiffany Trader

Hedge Funds (with Supercomputing help) Rank First Among Investors

May 22, 2017

In case you didn’t know, The Quants Run Wall Street Now, or so says a headline in today’s Wall Street Journal. Quant-run hedge funds now control the largest Read more…

By John Russell

IBM, D-Wave Report Quantum Computing Advances

May 18, 2017

IBM said this week it has built and tested a pair of quantum computing processors, including a prototype of a commercial version. That progress follows an an Read more…

By George Leopold

PRACEdays Reflects Europe’s HPC Commitment

May 25, 2017

More than 250 attendees and participants came together for PRACEdays17 in Barcelona last week, part of the European HPC Summit Week 2017, held May 15-19 at t Read more…

By Tiffany Trader

PGAS Use will Rise on New H/W Trends, Says Reinders

May 25, 2017

If you have not already tried using PGAS, it is time to consider adding PGAS to the programming techniques you know. Partitioned Global Array Space, commonly kn Read more…

By James Reinders

Exascale Escapes 2018 Budget Axe; Rest of Science Suffers

May 23, 2017

President Trump's proposed $4.1 trillion FY 2018 budget is good for U.S. exascale computing development, but grim for the rest of science and technology spend Read more…

By Tiffany Trader

Cray Offers Supercomputing as a Service, Targets Biotechs First

May 16, 2017

Leading supercomputer vendor Cray and datacenter/cloud provider the Markley Group today announced plans to jointly deliver supercomputing as a service. The init Read more…

By John Russell

HPE’s Memory-centric The Machine Coming into View, Opens ARMs to 3rd-party Developers

May 16, 2017

Announced three years ago, HPE’s The Machine is said to be the largest R&D program in the venerable company’s history, one that could be progressing tow Read more…

By Doug Black

What’s Up with Hyperion as It Transitions From IDC?

May 15, 2017

If you’re wondering what’s happening with Hyperion Research – formerly the IDC HPC group – apparently you are not alone, says Steve Conway, now senior V Read more…

By John Russell

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

HPE Launches Servers, Services, and Collaboration at GTC

May 10, 2017

Hewlett Packard Enterprise (HPE) today launched a new liquid cooled GPU-driven Apollo platform based on SGI ICE architecture, a new collaboration with NVIDIA, a Read more…

By John Russell

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

Google Pulls Back the Covers on Its First Machine Learning Chip

April 6, 2017

This week Google released a report detailing the design and performance characteristics of the Tensor Processing Unit (TPU), its custom ASIC for the inference Read more…

By Tiffany Trader

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

Since our first formal product releases of OSPRay and OpenSWR libraries in 2016, CPU-based Software Defined Visualization (SDVis) has achieved wide-spread adopt Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Last week, Google reported that its custom ASIC Tensor Processing Unit (TPU) was 15-30x faster for inferencing workloads than Nvidia's K80 GPU (see our coverage Read more…

By Tiffany Trader

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a ne Read more…

By Tiffany Trader

Leading Solution Providers

Facebook Open Sources Caffe2; Nvidia, Intel Rush to Optimize

April 18, 2017

From its F8 developer conference in San Jose, Calif., today, Facebook announced Caffe2, a new open-source, cross-platform framework for deep learning. Caffe2 is Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which w Read more…

By Tiffany Trader

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling Read more…

By Steve Campbell

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Eng Read more…

By Tiffany Trader

US Supercomputing Leaders Tackle the China Question

March 15, 2017

As China continues to prove its supercomputing mettle via the Top500 list and the forward march of its ambitious plans to stand up an exascale machine by 2020, Read more…

By Tiffany Trader

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu's Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural networ Read more…

By Tiffany Trader

DOE Supercomputer Achieves Record 45-Qubit Quantum Simulation

April 13, 2017

In order to simulate larger and larger quantum systems and usher in an age of "quantum supremacy," researchers are stretching the limits of today's most advance Read more…

By Tiffany Trader

Knights Landing Processor with Omni-Path Makes Cloud Debut

April 18, 2017

HPC cloud specialist Rescale is partnering with Intel and HPC resource provider R Systems to offer first-ever cloud access to Xeon Phi "Knights Landing" process Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This