At the Nexus of Grid, Cloud and HPC

By Dennis Barker

October 17, 2008

What’s the big difference between cloud computing and grid computing? The goal of cloud computing is to put system administrators out of work.

That’s one way of looking at it, at least. Steve Armentrout, CEO of Parabon Computation, says that was the perspective tossed out by a couple of Google and IBM reps at a panel discussion in which he recently participated. Armentrout suggests a less Dickensian way of looking at it: cloud computing is about “providing a datacenter that is fully automated.” (More on cloud versus grid later.)

Armentrout sees cloud and grid as complementary in some ways — bipartisan, you might say — but he is an unapologetic grid partisan — especially when it comes to his company’s collection of solutions. “We have no intention of changing our grid stripes,” he says. “What Parabon provides is grid software as a service. We enable individuals with grid applications to scale them across a large infrastructure without having to go out and buy hardware. They can just buy capacity as it’s needed. It’s a pay-as-you-go model.”

Basically, Parabon’s Frontier Grid Services offering is a high-performance computing utility. If you need a few thousand nodes to run a financial risk model or some other long and winding analysis, Parabon will hook you up to the resources you need. “We broker computation,” Armentrout says. Like its customers, the company doesn’t own datacenters. What it has is contracts with universities and institutions with big server farms and HPC clusters to aggregate their unused capacity. “All that compute power we use to provide computation on demand,” explains Armentrout.

There’s a lot of computational capability sitting around doing nothing, Armentrout says. “You often hear the estimate that standard servers are typically running at anywhere from 5 to 20 percent capacity. Just think of 80 percent capacity going to waste. Even in a virtualized environment, seldom do you see capacity usage at over 50 percent. All that idle capacity allows us to deploy across a university datacenter, for example, and execute large-scale jobs in the background. Frontier is our technology that lets us capture that unused capacity and make it available as a grid service.”

Parabon’s technology can be used, as just described, across worldwide “public” resources like campus networks — that’s the Parabon Computation Grid — but can also be applied to a company’s own network as the Frontier Enterprise Grid.
 
Parabon built its platform around the Frontier Grid Server, which provides grid services and shared resources to users and developers, whether using the Internet-based Parabon Computation Grid or an in-house Frontier Enterprise grid. The Frontier Grid Server manages execution of jobs across hundreds or thousands of compute nodes. “It can scale up to arbitrarily large grids,” Armentrout says. “Tens of thousands of machines.” Frontier always reserves excess capacity to handle unexpected scale-out demands, he says.

The Frontier Compute Engine is the agnostic agent application that runs on each grid node to actually do the work. It executes tasks only when the resource, the virtual machine in many cases, is not handling a primary task. “Frontier runs as a low-priority process,” Armentrout says, “so if running in a virtualized datacenter — a cloud, you could say — the Compute Engine backs off if a request comes in from the cloud application. It takes precedence. But when resources are not busy, we can fully saturate the datacenter during that unused period of time.”

For example, Parabon might have an arrangement with a research facility in Australia to use its cluster when the scientists are home at night. That could be prime work time for scientists on the other side of the globe. That’s when Frontier could saturate compute nodes to calculate solutions more quickly.

Parabon just released a browser-based interface called the Dashboard that provides an intuitive front-end to the Frontier Grid Platform. “It lets you easily monitor a job, kill a job, assign resources, plus some back-office and accounting functions like looking up how much you’re paying for use,” Armentrout says.

Parabon’s pricing structure is better explained by the company, but the basic idea is that customers pay for units of computational power using a formula that involves kilo-cap hours.

The company provides an API and suite of tools to simplify adapting applications to take advantage of Frontier grid capabilities. And there’s a collection of Frontier-ready programs for applications, including data mining and biological modeling. “It’s kind of like Apple’s App Store but for distributed applications,” Armentrout analogizes. 

Parabon has been around since 2000, when it introduced “the first commercial grid,” Armentrout says. Customers include not just scientific researchers, but also financial analysts, commercial enterprises with high-end analytical demands, bioinformatics, traditional HPC users and government agencies. “Our customers are doing modeling and simulation with very large models, immense data sets,” he explains. “We enable them to run not just one complex scenario but 10,000 scenarios. With Frontier you can explore an entire space of possibilities at once instead of running one simulation, then another, then another.”

Grid vs. Cloud: Parabon-Style

“In terms of grid vs. cloud, there’s lots of confusion around those two terms,” Armentrout says. “But, honestly, the fact that cloud has so much hype surrounding it now makes it easier for us to clarify to customers the benefits of grid computing. Grid, I think, is becoming clearer in people’s minds, while cloud is still, if I might say it, a ‘cloudy’ term.”

There are certainly commonalities, he says: computational utility, virtualized use of computing resources, eliminating the need for dedicated resources and dramatically improved price/performance. “But cloud computing is more about auto-provisioning virtual machines,” explains Armentrout. “It’s about software that lets you go out into a cloud infrastructure, a virtualized datacenter, and say give me one or two VMs and get them in an automated and orderly way. It’s about a datacenter that is completely automated. Sure, customers can scale up and down — that’s one of the benefits of the model — but they typically don’t scale in large-scale numbers. That’s the nature of most Web applications, which is typically what runs in the cloud. In that environment, you still have a lot of capacity that’s available.” 

On the other hand, he believes that grid computing is all about massive parallelization and running large-scale jobs on unused capacity rather than dedicated capacity. The goal is to accelerate large jobs from days to minutes and hours to seconds, and grid computing can enable computations that “just aren’t possible,” he says.

“The folks we’re talking to understand they need grid-scale compute capacity, and that’s not something they’ll get from a pure cloud approach,” Armentrout says. “We routinely run jobs on several thousand machines. It’s that mass parallelization that you just wouldn’t run in the cloud. You want a job done in 5 minutes, not days. Our grid service reaches out to thousands and thousands of boxes and returns an answer in minutes.”

“We’ve got a high-performance solution that works for our customers. We can take advantage of a cloud infrastructure, but we don’t need to chase the cloud phenomenon.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Weekly Twitter Roundup (Feb. 23, 2017)

February 23, 2017

Here at HPCwire, we aim to keep the HPC community apprised of the most relevant and interesting news items that get tweeted throughout the week. Read more…

By Thomas Ayres

HPE Server Shows Low Latency on STAC-N1 Test

February 22, 2017

The performance of trade and match servers can be a critical differentiator for financial trading houses. Read more…

By John Russell

HPC Financial Update (Feb. 2017)

February 22, 2017

In this recurring feature, we’ll provide you with financial highlights from companies in the HPC industry. Check back in regularly for an updated list with the most pertinent fiscal information. Read more…

By Thomas Ayres

Rethinking HPC Platforms for ‘Second Gen’ Applications

February 22, 2017

Just what constitutes HPC and how best to support it is a keen topic currently. Read more…

By John Russell

HPE Extreme Performance Solutions

O&G Companies Create Value with High Performance Remote Visualization

Today’s oil and gas (O&G) companies are striving to process datasets that have become not only tremendously large, but extremely complex. And the larger that data becomes, the harder it is to move and analyze it – particularly with a workforce that could be distributed between drilling sites, offshore rigs, and remote offices. Read more…

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu’s Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural network training and now they are sharing their implementation with the larger deep learning community. Read more…

By Tiffany Trader

IDC: Will the Real Exascale Race Please Stand Up?

February 21, 2017

So the exascale race is on. And lots of organizations are in the pack. Government announcements from the US, China, India, Japan, and the EU indicate that they are working hard to make it happen – some sooner, some later. Read more…

By Bob Sorensen, IDC

ExxonMobil, NCSA, Cray Scale Reservoir Simulation to 700,000+ Processors

February 17, 2017

In a scaling breakthrough for oil and gas discovery, ExxonMobil geoscientists report they have harnessed the power of 717,000 processors – the equivalent of 22,000 32-processor computers – to run complex oil and gas reservoir simulation models. Read more…

By Doug Black

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a new board design for NVLink-equipped Pascal P100 GPUs that will create another entrant to the space currently occupied by Nvidia's DGX-1 system, IBM's "Minsky" platform and the Supermicro SuperServer (1028GQ-TXR). Read more…

By Tiffany Trader

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu’s Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural network training and now they are sharing their implementation with the larger deep learning community. Read more…

By Tiffany Trader

IDC: Will the Real Exascale Race Please Stand Up?

February 21, 2017

So the exascale race is on. And lots of organizations are in the pack. Government announcements from the US, China, India, Japan, and the EU indicate that they are working hard to make it happen – some sooner, some later. Read more…

By Bob Sorensen, IDC

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a new board design for NVLink-equipped Pascal P100 GPUs that will create another entrant to the space currently occupied by Nvidia's DGX-1 system, IBM's "Minsky" platform and the Supermicro SuperServer (1028GQ-TXR). Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

Drug Developers Use Google Cloud HPC in the Fight Against ALS

February 16, 2017

Within the haystack of a lethal disease such as ALS (amyotrophic lateral sclerosis / Lou Gehrig’s Disease) there exists, somewhere, the needle that will pierce this therapy-resistant affliction. Read more…

By Doug Black

Azure Edges AWS in Linpack Benchmark Study

February 15, 2017

The “when will clouds be ready for HPC” question has ebbed and flowed for years. Read more…

By John Russell

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling was pretty much the exclusive realm of the Cray-2 and IBM mainframe class products. That’s changing. We are now seeing an emergence of x86 class server products with exotic plumbing technology ranging from Direct-to-Chip to servers and storage completely immersed in a dielectric fluid. Read more…

By Steve Campbell

Cray Posts Best-Ever Quarter, Visibility Still Limited

February 10, 2017

On its Wednesday earnings call, Cray announced the largest revenue quarter in the company’s history and the second-highest revenue year. Read more…

By Tiffany Trader

For IBM/OpenPOWER: Success in 2017 = (Volume) Sales

January 11, 2017

To a large degree IBM and the OpenPOWER Foundation have done what they said they would – assembling a substantial and growing ecosystem and bringing Power-based products to market, all in about three years. Read more…

By John Russell

US, China Vie for Supercomputing Supremacy

November 14, 2016

The 48th edition of the TOP500 list is fresh off the presses and while there is no new number one system, as previously teased by China, there are a number of notable entrants from the US and around the world and significant trends to report on. Read more…

By Tiffany Trader

Lighting up Aurora: Behind the Scenes at the Creation of the DOE’s Upcoming 200 Petaflops Supercomputer

December 1, 2016

In April 2015, U.S. Department of Energy Undersecretary Franklin Orr announced that Intel would be the prime contractor for Aurora: Read more…

By Jan Rowell

Enlisting Deep Learning in the War on Cancer

December 7, 2016

Sometime in Q2 2017 the first ‘results’ of the Joint Design of Advanced Computing Solutions for Cancer (JDACS4C) will become publicly available according to Rick Stevens. He leads one of three JDACS4C pilot projects pressing deep learning (DL) into service in the War on Cancer. Read more…

By John Russell

D-Wave SC16 Update: What’s Bo Ewald Saying These Days

November 18, 2016

Tucked in a back section of the SC16 exhibit hall, quantum computing pioneer D-Wave has been talking up its new 2000-qubit processor announced in September. Forget for a moment the criticism sometimes aimed at D-Wave. This small Canadian company has sold several machines including, for example, ones to Lockheed and NASA, and has worked with Google on mapping machine learning problems to quantum computing. In July Los Alamos National Laboratory took possession of a 1000-quibit D-Wave 2X system that LANL ordered a year ago around the time of SC15. Read more…

By John Russell

IBM Wants to be “Red Hat” of Deep Learning

January 26, 2017

IBM today announced the addition of TensorFlow and Chainer deep learning frameworks to its PowerAI suite of deep learning tools, which already includes popular offerings such as Caffe, Theano, and Torch. Read more…

By John Russell

HPC Startup Advances Auto-Parallelization’s Promise

January 23, 2017

The shift from single core to multicore hardware has made finding parallelism in codes more important than ever, but that hasn’t made the task of parallel programming any easier. Read more…

By Tiffany Trader

CPU Benchmarking: Haswell Versus POWER8

June 2, 2015

With OpenPOWER activity ramping up and IBM’s prominent role in the upcoming DOE machines Summit and Sierra, it’s a good time to look at how the IBM POWER CPU stacks up against the x86 Xeon Haswell CPU from Intel. Read more…

By Tiffany Trader

Leading Solution Providers

Nvidia Sees Bright Future for AI Supercomputing

November 23, 2016

Graphics chipmaker Nvidia made a strong showing at SC16 in Salt Lake City last week. Read more…

By Tiffany Trader

BioTeam’s Berman Charts 2017 HPC Trends in Life Sciences

January 4, 2017

Twenty years ago high performance computing was nearly absent from life sciences. Today it’s used throughout life sciences and biomedical research. Genomics and the data deluge from modern lab instruments are the main drivers, but so is the longer-term desire to perform predictive simulation in support of Precision Medicine (PM). There’s even a specialized life sciences supercomputer, ‘Anton’ from D.E. Shaw Research, and the Pittsburgh Supercomputing Center is standing up its second Anton 2 and actively soliciting project proposals. There’s a lot going on. Read more…

By John Russell

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

IDG to Be Bought by Chinese Investors; IDC to Spin Out HPC Group

January 19, 2017

US-based publishing and investment firm International Data Group, Inc. (IDG) will be acquired by a pair of Chinese investors, China Oceanwide Holdings Group Co., Ltd. Read more…

By Tiffany Trader

Dell Knights Landing Machine Sets New STAC Records

November 2, 2016

The Securities Technology Analysis Center, commonly known as STAC, has released a new report characterizing the performance of the Knight Landing-based Dell PowerEdge C6320p server on the STAC-A2 benchmarking suite, widely used by the financial services industry to test and evaluate computing platforms. The Dell machine has set new records for both the baseline Greeks benchmark and the large Greeks benchmark. Read more…

By Tiffany Trader

What Knights Landing Is Not

June 18, 2016

As we get ready to launch the newest member of the Intel Xeon Phi family, code named Knights Landing, it is natural that there be some questions and potentially some confusion. Read more…

By James Reinders, Intel

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling was pretty much the exclusive realm of the Cray-2 and IBM mainframe class products. That’s changing. We are now seeing an emergence of x86 class server products with exotic plumbing technology ranging from Direct-to-Chip to servers and storage completely immersed in a dielectric fluid. Read more…

By Steve Campbell

KNUPATH Hermosa-based Commercial Boards Expected in Q1 2017

December 15, 2016

Last June tech start-up KnuEdge emerged from stealth mode to begin spreading the word about its new processor and fabric technology that’s been roughly a decade in the making. Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Share This