What I Learned at NGDC: Technology is Ready, Users are Not

By Derrick Harris

August 13, 2007

If I had to boil what I observed at last week’s Next Generation Data Center conference into one thought, it would be this: Although technologies to virtualize, optimize and automate datacenters do exist and are mature, organizations are still leery about making the transformation, despite coveting the associated improvements in performance, flexibility and overall efficiency.

As evidence that these technologies are mature, one really need look no further than the event’s opening keynotes, in which Amazon CTO Werner Vogels and eBay distinguished research scientist Paul Strong discussed the Web-scale datacenters being operated by their respective employers. Strong, for his part, delved a little deeper into the nitty gritty details (see here and here for more on this), all the while, however, stressing the importance of building a datacenter that: (1) runs processes driven by SLAs; (2) operates as a value center rather than a cost center; and (3) enables the rolling out of new utilities, platforms, etc. To achieve this next-generation datacenter, he said, many technologies can and should be considered, including (but certainly not limited to) grid computing, utility computing, real-time solutions and virtualization.

Now, it’s unlikely that most organizations have the resources (or the need — said Strong, “If we don’t keep up, our business is gone.”) to develop and/or manage a datacenter like eBay’s — a highly automated and virtualized environment consisting of several thousand blades servers — but that doesn’t mean companies with less demand on their infrastructures can’t learn some lessons from the online auction leader. For starters, said Strong, automation is the key to efficiency and, in point he made sure to drive home, it is important to “manage relationships, not things.” This advice should be particularly relevant as today’s average datacenters continue to evolve toward advanced models like that of eBay. After all, when you’re staring down thousands of physical machines (and likely significantly more virtual ones) you can’t possibly expect to manage each one individually.

As for Vogels, whose presentation kicked off the doubleheader, he targeted his comments toward those companies who aren’t too keen on managing their own resources, and he used the opportunity to push Amazon’s stable of Web services. For most companies, he estimates, 70 percent of time and money expenditures go toward “heavy lifting” operations like maintenance, load balancing or software management, among others, none of which offer much in terms of innovation or helping differentiate your company from competitors. Unless you’re in an industry where having a customized, highly efficient datacenter directly translates into dollars, he suggested, it might be a waste of resources on all levels. “At [Amazon’s] scale, datacenters matter,” he stated. “They don’t for everyone.”

Following his statement that “I hate datacenters,” Vogels cited the recent power outage at a popular San Francisco datacenter — which led to temporary shutdowns of several Web 2.0 leaders, including Craigslist, Second Life and Netflix — as one example of what can go wrong. Power-wise, he added, even if you have generators to ensure you keep running, you still need batteries to handle the gap in time between the power going out and the generators kicking in. Still in the realm of possible physical issues, Vogels noted that datacenter managers still need to worry about issues such as sufficient cooling and how to handle a fire or other disasters. And that’s not even addressing business-side concerns, such as whether one datacenter is enough, or how you’re going to push out enough bandwidth to handle demand. Often times, he said, companies need to overprovision to handle peak loads or in case they become successful.

His solution? Utilize services such as Amazon’s Elastic Compute Cloud (EC2), as well its other Web services offerings, to handle your computing needs, paying only for what you actually use. Vogels talked about this notion as the “push versus pull” model of resource management, where “pushing” refers to the old-style method of preemptively pushing resources toward problems and “pulling” refers to the more progressive concept of pulling in resources from a centralized pool as needed — just like Amazon itself does. For organizations that don’t necessarily benefit from managing their own datacenters, he said, this utility model will allow them to get the computing resources they need elsewhere, thus freeing up time and money to spend on innovation.

However, while Vogels’ solution to datacenter woes might sound logical and easy enough to incorporate, I wouldn’t hold my breath waiting for this idea to gain mainstream acceptance, much less adoption. Why, you ask? Because organizations are having a difficult enough time taking advantage of next-generation tools within their own walls — something far less scary than the notion of relying on someone else to make sure their applications get the attention they deserve. This was made abundantly clear during a presentation by OGF President Mark Linesch, who was simply trying to lay out the business case for and current status of grid computing technologies.

During Linesch’s presentation, an audience member (who actually has some firsthand grid experience under his belt) asked how he is supposed to sell the idea of grid or virtualization to his applications developers, some of whom still oppose running their applications on distributed or virtualized platforms. (In fact, this gentleman just denied a request for 60-plus servers to develop and test a new application, instead preferring the work to be done on existing, virtually partitioned machines.) Linesch, backed up by Ravi Subramaniam, who has plenty of insight to offer after years of managing Intel’s in-house grid, gave really the only answer one can give in this situation, regardless of how frustrating it might be to someone desperately seeking a cost-efficient, dynamic IT platform for his/her organization: You have to start slow, showing success in one area at a time — perhaps with just one application — and illustrate how that translates into other areas. Not exactly the best way to show off a technology’s full range of capabilities, but not exactly uncommon, either.

This point was hammered home when I sat down to speak with Jay Fry and Ken Oestreich of Cassatt, vice president of marketing and director of product management and marketing, respectively. With its capabilities in areas like capacity on-demand, application virtualization, service level automation, utility metering, etc., Cassatt’s Collage software certainly falls under the “next-generation” umbrella, but customers aren’t always ready to experience it in full force from the get-go. In fact, customers have been known to ask for a pared down version of Collage, something Cassatt might have to do in order to show them — one step at a time — that its software is for real.

I was happy to hear, though, that Cassatt is making inroads on another front: the battle to ease customers’ minds about the cultural and organization changes that come along with the technical changes of a shared IT platform. Gone are the siloed applications and their siloed personnel. Gone are the days of server hugging. Gone are the days of low utilization and high overhead. While these all sound like great things, that kind of change apparently can be quite foreboding for IT departments, which is why many are hesitant to cross over into the promised land. Well, Cassatt and consultancy partner BearingPoint have been walking customers through this process, which they believe needs to be done in parallel, in their New York-based customer experience center, and the reaction has been very positive thus far. You can read more about the Cassatt/BearingPoint partnership here, and you can expect to see more about Cassatt’s take on utility computing in the weeks to come.

Speaking of application virtualization and its associated functionalities, the topic came up in a panel discussion featuring three distinct virtualization users and, wouldn’t you know, it seems to be a little much for them at this point. When the topic of “virtualization 2.0” was brought up, which was defined as including the grid-like abilities (e.g., high availability, SLA management, scalability, etc.) often associated with application virtualization solutions, the response was not overly positive. While Brian Harris, president and founder of Virtual Ngenuity, stated that he believes these functionalities are currently driving business decisions, two of his fellow panelists showed that this might not be entirely the case. Richard Robinson, chief operations officer for Department of Telecommunications and Information Services, City and County of San Francisco, commented that while his department is working toward these goals, they are not there yet and might not be for quite a while. After all, he noted, the “if it’s not broken, don’t fix it” axiom carries a lot of weight in the local government sector. Sudip Chahal, a senior architect in Intel’s IT Strategy, Architecture and Innovation organization, espoused his belief that this type of architecture isn’t well-suited for traditional business applications — a view not shared by audience member Dave Pearson of Oracle or, I would assume, most of the distributed IT community.

Of course, there was more going on at LinuxWorld/NGDC than just vendors showing off to and discussing with skeptical end-users their cutting-edge technologies. For example, some cool news also came out of the show, such as Appistry adding power-saving functionality to its Enterprise Application Fabric; ServePath announcing high-performance hosting via its virtualized GoGrid service; EnterpriseDB challenging Oracle with GridSQL; and IBM tackling your mountains of widely dispersed data with its grid- and virtualization-powered Information Server Blade, which Big Blue says has demonstrated significant improvements in batch process performance, hardware price performance and budget expenditure. If you’re still hungry for more after reading these announcements, don’t fret, as we’ll have more on all of them — as well as a look at the growing grid hosting business — in the weeks and months to come.

Outside of NGDC news, be sure to check these very noteworthy items: “GigaSpaces Powers Sun’s Market Data Solution”; “NCAR Adds Resources to TeraGrid”; “Imense Using Grid to Become ‘Google of Image Searching’”; “Trigence Intros Optimized App Virtualization Software”; “Sun Releases Fastest Commodity Microprocessor”; and “Layered Tech Announces Super Grid.”

—–

Comments about GRIDtoday are welcomed and encouraged. Write to me, Derrick Harris, at editor@gridtoday.com.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

ExxonMobil, NCSA, Cray Scale Reservoir Simulation to 700,000+ Processors

February 17, 2017

In a scaling breakthrough for oil and gas discovery, ExxonMobil geoscientists report they have harnessed the power of 717,000 processors – the equivalent of 22,000 32-processor computers – to run complex oil and gas reservoir simulation models. Read more…

By Doug Black

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a new board design for NVLink-equipped Pascal P100 GPUs that will create another entrant to the space currently occupied by Nvidia's DGX-1 system, IBM's "Minsky" platform and the Supermicro SuperServer (1028GQ-TXR). Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

Drug Developers Use Google Cloud HPC in the Fight Against ALS

February 16, 2017

Within the haystack of a lethal disease such as ALS (amyotrophic lateral sclerosis / Lou Gehrig’s Disease) there exists, somewhere, the needle that will pierce this therapy-resistant affliction. Read more…

By Doug Black

HPE Extreme Performance Solutions

Object Storage is the Ideal Storage Method for CME Companies

The communications, media, and entertainment (CME) sector is experiencing a massive paradigm shift driven by rising data volumes and the demand for high-performance data analytics. Read more…

Weekly Twitter Roundup (Feb. 16, 2017)

February 16, 2017

Here at HPCwire, we aim to keep the HPC community apprised of the most relevant and interesting news items that get tweeted throughout the week. Read more…

By Thomas Ayres

Alexander Named Dep. Dir. of Brookhaven Computational Initiative

February 15, 2017

Francis Alexander, a physicist with extensive management and leadership experience in computational science research, has been named Deputy Director of the Computational Science Initiative at the U.S. Read more…

Here’s What a Neural Net Looks Like On the Inside

February 15, 2017

Ever wonder what the inside of a machine learning model looks like? Today Graphcore released fascinating images that show how the computational graph concept maps to a new graph processor and graph programming framework it’s creating. Read more…

By Alex Woodie

Azure Edges AWS in Linpack Benchmark Study

February 15, 2017

The “when will clouds be ready for HPC” question has ebbed and flowed for years. Read more…

By John Russell

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a new board design for NVLink-equipped Pascal P100 GPUs that will create another entrant to the space currently occupied by Nvidia's DGX-1 system, IBM's "Minsky" platform and the Supermicro SuperServer (1028GQ-TXR). Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

Drug Developers Use Google Cloud HPC in the Fight Against ALS

February 16, 2017

Within the haystack of a lethal disease such as ALS (amyotrophic lateral sclerosis / Lou Gehrig’s Disease) there exists, somewhere, the needle that will pierce this therapy-resistant affliction. Read more…

By Doug Black

Azure Edges AWS in Linpack Benchmark Study

February 15, 2017

The “when will clouds be ready for HPC” question has ebbed and flowed for years. Read more…

By John Russell

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling was pretty much the exclusive realm of the Cray-2 and IBM mainframe class products. That’s changing. We are now seeing an emergence of x86 class server products with exotic plumbing technology ranging from Direct-to-Chip to servers and storage completely immersed in a dielectric fluid. Read more…

By Steve Campbell

Cray Posts Best-Ever Quarter, Visibility Still Limited

February 10, 2017

On its Wednesday earnings call, Cray announced the largest revenue quarter in the company’s history and the second-highest revenue year. Read more…

By Tiffany Trader

HPC Cloud Startup Launches ‘App Store’ for HPC Workflows

February 9, 2017

“Civilization advances by extending the number of important operations which we can perform without thinking about them,” Read more…

By Tiffany Trader

Intel and Trump Announce $7B for Fab 42 Targeting 7nm

February 8, 2017

In what may be an attempt by President Trump to reset his turbulent relationship with the high tech industry, he and Intel CEO Brian Krzanich today announced plans to invest more than $7 billion to complete Fab 42. Read more…

By John Russell

For IBM/OpenPOWER: Success in 2017 = (Volume) Sales

January 11, 2017

To a large degree IBM and the OpenPOWER Foundation have done what they said they would – assembling a substantial and growing ecosystem and bringing Power-based products to market, all in about three years. Read more…

By John Russell

US, China Vie for Supercomputing Supremacy

November 14, 2016

The 48th edition of the TOP500 list is fresh off the presses and while there is no new number one system, as previously teased by China, there are a number of notable entrants from the US and around the world and significant trends to report on. Read more…

By Tiffany Trader

Lighting up Aurora: Behind the Scenes at the Creation of the DOE’s Upcoming 200 Petaflops Supercomputer

December 1, 2016

In April 2015, U.S. Department of Energy Undersecretary Franklin Orr announced that Intel would be the prime contractor for Aurora: Read more…

By Jan Rowell

D-Wave SC16 Update: What’s Bo Ewald Saying These Days

November 18, 2016

Tucked in a back section of the SC16 exhibit hall, quantum computing pioneer D-Wave has been talking up its new 2000-qubit processor announced in September. Forget for a moment the criticism sometimes aimed at D-Wave. This small Canadian company has sold several machines including, for example, ones to Lockheed and NASA, and has worked with Google on mapping machine learning problems to quantum computing. In July Los Alamos National Laboratory took possession of a 1000-quibit D-Wave 2X system that LANL ordered a year ago around the time of SC15. Read more…

By John Russell

Enlisting Deep Learning in the War on Cancer

December 7, 2016

Sometime in Q2 2017 the first ‘results’ of the Joint Design of Advanced Computing Solutions for Cancer (JDACS4C) will become publicly available according to Rick Stevens. He leads one of three JDACS4C pilot projects pressing deep learning (DL) into service in the War on Cancer. Read more…

By John Russell

HPC Startup Advances Auto-Parallelization’s Promise

January 23, 2017

The shift from single core to multicore hardware has made finding parallelism in codes more important than ever, but that hasn’t made the task of parallel programming any easier. Read more…

By Tiffany Trader

IBM Wants to be “Red Hat” of Deep Learning

January 26, 2017

IBM today announced the addition of TensorFlow and Chainer deep learning frameworks to its PowerAI suite of deep learning tools, which already includes popular offerings such as Caffe, Theano, and Torch. Read more…

By John Russell

CPU Benchmarking: Haswell Versus POWER8

June 2, 2015

With OpenPOWER activity ramping up and IBM’s prominent role in the upcoming DOE machines Summit and Sierra, it’s a good time to look at how the IBM POWER CPU stacks up against the x86 Xeon Haswell CPU from Intel. Read more…

By Tiffany Trader

Leading Solution Providers

Nvidia Sees Bright Future for AI Supercomputing

November 23, 2016

Graphics chipmaker Nvidia made a strong showing at SC16 in Salt Lake City last week. Read more…

By Tiffany Trader

BioTeam’s Berman Charts 2017 HPC Trends in Life Sciences

January 4, 2017

Twenty years ago high performance computing was nearly absent from life sciences. Today it’s used throughout life sciences and biomedical research. Genomics and the data deluge from modern lab instruments are the main drivers, but so is the longer-term desire to perform predictive simulation in support of Precision Medicine (PM). There’s even a specialized life sciences supercomputer, ‘Anton’ from D.E. Shaw Research, and the Pittsburgh Supercomputing Center is standing up its second Anton 2 and actively soliciting project proposals. There’s a lot going on. Read more…

By John Russell

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

Dell Knights Landing Machine Sets New STAC Records

November 2, 2016

The Securities Technology Analysis Center, commonly known as STAC, has released a new report characterizing the performance of the Knight Landing-based Dell PowerEdge C6320p server on the STAC-A2 benchmarking suite, widely used by the financial services industry to test and evaluate computing platforms. The Dell machine has set new records for both the baseline Greeks benchmark and the large Greeks benchmark. Read more…

By Tiffany Trader

IDG to Be Bought by Chinese Investors; IDC to Spin Out HPC Group

January 19, 2017

US-based publishing and investment firm International Data Group, Inc. (IDG) will be acquired by a pair of Chinese investors, China Oceanwide Holdings Group Co., Ltd. Read more…

By Tiffany Trader

What Knights Landing Is Not

June 18, 2016

As we get ready to launch the newest member of the Intel Xeon Phi family, code named Knights Landing, it is natural that there be some questions and potentially some confusion. Read more…

By James Reinders, Intel

KNUPATH Hermosa-based Commercial Boards Expected in Q1 2017

December 15, 2016

Last June tech start-up KnuEdge emerged from stealth mode to begin spreading the word about its new processor and fabric technology that’s been roughly a decade in the making. Read more…

By John Russell

Intel and Trump Announce $7B for Fab 42 Targeting 7nm

February 8, 2017

In what may be an attempt by President Trump to reset his turbulent relationship with the high tech industry, he and Intel CEO Brian Krzanich today announced plans to invest more than $7 billion to complete Fab 42. Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Share This