When Former Vice President Al Gore won the Nobel Peace Prize in 2007 for his work on climate change, he shared the prize with 34 researchers on the Intergovernmental Panel for Climate Change, 11 of whom do their research at the University of California Irvine (UC Irvine), School of Physical Sciences. Three other researchers have received the Nobel Prize during their tenure at UC Irvine: Frank Sherwood Rowland (Chemistry, 1995), Frederick Reines (Physics, 1995) and Irwin Rose (Chemistry, 2004).

One of the 10 universities in the University of California system, UC Irvine prides itself on its distinguished history of scientific research.

Yet until a few years ago, UC Irvine did not have sufficient homegrown compute power to support its award-winning Nobel groups or its young and innovative researchers in such areas as atmospheric chemistry, bioinformatics, biopharmaceutical engineering and other disciplines that focus on chemistry and synthetic chemistry. Its researchers had to outsource their data to high-performance computing (HPC) labs such as the Fermi Lab in Chicago, the National Center for Atmospheric Research in Boulder, Colorado, the Stanford Linear Accelerator Center in Menlo Park, California and the San Diego Supercomputer Center.

“Our researchers were not in charge of their own destiny. They needed immediate access to large computational resources in order to quickly follow up on exciting results from both virtual and real-world experiments. The weeks to months of lag between applying for and possibly getting time at a large supercomputer center is a drag on the pace of research and outright kills many applications of scientific computing,” says Dr. Nathan R. M. Crawford, director of the Chemistry Modeling Facility and GreenPlanet HPCC, School of Physical Sciences, UC Irvine.

Seeking partners in progress

Each of the large research groups at UC Irvine got together with Ronald D. Hubbard, former executive director, GreenPlanet HPCC, School of Physical Sciences, UC Irvine, and they decided that UC Irvine needed an HPC solution. Hubbard recalls, “We put together a plan and we decided that the only way we could make it work was by having partners in progress, not just vendors.” Hubbard was also recently named a 2010 laureate by the ComputerWorld Honors program for his work on a case study outlining collaboration between UC Irvine’s dark particle physicists and Intel Corp. “Dell and Intel rose to the challenge through their CEOs, Michael Dell and Craig Barrett, who committed themselves to high-performance computing at UC Irvine. They put us in a position where we could get their advanced tools and their advanced products-not necessarily off-the-shelf products-by opening their engineering and design-build processes to us and ultimately allowing us to do work that we couldn’t have done without their help.”

UC Irvine’s HPC solution, GreenPlanet, started with 88 Dell PowerEdge 1950 servers and 14 PowerEdge 2950 servers with Intel Xeon processors. QLogic TrueScale InfiniBand interconnect, sourced through Dell, provided high thoughput, low latency connectivity among the nodes. Dell PowerVault MD1220 and MD1000 direct attached storage arrays delivered rack-dense, scalable storage.

Tapping federal grants

With the infrastructure in place, federal money followed.

“Due to our activity in the science community with the National Science Foundation, the National Institutes of Health, the Department of Energy, the Department of Defense and NASA, we’ve applied for and received grants to expand GreenPlanet over these past 29 months from 102 to over 300 nodes, with another 20 on order,” says Crawford. “We applied for 11 grants and got 10, with a dozen more in the pipeline.”

UC Irvine’s federal grants span a wide spectrum of scientific activities. A $20 million grant for UC Irvine’s Chemistry at the Space Time Limit (C@STL) project, will enable researchers to look at photographs of chemical reactions within cells for the very first time using high resolution video images. Other projects on the macro-end of the spectrum will gather data about the farthest novas in the universe.

“In between, we have really strong thrusts in atmospheric, synthetic, theoretical and pharmaceutical chemistry, and a great deal of climate, weather and geologic modeling that takes place from the top of the atmosphere via satellite down to deep underground,” says Crawford.

150% of the work in 50% of the space

The 350+ servers in the expanded cluster include Dell PowerEdge R710, R610 and R410 servers with Intel Xeon 5500 and 5600 series processors. “Currently, we are procuring Dell PowerEdge C6100 servers purpose-made by Dell with Intel Xeon 5600 series processors, Intel’s newest and most advanced processors,” says Crawford. “I am recommending it as the default for new node purchases.”

With four two-socket server nodes in a 2U rack chassis, the PowerEdge C6100 is a hyper-scale-inspired building block for HPC environments and cloud builders.

“Through the efforts of Dell and Intel, we’ve been able to accelerate our activities, plus we’re seeing great energy and space savings per amount of work done,” says Crawford. “In fact, with the advancements that Dell has embedded in the C6100 series servers running Intel Xeon X5600 series processors, we can get 150 percent of the work in 50 percent of the space, and this at a price that is less than the PowerEdge R610 nodes with X5500 series Xeons that we were buying a year ago. There is more than enough headroom in the QLogic TrueScale QDR InfiniBand Fabric to handle the 50 percent increase in message-passing traffic per node that the extra processor cores produce. The partnership between Dell, Intel and QLogic has definitely allowed us to increase the throughput of the large-scale, highly-parallel applications that our research requires.”

The Dell and Intel solution also saves 28 percent in energy consumption. “Anytime a university like ours searches for new partnerships, we’re going to need a lot of ingredients,” says Crawford. “What Dell has been able to bring through their partners are the roadmaps and push to do more dense and scalable research computing. Without that, GreenPlanet would be a collection of disparate, expensive boxes, and a headache to expand and maintain. It’s through the Dell process, in close coordination with the processes of their partners, that we’ve been able to be so successful in a very short period of time. GreenPlanet was designed to be an enabler of new discoveries. In order to make that design work, it takes a lot of people, a lot of process and a lot of production. Dell, Intel and QLogic have allowed all that to happen.”

Simplifying acquisition and deployment

In addition to opening up its design and build processes to enable UC Irvine to obtain the advanced tools that it needed to do more science in less time, Dell worked with Intel to bring UC Irvine into Intel’s Cluster Ready Program. Intel Cluster Ready certification includes comprehensive testing and integration work, enabling Dell HPC clusters with Intel Cluster Ready certification to simplify acquisition and deployment.

UC Irvine also took advantage of the Dell Hardware Customization service that integrates hardware, images, applications, peripherals and documents with systems as they’re being built.

“With the integration that Dell does, and getting the product pre-assembled and put into our cluster, we have become more efficient and proficient in the way we monitor and manage on a daily basis,” says Crawford. “GreenPlanet HPC is powered by Linux. We run multiple applications. We run very diverse software across more than 30 different professors and their research groups, with many simulations that take days to or weeks to run. Through Dell’s and Intel’s expertise, we’ve been able to integrate all these into the cluster by using the Intel Cluster Ready Program and Dell Hardware Customization services. This has helped us to monitor and manage all these divergent activities on an open platform.”

Faster cures for diseases

With the acceleration of science that UC Irvine is achieving with its GreenPlanet cluster, the result could be faster cures for diseases known as neuro degenerative diseases, such as ALS, Multiple Sclerosis, Alzheimer’s, Parkinson’s, Huntington’s Disease, and cell degeneration, which are caused by the disturbed function of ion channel subunits or the proteins that regulate them.

Douglas Tobias, professor of chemistry at UC Irvine, is working on the computational chemistry that simulates molecular scale details of how proteins in voltage-gated ion channels open and close in response to changes in voltage. This is the basis of what’s called the action potential, which is the electrical signal in muscles and nerves, and excitable tissues in general.

“When I started my research, I was very happy to be to be doing cutting-edge research by simulating approximately 800 atoms,” says Tobias. “And now with GreenPlanet we’re doing routinely a quarter of a million atoms and for two or three orders of magnitude longer in time. The type of system that we can study has been completely changed. We’ve gone from small molecules to large assemblies of macro-molecules. This opens up the landscape of the type of questions that you can ask. The more we can sample the underlying distribution or the dynamics, the more confident we can be in the results. And sometimes just running for a longer time lets you see things that you didn’t see before, and couldn’t have imagined seeing before. It’s really transformed our research.”

Developing new drugs

Rommie Amaro, assistant professor of pharmaceutical science and computer science, is studying a variety of infectious diseases. “In my lab we are using different computational simulations to try to understand how enzymes and biomolecules within these diseases behave and how we can come up with new drugs to target these specific enzymes,” she says. “We study systems with hundreds of thousands of atoms and simulate their behavior. So, it’s very important for us to have access to a large number of processors that have very fast interconnect. That allows us to get results much more quickly. And the great thing about GreenPlanet is it satisfies those constraints for us.”

“We’re so happy and so excited to have Dr. Rommie Amaro here, who is doing research at the leading edge of bio-synthetic research,” says Crawford. “We are very hopeful that those discoveries will lead to compounds and groups of compounds that will be converted into new medicines for new applications in the very near future.”

Learn how Dell’s energy-efficient power and cooling solutions can save capital, operating costs and most importantly, maximize performance per watt.

To see how Dell is helping to enable research discoveries visit DellHPCSolutions.com