Visit additional Tabor Communication Publications
December 12, 2011
At Microsoft® Bing™ Maps, technology imitates art. The Bing Imagery Technologies team takes diverse satellite, aerial, bird’s eye, and street-side imagery and stitch them together to form visual mosaics for Bing Maps.
Given its compute- and data-intensive mission, and the huge scale of its operations, the team requires enormous amounts of processing power, data storage capacity, and memory. It measures its storage needs in petabytes, its memory needs in terabytes, and its processing needs in tens of thousands of cores. At the same time, the team must work continuously to drive down the cost of processing this data.
When it came time to launch a new imagery-processing site at the edge of the Colorado Rockies, the Bing Imagery Technologies team (BITs) studied its options carefully. It wanted to find a no-frills data center infrastructure that was highly efficient, highly reliable, extremely scalable, and quick to deploy. It found its answer in Dell™ Modular Data Centers, which are helping the team build a highly efficient, state of the art imagery processing microsite.
The route to the Modular Data Center
The team’s road to a more efficient microsite began when the organization was making plans to move its legacy containerized data-processing infrastructure to a new site that could accommodate additional capacity and ongoing growth.
The team found a workable, eight-acre patch of ground in Longmont, Colorado, northwest of Denver. The requirements for the microsite were aggressive. Bing Maps needed an extremely cost-effective, highly efficient solution to power high-performance compute-and storage-intensive geospatial applications running a customized parallel processing system built on Windows Server 2008 and SQL Server 2008. The solution needed to be up to the challenges that come with processing and storing terabytes of data on an hourly basis.
“We produce hundreds of terabytes each day,” notes Ryan Tracy, IT operations lead for Bing Imagery Technologies. “We generate about ten terabytes per hour. Our record is three petabytes in four days. So we have a significant amount of processing going on.”
The requirements didn’t stop there. The team needed a modular solution that could be quickly deployed to add capacity to support growth and new partner needs. In addition to creating imagery for Bing Maps, the organization produces geospatial imagery for various strategic partners.
On the power and cooling front, the team wanted a modular solution that could make maximum use of free air cooling to capitalize on Colorado’s climate. What’s more, it wanted to achieve an extremely low power usage effectiveness (PUE) ratio—a key measure of data center energy efficiency.
Technology at Work
Choosing the Dell solution
Bing Maps personnel talked to multiple vendors before settling on Dell. At a product level, they liked the modularity, superior PUE, low overall power utilization, and high density of the Dell Modular Data Center, according to Clark. At an engineering level, they were impressed with the expertise of the Dell Data Center Solutions team who designed the Modular Data Center and customized the solution, tailoring it to the needs of Bing Maps.
“Dell had a solution that was designed for high efficiency, density, and low cost,” Clark says. “Their container allowed us to have a flexible IT unit, which gave us room to customize or modify to meet our needs. It was a clean and simple solution, and it was obvious that we were not going to be paying for items within the solution that we didn’t need.”
The Dell DCS engineers on the project went to great lengths to accommodate the specific needs of Bing Maps and the custom geospatial applications it runs, Clark says. For example, they changed the server design to meet the unique storage and I/O needs of the BITs processing environment and they integrated InfiniBand networking infrastructure into the solution.
“Dell closely partnered with us for the duration of the project on all aspects,” he says. “We really challenged everything to keep costs low. The Dell guys were cut from the same cloth. We just made sure that everything that was needed was really needed. Dell realized that we were unorthodox, and they were willing to jump in with us.”
Fast deployment speed
Rapid deployment was another over-arching need for the team. The organization wanted the ability to bring on new IT capacity quickly to support new products, new partnerships, and ongoing growth.
In its Longmont experience, the team learned that it can now have a microsite up and running within 60 days, assuming all government permits are in place, according to Clark.
“We have the design now, and it can be put anywhere, as long as you can find land that is relatively flat and has power and networking,” he says. “We now have a reference site, we have the design, and we have a great partner in Dell that can produce the IT gear and the containers in the time-frame that we put in place. And we’ve proven that we can turn those containers on in three weeks, from dropping them from a truck to turning them live. So everything’s up and running within 60 days. That’s a pretty amazing feat.”
“In working with Dell, we were able to build a microsite that’s cost-effective, reliable, agile, and rugged, that could be located in extreme environments within various countries around the globe,” he notes.
‘Better all the way around’
“When I first looked at the design of the Dell Modular Data Center, I saw that it’s very clean and it’s very elegant,” Clark says. “It’s very well thought-out. And when you get inside the container, it’s very quiet.”
The quiet environment is made possible by the Dell Modular Data Center’s use of a central fan system that cools the environment, rather than fans in each of the servers.
“I thought that was a smart and clever approach,” Clark says of the central fan. “And having the evaporative cooling system in a dry climate like the Colorado Front Range is perfect. It makes sense. It drives lower operating costs. It’s just better all the way around.”
Ultimately, “better all the way around” describes the feeling of the Bing Maps’ team when it comes to working with the Dell Data Center Solutions team to design and deploy Modular Data Centers at the Bing Maps Imagery microsite in Colorado.
“They had the best solution and the best team, which was well aligned with our site philosophy,” Clark says. “The whole Dell team was just fantastic on all fronts.”
Watch the time-lapse video of the site being built!
Watch a quick walkthrough video on a Dell Modular Data Center.
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
The Xeon Phi coprocessor might be the new kid on the high performance block, but out of all first-rate kickers of the Intel tires, the Texas Advanced Computing Center (TACC) got the first real jab with its new top ten Stampede system.We talk with the center's Karl Schultz about the challenges of programming for Phi--but more specifically, the optimization...
Although Horst Simon was named Deputy Director of Lawrence Berkeley National Laboratory, he maintains his strong ties to the scientific computing community as an editor of the TOP500 list and as an invited speaker at conferences.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
May 10, 2013 |
Program provides cash awards up to $10,000 for the best open-source end-user applications deployed on 100G network.
May 09, 2013 |
The Japanese government has revealed its plans to best its previous K Computer efforts with what they hope will be the first exascale system...
May 08, 2013 |
For engineers looking to leverage high-performance computing, the accessibility of a cloud-based approach is a powerful draw, but there are costs that may not be readily apparent.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.