Open Cloud Test Bed Bolsters Big Data Innovation

By Tiffany Trader

April 28, 2014

On Friday, in the Western Massachusetts town of Holyoke, Governor Deval Patrick and officials from industry, government and academia joined together for the official launch of a $3 million capital investment, known as the Massachusetts Open Cloud (MOC) project. The public-private initiative will establish a new cloud computing infrastructure to serve as a virtual laboratory for big data researchers and innovators across the state. One of the first aims of the MOC Project will be to study the feasibility of operating HPC applications in this open cloud environment.

MassachusettsOpenCloud-logos-in-cloud-500xAlthough the MOC will begin as a research collaboration and testbed, the ultimate goal is for it to become an independent non-profit entity. Guiding its direction as it evolves from prototype to production to self-sustaining operation will be a wide range of stakeholders – a mix of academic, industry, government and non-profit members. The initial $3 million in funding is being provided by the Commonwealth of Massachusetts under a Mass Tech Collaborative Matching Grant Award with another $16 million in matching funds coming from a mix of federal, industry and philanthropic sources.

The physical resources for the cloud, which will be implemented using an OpenStack framework, will be hosted at the Massachusetts Green High Performance Computing Center (MGHPCC), the Holyoke, Mass., datacenter that was created in 2012 by Boston University (BU), Harvard University, MIT, Northeastern University, and the University of Massachusetts (Umass). The 90,000 square foot, 10 megawatt facility is located on an 8.6 acre former industrial site just a few blocks from City Hall in Holyoke, Mass.

The five founding universities are also principal MOC partners along with Massachusetts Green High-Performance Computing Center (MGHPCC) and Oak Ridge National Laboratory (ORNL). Boston University’s Hariri Institute for Computing and Computational Science and Engineering is leading the project with operational leadership coming from Harvard University, development leadership from Northeastern University, community building from MIT, and related research falling under the purview of all five universities.

The initiative is based on the “Open Cloud eXchange” model, where hardware, software and other services can be supplied, purchased and resold by many participants, ranging from existing providers to startup innovators. As with popular public clouds from vendors like Amazon and Google, MOC provides access to massive off-site computational resources, but unlike those proprietary clouds, where all of the technology is controlled by a single provider, MOC uses an open and customizable approach to the design and operation of cloud computing.

It makes sense that a program designed to spur innovation would itself be on the cutting-edge. “The MOC will be the first realization of this model,” reports Orran Krieger, director of the Cloud Computing Initiative at the Hariri Institute. “If it’s successful, we expect other clouds to follow our model, fundamentally changing the nature of cloud computing.” Krieger is also a College of Arts & Sciences research professor of computer science at Boston University.

Krieger and fellow Boston University researcher Azer Bestavros were instrumental in designing the model, and seeing it to fruition.

“The Open Cloud Exchange (OCX) is envisioned as a public cloud marketplace in which many stakeholders, rather than just a single provider, participate in implementing and operating the cloud. This ecosystem would enable innovation from the broader academic and industry communities, resulting in a much healthier and more efficient cloud marketplace,” the duo write in a recent paper.

They make the case that a closed cloud stifles innovation, whereas an open cloud promotes it. With a comparison to open source software, the authors are confident an open cloud will also enhance security.

“Th[e] single-provider model results in strong homogeneity in cloud providers’ offerings,” they write. “This not only limits research but also results in a monoculture that’s susceptible to security threats. Security concerns also arise because public clouds are designed under the assumption that the cloud operator is fundamentally trusted. No documented technologies or policies keep a cloud provider, or even a disgruntled employee, from instrospecting on a customer’s network traffic, computers or even private datasets. We argue that an open cloud would deliver some of the same long-recognized security benefits as has open source software development.”

According to the project’s website, the MOC has two overarching goals:

  • To create an improved computing resource for cloud and big data users in the Commonwealth.
  • To create a new model of cloud computing that enables research and technology companies to innovate and profit in the cloud and big data sectors.

Major tasks and milestones have been divided into three main categories:

  • Deploying, operating, and maintaining a production cloud service with technology partners.
  • Enhancing OpenStack to enable multiple competing providers to participate in a shared cloud.
  • Working with a broad industry and academic community to enable new workloads and users to exploit the cloud.

The MOC project is already enjoying strong industry support. Founding vendor partners include Cisco, DataDirect Networks, EMC, SGI, Red Hat, Juniper Networks, Canonical, Dell, Intel, Mellanox, Brocade, Mathworks, Plexxi, Cambridge Computer Services, Enterprise DB, and Riverbed.

DDN’s role in the project will be to contribute its Web Object Scaler (WOS) software to enable a low latency object storage service. DDN’s Chief Technology Officer Jean-Luc Chatelain notes that while the initial interface to the WOS cloud will be through OpenStack APIs, the company expects to be able to test multiple interfaces and configurations at large scale. (WOS can host up 32 trillion objects, according to the company.) In an interview with HPCwire, Dave Fellinger, DDN Chief Scientist, emphasized the importance of data management in a federated environment to enable collaboration. The MOC Project has a long-term goal of federating with other datacenters, and DDN’s WOS platform has been designed to enable this service, without involving servers or any external management devices.

As Fellinger further explains, WOS was designed to be an efficient means of data replication and data recovery. The software establishes on each storage node a demon that runs across the entire storage cluster, tracking and recovering data on a peer to peer basis. If a disk drive fails, the node will retrieve the objects and replace them automatically without any server involvement.

SGI is another company that’s excited to be collaborating on the Massachusetts Open Cloud project. “SGI’s computational infrastructure is purpose-built to handle workflows that lie at the intersection of HPC and Big Data – High Performance Data Analysis is a growing area for data scientists in both academic and commercial settings, and we are proud to help Massachusetts grow its leadership efforts in these areas,” Jorge Titinger, president and CEO of SGI states. “As a result of this collaboration SGI expects to contribute to further advances in Knowledge Discovery, the science of extracting exceptional insights from very large and continuously dynamic data sets, paving the way for real Big Data innovation.”

Governor Patrick also announced the release of the 2014 Mass Big Data Report, which identifies and outlines the opportunities for innovation and growth within the big data industry, providing strategic recommendations for policymakers. According to findings in the report, the global big data market is expected to hit $48 billion by 2017, up from $11.6 billion in 2012. Hardware and services are expected to continue to account for the greatest share of revenue, however, the fastest growing sector, according to the report, is likely to be in big data-enabled applications. In Mass., big data applications are especially well-represented in healthcare, life sciences and financial services, and local firms are seeking to fill as many as 3,000 jobs related to these fields over the next 12 months.

“Investment in the Massachusetts Open Cloud will help keep our Commonwealth at the forefront of big data research nationally, expanding opportunities for innovators to build advanced cloud computing solutions,” said Pamela Goldberg, CEO of the Massachusetts Technology Collaborative. “As cited in the 2014 Mass Big Data Report, we must continue developing cross-sector collaborations like the Massachusetts Open Cloud, in order to spur innovation and foster industry growth.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Summit Now Offers Virtual Tours

August 10, 2020

Summit, the second most powerful publicly ranked supercomputer in the world, now has a virtual tour. The tour, implemented by 3D platform Matterport, allows users to virtually “walk” around the massive supercomputer Read more…

By Oliver Peckham

Supercomputer Simulations Examine Changes in Chesapeake Bay

August 8, 2020

The Chesapeake Bay, the largest estuary in the continental United States, weaves its way south from Maryland, collecting waters from West Virginia, Delaware, DC, Pennsylvania and New York along the way. Like many major e Read more…

By Oliver Peckham

Student Success from ‘Scratch’: CHPC’s Proof is in the Pudding

August 7, 2020

Happy Sithole, who directs the South African Centre for High Performance Computing (SA-CHPC), called the 13th annual CHPC National conference to order on December 1, 2019, at the Birchwood Conference Centre in Kempton Pa Read more…

By Elizabeth Leake

New GE Simulations on Summit to Advance Offshore Wind Power

August 6, 2020

The wind energy sector is a frequent user of high-power simulations, with researchers aiming to optimize wind flows and energy production from the massive turbines. Now, researchers at GE are preparing to undertake a lar Read more…

By Oliver Peckham

Research: A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic

August 5, 2020

Within the past years, hardware vendors have started designing low precision special function units in response to the demand of the machine learning community and their demand for high compute power in low precision for Read more…

By Hartwig Anzt and Jack Dongarra

AWS Solution Channel

AWS announces the release of AWS ParallelCluster 2.8.0

AWS ParallelCluster is a fully supported and maintained open source cluster management tool that makes it easy for scientists, researchers, and IT administrators to deploy and manage High Performance Computing (HPC) clusters in the AWS cloud. Read more…

Intel® HPC + AI Pavilion

Supercomputing the Pandemic: Scientific Community Tackles COVID-19 from Multiple Perspectives

Since their inception, supercomputers have taken on the biggest, most complex, and most data-intensive computing challenges—from confirming Einstein’s theories about gravitational waves to predicting the impacts of climate change. Read more…

Implement Photonic Tensor Cores for Machine Learning?

August 5, 2020

Researchers from George Washington University have reported an approach for building photonic tensor cores that leverages phase change photonic memory to implement a neural network (NN). Their novel architecture, reporte Read more…

By John Russell

Summit Now Offers Virtual Tours

August 10, 2020

Summit, the second most powerful publicly ranked supercomputer in the world, now has a virtual tour. The tour, implemented by 3D platform Matterport, allows use Read more…

By Oliver Peckham

HPE Keeps Cray Brand Promise, Reveals HPE Cray Supercomputing Line

August 4, 2020

The HPC community, ever-affectionate toward Cray and its eponymous founder, can breathe a (virtual) sigh of relief. The Cray brand will live on, encompassing th Read more…

By Tiffany Trader

Machines, Connections, Data, and Especially People: OAC Acting Director Amy Friedlander Charts Office’s Blueprint for Innovation

August 3, 2020

The path to innovation in cyberinfrastructure (CI) will require continued focus on building HPC systems and secure connections between them, in addition to the Read more…

By Ken Chiacchia, Pittsburgh Supercomputing Center/XSEDE

Nvidia Said to Be Close on Arm Deal

August 3, 2020

GPU leader Nvidia Corp. is in talks to buy U.K. chip designer Arm from parent company Softbank, according to several reports over the weekend. If consummated Read more…

By George Leopold

Intel’s 7nm Slip Raises Questions About Ponte Vecchio GPU, Aurora Supercomputer

July 30, 2020

During its second-quarter earnings call, Intel announced a one-year delay of its 7nm process technology, which it says it will create an approximate six-month shift for its CPU product timing relative to prior expectations. The primary issue is a defect mode in the 7nm process that resulted in yield degradation... Read more…

By Tiffany Trader

PEARC20 Plenary Introduces Five Upcoming NSF-Funded HPC Systems

July 30, 2020

Five new HPC systems—three National Science Foundation-funded “Capacity” systems and two “Innovative Prototype/Testbed” systems—will be coming onlin Read more…

By Ken Chiacchia, Pittsburgh Supercomputing Center/XSEDE

Nvidia Dominates Latest MLPerf Training Benchmark Results

July 29, 2020

MLPerf.org released its third round of training benchmark (v0.7) results today and Nvidia again dominated, claiming 16 new records. Meanwhile, Google provided e Read more…

By John Russell

$39 Billion Worldwide HPC Market Faces 3.7% COVID-related Drop in 2020

July 29, 2020

Global HPC market revenue reached $39 billion in 2019, growing a healthy 8.2 percent over 2018, according to the latest analysis from Intersect360 Research. A 3 Read more…

By Tiffany Trader

Supercomputer Modeling Tests How COVID-19 Spreads in Grocery Stores

April 8, 2020

In the COVID-19 era, many people are treating simple activities like getting gas or groceries with caution as they try to heed social distancing mandates and protect their own health. Still, significant uncertainty surrounds the relative risk of different activities, and conflicting information is prevalent. A team of Finnish researchers set out to address some of these uncertainties by... Read more…

By Oliver Peckham

Supercomputer-Powered Research Uncovers Signs of ‘Bradykinin Storm’ That May Explain COVID-19 Symptoms

July 28, 2020

Doctors and medical researchers have struggled to pinpoint – let alone explain – the deluge of symptoms induced by COVID-19 infections in patients, and what Read more…

By Oliver Peckham

Intel’s 7nm Slip Raises Questions About Ponte Vecchio GPU, Aurora Supercomputer

July 30, 2020

During its second-quarter earnings call, Intel announced a one-year delay of its 7nm process technology, which it says it will create an approximate six-month shift for its CPU product timing relative to prior expectations. The primary issue is a defect mode in the 7nm process that resulted in yield degradation... Read more…

By Tiffany Trader

Nvidia Said to Be Close on Arm Deal

August 3, 2020

GPU leader Nvidia Corp. is in talks to buy U.K. chip designer Arm from parent company Softbank, according to several reports over the weekend. If consummated Read more…

By George Leopold

Supercomputer Simulations Reveal the Fate of the Neanderthals

May 25, 2020

For hundreds of thousands of years, neanderthals roamed the planet, eventually (almost 50,000 years ago) giving way to homo sapiens, which quickly became the do Read more…

By Oliver Peckham

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

By Doug Black

Neocortex Will Be First-of-Its-Kind 800,000-Core AI Supercomputer

June 9, 2020

Pittsburgh Supercomputing Center (PSC - a joint research organization of Carnegie Mellon University and the University of Pittsburgh) has won a $5 million award Read more…

By Tiffany Trader

HPE Keeps Cray Brand Promise, Reveals HPE Cray Supercomputing Line

August 4, 2020

The HPC community, ever-affectionate toward Cray and its eponymous founder, can breathe a (virtual) sigh of relief. The Cray brand will live on, encompassing th Read more…

By Tiffany Trader

Leading Solution Providers

Contributors

Nvidia’s Ampere A100 GPU: Up to 2.5X the HPC, 20X the AI

May 14, 2020

Nvidia's first Ampere-based graphics card, the A100 GPU, packs a whopping 54 billion transistors on 826mm2 of silicon, making it the world's largest seven-nanom Read more…

By Tiffany Trader

Australian Researchers Break All-Time Internet Speed Record

May 26, 2020

If you’ve been stuck at home for the last few months, you’ve probably become more attuned to the quality (or lack thereof) of your internet connection. Even Read more…

By Oliver Peckham

15 Slides on Programming Aurora and Exascale Systems

May 7, 2020

Sometime in 2021, Aurora, the first planned U.S. exascale system, is scheduled to be fired up at Argonne National Laboratory. Cray (now HPE) and Intel are the k Read more…

By John Russell

‘Billion Molecules Against COVID-19’ Challenge to Launch with Massive Supercomputing Support

April 22, 2020

Around the world, supercomputing centers have spun up and opened their doors for COVID-19 research in what may be the most unified supercomputing effort in hist Read more…

By Oliver Peckham

Joliot-Curie Supercomputer Used to Build First Full, High-Fidelity Aircraft Engine Simulation

July 14, 2020

When industrial designers plan the design of a new element of a vehicle’s propulsion or exterior, they typically use fluid dynamics to optimize airflow and in Read more…

By Oliver Peckham

John Martinis Reportedly Leaves Google Quantum Effort

April 21, 2020

John Martinis, who led Google’s quantum computing effort since establishing its quantum hardware group in 2014, has left Google after being moved into an advi Read more…

By John Russell

$100B Plan Submitted for Massive Remake and Expansion of NSF

May 27, 2020

Legislation to reshape, expand - and rename - the National Science Foundation has been submitted in both the U.S. House and Senate. The proposal, which seems to Read more…

By John Russell

Google Cloud Debuts 16-GPU Ampere A100 Instances

July 7, 2020

On the heels of the Nvidia’s Ampere A100 GPU launch in May, Google Cloud is announcing alpha availability of the A100 “Accelerator Optimized” VM A2 instance family on Google Compute Engine. The instances are powered by the HGX A100 16-GPU platform, which combines two HGX A100 8-GPU baseboards using... Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This