Open Cloud Test Bed Bolsters Big Data Innovation

By Tiffany Trader

April 28, 2014

On Friday, in the Western Massachusetts town of Holyoke, Governor Deval Patrick and officials from industry, government and academia joined together for the official launch of a $3 million capital investment, known as the Massachusetts Open Cloud (MOC) project. The public-private initiative will establish a new cloud computing infrastructure to serve as a virtual laboratory for big data researchers and innovators across the state. One of the first aims of the MOC Project will be to study the feasibility of operating HPC applications in this open cloud environment.

MassachusettsOpenCloud-logos-in-cloud-500xAlthough the MOC will begin as a research collaboration and testbed, the ultimate goal is for it to become an independent non-profit entity. Guiding its direction as it evolves from prototype to production to self-sustaining operation will be a wide range of stakeholders – a mix of academic, industry, government and non-profit members. The initial $3 million in funding is being provided by the Commonwealth of Massachusetts under a Mass Tech Collaborative Matching Grant Award with another $16 million in matching funds coming from a mix of federal, industry and philanthropic sources.

The physical resources for the cloud, which will be implemented using an OpenStack framework, will be hosted at the Massachusetts Green High Performance Computing Center (MGHPCC), the Holyoke, Mass., datacenter that was created in 2012 by Boston University (BU), Harvard University, MIT, Northeastern University, and the University of Massachusetts (Umass). The 90,000 square foot, 10 megawatt facility is located on an 8.6 acre former industrial site just a few blocks from City Hall in Holyoke, Mass.

The five founding universities are also principal MOC partners along with Massachusetts Green High-Performance Computing Center (MGHPCC) and Oak Ridge National Laboratory (ORNL). Boston University’s Hariri Institute for Computing and Computational Science and Engineering is leading the project with operational leadership coming from Harvard University, development leadership from Northeastern University, community building from MIT, and related research falling under the purview of all five universities.

The initiative is based on the “Open Cloud eXchange” model, where hardware, software and other services can be supplied, purchased and resold by many participants, ranging from existing providers to startup innovators. As with popular public clouds from vendors like Amazon and Google, MOC provides access to massive off-site computational resources, but unlike those proprietary clouds, where all of the technology is controlled by a single provider, MOC uses an open and customizable approach to the design and operation of cloud computing.

It makes sense that a program designed to spur innovation would itself be on the cutting-edge. “The MOC will be the first realization of this model,” reports Orran Krieger, director of the Cloud Computing Initiative at the Hariri Institute. “If it’s successful, we expect other clouds to follow our model, fundamentally changing the nature of cloud computing.” Krieger is also a College of Arts & Sciences research professor of computer science at Boston University.

Krieger and fellow Boston University researcher Azer Bestavros were instrumental in designing the model, and seeing it to fruition.

“The Open Cloud Exchange (OCX) is envisioned as a public cloud marketplace in which many stakeholders, rather than just a single provider, participate in implementing and operating the cloud. This ecosystem would enable innovation from the broader academic and industry communities, resulting in a much healthier and more efficient cloud marketplace,” the duo write in a recent paper.

They make the case that a closed cloud stifles innovation, whereas an open cloud promotes it. With a comparison to open source software, the authors are confident an open cloud will also enhance security.

“Th[e] single-provider model results in strong homogeneity in cloud providers’ offerings,” they write. “This not only limits research but also results in a monoculture that’s susceptible to security threats. Security concerns also arise because public clouds are designed under the assumption that the cloud operator is fundamentally trusted. No documented technologies or policies keep a cloud provider, or even a disgruntled employee, from instrospecting on a customer’s network traffic, computers or even private datasets. We argue that an open cloud would deliver some of the same long-recognized security benefits as has open source software development.”

According to the project’s website, the MOC has two overarching goals:

  • To create an improved computing resource for cloud and big data users in the Commonwealth.
  • To create a new model of cloud computing that enables research and technology companies to innovate and profit in the cloud and big data sectors.

Major tasks and milestones have been divided into three main categories:

  • Deploying, operating, and maintaining a production cloud service with technology partners.
  • Enhancing OpenStack to enable multiple competing providers to participate in a shared cloud.
  • Working with a broad industry and academic community to enable new workloads and users to exploit the cloud.

The MOC project is already enjoying strong industry support. Founding vendor partners include Cisco, DataDirect Networks, EMC, SGI, Red Hat, Juniper Networks, Canonical, Dell, Intel, Mellanox, Brocade, Mathworks, Plexxi, Cambridge Computer Services, Enterprise DB, and Riverbed.

DDN’s role in the project will be to contribute its Web Object Scaler (WOS) software to enable a low latency object storage service. DDN’s Chief Technology Officer Jean-Luc Chatelain notes that while the initial interface to the WOS cloud will be through OpenStack APIs, the company expects to be able to test multiple interfaces and configurations at large scale. (WOS can host up 32 trillion objects, according to the company.) In an interview with HPCwire, Dave Fellinger, DDN Chief Scientist, emphasized the importance of data management in a federated environment to enable collaboration. The MOC Project has a long-term goal of federating with other datacenters, and DDN’s WOS platform has been designed to enable this service, without involving servers or any external management devices.

As Fellinger further explains, WOS was designed to be an efficient means of data replication and data recovery. The software establishes on each storage node a demon that runs across the entire storage cluster, tracking and recovering data on a peer to peer basis. If a disk drive fails, the node will retrieve the objects and replace them automatically without any server involvement.

SGI is another company that’s excited to be collaborating on the Massachusetts Open Cloud project. “SGI’s computational infrastructure is purpose-built to handle workflows that lie at the intersection of HPC and Big Data – High Performance Data Analysis is a growing area for data scientists in both academic and commercial settings, and we are proud to help Massachusetts grow its leadership efforts in these areas,” Jorge Titinger, president and CEO of SGI states. “As a result of this collaboration SGI expects to contribute to further advances in Knowledge Discovery, the science of extracting exceptional insights from very large and continuously dynamic data sets, paving the way for real Big Data innovation.”

Governor Patrick also announced the release of the 2014 Mass Big Data Report, which identifies and outlines the opportunities for innovation and growth within the big data industry, providing strategic recommendations for policymakers. According to findings in the report, the global big data market is expected to hit $48 billion by 2017, up from $11.6 billion in 2012. Hardware and services are expected to continue to account for the greatest share of revenue, however, the fastest growing sector, according to the report, is likely to be in big data-enabled applications. In Mass., big data applications are especially well-represented in healthcare, life sciences and financial services, and local firms are seeking to fill as many as 3,000 jobs related to these fields over the next 12 months.

“Investment in the Massachusetts Open Cloud will help keep our Commonwealth at the forefront of big data research nationally, expanding opportunities for innovators to build advanced cloud computing solutions,” said Pamela Goldberg, CEO of the Massachusetts Technology Collaborative. “As cited in the 2014 Mass Big Data Report, we must continue developing cross-sector collaborations like the Massachusetts Open Cloud, in order to spur innovation and foster industry growth.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

ASC17 Makes Splash at Wuxi Supercomputing Center

April 24, 2017

A record-breaking twenty student teams plus scores of company representatives, media professionals, staff and student volunteers transformed a formerly empty hall inside the Wuxi Supercomputing Center into a bustling hub of HPC activity, kicking off day one of 2017 Asia Student Supercomputer Challenge (ASC17). Read more…

By Tiffany Trader

Musk’s Latest Startup Eyes Brain-Computer Links

April 21, 2017

Elon Musk, the auto and space entrepreneur and severe critic of artificial intelligence, is forming a new venture that reportedly will seek to develop an interface between the human brain and computers. Read more…

By George Leopold

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Engine (GCE) job. Sutherland ran the massive mathematics workload on 220,000 GCE cores using preemptible virtual machine instances. Read more…

By Tiffany Trader

NERSC Cori Shows the World How Many-Cores for the Masses Works

April 21, 2017

As its mission, the high performance computing center for the U.S. Department of Energy Office of Science, NERSC (the National Energy Research Supercomputer Center), supports a broad spectrum of forefront scientific research across diverse areas that includes climate, material science, chemistry, fusion energy, high-energy physics and many others. Read more…

By Rob Farber

HPE Extreme Performance Solutions

Remote Visualization Optimizing Life Sciences Operations and Care Delivery

As patients continually demand a better quality of care and increasingly complex workloads challenge healthcare organizations to innovate, investing in the right technologies is key to ensuring growth and success. Read more…

Nvidia P100 Shows 1.3-2.3x Speedup Over K80 GPU on Financial Apps

April 20, 2017

When it comes to the true performance of the latest silicon, every end user knows that the best processor is the one that works best for their application. Read more…

By Tiffany Trader

Quantum Adds Global Smarts to StorNext File System

April 20, 2017

Companies that use Quantum’s StorNext platform to store massive amounts of data this week got a glimpse of new storage capabilities that should make it easier to access their data horde from anywhere in the world. Read more…

By Alex Woodie

Scaling an HPC Career in Nepal Can Be a Steep Climb

April 20, 2017

Umesh Upadhyaya works as an IT Associate at the International Centre for Integrated Mountain Development (ICIMOD) in Nepal, which supports the country’s one and only HPC facility. He is directly involved in an initiative that focuses on climate change and atmosphere modeling Read more…

By Nages Sieslack

Hyperion (IDC) Paints a Bullish Picture of HPC Future

April 20, 2017

Hyperion Research – formerly IDC’s HPC group – yesterday painted a fascinating and complicated portrait of the HPC community’s health and prospects at the HPC User Forum held in Albuquerque, NM. HPC sales are up and growing ($22 billion, all HPC segments, 2016). Read more…

By John Russell

ASC17 Makes Splash at Wuxi Supercomputing Center

April 24, 2017

A record-breaking twenty student teams plus scores of company representatives, media professionals, staff and student volunteers transformed a formerly empty hall inside the Wuxi Supercomputing Center into a bustling hub of HPC activity, kicking off day one of 2017 Asia Student Supercomputer Challenge (ASC17). Read more…

By Tiffany Trader

NERSC Cori Shows the World How Many-Cores for the Masses Works

April 21, 2017

As its mission, the high performance computing center for the U.S. Department of Energy Office of Science, NERSC (the National Energy Research Supercomputer Center), supports a broad spectrum of forefront scientific research across diverse areas that includes climate, material science, chemistry, fusion energy, high-energy physics and many others. Read more…

By Rob Farber

Hyperion (IDC) Paints a Bullish Picture of HPC Future

April 20, 2017

Hyperion Research – formerly IDC’s HPC group – yesterday painted a fascinating and complicated portrait of the HPC community’s health and prospects at the HPC User Forum held in Albuquerque, NM. HPC sales are up and growing ($22 billion, all HPC segments, 2016). Read more…

By John Russell

Knights Landing Processor with Omni-Path Makes Cloud Debut

April 18, 2017

HPC cloud specialist Rescale is partnering with Intel and HPC resource provider R Systems to offer first-ever cloud access to Xeon Phi "Knights Landing" processors. The infrastructure is based on the 68-core Intel Knights Landing processor with integrated Omni-Path fabric (the 7250F Xeon Phi). Read more…

By Tiffany Trader

CERN openlab Explores New CPU/FPGA Processing Solutions

April 14, 2017

Through a CERN openlab project known as the ‘High-Throughput Computing Collaboration,’ researchers are investigating the use of various Intel technologies in data filtering and data acquisition systems. Read more…

By Linda Barney

DOE Supercomputer Achieves Record 45-Qubit Quantum Simulation

April 13, 2017

In order to simulate larger and larger quantum systems and usher in an age of “quantum supremacy,” researchers are stretching the limits of today’s most advanced supercomputers. Read more…

By Tiffany Trader

Penguin Takes a Run at the Big Cloud Providers

April 12, 2017

HPC specialist Penguin Computing recently re-ran benchmarks from a study of its larger brethren and says the results show its ‘public cloud’ – Penguin on Demand (POD) – is among the leaders in cost and performance. Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Google Pulls Back the Covers on Its First Machine Learning Chip

April 6, 2017

This week Google released a report detailing the design and performance characteristics of the Tensor Processing Unit (TPU), its custom ASIC for the inference phase of neural networks (NN). Read more…

By Tiffany Trader

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Read more…

By John Russell

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the campaign. Read more…

By John Russell

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its assets. Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

For IBM/OpenPOWER: Success in 2017 = (Volume) Sales

January 11, 2017

To a large degree IBM and the OpenPOWER Foundation have done what they said they would – assembling a substantial and growing ecosystem and bringing Power-based products to market, all in about three years. Read more…

By John Russell

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

In this contributed perspective piece, Intel’s Jim Jeffers makes the case that CPU-based visualization is now widely adopted and as such is no longer a contrarian view, but is rather an exascale requirement. Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a new board design for NVLink-equipped Pascal P100 GPUs that will create another entrant to the space currently occupied by Nvidia's DGX-1 system, IBM's "Minsky" platform and the Supermicro SuperServer (1028GQ-TXR). Read more…

By Tiffany Trader

Leading Solution Providers

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

IBM Wants to be “Red Hat” of Deep Learning

January 26, 2017

IBM today announced the addition of TensorFlow and Chainer deep learning frameworks to its PowerAI suite of deep learning tools, which already includes popular offerings such as Caffe, Theano, and Torch. Read more…

By John Russell

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling was pretty much the exclusive realm of the Cray-2 and IBM mainframe class products. That’s changing. We are now seeing an emergence of x86 class server products with exotic plumbing technology ranging from Direct-to-Chip to servers and storage completely immersed in a dielectric fluid. Read more…

By Steve Campbell

BioTeam’s Berman Charts 2017 HPC Trends in Life Sciences

January 4, 2017

Twenty years ago high performance computing was nearly absent from life sciences. Today it’s used throughout life sciences and biomedical research. Genomics and the data deluge from modern lab instruments are the main drivers, but so is the longer-term desire to perform predictive simulation in support of Precision Medicine (PM). There’s even a specialized life sciences supercomputer, ‘Anton’ from D.E. Shaw Research, and the Pittsburgh Supercomputing Center is standing up its second Anton 2 and actively soliciting project proposals. There’s a lot going on. Read more…

By John Russell

HPC Startup Advances Auto-Parallelization’s Promise

January 23, 2017

The shift from single core to multicore hardware has made finding parallelism in codes more important than ever, but that hasn’t made the task of parallel programming any easier. Read more…

By Tiffany Trader

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu’s Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural network training and now they are sharing their implementation with the larger deep learning community. Read more…

By Tiffany Trader

US Supercomputing Leaders Tackle the China Question

March 15, 2017

Joint DOE-NSA report responds to the increased global pressures impacting the competitiveness of U.S. supercomputing. Read more…

By Tiffany Trader

IDG to Be Bought by Chinese Investors; IDC to Spin Out HPC Group

January 19, 2017

US-based publishing and investment firm International Data Group, Inc. (IDG) will be acquired by a pair of Chinese investors, China Oceanwide Holdings Group Co., Ltd. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This