“If the light is green, you’re burning service units (SUs). If it’s not, you’re not.” – Jeremy Fischer, Jetstream senior technical advisor.
The concept behind the capabilities of new XSEDE resource Jetstream is all about usability, hence, the green light question. In the user interface, when you start a virtual machine (VM) image, a green light comes on. Now, after a few months in early operations mode, Jetstream’s green light is officially on starting in early Fall 2016.
Indiana University and the National Science Foundation (NSF) have announced that Jetstream, the first-ever NSF-supported cloud resource for science and engineering research, will go into full operations as of September 1. Read the full announcement here.
“NSF, the IU Pervasive Technology Institute (PTI), and our many partners are embarking on a journey to operate a system that is both a production cloud and a first of a kind pilot project,” said Craig Stewart, executive director of the Indiana University Pervasive Technology Institute and Deputy Director of XSEDE Community Infrastructure. “Both design and operations are based on cloud standards, and from the end user’s standpoint this should feel like a production cloud. It’s on all the time, it works all the time. At the same time, this is a first of a kind system for the NSF and thus a pilot project in which we are supporting some very important research while learning a lot about operating a cloud system,” said Stewart.
Jetstream is an open, standards-based cloud platform designed to support science and engineering research, development, and education, while commercial services are typically designed to support businesses and web hosting.
Stewart continued, “What’s special about Jetstream is that it is a managed science cloud – a cloud managed for science. Commercial cloud services are not.”
Jetstream, in that way, is just like any other XSEDE resource: help, in the form of Extended Collaborative Support Services (ECSS), is available, as is Fischer and others across the project. ECSS partners XSEDE users with cyberinfrastructure experts for a period of time ranging from a few months to a year to help researchers fundamentally advance their use of XSEDE resources.
“We want Jetstream to be extremely user friendly, while also allowing flexibility for adapting to what the researcher might want to do,” said Jeremy Fischer, senior technical advisor for the system, which is a distributed science and engineering cloud run by Indiana University, the Texas Advanced Computing Center (TACC), University of Arizona, and several funded and collaborating partners. Fischer hosted a tutorial at XSEDE16 in Miami in which he allowed the attendees to register with training accounts in order to see what it’s like running jobs in Jetstream. Stewart presented a paper at XSEDE16 on the system, which won the “Phil Andrews Best Technology Paper” award, titled “Jetstream – Performance, Early Experiences, and Early Results.”
Much of the strength of Jetstream is its “cloudy-ness,” as Fischer puts it.
“Jetstream is not like a typical HPC supercomputer. It’s a cluster providing VMs that allows researchers to interact via a graphical user interface (GUI) via virtual network computing (VNC) and soon any web browser.”
Fischer sees virtual machines as “democratizing” for users. Jetstream has no queues like traditional parallel computing and is intended primarily for interactive use, including the ability to suspend and resume running VMs.
“Traditional HPC resources are hard to learn to use if you are not training in a discipline that traditionally uses such systems,” Fischer says. “People new to advanced computing aren’t used to living on the command line.”
However, because the user is running their own VM, Fischer notes that command line interaction is easily possible.
As of early July 2016, Jetstream had more than 250 users on more than 100 projects, while still officially in “early operations” mode. Users are able to access support to it – even if they missed the tutorial at XSEDE16 – by taking part in online classes through the Cornell Virtual Workshop, or other training, education and outreach means found through the XSEDE portal. More personalized help can come from XSEDE Campus Champions, numbering over 250 across the country; additionally, detailed documentation can be found in User Guides available on the XSEDE website.
“Usability is of the utmost concern with Jetstream,” said Fischer. “We want to enable all users from experts to new researchers to fully grasp Jetstream’s capabilities and further their research.”
Concluded Stewart: “What sets this system apart is its support for the full range of disciplines served by XSEDE, and for small schools—particularly minority-serving institutions—with constrained budgets.”
A Deeper Dive
- Jetstream offers “self-serve” cloud services, enabling researchers or students to select a pre-existing VM image or to create a new virtual environment for personalized research computing.
- Provides virtual desktop services to tablet devices, increasing cyberinfrastructure access for users at resource-limited institutions.
- Authenticates through XSEDE’s partnership with Globus, allowing for a simple mechanism to get into the Jetstream user home page. Once in, the user is in the “Atmosphere” interface, where virtual machines (VMs) can be accessed.
Enables data transfers via Globus Connect, storage and dissemination via the IUScholarWorks digital repository, and discoverability via a Digital Object Identifier (DOI). - Hosts persistent Science Gateways.
- Each physical system has the following specifications: 320 Dell M630 blades with a total of 640 CPUs, 15,360 cores, 258 TFLOPS peak processing capability and 40 TB RAM. Plus, 20 Dell R730 servers for storage nodes, with a total of 40 CPUs, 960 processing cores, 1.2 TB RAM, 16 TB local storage, 960 TB of storage, and peak processing capability of 16.1 TFLOPS.