If there’s one mandate that all IT and business professionals can agree on, it’s the need for security. For the largest systems in the world, keeping IT assets safe presents a unique set of challenges. Take the NCSA Blue Waters supercomputer as an example. The machine must be open and accessible to a collaborative-leaning scientific community while guarding against malicious activity. The person in charge of this balancing act is Adam Slagell. As Chief Information Security Officer for the National Center for Supercomputing Applications (NCSA) at the University of Illinois, Slagell is part of a team responsible for securing this massive resource.

Blue Waters is a leadership-class machine capable of sustained petascale performance on a range of scientific and engineering applications. The 11.6 petaflop (peak) system is comprised of 237 Cray XE6 racks, 32 Cray XK7 racks with NVIDIA Kepler GPUs, and 7 I/O racks. It also includes 1.5 petabytes of high-speed memory, 25 petabytes of usable online storage, and 300 petabytes near-line tape capacity.

As one of the largest computational systems in the world, Blue Waters faces some unique security challenges as Slagell can attest to:

“Traditional security technologies like inline intrusion prevention systems, stateful firewalls, and security appliances can take a 10 Gbps connection down to 500Mbps easily,” observes Slagell. “NCSA, which has well over 100 Gbps of external WAN connections (and plans to go to 300 Gbps), can’t come close to operating efficiently with those kinds of bottlenecks. Instead NCSA relies on passive monitoring techniques, making heavy use of ICSI’s open-source Bro network security monitor to understand and protect its network.”

The open-source traffic analyzer, Bro, was developed over many years by the International Computer Science Institute (ICSI) – an independent, non-profit computer science research center. Current ICSI Director Vern Paxson created the first version in 1995 at the Lawrence Berkeley National Laboratory. Work on Bro continues under the stewardship of ICSI with funding from the National Science Foundation.

Slagell explains that Bro has evolved to bridge the traditional gap between academic research and operations. The cyber-security tool has been deployed by major universities, research labs, supercomputing centers, open-science communities, as well as industry sites.

“Bro remains unique in its analysis capabilities as it is not limited to any particular detection strategy – a major restriction of traditional intrusion detection systems,” writes Slagell. “Bro instead provides a flexible platform for implementing a range of sophisticated, in-depth traffic analyses that are tailored to the needs of individual sites.”

To say that Bro has evolved over the years is an understatement. Before Blue Waters, NCSA only had to monitor a single optical link. With the addition of a brand-new datacenter housing a multi-million dollar supercomputer, NCSA’s monitoring responsibilities grew to dozens of 10G connections with planned 100G connections. The center moved from a single host running Bro to a cluster of 80 Bro workers, with plans for a 20 more.

In addition to increasing Bro’s scalability, there have been other improvements as well, for example in the area of intrusion detection. The developer team optimized this capability, adding policy-driven enhancements that allow the system to respond more quickly to potential incidents. Bro can also monitor traffic across multiple network zone boundaries.

For the development team at ICSI, Bro’s deployment at one of the world’s most prominent supercomputer centers is a validation of years of hard work. They now have confirmation that Bro is suitable for large-scale deployments in operational settings. But there’s no time for resting on laurels in the cat and mouse game of security. ICSI’s Networking and Security professionals are already focused on future iterations of Bro that can monitor for more sophisticated attacks over faster networks.