Scientists Using Intel-Cray ‘Theta’ Supercomputer to Map Brain Function

September 14, 2017

Sept. 14, 2017 — A neuroscientist and a computational scientist walk into a synchrotron facility to study a mouse brain… Sounds like a great set-up for a comedy bit, but there is no punchline. The result is cutting-edge science that can only be accomplished in a facility as scientifically integrated as the U.S. Department of Energy’s (DOE) Argonne National Laboratory.

At a casual, or even a more attentive glance, Doga Gursoy and Bobby Kasthuri would seem at opposite ends of the research spectrum. Gursoy is an assistant computational scientist at Argonne’s Advanced Photon Source (APS), a DOE Office of Science User Facility; Kasthuri, an Argonne neuroscientist.

But together, they are using Argonne’s vast arsenal of innovative technologies to map the intricacies of brain function at the deepest levels, and describing them in greater detail than ever before through advanced data analysis techniques.

Gursoy and Kasthuri are among the first group of researchers to access Theta, the new 9.65 petaflops Intel-Cray supercomputer housed at the Argonne Leadership Computing Facility (ALCF), also a DOE Office of Science User Facility. Theta’s advanced and flexible software platform supports the ALCF Data Science Program (ADSP), a new initiative targeted at big data problems, like Gursoy and Kasthuri’s brain connectome project.

ADSP projects explore and improve a variety of computational methods that will enable data-driven discoveries across all scientific disciplines.

“By developing and demonstrating rapid analysis techniques, such as data mining, graph analytics and machine learning, together with workflows that will facilitate productive usage on our systems for applications, we will pave the way for more and more science communities to use supercomputers for their big data challenges in the future,” said Venkat Vishwanath, ALCF Data Sciences Group Lead.

All about the connections

This new ADSP study of connectomes maps the connections of every neuron in the brain, whether human or mouse. Determining the location of every cell in the brain and how they communicate with each other is a daunting task, as each cell makes thousands of connections. The human brain, for example, has some 100 billion neurons, creating 100 trillion connections. Even the average mouse brain has 75 million neurons.

This ALCF award targets big data problems and our application of brain imaging does just that,” said Gursoy, assistant computational scientist in the X-Ray Science Division of Argonne’s Advanced Photon Source. “The basic goal is simple — we would like to be able to image all of the neurons in the brain — but the datasets from X-rays and electron microscopes are extremely large. They are at the tera- and petabyte scales. So we would like to use Theta to build the software and codebase infrastructure in order to analyze that data.”

This research was supported by the U.S. Department of Energy’s Office of Science. A portion of the work was also supported by Argonne’s Laboratory-Directed Research and Development (LDRD) program.

The process begins with two imaging techniques that will provide the massive sets of data for analysis by Theta. One is at the APS, where full brains can be analyzed at submicron resolution — in this case, the brain of a petite shrewmouse — through X-ray microtomography, a high-resolution 3-D imaging technique. Argonne’s X-ray Sciences Division of the APS provides the expertise for the microtomography research. Much like a CT scanner, it produces images as micro-thin slices of a material whose structure can be meticulously scrutinized. While this resolution provides a detailed picture of blood vessels and cell bodies, the researchers aim to go still deeper.

That depth of detail requires the use of an electron microscope, which transmits a short-wavelength electron beam to deliver resolution at the nanometer scale. This will allow for the capture of all the synaptic connections between individual neurons at small targeted regions guided by the X-ray microtomography.

For years, scientists at the APS have used these techniques to deepen our understanding of a wide variety of materials, from soil samples to new materials to biological matter,” said Kamel Fezzaa from sector 32-ID at the APS. “By coordinating our efforts with Argonne high-speed computing capabilities for this project, we are able to provide some truly revolutionary images that could provide details about brain functions that we have never before been able to observe.”

Both techniques can produce petabytes of information a day and, according to the researchers, the next generations of both microscopes will increase that amount dramatically.

Images produced by these datasets have to be processed, reconstructed and analyzed. Through the ADSP, Gursoy and Kasthuri are developing a series of large-scale data and computational steps — a pipeline — that integrates exascale computational approaches into an entirely new set of tools for brain research.

Taming of the shrew

The first case study for this pipeline is the reconstruction of an entire adult shrewmouse brain, which, they estimate, will produce one exabyte of data, or one billion gigabytes. And the studies only get bigger from there.

Machine learning will go through these datasets and help come up with predictive models. For this project, it can help with segmentation or reconstruction of the brain and help classify or identify features of interest,” said Vishwanath.

Lessons learned from the smaller shrewmouse brain will be applied to a large mouse brain, which constitutes a 10-fold increase in volume. Comparisons between the two will reveal how organizational structures form during development, from embryo to adult, and how they evolve. The reconstruction of a non-human primate brain, with a volume 100 times larger than a mouse brain, is being considered for a later study.

A neuroscientist and a computational scientist leave a synchrotron facility with studies from a mouse brain . . .  armed with new techniques to analyze this data. The images produced by their work will provide a clearer understanding of how even the smallest changes to the brain play a role in the onset and evolution of neurological diseases, such as Alzheimer’s and autism, and perhaps lead to improved treatments or even a cure.

Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation’s first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America’s scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science.

The U.S. Department of Energy’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.


Source: John Spizzirri, Argonne National Laboratory

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip using standard CMOS fabrication. At Hot Chips 31 in Stanfor Read more…

By Tiffany Trader

Talk to Me: Nvidia Claims NLP Inference, Training Records

August 15, 2019

Nvidia says it’s achieved significant advances in conversation natural language processing (NLP) training and inference, enabling more complex, immediate-response interchanges between customers and chatbots. And the co Read more…

By Doug Black

Trump Administration and NIST Issue AI Standards Development Plan

August 14, 2019

Efforts to develop AI are gathering steam fast. On Monday, the White House issued a federal plan to help develop technical standards for AI following up on a mandate contained in the Administration’s AI Executive Order Read more…

By John Russell

AWS Solution Channel

Efficiency and Cost-Optimization for HPC Workloads – AWS Batch and Amazon EC2 Spot Instances

High Performance Computing on AWS leverages the power of cloud computing and the extreme scale it offers to achieve optimal HPC price/performance. With AWS you can right size your services to meet exactly the capacity requirements you need without having to overprovision or compromise capacity. Read more…

HPE Extreme Performance Solutions

Bring the combined power of HPC and AI to your business transformation

FPGA (Field Programmable Gate Array) acceleration cards are not new, as they’ve been commercially available since 1984. Typically, the emphasis around FPGAs has centered on the fact that they’re programmable accelerators, and that they can truly offer workload specific hardware acceleration solutions without requiring custom silicon. Read more…

IBM Accelerated Insights

Cloudy with a Chance of Mainframes

[Connect with HPC users and learn new skills in the IBM Spectrum LSF User Community.]

Rapid rates of change sometimes result in unexpected bedfellows. Read more…

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a good understanding of the early universe, its fate billions Read more…

By Rob Johnson

Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip Read more…

By Tiffany Trader

Scientists to Tap Exascale Computing to Unlock the Mystery of our Accelerating Universe

August 14, 2019

The universe and everything in it roared to life with the Big Bang approximately 13.8 billion years ago. It has continued expanding ever since. While we have a Read more…

By Rob Johnson

AI is the Next Exascale – Rick Stevens on What that Means and Why It’s Important

August 13, 2019

Twelve years ago the Department of Energy (DOE) was just beginning to explore what an exascale computing program might look like and what it might accomplish. Today, DOE is repeating that process for AI, once again starting with science community town halls to gather input and stimulate conversation. The town hall program... Read more…

By Tiffany Trader and John Russell

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

Lenovo Drives Single-Socket Servers with AMD Epyc Rome CPUs

August 7, 2019

No summer doldrums here. As part of the AMD Epyc Rome launch event in San Francisco today, Lenovo announced two new single-socket servers, the ThinkSystem SR635 Read more…

By Doug Black

Building Diversity and Broader Engagement in the HPC Community

August 7, 2019

Increasing diversity and inclusion in HPC is a community-building effort. Representation of both issues and individuals matters - the more people see HPC in a w Read more…

By AJ Lauer

Xilinx vs. Intel: FPGA Market Leaders Launch Server Accelerator Cards

August 6, 2019

The two FPGA market leaders, Intel and Xilinx, both announced new accelerator cards this week designed to handle specialized, compute-intensive workloads and un Read more…

By Doug Black

High Performance (Potato) Chips

May 5, 2006

In this article, we focus on how Procter & Gamble is using high performance computing to create some common, everyday supermarket products. Tom Lange, a 27-year veteran of the company, tells us how P&G models products, processes and production systems for the betterment of consumer package goods. Read more…

By Michael Feldman

Supercomputer-Powered AI Tackles a Key Fusion Energy Challenge

August 7, 2019

Fusion energy is the Holy Grail of the energy world: low-radioactivity, low-waste, zero-carbon, high-output nuclear power that can run on hydrogen or lithium. T Read more…

By Oliver Peckham

Cray, AMD to Extend DOE’s Exascale Frontier

May 7, 2019

Cray and AMD are coming back to Oak Ridge National Laboratory to partner on the world’s largest and most expensive supercomputer. The Department of Energy’s Read more…

By Tiffany Trader

Graphene Surprises Again, This Time for Quantum Computing

May 8, 2019

Graphene is fascinating stuff with promise for use in a seeming endless number of applications. This month researchers from the University of Vienna and Institu Read more…

By John Russell

AMD Verifies Its Largest 7nm Chip Design in Ten Hours

June 5, 2019

AMD announced last week that its engineers had successfully executed the first physical verification of its largest 7nm chip design – in just ten hours. The AMD Radeon Instinct Vega20 – which boasts 13.2 billion transistors – was tested using a TSMC-certified Calibre nmDRC software platform from Mentor. Read more…

By Oliver Peckham

TSMC and Samsung Moving to 5nm; Whither Moore’s Law?

June 12, 2019

With reports that Taiwan Semiconductor Manufacturing Co. (TMSC) and Samsung are moving quickly to 5nm manufacturing, it’s a good time to again ponder whither goes the venerable Moore’s law. Shrinking feature size has of course been the primary hallmark of achieving Moore’s law... Read more…

By John Russell

Deep Learning Competitors Stalk Nvidia

May 14, 2019

There is no shortage of processing architectures emerging to accelerate deep learning workloads, with two more options emerging this week to challenge GPU leader Nvidia. First, Intel researchers claimed a new deep learning record for image classification on the ResNet-50 convolutional neural network. Separately, Israeli AI chip startup Hailo.ai... Read more…

By George Leopold

Cray Wins NNSA-Livermore ‘El Capitan’ Exascale Contract

August 13, 2019

Cray has won the bid to build the first exascale supercomputer for the National Nuclear Security Administration (NNSA) and Lawrence Livermore National Laborator Read more…

By Tiffany Trader

Leading Solution Providers

ISC 2019 Virtual Booth Video Tour

CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
GOOGLE
GOOGLE
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
VERNE GLOBAL
VERNE GLOBAL

Nvidia Embraces Arm, Declares Intent to Accelerate All CPU Architectures

June 17, 2019

As the Top500 list was being announced at ISC in Frankfurt today with an upgraded petascale Arm supercomputer in the top third of the list, Nvidia announced its Read more…

By Tiffany Trader

Top500 Purely Petaflops; US Maintains Performance Lead

June 17, 2019

With the kick-off of the International Supercomputing Conference (ISC) in Frankfurt this morning, the 53rd Top500 list made its debut, and this one's for petafl Read more…

By Tiffany Trader

AMD Launches Epyc Rome, First 7nm CPU

August 8, 2019

From a gala event at the Palace of Fine Arts in San Francisco yesterday (Aug. 7), AMD launched its second-generation Epyc Rome x86 chips, based on its 7nm proce Read more…

By Tiffany Trader

A Behind-the-Scenes Look at the Hardware That Powered the Black Hole Image

June 24, 2019

Two months ago, the first-ever image of a black hole took the internet by storm. A team of scientists took years to produce and verify the striking image – an Read more…

By Oliver Peckham

Cray – and the Cray Brand – to Be Positioned at Tip of HPE’s HPC Spear

May 22, 2019

More so than with most acquisitions of this kind, HPE’s purchase of Cray for $1.3 billion, announced last week, seems to have elements of that overused, often Read more…

By Doug Black and Tiffany Trader

Chinese Company Sugon Placed on US ‘Entity List’ After Strong Showing at International Supercomputing Conference

June 26, 2019

After more than a decade of advancing its supercomputing prowess, operating the world’s most powerful supercomputer from June 2013 to June 2018, China is keep Read more…

By Tiffany Trader

In Wake of Nvidia-Mellanox: Xilinx to Acquire Solarflare

April 25, 2019

With echoes of Nvidia’s recent acquisition of Mellanox, FPGA maker Xilinx has announced a definitive agreement to acquire Solarflare Communications, provider Read more…

By Doug Black

Qualcomm Invests in RISC-V Startup SiFive

June 7, 2019

Investors are zeroing in on the open standard RISC-V instruction set architecture and the processor intellectual property being developed by a batch of high-flying chip startups. Last fall, Esperanto Technologies announced a $58 million funding round. Read more…

By George Leopold

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This