The Revolution in the Lab is Overwhelming IT

By John Russell

October 5, 2015

Sifting through the vast treasure trove of data spilling from modern life science instruments is perhaps the defining challenge for biomedical research today. NIH, for example, generates about 1.5PB of data a month, and that excludes NIH-funded external research. Not only have DNA sequencers become extraordinarily powerful, but also they have proliferated in size and type, from big workhorse instruments like the Illumina HiSeq X Ten, down to reliable bench-top models (MiSeq) suitable for small labs, and there are now in advanced development USB-stick sized devices that plug into a USB port.

“The flood of sequence data, human and non-human that may impact human health, is certainly growing and in need of being integrated, mined, and understood. Further, there are emerging technologies in imaging and high resolution structure studies that will be generating a huge amount of data that will need to be analyzed, integrated, and understood,”[i] said Jack Collins, Director of the Advanced Biomedical Computing Center at the Frederick National Laboratory for Cancer Research, NCI.

Here are just a few of the many feeder streams to the data deluge:

  • DNA Sequencers. An Illumina (NASDAQ: ILMN) top-of-the-line HiSeq Ten Series can generate a full human genome in just 18 hours (and generate 3TB) and deliver 18000 genomes in a year. File size for a single whole genome sample may exceed 75GB.
  • Live cell imaging. High throughput imaging in which robots screen hundreds of millions of compounds on live cells typically generate tens of terabytes weekly.
  • Confocal imaging. Scanning 100s of tissue section, with sometimes many scans per section, each with 20-40 layers and multiple fluorescent channels can produce on the order of 10TB weekly.
  • Structural Data. Advanced investigation into form and structure is driving huge and diverse datasets derived from many sources.

Broadly, the flood of data from various LS instruments stresses virtually every part of most research computational environment (cpu, network, storage, system and application software). Indeed, important research and clinical work can be delayed or not attempted because although generating the data is feasible, the time required to perform the data analysis can be impractical. Faced with these situations, research organizations are forced to retool the IT infrastructure.

“Bench science is changing month to month while IT infrastructure is refreshed every 2-7 years. Right now IT is not part of the conversation [with life scientists] and running to catch up,” noted Ari Berman, GM of Government Services, the BioTeam consulting firm and a member of Tabor EnterpriseHPC Conference Advisory Board.

The sheer volume of data is only one aspect of the problem. Diversity in files and data types further complicates efforts to build the “right” infrastructure. Berman noted in a recent presentation that life sciences generates massive text files, massive binary files, large directories (many millions of files), large files ~600Gb and very many small files ~30kb or less. Workflows likewise vary. Sequencing alignment and variant calling offer one set of challenges; pathway simulation presents another; creating 3D models – perhaps of the brain and using those to perform detailed neurosurgery with real-time analytic feedback

“Data piles up faster than it ever has before. In fact, a single new sequencer can typically generate terabytes of data a day. And as a result, an organization or lab with multiple sequencers is capable of producing petabytes of data in a year. The data from the sequencers must be analyzed and visualized using third-party tools. And then it must be managed over time,” said Berman.

Human Brain ProjectAn excellent, though admittedly high-end, example of the growing complexity of computational tools being contemplated and developed in life science research is presented by the European Union Human Brain Project[ii] (HBP). Among its lofty goals are creation of six information and communications technology (ICT) platforms intended to enable “large-scale collaboration and data sharing, reconstruction of the brain at different biological scales, federated analysis of clinical data to map diseases of the brain, and development of brain-inspired computing systems.”

The elements of the planned HPC platform include[iii]:

  • Neuroinformatics: a data repository, including brain atlases.
  • Brain Simulation: building ICT models and simulations of brains and brain components.
  • Medical Informatics: bringing together information on brain diseases.
  • Neuromorphic Computing: ICT that mimics the functioning of the brain.
  • Neurorobotics: testing brain models and simulations in virtual environments.
  • HPC Infrastructure: hardware and software to support the other Platforms.

(Tellingly HBP organizers have recognized the limited computational expertise of many biomedical researchers and also plan to develop technical support and training programs for users of the platforms.)

There is broad agreement in the life sciences research community that there is no single best HPC infrastructure to handle the many LS use cases. The best approach is to build for the dominant use cases. Even here, said Berman, building HPC environments for LS is risky, “The challenge is to design systems today that can support unknown research requirements over many years.” And of course, this all must be accomplished in a cost-constrained environment.

Vertical Focus: HPC in BioIT“Some lab instruments know how to submit jobs to clusters. You need heterogeneous systems. Homogeneous clusters don’t work well in life sciences because of the varying uses cases. Newer clusters are kind of a mix and match of things we have fat nodes with tons of cpus and thin nodes with really fast cpus, [for example],” said Berman.

Just one genome, depending upon the type of sequencing and the coverage, can generate 100GB of data to manage. Capturing, analyzing, storing, and presenting the accumulating data requires a hybrid HPC infrastructure that blends traditional cluster computing with emerging tools such as iRODS (Integrated Rule-Oriented Data System) and Hadoop. Unsurprisingly the HPC infrastructure is always a work in progress

Here’s a snapshot of the two of the most common genomic analysis pipelines:

  1. DNA Sequencing. DNA extracted from tissue samples is run through the high-throughput NGS instruments. These modern sequencers generate hundreds of millions of short DNA sequences for each sample, which must then be ‘assembled’ into proper order to determine the genome. Researchers use parallelized computational workflows to assemble the genome and perform quality control on the reassembly—fixing errors in the reassembly.
  2. Variant Calling. DNA variations (SNPs, haplotypes, indels, etc) for an individual are detected, often using large patient populations to help resolve ambiguities in the individual’s sequence data. Data may be organized into a hybrid solution that uses a relational database to store canonical variations, high-performance file systems to hold data, and a Hadoop-based approach for specialized data-intensive analysis. Links to public and private databases help researchers identify the impact of variations including, for example, whether variants have known associations with clinically relevant conditions.

The point is that life science research – and soon healthcare delivery – has been transformed by productivity leaps in the lab that now are creating immense computational challenges. (next Part 2: Storage Strategies)

[i] Presented on a panel at Leverage Big Data conference, March 2015; http://www.leveragebigdata.com;

[ii] https://www.humanbrainproject.eu/

[iii] https://www.humanbrainproject.eu/discover/the-project/platforms;jsessionid=emae995mioyqxt99x2a14ljg

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

AI Thought Leaders on Capitol Hill

July 14, 2018

On Thursday, July 12, the House Committee on Science, Space, and Technology heard from four academic and industry leaders – representatives from Berkeley Lab, Argonne Lab, GE Global Research and Carnegie Mellon University – on the opportunities springing from the intersection of machine learning and advanced-scale computing. Read more…

By Tiffany Trader

HPC Serves as a ‘Rosetta Stone’ for the Information Age

July 12, 2018

In an age defined and transformed by its data, several large-scale scientific instruments around the globe might be viewed as a ‘mother lode’ of precious data. With names seemingly created for a ‘techno-speak’ glossary, these interferometers, cyclotrons, sequencers, solenoids, satellite altimeters, and cryo-electron microscopes are churning out data in previously unthinkable and seemingly incomprehensible quantities -- billions, trillions and quadrillions of bits and bytes of electro-magnetic code. Read more…

By Warren Froelich

Can Markov Logic Take Machine Learning to the Next Level?

July 11, 2018

Advances in machine learning, including deep learning, have propelled artificial intelligence (AI) into the public conscience and forced executives to create new business plans based on data. However, the scarcity of hig Read more…

By Alex Woodie

HPE Extreme Performance Solutions

Introducing the First Integrated System Management Software for HPC Clusters from HPE

How do you manage your complex, growing cluster environments? Answer that big challenge with the new HPC cluster management solution: HPE Performance Cluster Manager. Read more…

IBM Accelerated Insights

ORNL Summit Supercomputer Is Officially Here

Oak Ridge National Laboratory (ORNL) together with IBM and Nvidia celebrated the official unveiling of the Department of Energy (DOE) Summit supercomputer today at an event presided over by DOE Secretary Rick Perry. Read more…

CSIR, Nvidia Partner to Launch GPU-Powered AI Center in India

July 10, 2018

As reported by a number of Indian news outlets, India’s Council of Scientific and Industrial Research (CSIR) is partnering with Nvidia to establish a new, AI-focused Centre of Excellence in New Delhi, India's capital. Read more…

By Oliver Peckham

AI Thought Leaders on Capitol Hill

July 14, 2018

On Thursday, July 12, the House Committee on Science, Space, and Technology heard from four academic and industry leaders – representatives from Berkeley Lab, Argonne Lab, GE Global Research and Carnegie Mellon University – on the opportunities springing from the intersection of machine learning and advanced-scale computing. Read more…

By Tiffany Trader

HPC Serves as a ‘Rosetta Stone’ for the Information Age

July 12, 2018

In an age defined and transformed by its data, several large-scale scientific instruments around the globe might be viewed as a ‘mother lode’ of precious data. With names seemingly created for a ‘techno-speak’ glossary, these interferometers, cyclotrons, sequencers, solenoids, satellite altimeters, and cryo-electron microscopes are churning out data in previously unthinkable and seemingly incomprehensible quantities -- billions, trillions and quadrillions of bits and bytes of electro-magnetic code. Read more…

By Warren Froelich

Tsinghua Powers Through ISC18 Field

July 10, 2018

Tsinghua University topped all other competitors at the ISC18 Student Cluster Competition with an overall score of 88.43 out of 100. This gives Tsinghua their s Read more…

By Dan Olds

HPE, EPFL Launch Blue Brain 5 Supercomputer

July 10, 2018

HPE and the Ecole Polytechnique Federale de Lausannne (EPFL) Blue Brain Project yesterday introduced Blue Brain 5, a new supercomputer built by HPE, which displ Read more…

By John Russell

Pumping New Life into HPC Clusters, the Case for Liquid Cooling

July 10, 2018

High Performance Computing (HPC) faces some daunting challenges in the coming years as traditional, industry-standard systems push the boundaries of data center Read more…

By Scott Tease

Meet the ISC18 Cluster Teams: Up Close & Personal

July 6, 2018

It’s time to meet your ISC18 Student Cluster Competition teams. While I was able to film them live at the ISC show, the trick was finding time to edit the vid Read more…

By Dan Olds

PRACEdays18 Keynote Allan Williams (Australia/NCI): We’re Open for Business Down Under!

July 5, 2018

The University of Ljubljana in Slovenia hosted the third annual EHPCSW18 and fifth annual PRACEdays18 events which opened with a plenary session on May 29, 2018 Read more…

By Elizabeth Leake (STEM-Trek for HPCwire)

HPC Under the Covers: Linpack, Exascale & the Top500

June 28, 2018

HPCers can get painted as a monolithic bunch by outsiders, but internecine disagreements abound over the HPCest of HPC jargon, as was evident at ISC this week. Read more…

By Tiffany Trader

Leading Solution Providers

SC17 Booth Video Tours Playlist

Altair @ SC17

Altair

AMD @ SC17

AMD

ASRock Rack @ SC17

ASRock Rack

CEJN @ SC17

CEJN

DDN Storage @ SC17

DDN Storage

Huawei @ SC17

Huawei

IBM @ SC17

IBM

IBM Power Systems @ SC17

IBM Power Systems

Intel @ SC17

Intel

Lenovo @ SC17

Lenovo

Mellanox Technologies @ SC17

Mellanox Technologies

Microsoft @ SC17

Microsoft

Penguin Computing @ SC17

Penguin Computing

Pure Storage @ SC17

Pure Storage

Supericro @ SC17

Supericro

Tyan @ SC17

Tyan

Univa @ SC17

Univa

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This