This post was contributed by Sanchit Misra, PhD, Vasimuddin Md., PhD,  Saurabh Kalikar, PhD, and Narendra Chaudhary, PhD, research scientists at Intel Labs.

Introduction

We are in the epoch of digital biology, that is fueled by the convergence of three revolutions:

• • the measurement of biological systems at high resolution,
  • novel data science (AI and data management) techniques that can be applied on this data, and
  • widespread use of the massive public data repositories, large collaborative projects, and consortia, which in turn promote the use of cloud due to easy data access.

Genomics is a primary example of this trend, where high-throughput next-generation sequencing (NGS) devices are being used to sequence DNA, mRNA, regulatory regions, the gut microbiome, etc. Computational workflows are also being developed, standardized rapidly, and scaled by running on the cloud. With the enormous quantities of genomic data being collected, processing times are often in the order of billions of core hours, and the cost of processing increase commensurately. As a result, customers are looking for optimized tools and systems that incur the shortest runtimes and lowest costs.

Intel’s Open Omics Acceleration Framework (in short, Open Omics) is an open-sourced high throughput framework for accelerating omics pipelines. Intel is developing this framework with the following characteristics:

• Community driven: Open Omics framework is being built based on extensive discussions with thought leaders in digital biology to understand the requirements of the user community. Moreover, Intel is building the framework with a modular design. This enables the developer community to use efficient modules to achieve faster performance for existing and new software tools in a productive manner.
• Open-sourced: so that anyone can customize it for variations in use-cases.
• Hardware accelerated: uses the underlying hardware efficiently to reduce cloud costs.
• Supports full application stack: The application layer supports a wide range of applications in genomics, single cell analysis, and drug discovery. The middleware layer has scalable and efficient implementations of key building blocks, such as data management and key compute motifs. All of this is optimized for the processor, memory, storage, and networking.

In this blog, we showcase the first version of Open Omics and benchmark three applications that are used in processing NGS data – sequence alignment tools BWA-MEM, minimap2, and single cell ATAC-Seq on Xeon-based Amazon Elastic Compute Cloud (Amazon EC2) Instances.

Applications benchmarked for this blog

BWA-MEM and Minimap2 are popular software tools for aligning short reads and long reads to a reference sequence. The Open Omics version of BWA-MEM is called BWA-MEM2 and that of minimap2 is called mm2-fast. These are efficient architecture-aware implementations of original tools that were built in collaboration with Prof. Heng Li. They are both drop-in replacements that significantly reduce runtime and cloud costs while maintaining command line interface and output identical to original tools [2,3] and have been open sourced. Open Omics BWA-MEM has been used by more than 40 peer-reviewed genomics studies already, including research on Covid-19 [5,6], gut microbiome [7], and cancer [8].

ATAC-seq assays are used for identifying accessible chromatin regions in the DNA. ATACWorks [1] is a toolkit that is used to de-noise and identify accessible chromatin regions, and it uses deep learning on 1D data. The Open Omics version of ATACWorks builds an efficient 1D dilated convolution layer and demonstrates reduced precision (BFloat16) training to achieve significant performance gain without any loss of accuracy [4].

Benchmarking the Open Omics Acceleration framework on AWS

Amazon EC2 Instances used in this benchmarking

The four types of Amazon EC2 Instances used in this benchmarking study are detailed in the following table.

Table 1: Details of the Amazon EC2 instance types used for benchmarking. On-Demand and Spot pricing are from the publish date for the US-East (Virginia) Region, and is subject to change over time. Please consult the Amazon EC2 pricing page for current pricing in your region.
Instance names	On-Demand hourly rate	Spot hourly rate	Number of vCPUs	Memory
c5.12xlarge	$2.04	$0.4984	48	96 GiB
m5.12xlarge	$2.304	$0.4933	48	192 GiB
c6i.16xlarge	$2.72	$0.7602	64	128 GiB
m6i.16xlarge	$3.072	$0.7406	64	256 GiB

Prerequisites

An AWS account with permissions to provision Amazon S3 buckets for input and output data storage, as well as sufficient permissions/limits to provision Amazon EC2 C5, M5, C6i, and M6i Instances.

How to benchmark Open Omics Acceleration Framework on AWS

The configuration details and steps used for benchmarking baseline and Open Omics versions of all three applications on EC2 Instances are detailed at IntelLab’s GitHub page. Typical process involves launching the corresponding EC2 Instances, connecting to the instances, installing the software, downloading the datasets, and executing the baseline and Open Omics versions. In the following subsections, we report results for the three applications on on-demand instances with dedicated tenancy. Compared to on-demand costs shown, the EC2 Spot Instances can provide nearly 75% cost savings.

Benchmarking Results: BWA-MEM

We used m5.12xlarge and m6i.16xlarge instances, with 48 and 64 threads (one thread per vCPU), respectively. The m-instance types were used because they provide 4 GB memory per vCPU that is required to run Open Omics BWA-MEM.

Figure 1 shows that on the same instance type (m5), Open Omics BWA-MEM achieves 1.8-2.3x speedup over the baseline BWA-MEM. Using the m6i instance type gives further performance gain, achieving 2.6-3.5x over baseline BWA-MEM on m5. The performance reported here of Open Omics BWA-MEM on m6i instance is ~1.7x faster than best performance on latest GPU. Please refer to this blog post and this video for a comparison.

The speedups are lower for ERR194147 dataset because it has reads of length ~100, providing less scope of parallelization. In comparison, the other two datasets that have reads of length ~150. A majority of the modern short read sequencers have read lengths ≥ 150 and they are expected to grow further. Therefore, we can expect higher speedups in the future.

Figure 2 shows the price-performance chart for BWA-MEM. It demonstrates that Open Omics BWA-MEM achieves significant cost cuttings compared to baseline BWA-MEM. Moreover, the m6i instances not only provide faster performance compared to m5 instances, they also incur lower costs.

Benchmarking Results: Minimap2

For this experiment, we used c5.12xlarge and c6i.16xlarge instances, using 48 and 64 threads (one thread per vCPU), respectively. Figure 3 shows that on the same instance type (c5), Open Omics minimap2 achieves 1.5-1.9x speedup over the baseline minimap2. Using the c6i instance type, gives further performance gain achieving 2-2.4x over baseline minimap2 on c5.

The price-performance chart shown in Figure 4 clearly demonstrates that Open Omics minimap2 costs nearly the same on the c6i and c5 instances, while achieving significant cost savings over the baseline minimap2 running on c5.

Benchmarking Results: ATAC-Seq data analysis

Figure 5 compares the execution time of the baseline and Open Omics versions of ATACWorks on c5 and c6i instances. The baseline version of ATACWorks is created by replacing the CUDA based deep learning modules with Intel^® oneDNN library. Open Omics version uses Intel’s new optimized implementation of the 1D convolutions. The chart shows…

Read the full blog to learn more. Reminder: You can learn a lot from AWS HPC engineers by subscribing to the HPC Tech Short YouTube channel, and following the AWS HPC Blog channel.