Data is flowing out of every instrument – faster and bigger each day – creating massive quantities of data that needs to be captured, stored, cleaned, managed, shared, analyzed and archived for long retention periods.
How can we:
- Solve this big data issue to avoid a situation where data preparation is becoming a bottleneck?
- Deploy comprehensive compute capabilities that support rapidly evolving frameworks and applications?
- Enable collaboration among organizations across global borders?
- Turn massive data sets into medically actionable insights?
Frank Lee, PhD, IBM Global Industry Leader for Healthcare and Life Sciences, and his extended team, designed a high-performance data and AI architecture that helps world-class organizations solve some of the biggest genomics and imaging research challenges. Frank has the privilege to meet and partner with healthcare and life sciences organizations around the globe on a daily basis. We met with Frank recently to get his perspective, based on more than twenty years of industry expertise, on proven practices that help organizations transform biomedical research data into insights that drive precision medicine.
Q: You meet with so many healthcare and life sciences organizations around the globe that are trying to accelerate discoveries in genomics medicine. What are the key data challenges they all face?
Frank: “Genomics medicine is transforming the healthcare industry. Hospitals, medical centers, research institutions and pharmaceuticals companies are using genomic technology to understand the biology of disease, improve the diagnosis of patients and develop new drugs and treatments for better outcomes.
Along with that, there are three key challenges. The first one is the amount of data that these institutions need to deal with. For example, the data coming from the new generation sequencer is growing at tremendous speed and volume. This data is flowing at a rate of Terabytes per day, or even per hour now, and it can be very challenging to store the data, analyze the data, and complete the analysis of the data in a timely manner.
The second challenge is the complexity of the data. We’re not talking about just one set of data.
You need to put a lot of data together into one coherent data set that then can be analyzed. This compels organizations to develop common best practices that allow them to put the data together and solve the complexity issue.
The third challenge is what I call the provenance of the data. We know that data is only relevant, usable and useful when we put it into context. For example, we need to have a clear understanding about the provenance of metadata – how the data that an application generates relates to patient data. Without having this information, the data would be pretty much useless, so we need to pull all this data together in the right way and label it, solve the complexity issue and at the same time store and analyze it as fast as possible.”
Q – What was the vision behind the reference architecture you designed?
Frank: “The architecture we designed is tackling all the data challenges at the same time. We wanted to address the inefficiencies and limitations of the current healthcare model by providing a solid IT foundation that will help healthcare organizations of all sizes modernize and transform their current capabilities with greater performance, scalability and flexibility. We provide a cost-effective infrastructure that supports more data types, applications and frameworks that keep evolving, which requires much more comprehensive compute capabilities.
Our architecture for high-performance data and AI platforms is deployed for cloud-scale data management, multi-cloud workload orchestration and converged HPC with deep learning. It is based on a data hub which helps to manage the ocean of data that is siloed in disparate systems using advanced tiering functions, peering, and cataloguing. The advanced capabilities allow the data to be captured very rapidly, stored safely, accessed at all times and be shared globally in the most secure way with collaborators or partners wherever the data is needed.
We also provide a very efficient scalable computational capability based on a shared infrastructure to orchestrate applications and deploy policy-driven resource management with critical functions like parallel computing and pipelining for faster time to insights and better outcomes.
We can extend the capabilities of the data hub and orchestrator and have them work together, not only within the research institution or the hospital data center, but also extended all the way to the cloud.
Our vision is becoming reality, enabling effective collaboration between research and clinical teams, providing organizations with comprehensive data analysis capabilities and enhancing the HPC platform to support the next phase of their precision medicine journey.”
For the last several years, leading healthcare organizations have implemented IBM’s high-performance compute and storage systems to support research and clinical applications and help them make personalized healthcare a reality. These real-life use cases demonstrate the importance of having a powerful underlying infrastructure to fully exploit the value of data and achieve high performance, low cost, ease of use and collaboration.
More from Frank Lee
Watch Frank Lee present best practices for high performance genomics and imaging that will help you derive faster clinical insights:
To learn more about the critical IT functions you’ll need to manage your oceans of data and put guard rails around the jungle of applications, read the smart tips guide to high performance data and AI architecture.
Frank Lee, PhD
Global Industry Leader for Healthcare and Life Sciences, IBM Systems
Chief Architect of High Performance Data & AI for Healthcare
IBM Storage CTO Office Member
Dr. Frank Lee is the Healthcare and Life Science industry leader for IBM Systems Group with over twenty years’ experience in scientific research and information technology. His work includes the creation of an industry reference architecture and its implementation as HPC, cloud, big data and AI platforms for dozens of clients and partners worldwide. As an advocate for the transformation of the industry towards precision medicine, Frank has spoken in dozens of conferences and published in IBM System Journals, Redbooks, research papers and HIMSS report. After encountering gaps in technologies, Frank drove innovation with inventions in metadata and provenance management. Frank also brings expertise in genomics, including participation in the Human Genome Project as a research associate and training as a molecular biologist at Washington University.