Python Snakes Its Way Into HPC

By Nicole Hemsoth

November 17, 2010

Interpreted programming languages usually don’t find too many friends in high performance computing. Yet Python, one of the most popular general-purpose interpreted languages, has garnered a small community of enthusiastic followers. True believers got the opportunity to hear about the language in the HPC realm in a tutorial session on Monday and a BoF session on Wednesday. Argonne National Lab’s William Scullin, who participated in both events, talked with HPCwire about the status of Python in this space and what developers might look forward to.

HPCwire: Python is not a language normally associated with high performance or scientific computing. What does it have to offer this user community not being fulfilled by traditional languages, like C, Fortran or other high productivity, interpreted languages like MATLAB?

William Scullin: In a way, Python’s growing adoption in the high performance and scientific computing space is a homecoming. Guido Van Rossum originally began Python as a way of providing an administrative scripting language for the Amoeba distributed operating system. Then as now, it combines simple, easy to learn and maintain syntax with access to the same powerful libraries and function calls you would find in any C or Fortran implementation. While there has always been an emphasis on reducing the time it takes to perform a computation, Python has truly shined in improving scientific computing by taking the work out of programming and reducing the time to solution.

Often, projects fail when they try to be all things to all people. MATLAB, Mathematica, SPSS, and Maple are all very useful tools, in part because they are focused on meeting the needs of a well funded community with very specific goals. Python, arising from a very diverse community that ranges from astrophysicists to game programmers to web designers to entry level computer science students, has been very successful due to the diversity of users. The standard library has become amazingly extensive without becoming inconsistent.

Likewise, the amount of software that has come out of the community is amazing, most of which is open source, and the vast majority of which follows the same coding guidelines as the core modules. This makes it possible to easily develop an interface to an embedded microcontroller to turn off the desk lamp when your simulation finally ends and automatically push results to a web server in less than an hour — or alternately turn on a coffee pot and resubmit your job when the simulation fails — all in one language.

HPCwire: Obviously, performance is a driving issue in HPC. How is the issue of execution performance being addressed?

Scullin: Performance is a matter of perspective. A favorite maxim in the Python community is that the greatest performance improvement comes from going to the working from the non-working state. A second maxim, from Knuth, is that premature optimization is the root of all evil. While the execution speed of a Python application may not be as fast as one written in C, C++, or Fortran, its ease of use and low learning curve sharply improves overall time to solution. It’s a question of developer time versus compute time.

Side stepping the issue, it’s ridiculously easy to extend Python with modules written in C, C++, and Fortran. It’s common in our community to utilize compiled high performance numerical kernels, then use Python to handle areas like I/O, workflow management, computational analysis, and steering. When areas become performance bottlenecks, those areas tend to be rewritten in C.

Conversely, I’ve seen C and Fortran projects where code complexity has prevented maintenance and functionality, leading to thousands of lines of compiled code being replaced with less than a hundred lines of Python. In many ways, Python is coming to fulfill the roles that frameworks like Cactus and Samurai sought to fill at the start of the decade — letting scientists worry about their problems while letting the language and interpreter do the heavy lifting.

HPCwire: Do you think a compiled implementation of Python would be a step in the right direction?

Scullin: There will always be a place for the interpreted reference implementation, especially in development, but if a Python compiler comes along that provides better performance without compromising the language, I can’t see it finding much resistance.

That said, there are currently projects such as Unladen Swallow, PyPy, Stackless Python, Jython, and Iron Python that provide alternatives to the CPython interpreter. Unladen Swallow, backed in part by Google, and PyPy both seek to close the performance gap with compiled languages. Unladen Swallow is particularly exciting as it’s backended into the Low Level Virtual Machine, which is the basis for multiple compilers including Clang, currently the default compiler under Apple’s OS X. This makes a Python compiler more a matter of when than if.

HPCwire: Can you describe some of the more important Python initiatives — language extensions, libraries, tools, etc. — that are aimed at the HPC domain?

Scullin: I cannot speak highly enough of NumPy, which is almost the Swiss army knife of Python for scientific and high performance computing. It’s been under active development for years now with each release providing better performance, automatic integration of popular high performance libraries like BLAS and LAPACK, more features, and greater portability. NumPy is further extended by SciPy, which provides additional tools and lab kits addressing almost every science domain.

Likewise, I think very highly of mpi4py, PyMPI, PyCUDA and its sister PyOpenCL, petsc4py, and PyTrilinos. All of these keep improving the options we have to accelerate our code using the very same tools and interfaces that are available through traditional compiled languages with none of the complexity.

HPCwire: Are there vendors out there with commercially-supported solutions?

Scullin: Indeed, and more importantly, most of them are active contributors to and supporters of the Python community. I can no longer count the number of consulting firms that provide Python solutions. It’s also been very encouraging seeing vendors add Python support to their products. Two companies well known in the HPC space, Rogue Wave and ParaTools, have both been very responsive.

Rogue Wave has provided access to their mathematical libraries, IMSL via PyIMSL. Furthermore, they have brought a number of people into the Python community via PyIMSL Studio which they market officially as a prototyping tool. I’ve encountered PyIMSL studio users so happy with their prototype Python applications with PyIMSL Studio, that they ran with the Python code as production code. I should also mention that while the TotalView debugger is not officially a Python tool, it’s seen a lot of use by Python HPC users and it will be interesting to see where it goes since Rogue Wave’s acquisition of Acumen.

ParaTools, a major contributor to the TAU Performance System and a leading consultant in the area of parallel and high performance codes has done a very good job of adding Python support to TAU.

Without hesitation, I have used their tools with C, Fortran, and Python and found their support to be helpful and responsive regardless of language.

While not directly in the HPC market, Enthought, deserves special mention. They host an array of Python projects with engineering and science applications. They provide a commercial packaging of the Python interpreter with commonly used libraries and utilities along with technical support as the Enthought Python Distribution. Most of all, they are active developers of NumPy and SciPy. Without their support and involvement, I am not sure that NumPy would have come together as nicely as it has.

While relatively new, I’ll also be interested to see what the future holds for MBA Sciences’s SPM.Python toolkit for bringing parallelism into serial Python programs. I’ll be keeping a close watch on PiCloud, a firm which provides an amazingly easy to user cloud computing platform that makes running Python codes on a compute cloud ridiculously easy. PiCloud users have their computations offloaded without any serious code changes, having to be involved in any aspect of setting up a cloud infrastructure, or doing any server management. They’ve seriously made it as simple as coding and running.

Finally, though it hasn’t been making a lot of noise lately, NVIDIA has been putting effort behind Copperhead, which while not a complete Python, allows for the rapid development of CUDA kernels in Python-like code.

HPCwire: Do you think most uses of Python in HPC will eventually involve either integration with C or Fortran or source code translation to those languages?

Scullin: I believe that HPC users will continue to choose the best possible tool to address a need in a given situation. Python is flexible enough that there will be continued integration with C, Fortran, and other languages. At the same time, interpreter performance is being rapidly addressed, which makes the issues that come with language translation into C and Fortran cause that sort of project to be less attractive to active Python developers. What will be interesting to watch is how codes written in a mix of C, C++, Fortran, Python and other languages perform and evolve as the LLVM platform continues to mature.

HPCwire: Can you point to any successful case studies or projects where Python has been employed in this arena?

Scullin: At Argonne, we are involved in the development of GPAW, a density-functional theory Python code based on the projector-augmented wave method. Originating out of an international collaboration, it is mostly a mix of C and Python with the vast majority of the code being Python. It has been run at scale successfully and routinely on our Blue Gene platform. While the porting of any application to platforms like the Cray XT series or the Blue Gene is an interesting exercise in computer science, it’s far more remarkable that the performance has been on par from what I’ve seen in C or C++ codes. Moreover, it is being used to produce reliable data used to generate publications.

The other community that a lot of people think of when looking for successful Python applications in the HPC space is bioinformatics. While I’ve not been involved with many bioinformatics codes, the last four or five years have seen a rising number of chemists and biologists appearing on Python-related mailing lists and at conferences discussing how they have been using Python to power their science. While Perl still holds sway in the field, Python is quickly becoming almost as popular.

HPCwire: For those HPC developers interested in learning more about what’s available in the Python ecosystem, can you point to some resources they could tap into?

Scullin: Depending on their particular interests, one of the best places to start is by visiting www.scipy.org. From there, you can find links to numerous mailing lists, information about conferences, code recipes, documentation, and much more. In the Chicago and Bay Areas there are very active Python users groups with sizable memberships with an interest in HPC and scientific computing. Finally, given Python’s ease of use, one of the best things you can do is to spend an afternoon with the interpreter, simply playing with code and seeing what the language can do for you without any effort. The joy of doing powerful things with simple code is one of the most admirable traits of the language.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

What’s New in HPC Research: Wind Farms, Gravitational Lenses, Web Portals & More

February 19, 2019

In this bimonthly feature, HPCwire highlights newly published research in the high-performance computing community and related domains. From parallel programming to exascale to quantum computing, the details are here. Read more…

By Oliver Peckham

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from the nanoscale to the astronomic, from calculating quantum effe Read more…

By Ken Strandberg

What Will IBM’s AI Debater Learn from Its Loss?

February 14, 2019

The utility of IBM’s latest man-versus-machine gambit is debatable. At the very least its Project Debater got us thinking about the potential uses of artificial intelligence as a way of helping humans sift through al Read more…

By George Leopold

HPE Extreme Performance Solutions

HPE Systems With Intel Omni-Path: Architected for Value and Accessible High-Performance Computing

Today’s high-performance computing (HPC) and artificial intelligence (AI) users value high performing clusters. And the higher the performance that their system can deliver, the better. Read more…

IBM Accelerated Insights

Medical Research Powered by Data

“We’re all the same, but we’re unique as well. In that uniqueness lies all of the answers….”

  • Mark Tykocinski, MD, Provost, Executive Vice President for Academic Affairs, Thomas Jefferson University

Getting the answers to what causes some people to develop diseases and not others is driving the groundbreaking medical research being conducted by the Computational Medicine Center at Thomas Jefferson University in Philadelphia. Read more…

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst of bankruptcy proceedings. According to Dutch news site Drimb Read more…

By Tiffany Trader

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from th Read more…

By Ken Strandberg

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

UC Berkeley Paper Heralds Rise of Serverless Computing in the Cloud – Do You Agree?

February 13, 2019

Almost exactly ten years to the day from publishing of their widely-read, seminal paper on cloud computing, UC Berkeley researchers have issued another ambitious examination of cloud computing - Cloud Programming Simplified: A Berkeley View on Serverless Computing. The new work heralds the rise of ‘serverless computing’ as the next dominant phase of cloud computing. Read more…

By John Russell

Iowa ‘Grows Its Own’ to Fill the HPC Workforce Pipeline

February 13, 2019

The global workforce that supports advanced computing, scientific software and high-speed research networks is relatively small when you stop to consider the magnitude of the transformative discoveries it empowers. Technical conferences provide a forum where specialists convene to learn about the latest innovations and schedule face-time with colleagues from other institutions. Read more…

By Elizabeth Leake, STEM-Trek

Trump Signs Executive Order Launching U.S. AI Initiative

February 11, 2019

U.S. President Donald Trump issued an Executive Order (EO) today launching a U.S Artificial Intelligence Initiative. The new initiative - Maintaining American L Read more…

By John Russell

Celebrating Women in Science: Meet Four Women Leading the Way in HPC

February 11, 2019

One only needs to look around at virtually any CS/tech conference to realize that women are underrepresented, and that holds true of HPC. SC hosts over 13,000 H Read more…

By AJ Lauer

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Assessing Government Shutdown’s Impact on HPC

February 6, 2019

After a 35-day federal government shutdown, the longest in U.S. history, government agencies are taking stock of the damage -- and girding for a potential secon Read more…

By Tiffany Trader

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This