Visit additional Tabor Communication Publications
November 18, 2009
At SC09 this week, Mitrionics announced it has started to work on an experimental compiler that aims to make parallel programming architecture-agnostic. The goal of the work is to extend the Mitrion-C platform for FPGAs to multicore CPUs, cluster architectures, and eventually even GPGPUs. We asked Stefan Möhl, Mitrionics' chief science officer and co-founder, to explain what's behind the new technology and what prompted the decision to add support for other parallel architectures.
HPCwire: Can you tell us about the new programming capabilities of the Mitrionics platform that you announced here at the Supercomputing Conference?
Stefan Möhl: Well, we haven't added new programming capabilities to the Mitrionics' Accelerated Computing Platform yet. We are still in the proof-of-concept stage with this new compiler, but things look very promising. For this proof-of-concept compiler, the news is that existing Mitrion-C code, originally written for the MVP on FPGAs, will now also run on multicores and clusters. This initial proof-of-concept was made only to prove that the basic principles work, so there are limits to what code we can currently run. A production version of a portable programming language will require changes to Mitrion-C to make it less focused on what is needed for FPGA acceleration.
HPCwire: How does it work?
Möhl: The main challenge when porting between parallel architectures is that the level of granularity of the parallelism differs. For example, to parallelize code for vector processors, you would have to parallelize inner-most loops. To parallelize code for clusters, you would have to parallelize the outer-most loops. Doing general automatic parallelization (parallelization without re-writing the code) has not been solved, even after decades of research. Nor is there a general automatic way to transform one kind of parallelism into another.
Mitrion-C was originally developed as a programming language for the Mitrion Virtual Processor (MVP). The MVP is a hardware design for a compute engine specifically developed for high-performance execution in FPGAs. As such, it is full MIMD ((Multiple Instruction stream, Multiple Data stream) at the individual instruction level, so it potentially executes every single instruction of the program in parallel. This can be thought of as a limit-case for parallelism. Mitrion-C is a C-family language that supports and aids the programmer in specifying the kind of parallelism that the MVP requires. It is roughly as similar to ANSI-C as Java or C# are, so it isn't too unusual to use.
The trick that makes Mitrion-C work for parallel portability comes from an important asymmetry in parallelization. Though automatic parallelization without code re-writes is very hard to achieve, general automatic sequentialization is much, much easier. Trivially, operating systems have run multiple programs in parallel on sequential processors for many years. For efficient execution, there are of course many optimization considerations, but it is still much easier than automatic parallelization. This property is what we use to port Mitrion-C between platforms. Since the code is fully parallel from the start, we never parallelize at all, we only sequentialize. So for a cluster, instead of parallelizing outer-most loops, we sequentialize everything except the outer-most loops. And for a vector processor, we sequentialize everything except for the inner-most loops.
HPCwire: So if you don't have the parallelization problem, how can you handle the various memory architectures of multicore CPUs, GPGPUs and clusters, and so on?
Möhl: Our FPGA background has required us to consider these issues carefully from the start. FPGAs are usually connected to the system on data buses designed for devices with an order of magnitude less performance than FPGAs. So Mitrion-C was designed from the start to allow programmers to manage both memory latency and raw memory bandwidth in an effective manner. This issue will become increasingly important also for multicores and manycores, since increasing core counts without increasing clock-frequencies of data buses will put them in the same situation FPGAs have always been in.
Another important aspect comes from the diversity of FPGA cards. There are almost no two FPGA cards with the same memory sub-system, so we had to design Mitrion-C to have a memory model that addresses this from the start.
In Mitrion-C, there is no assumption of a single monolithic memory space. Instead, each collection may have its own address space, and different ones for different memory size and bandwidth requirements. This allows programmers to manually stage data from few, large and slow memories to many, small and fast memories in any number of levels. There are also several different built-in types for multi-dimensional data collections that let programmers specify what kind of access patterns a collection should permit. This helps the programmer in making correct and efficient programs, and also lets the compiler know what types of memory to place the data collection in. Of course, you can still write a program that requires more, larger or faster memories than a particular system has, but Mitrion-C will at least make you aware of what you demand of the system.
HPCwire: Can the exact same Mitrion-C source code be compiled to any of target architectures?
Möhl: Yes. You will need to parameterize for the number of cores you want to run on in a cluster or multicore, or how much unroll you want for loops in an FPGA, but other than that, the same code works without changes. However, not all algorithms will be efficient on all architectures, so the programmer will in some cases need to consider what platform to run the algorithms in, or change the algorithms to suit the available platform.
HPCwire: Mitrionics has focused on FPGA software development since its inception in 2001. What prompted the decision to target other architectures?
Möhl: Well, we are actually still focused on FPGAs. What prompted this is a customer interest in running Mitrion-C on standard processors and not only the Mitrion Virtual Processor. Customers want to be able to write an algorithm once and make efficient use of it on systems with and without FPGAs. They would also like to avoid having their code "locked in" to FPGAs. So we set up an experiment at Mitrionics to see what can be done with Mitrion-C on other platforms. And, as it turns out, very much can be done!
HPCwire: There are already a number of programming environments and languages that target multicore CPUs and GPGPUs and clusters. What does Mitrionics brings to the table?
Möhl: Three main things. First, Mitrion-C is a single, coherent language that maintains the same style of programming regardless of what platform you run it on. Programming languages like MPI, OpenMP, OpenCL and CUDA are really several different languages mixed together. There is the base-line C-code which is purely sequential, then there are added parts for clusters (in the case of MPI), multicores (in the case of OpenMP), or GPUs (in the case of OpenCL and CUDA). Often, you even have to combine them, such as with MPI+OpenMP. The additions introduce completely different ways of doing things than what the sequential C code does. They are not just added syntax in the sequential C paradigm. That means that you are really writing in several different languages at the same time, and need to learn them all to be able to do it properly. It also complicates the code dramatically.
Second, the fact that you have separate syntax for each architecture means that you need to re-write your code to move it between architectures. With our solution, software developers can make a single investment in writing code, and then use it on any architecture depending on what is optimal under current circumstances. With a universal programming language that can be used to target any architecture without changing syntax, it also becomes possible to explore the benefits and possibilities of different architectures much faster, in the end resulting in more efficient code.
Finally, and perhaps most tantalizing, is that the portability is not limited to the architectures that are popular today. History has seen a wide range of architectures -- from the old scalar processors, vector processors, Thinking Machines, MasPar and SIMD, the Multi-Threaded Architecture, large shared memory machines, MPPs, and clusters to today's FPGAs, GPUs, Cell, multicores and several others. Each new generation has required code re-writes. Though this is not yet proven, there is good hope that Mitrion-C would be efficient without re-writes on most of the historical popular parallel architectures. If that is the case, it bodes well for parallel architectures of the future too. Though we probably won't be able to say "Never again!" to re-writes for all eternity, Mitrion-C holds the promise of dramatically reduce the number of re-writes we will need to do in the future.
Jun 19, 2013 |
Supercomputer architectures have evolved considerably over the last 20 years, particularly in the number of processors that are linked together. One aspect of HPC architecture that hasn't changed is the MPI programming model.
Jun 18, 2013 |
The world's largest supercomputers, like Tianhe-2, are great at traditional, compute-intensive HPC workloads, such as simulating atomic decay or modeling tornados. But data-intensive applications--such as mining big data sets for connections--is a different sort of workload, and runs best on a different sort of computer.
Jun 18, 2013 |
Researchers are finding innovative uses for Gordon, the 285 teraflop supercomputer housed at the San Diego Supercomputer Center (SDSC) that has a unique Flash-based storage system. Since going online, researchers have put the incredibly fast I/O to use on a wide variety of workloads, ranging from chemistry to political science.
Jun 17, 2013 |
The advent of low-power mobile processors and cloud delivery models is changing the economics of computing. But just as an economy car is good at different things than a full size truck, an HPC workload still has certain computing demands that neither the fastest smartphone nor the most elastic cloud cluster can fulfill.
Jun 14, 2013 |
For all the progress we've made in IT over the last 50 years, there's one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?
Join our webinar to learn how IT managers can migrate to a more resilient, flexible and scalable solution that grows with the data center. Mellanox VMS is future-proof, efficient and brings significant CAPEX and OPEX savings. The VMS is available today.