Gedae Promises Performance and Portability

By John E. West

September 11, 2008

Developers in HPC have long lamented that you can either have performance or portability, but not both. You can get middling performance on many architectures with a single, portable code, but to get great performance, you need to tune your application for the features of a particular machine. For example, machine A, with a giant cache and slow inter-processor communication demands one approach to solving the problem, while machine B with vector processors, mid-sized cache, and fast interconnect demands another. This creates headaches for developers who have to manage multiple versions of an application for various platforms, inserting complex target-dependent logic into the application build process to ensure that users are able to get at the special features of their chosen platform.

For a while it looked like the need for platform-specific tuning might gradually fade away as the industry concentrated on the x86 architecture, but the rise of new architectures and re-introduction of hardware acceleration reminds us that in HPC the only constant is change. That’s why development platforms like Gedae, pronounced “JEE-day,” are getting renewed attention. Gedae is both the name of the product and the name of the company. The platform strikes a middle ground between performance and portability. Yes, you can separate implementation and specification, but you can also get into the implementation and tune to your heart’s content. Gedae CEO Bill Lundgren is sure his company is on to something big, and he expects this approach to become the standard technique for creating efficient, portable, parallel applications on everything from desktop computers to Top 10 machines.

Gedae is a complete application development system. Development starts with the expression of application functionality at a very high level using a visual “widget-on-a-string” editor, which CEO Bill Lundgren describes as a “thinking tool” for scientists and engineers. This type of environment is most typically associated with data flow computing, and that’s where Gedae’s roots are. But the company has worked hard to expand the basic data flow approach, which originated in real-time signal processing, to include the expressiveness needed to specify a wide range of computational problems. As a result, today Gedae is used in applications ranging from image and radar processing to fluid flow simulation and distributed battle simulation.

The developer then moves on to specifying how the functionality should be implemented and creates a separate implementation specification to express the problem- or machine-specific details of the application. For example, while the functional specification might say “multiply matrix A and matrix B,” the implementation file contains details of domain decomposition and architecture-specific tuning parameters (e.g., tile size). Because implementation and high-level behavior are separated, it is possible for scientists and engineers to work just on what their algorithm should do, while computer and computational scientists can be left to focus on how the final application should get those things done.

The Gedae compiler then takes both files as input and starts working on creating a parallel application. The compiler takes into account what machine it is compiling for, and uses information about the machine supplied to it in a hardware model to make very specific decisions about how to compile the application for best performance, lengthening vectors for machine A, and favoring fewer but larger messages on machine B.

The compiler also handles thread definition, task scheduling, and decomposition (following the rules built into the implementation file). This level of automation allows the compiler to worry about concurrency control and avoid deadlocks, in addition to optimizing memory sharing between threads or tasks. After the application is compiled, Gedae can monitor the performance of the application as it runs, and developers can iteratively refine the hardware-specific details based on actual application performance for better performance.

All of these features emphasize Gedae’s belief that the compiler should do as much work as possible. “It is absolutely essential to have a new language for creating parallel programs,” says Lundgren. “We also have to have a high degree of automation to produce reliable and accurate high performance codes, but to get there the compiler needs to have access to the same information the developer has.”

Gedae got its start in 1987 at RCA as a project to support the DoD’s need to program a set of newly acquired Connection Machines. The compiler was designed from the beginning to support multiple hardware architectures (“targets”), and as it was developed, it was also used to create applications for other computers of the day, including BBN’s Butterfly. This was at a time when computer scientists were just starting to get excited about object-oriented computing, and the first versions of Gedae were built around an object engine that executed applications in a managed runtime. While the system worked and was well-received by users, the object engine made many runtime decisions and added a lot of execution overhead that made Gedae inappropriate in many scenarios.

In 1995 Gedae was re-engineered and adapted for better performance. Part of the new mandate was for Gedae to create code for embedded systems, which meant moving away from a managed runtime. Efficient execution became a driving force, and Gedae added a compiler. This is also when the original data flow model was expanded with the semantics needed to express a much wider range of problems than data flow graphs can typically express.

Developers today are using Gedae to build applications targeted for IBM’s BlueGene and Cell processors, Intel’s multicore processors, and a variety of other specialized hardware, for applications in medical, defense and scientific domains. Gedae offers the features that Lundgren believes are essential to performance-oriented programming in multi-processor and multicore environments.

“Gedae separates functionality from implementation, but still exposes the knobs that developers want to tune to get the best performance,” he says. “Those knobs have good defaults on each platform, ensuring good performance for developers that lack the time or knowledge to do detailed performance tuning. Gedae’s compile approach keeps the code lean, without time consuming conditionals and library calls that conventional approaches need, and provides the observability tools developers require to understand what the compiler did and how to tweak the implementation in response.”

As Gedae plans out the near term for its namesake product, the company continues to focus on reducing complexity for developers and improving performance. An important upcoming change is the addition of more and better automation, including a rules-based engine for generating code that will adapt algorithms and implementation details based on a specific target platform, eliminating the need for developers to bring this knowledge to the process. This engine will also allow for the feedback of information gathered during profiling runs, enabling further support for performance tuning by the compiler. Another significant new feature in Gedae is the planned addition of a high level language, similar to MATLAB, to provide users with another path for specifying functionality.

“We are seeing an increasing number of cases from customers, especially in the last year, in which a Gedae-compiled application is outperforming hand-tuned code in real applications,” says Lundgren. He also believes that the success of Gedae thus far, and more importantly the success of the application development model, are strong indicators that this approach will be an important part of the application developer’s toolkit in the future.

As Lundgren puts it, “I expect this technique will become pervasive in the software development community. Whether Gedae itself becomes pervasive is up for debate, but we have proven that the strong separation of implementation and functionality works in practice and offers significant advantages for performance and productivity.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion XL — were added to the benchmark suite as MLPerf continues Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing power it brings to artificial intelligence.  Nvidia's DGX Read more…

Call for Participation in Workshop on Potential NSF CISE Quantum Initiative

March 26, 2024

Editor’s Note: Next month there will be a workshop to discuss what a quantum initiative led by NSF’s Computer, Information Science and Engineering (CISE) directorate could entail. The details are posted below in a Ca Read more…

Waseda U. Researchers Reports New Quantum Algorithm for Speeding Optimization

March 25, 2024

Optimization problems cover a wide range of applications and are often cited as good candidates for quantum computing. However, the execution time for constrained combinatorial optimization applications on quantum device Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at the network layer threatens to make bigger and brawnier pro Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as the the ability to train 1 trillion pa Read more…

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HB Read more…

Nvidia Looks to Accelerate GenAI Adoption with NIM

March 19, 2024

Today at the GPU Technology Conference, Nvidia launched a new offering aimed at helping customers quickly deploy their generative AI applications in a secure, s Read more…

The Generative AI Future Is Now, Nvidia’s Huang Says

March 19, 2024

We are in the early days of a transformative shift in how business gets done thanks to the advent of generative AI, according to Nvidia CEO and cofounder Jensen Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

Intel Won’t Have a Xeon Max Chip with New Emerald Rapids CPU

December 14, 2023

As expected, Intel officially announced its 5th generation Xeon server chips codenamed Emerald Rapids at an event in New York City, where the focus was really o Read more…

IBM Quantum Summit: Two New QPUs, Upgraded Qiskit, 10-year Roadmap and More

December 4, 2023

IBM kicks off its annual Quantum Summit today and will announce a broad range of advances including its much-anticipated 1121-qubit Condor QPU, a smaller 133-qu Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire