Network-Attached Memory: Virtualization for Java Environments

By Dennis Barker

October 6, 2008

Easing application scalability across a cluster is a problem being solved in a variety of ways. Terracotta does it with memory – network-attached memory, to be exact. The company’s infrastructure software creates an expandable/retractable pool of shared memory that Java applications can tap to meet increasing demand.

Network-attached memory is analogous to network-attached storage (NAS) in that it provides a service to thousands of connected clients transparently. As NAS is transparent underneath the file system, network-attached memory is transparent underneath the Java language, says Jeff Hartley, vice president of products and marketing at Terracotta. Objects are manipulated and kept consistent in memory like files are in NAS, but in Terracotta’s memory pool, everything can be massively scaled out.

“Another way to think of it is virtualization for the Java environment,” Hartley says. “In the same way that a hypervisor slices a machine into several logical machines, Terracotta takes many physical machines and connects them as one logical machine.”

Developers don’t have to change their applications to get clustering behavior. A clustered application looks the same as a Java application. Essentially, users tell Terracotta what to do in a config file, Hartley says, and the software injects that behavior into the application at runtime. Not only is there is no API required to send messages across the cluster, he says, there is no API at all. Terracotta uses plain old Java objects and plugs into familiar frameworks like Spring, Hibernate and EHcache so developers can continue using the same tools, stacks and development models.

“We provide an open source clustering solution,” says Hartley. “It’s scalability and high availability for Java enterprise apps, without having to change your application code. We actually hook into the Java virtual machine and share data.”

Beside simplified clustering and scalable performance, Hartley says Terracotta’s approach also brings “high availability without tradeoffs” — the possibility of reducing database bottlenecks, achieving better use of hardware, lowering maintenance costs, and, because the software takes care of adding clustering capabilities, focusing on developing new applications rather than retooling old ones.

“We’re working at the level of memory. If you have App Server A die, the user gets sent to App Server B, and all the data is sent from memory to B without the user even realizing it,” Hartley says. “We provide high availability by putting everything in memory instead of adding racks of servers.”

When you add a server to handle demand, it just “joins the group.” “You don’t have to implement anything. Just add the servers and we make them members of the cluster’s shared memory pool. Our server keeps your server’s data in sync,” Hartley says. Changes to one virtual machine are instantaneously reflected to every virtual machine throughout the cluster that needs to know.

Because data can be shared between Java virtual machines and processed at in-memory speeds, some customers use the software to take load off their databases. Transient data like user session info or shopping cart info can be kept and processed in memory, while only critical results are sent to and kept in the database. “Terracotta is used to handle the work-in-progress data while a process is running, and only the completed data goes to the database. As a result, some of our users have been able to reduce database utilization by as much as 70 percent and not have to buy more database licenses to meet increased workload,” Hartley says.

Eliminating mundane work for IT staff is one of Terracotta’s other major selling points. As company CTO Ari Zilka explains in a video tour of the software, “You don’t have to write the plumbing or maintain the plumbing. …You can run an app on two servers at midnight and on 20 servers at noon. It frees IT to run apps the way they need to run them” and focus development time on more important business issues, he says.

Deployment involves two primary components: client nodes and the Terracotta server array. Nodes run on standard Java VMs, and each node corresponds to a Java process in the cluster (e.g., the application server). Terracotta is loaded into each VM at startup. The Terracotta server array provides the intelligence to orchestrate all the nodes in the cluster, synchronize activity between them, replicate data and handle storing data to disk. The array can run in an active-passive pair configuration for high availability that can achieve tens of thousands of requests per second, the company says. Running multiple instances of the Terracotta server in active-passive mode guarantees that a failure won’t compromise the cluster, the company says.

“We’re a very fundamental technology. We’re not a grid solution, we’re not a cloud solution. We’re network-attached memory — distributed memory that sits right below the applications, and can be used for all kinds of things,” explains Hartley.

Demanding Customers

Customers running Terracotta typically have one thing in common: unpredictable numbers of demanding users. (Zilka used to be chief architect at Walmart.com, so he knows about building infrastructure to deal with traffic spikes of epic proportion.) The user list includes big names familiar with meeting heavy online demand, including Adobe, MapQuest, BBC, Electronic Arts, PartyGaming (PartyPoker.com, etc.), and financial services companies like JP Morgan and Mizuho Securities.

An online multiplayer gaming company that wishes to remain unnamed uses clustered servers to not just host games, but also to coordinate and track player activities. It chose Terracotta as its scale-out solution for several reasons: (1) it works behind the scenes, at byte-code level, providing distributed heap memory across multiple Java VMs; (2) developers can use the standard Java semantics for synchronizing access to shared objects; (3) “impressive horizontal scalability” enabled by just adding more server nodes and not having to use databases and caches to manage shared data; (4) no single point of failure; and (5) it’s open source, so the software doesn’t add to the bottom line and doesn’t require a purchase order to get started.

Mark Turansky, a software architect currently working in the health care industry, has written about his experience using Terracotta. “With enabling software like Terracotta, clustering becomes easy. You’ve still got to design your software to take advantage of parallelism, but the act of running programs in parallel is no longer difficult. …It invisibly and magically clusters your Java classes via configuration,” Turansky writes. “Distributing code and running massively parallel programs used to be difficult. It required complex architectures and expensive application servers. This is accidental complexity. Advances in software development — like Terracotta, GridGain, Spring, and other FOSS [free open source] programs — dramatically reduce if not eliminate the accidental complexity of distributing your programs to a cluster of machines.”

Gnip, a service that aggregates and distributes feeds from sites like Twitter and Digg to users of services like Plaxo, runs its system on Amazon’s EC2 but chose Terracotta for node replication at the memory level. A post on the company’s site explains: “The prospect of just writing our app and thinking of it as a single thing, rather than ‘how does all this state get replicated across n number of nodes’ was soooo appealing.”

Terracotta has just released version 2.7, which the company says is more tightly integrated with frameworks like Spring and Glassfish, and adds better management and visualization features, improved garbage collection and the ability to apply hot patches. It is available for download at www. terracotta.org.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion XL — were added to the benchmark suite as MLPerf continues Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing power it brings to artificial intelligence.  Nvidia's DGX Read more…

Call for Participation in Workshop on Potential NSF CISE Quantum Initiative

March 26, 2024

Editor’s Note: Next month there will be a workshop to discuss what a quantum initiative led by NSF’s Computer, Information Science and Engineering (CISE) directorate could entail. The details are posted below in a Ca Read more…

Waseda U. Researchers Reports New Quantum Algorithm for Speeding Optimization

March 25, 2024

Optimization problems cover a wide range of applications and are often cited as good candidates for quantum computing. However, the execution time for constrained combinatorial optimization applications on quantum device Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at the network layer threatens to make bigger and brawnier pro Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as the the ability to train 1 trillion pa Read more…

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HB Read more…

Nvidia Looks to Accelerate GenAI Adoption with NIM

March 19, 2024

Today at the GPU Technology Conference, Nvidia launched a new offering aimed at helping customers quickly deploy their generative AI applications in a secure, s Read more…

The Generative AI Future Is Now, Nvidia’s Huang Says

March 19, 2024

We are in the early days of a transformative shift in how business gets done thanks to the advent of generative AI, according to Nvidia CEO and cofounder Jensen Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

Intel Won’t Have a Xeon Max Chip with New Emerald Rapids CPU

December 14, 2023

As expected, Intel officially announced its 5th generation Xeon server chips codenamed Emerald Rapids at an event in New York City, where the focus was really o Read more…

IBM Quantum Summit: Two New QPUs, Upgraded Qiskit, 10-year Roadmap and More

December 4, 2023

IBM kicks off its annual Quantum Summit today and will announce a broad range of advances including its much-anticipated 1121-qubit Condor QPU, a smaller 133-qu Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire