Tag: Hadoop

Adapting Hadoop to HPC Environments

Feb 14, 2014 |

MapReduce is well known for its relative ease of use in today’s ubiquitous world of parallelism. The beauty of the model is in its ability to absolve or abstract away details of parallelism, fault tolerance, synchronization, and input management from the user. A user typically writes her algorithm using two uniform functions — a map Read more…

How HPC is Hacking Hadoop

Feb 11, 2014 |

Although the trend may be quiet and distributed across only a relative few supercomputing sites, Hadoop and HPC are hopping hand-in-hand more frequently. These two technology areas aren’t necessarily made for another—there are limitations in what Hadoop can do. But a stretch of recent research has been pushing the possibilities, especially when it comes to Read more…

Cray Advances Hadoop for HPC

Feb 4, 2014 |

In a recent blog entry, Mike Boros, Hadoop Product Marketing Manager at Cray, Inc., writes about the company’s positioning of Hadoop for scientific big data. Like the old adage, “when the only tool you have is a hammer, every problem begins to resemble a nail,” Boros suggests that the Law of the Instrument may be true Read more…

SC13 Spotlights Three Main Trends

Dec 18, 2013 |

Collecting my thoughts after a thrilling and enlightening Supercomputing Conference in Denver, I want to discuss some of the key trends and highlights observed. The annual conference is always a hotbed of product announcements and this year, the 25th running, was no exception. There are many articles covering the key announcements, so I instead want Read more…

Cray Targets Oil and Gas Sector’s Big Data Needs

Sep 25, 2013 |

Supercomputer-maker Cray is helping oil and gas companies benefit from the most-advanced reservoir modeling approach yet. Called Permanent Reservoir Monitoring, or PRM, the technique requires innovative data warehousing technology and data analysis techniques.

Accelerate Hadoop MapReduce Performance using Dedicated OrangeFS Servers

Sep 9, 2013 |

Recent tests performed at Clemson University achieved a 25 percent improvement in Apache Hadoop Terasort run times by replacing Hadoop Distributed File System (HDFS) with an OrangeFS configuration using dedicated servers. Key components included extension of the MapReduce “FileSystem” class and a Java Native Interface (JNI) shim to the OrangeFS client. No modifications of Hadoop were required, and existing MapReduce jobs require no modification to utilize OrangeFS.

Cray Bundles Intel’s Hadoop with CS300 Line of Supercomputers

Jun 20, 2013 |

This month, Cray will begin delivery of a new big data analytics cluster that combines a of its entry-level CS300 system that’s been optimized to run Intel’s Hadoop distribution. Cray says the new system will provide customers with a “turnkey” Hadoop cluster that can tackle big data problems that would be difficult to solve using commodity hardware.

Cray Cracks Commercial HPC Code

Jun 20, 2013 |

During a conversation this week with Cray CEO, Peter Ungaro, we learned that the company has managed to extend its reach into the enterprise HPC market quite dramatically–at least in supercomputing business terms. With steady growth into these markets, however, the focus on hardware versus the software side of certain problems for such users is….

Hacking into the N-Queens Problem with Virtualization

Jun 19, 2013 |

Ruan Pethiyagoda, Cameron Boehmer, John S. Dvorak, and Tim Sze, trained at San Francisco’s Hack Reactor, an institute designed for intense fast paced learning of programming, put together a program based on the N-Queens algorithm designed by the University of Cambridge’s Martin Richards, and modified it to run in parallel across multiple machines.

TACC Longhorn Takes On Natural Language Processing

Jun 14, 2013 |

For all the progress we’ve made in IT over the last 50 years, there’s one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.