Although the trend may be quiet and distributed across only a relative few supercomputing sites, Hadoop and HPC are hopping hand-in-hand more frequently. These two technology areas aren’t necessarily made for another—there are limitations in what Hadoop can do. But a stretch of recent research has been pushing the possibilities, especially when it comes to Read more…
In a recent blog entry, Mike Boros, Hadoop Product Marketing Manager at Cray, Inc., writes about the company’s positioning of Hadoop for scientific big data. Like the old adage, “when the only tool you have is a hammer, every problem begins to resemble a nail,” Boros suggests that the Law of the Instrument may be true Read more…
Collecting my thoughts after a thrilling and enlightening Supercomputing Conference in Denver, I want to discuss some of the key trends and highlights observed. The annual conference is always a hotbed of product announcements and this year, the 25th running, was no exception. There are many articles covering the key announcements, so I instead want Read more…
Supercomputer-maker Cray is helping oil and gas companies benefit from the most-advanced reservoir modeling approach yet. Called Permanent Reservoir Monitoring, or PRM, the technique requires innovative data warehousing technology and data analysis techniques.
Recent tests performed at Clemson University achieved a 25 percent improvement in Apache Hadoop Terasort run times by replacing Hadoop Distributed File System (HDFS) with an OrangeFS configuration using dedicated servers. Key components included extension of the MapReduce “FileSystem” class and a Java Native Interface (JNI) shim to the OrangeFS client. No modifications of Hadoop were required, and existing MapReduce jobs require no modification to utilize OrangeFS.
This month, Cray will begin delivery of a new big data analytics cluster that combines a of its entry-level CS300 system that’s been optimized to run Intel’s Hadoop distribution. Cray says the new system will provide customers with a “turnkey” Hadoop cluster that can tackle big data problems that would be difficult to solve using commodity hardware.
During a conversation this week with Cray CEO, Peter Ungaro, we learned that the company has managed to extend its reach into the enterprise HPC market quite dramatically–at least in supercomputing business terms. With steady growth into these markets, however, the focus on hardware versus the software side of certain problems for such users is….
Ruan Pethiyagoda, Cameron Boehmer, John S. Dvorak, and Tim Sze, trained at San Francisco’s Hack Reactor, an institute designed for intense fast paced learning of programming, put together a program based on the N-Queens algorithm designed by the University of Cambridge’s Martin Richards, and modified it to run in parallel across multiple machines.
For all the progress we’ve made in IT over the last 50 years, there’s one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.
Today Intel announced some new pitches to push Lustre in front of enterprise eyeballs with usability features for Lustre and a total rip and replace for the native Hadoop file system designed to appeal to the HPC-oriented Hadoop set. We talked with Brent Gorda, former CEO and founder of Whamcloud, which Intel acquired just a tick under a year ago about how….