Increasingly, it’s a Big Data world we live in. Just in case you’ve been living under a rock and need proof of that, <a href=”http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/” target=”_blank”>a major retailer can use an unimaginable number of data points to predict the pregnancy of a teenage girl outside Minneapolis before she gets a chance to tell her family</a>. That’s just one example, but there are countless others that point to the idea that mining huge data volumes can uncover gold nuggets of actionable proportions (although sometimes they freak people out…)
<img style=”float: left;” src=”http://media2.hpcwire.com/hpcwire/OpenSFS_logoCROPPED.jpg” alt=”” width=”95″ height=”51″ />OpenSFS has chosen its Community Representative Director for 2013: Tommy Minyard, director of Advanced Computing Systems (ACS) at the Texas Advanced Computing Center (TACC). We got the new director’s views on Lustre’s opportunities in big data and exascale, maintaining a single source tree, and new features on the horizon.
We have identified five key ways that Univa creates business value for our customers for a modest investment. By upgrading Grid Engine an organization can reduce downtime, boost its jobs per day throughput and find ways to manage application license costs to help meet the budget cut requirements in capital spending imposed by the reality of tough times.
Several lectures from the VSCSE Summer School on Science Clouds are now available for viewing on YouTube.
With the ever increasing explosion in data for analysis and the need for fast insights on emerging trends, in-memory data grids (IMDGs) offer a highly attractive platform for hosting map/reduce analysis. In comparison to disk-based map/reduce platforms such as Hadoop, IMDGs reduce analysis times by reducing data motion while simplifying the development model. For applications which need to analyze fast-changing application data, such as shopping or financial trading data, IMDGs can provide near real-time results.
SGI, Microsoft warm up their Hadoop offerings.
Culling together massive data has provided some profound opportunities for a wide array of analytics projects but has created a number of complications for those who want to gain actionable intelligence from it. While the “big data” movement is still unfolding, a number of companies have emerged to help simplify access and use, especially of unstructured information. HPC stalwart Platform Computing entered the race to refine handling of vast datasets — not to mention the management behind such operations to stake their claim in this emerging space.
Another company has emerged from the woodwork to help bring the power of Hadoop to a hungry enterprise audience, this one focusing on refining the open source tool itself.
This week we checked in with experts from Microsoft, IBM, Yahoo, Facebook’s Hadoop engineering team and others to discuss some diverse issues related to big data, including changes in use, frameworks and applications. We also identify the year’s most profound trends and speculate on what lies ahead.
As the IBM Watson supercomputer prepares to battle human champions on Jeopardy, the Apache Software Foundation highlights the role of open source software keys to supercomputer trivia performance.