Visit additional Tabor Communication Publications
July 08, 2009
There were a couple of stories floating around the Intertubes in the past week or so that reminded me of how little we know about large classes of HPC applications. That's not a good thing.
The first story is about the arrest of former Goldman Sachs computer programmer, Sergey Aleynikov, who allegedly made off with proprietary trading software the firm used to execute high volume, low latency automated trading. The news was covered in Bloomberg and elsewhere, and is a compelling tale of espionage in the financial services industry. Here's the money quote, so to speak, from the Bloomberg report:
The proprietary code lets the firm do "sophisticated, high-speed and high-volume trades on various stock and commodities markets," prosecutors said in court papers. The trades generate "many millions of dollars" each year.
The second story concerns espionage too -- in this case, the more old-fashioned kind. Apparently the National Security Agency (NSA) is planning to construct a $2 billion dollar datacenter in Utah that will eventually consume 65 megawatts to power new supercomputers. The agency's current facility in Fort Meade, Maryland, is already power-constrained, preventing the NSA from installing any more supers at that site. The Salt Lake City Tribune reported that the new Utah datacenter will be used to support the NSA's intelligence-gathering mission. The Tribune used congressional budget documents to squeeze a bit more detail from the story:
The supercomputers in the center will be part of the NSA's signal intelligence program, which seeks to "gain a decisive information advantage for the nation and our allies under all circumstances" according to the documents.
Of course, the speculation is that the NSA is using these supers to perform domestic spying -- a program begun under the Bush administration and now being continued under the Obama regime. The Fort Meade machine is a Cray "Black Widow" super, which is reportedly sifting through emails and phone conversations to find out who's been naughty and nice.
The nexus of these two stories is that secrecy prevents the public from knowing the full breadth of HPC applications. In the case of the government, it's for national security reasons; for financial institutions, it's to maintain competitive advantage. But it's not just the financial industry and government intelligence domains (although these two areas are probably the most discrete in regard to technology transparency). You'll notice, for example, that the energy industry, commercial biotech companies, and automobile/aerospace manufacturers aren't giving public tours of their HPC datacenters either.
Even the TOP500 list reflects this secretive nature. The list is quickly becoming an anonymous record of supercomputing, where many of the users are only listed generically (for example, Financial Institution, Government, IT Provider, Semiconductor Company, and so on). This is especially true as you move toward the end of the list, where there are fewer public institutions. On the June 2009 TOP500 list, 78 of the bottom 100 supercomputers are listed anonymously.
By contrast, HPC being performed by national labs, supercomputing centers, and academic institutions is highly publicized, and tends to be very visible on the TOP500 list. These organizations are constantly on the prowl for grants, funding and other sorts of collaboration, so it pays for them to advertise what they're up to. Plus researchers tend to be a talkative bunch anyway. I suspect this is the reason that the public generally only associates supercomputing with applications like climate modeling and looking for galactic black holes.
What's the result of all this secrecy? Besides giving the public a skewed view of the industry, it also makes the technology invisible to a larger number of developers. Consider that most HPC apps are still implemented in legacy languages like Fortran and C, while "public" applications for personal computers or the Web are using more modern software frameworks, like Java, .NET, Python, etc. Even though HPC is not a volume industry in terms of software licenses, if more codes were public, you'd probably see a much more rapid development of libraries and tools (which is one reason why CUDA software has developed so quickly). Keeping software in silos makes for a lousy ecosystem.
The other aspect to secrecy is that it encourages the kind of bad behavior that Aleynikov and the US government are being accused of. There's nothing inherently wrong with protecting state secrets and proprietary IP, but eventually the whole model can become self defeating. Consider this: arguably, two of the biggest catastrophes of this young 21st century are that of US intelligence regarding 9/11 and Iraq and the collapse of the financial industry. Both institutions relied on keeping information siloed to such an extent that even the institutions themselves couldn't explain the data. And when the secrets are lies, nobody will know until it's too late.
Posted by Michael Feldman - July 08, 2009 @ 5:37 PM, Pacific Daylight Time
Michael Feldman is the editor of HPCwire.
No Recent Blog Comments
In quieter times, sounding the bell of funding big science with big systems tends to resonate further than when ears are already burning with sour economic and national security news. For exascale's future, however, the time could be ripe to instill some sense of urgency....
In a recent solicitation, the NSF laid out needs for furthering its scientific and engineering infrastructure with new tools to go beyond top performance, Having already delivered systems like Stampede and Blue Waters, they're turning an eye to solving data-intensive challenges. We spoke with the agency's Irene Qualters and Barry Schneider about..
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
May 22, 2013 |
At some point in the not-too-distant future, building powerful, miniature computing systems will be considered a hobby for high schoolers, just as robotics or even Lego-building are today. That could be made possible through recent advancements made with the Raspberry Pi computers.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.