Visit additional Tabor Communication Publications
January 19, 2011
Algorithmic trading is getting another cycle of press scrutiny, thanks mainly to a very well-researched article in Wired by Reuters financial blogger Felix Salmon and Ars Technica writer Jon Stokes. In it, they outline how pervasive these high-tech algorithms have become to the everyday running of financial trading. And the problem is no one knows how this software drives the market behavior -- not the investors, not the traders, and not even the people who run Wall Street.
The motivation for all this high-tech trading is, of course, money. And in this case, he who develops the fastest system usually wins. This often comes down to placing the trading servers in the same room as the stock exchange servers to get that millisecond edge on executing transactions. The code too is designed for maximum speed, being constantly tweaked to squeeze the last ounce of performance from the underlying computer chips. Appro recently launched a server based on overclocked Intel "Westmere" CPUs, to give high frequency traders that extra speed boost. But all that digitally enhanced speed means it's that much harder for humans to control.
That's partly because that computer-generated bids can be executed so quickly (10,000 bids per second for a single stock) and in such a complex manner that humans cannot comprehend the ramifications. The feedback loops become intertwined, such that the entire trading system exhibits emergent behavior, untraceable to any particular piece of code.
In a recent interview on NPR's Fresh Air program, Salmon declared. "The man danger about algorithmic trading is that we simply don't understand it." He says although the individual algorithms are controlled, and presumably understood, by their masters, the interactions between them are not.
In researching his article, Salmon talked with Michael Kearns, a CS Prof at the University of Pennsylvania, who has developed algorithms for various Wall Street firms. Kearns told him that the financial markets have become what he called an "automated adaptive dynamical system with feedback." That may sound very cool, but according to Kearns there is no science he's aware of that is able to understand such a system.
It should come as no surprise that occasionally such a system would run the financial markets into a ditch. That happened last May, with the so-called flash crash, when the Dow Jones Industrial Average plummeted 900 points in a matter of minutes -- before regaining most of its value. The cause was traced to a relatively obscure mutual fund company that decided to make a very large trade in a very short amount of time (about 20 minutes). The algorithms monitoring the market interpreted this as a panic and came to the same decision all at once: sell. The reason the mutual fund company decided to dump the shares in the first place was to hedge against the possibility of a future stock market drop. Talk about self-fulfilling prophesies.
In the wake of the flash crash, the Securities and Exchange Commission (SEC) announced some measures intended to prevent a reoccurrence. These include "circuit breakers" procedures, such as automatically halting trading when a stocks share price fluctuates by more than 10 percent in 5 minutes. The SEC is also considering other measures like limiting the size and speed of trades and requiring a complete audit trail of all transactions.
But Salmon considers those rather crude remedies for such a tightly wound system. The flash crash event was actually a rather simple example of what could go wrong. The interactions between all the analytic software inhabiting Wall Street datacenters is much more complex. For example, unlike that mutual fund company that executed the large trade all at once, algorithmic trading software tries to hide a big buy or sell events with a series of smaller transactions so as not to tip their hand.
Meanwhile, other algorithms are simultaneously monitoring the activity to discern the larger patterns that the other codes are trying to hide. In some cases, even more devious codes will purposely initiate transactions with no intention of executing them in order to confuse their rival software. It's very much algorithmic warfare, with no real thought given to collateral damage.
The quantitative analysts themselves have become somewhat innocent bystanders. The Wired article describes a typical quant shop, in this case Berkeley-based Voleon Capital Management, that specializes in statistical arbitrage. The idea is to process mounds of financial data, looking for patterns that would point to an profitable arbitrage opportunity. But the quants have no knowledge of the underlying fundamentals of the assets; they are simply looking for patterns. To them, it's just a pile of bits unrelated to any larger reality.
The software is becoming more sophisticated too. Salmon documents a recently launched service, called Dow Jones Lexicon, that mines the text in financial news stories and attempts to map keywords to market conditions, the idea being to help predict market trends based on external events. Although such software is in its nascent stage, this could add a whole other layer of complexity to trading models.
The fact that so much trading -- the majority, in fact -- is performed algorithmically suggests that the market is no longer balanced between investors and speculators. And since the speculation component is being propelled by superfast computers, the market has become increasingly volatile and unpredictable. Even before and after the May 2010 flash crash, there have been a number of examples of unexplained price fluctuations.
The University of Pennsylvania's Kearns suggests that we should to build a ginormous stock market simulator in order to provide some much-needed science for our market structures. In a recent Reuters blog by Salmon, Kearns is quoted about this at length. Although, the professor doesn't see a simulator as a magic bullet, in his estimation it's certainly the place to start.
Given the importance of the stock market to the economy, its increasing susceptibility to damaging volatility, and the lack of our understanding of current system, a simulator project seems like a no-brainer. Sounds like a nice little science project for the SEC.
Posted by Michael Feldman - January 19, 2011 @ 6:12 PM, Pacific Standard Time
Michael Feldman is the editor of HPCwire.
No Recent Blog Comments
In a recent solicitation, the NSF laid out needs for furthering its scientific and engineering infrastructure with new tools to go beyond top performance, Having already delivered systems like Stampede and Blue Waters, they're turning an eye to solving data-intensive challenges. We spoke with the agency's Irene Qualters and Barry Schneider about..
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
The Xeon Phi coprocessor might be the new kid on the high performance block, but out of all first-rate kickers of the Intel tires, the Texas Advanced Computing Center (TACC) got the first real jab with its new top ten Stampede system.We talk with the center's Karl Schultz about the challenges of programming for Phi--but more specifically, the optimization...
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
May 10, 2013 |
Program provides cash awards up to $10,000 for the best open-source end-user applications deployed on 100G network.
May 09, 2013 |
The Japanese government has revealed its plans to best its previous K Computer efforts with what they hope will be the first exascale system...
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.