In early April the SAS Institute (SAS) announced it had integrated its most advanced analytics software into database appliances from EMC Greenplum and Teradata Corporation. The new offerings marry high performance computing to “big data” and are designed to enable users to perform deep analysis on huge datasets hosted on purpose-built, parallel computing platforms.

Today SAS is considered the undisputed leader in advanced analytics — that according to IDC who, in 2009, pegged the company with a 34.7 percent market share in this category. A subset of business analytics, advanced analytics uses compute-intensive data mining and statistical software techniques to extract complex relationships from databases. For SAS, it’s a half a billion dollar business.

Competitors include IBM’s SPSS and lesser-used offerings from Microsoft, TIBCO, Oracle and others. Revolution Analytics, which recently developed an enterprise-capable version of R for statistical analysis, has only 100 or so deployments at this point, but its leverage of the popular open-source R language introduces a new model for advanced analytics users.

At the simplest level, advanced analytics allow you to develop models and then use them to ask “What if?” questions about your data. For example, developing a statistical model that associates buying behavior with customer profiles can then be applied to future behavior of customers. The application of that model is refered to as “scoring” and is the basis for predictive analytics.

That type of analysis is worlds away from traditional business intelligence, which is more about asking simple questions about data in one or two dimensions (e.g., How many shoes of Brand X do we have in stock?). That kind of analysis is fairly straightforward using a traditional database, needing only a small pipe to get the data in and out and a software component on the client to manage the interface.

“Business intelligence got shanghaied during the 1980s to just mean query and reporting,” says SAS CTO Keith Collins. “We are talking about much more than that.”

According to Collins, the high performance analytics SAS has in mind will be a “game changer” for the industry. He says it will do so by addressing both sides of the problem: the increasing size of enterprise datasets — terabytes, scaling to petabytes — and the need to get actionable intelligence from them in a timely manner. Traditionally, the compute- and data-intensive nature of advanced analytics tools has relegated their use to dataset samples, which not only requires extra time and effort, but also introduces inaccuracies associated with working on incomplete data.

The obvious solution is to put the compute next to the data, in this case, on the high performance data platforms themselves, thus eliminating the need to sample. And since these appliances are essentially HPC clusters (with added storage and software needed to house large databases), the CPUs and memory can be used to run the analytics natively. The data prep, model creation and scoring as well as the actual analytics are performed on the appliance servers, and in parallel fashion.

Conveniently, this can be done within the existing SAS language environment. Customers with legacy code can apply those applications to this new high performance environment with the trivial specification of HP (high performance) at the time of invocation. All of this is made possible by the invention of relatively inexpensive database appliances, which, like the HPC industry in general, has moved from SMP architectures to distributed clustered platforms employing commodity parts, Linux, and x86 CPUs.

In the case of Teradata and Greenplum, the basic appliance hardware is very similar, both based on dual-socket 2.93 GHz Westmere Xeon CPUs and outfitted with 48 GB of memory per node. The Teradata platform uses a proprietary system interconnect called BYNET, while the Greenplum machines rely on standard 10Gig Ethernet.

Storage-wise, the Teradata platform sports 1 and 2 TB SATA drives, and can scale from 45 TB on a single server instance up to 186 PB on 4,096 nodes. Alternatively, the company offers a performance version that uses SSD technology and tops out at 24 TB of total capacity.

Greenplum also has capacity and performance models of its appliance, employing both hard drives and SSDs accordingly. In this case, though, the spinning drives are Serial Attached SCSI. In Greenplum’s high capacity configuration, its appliance scales from 31 TB in a quarter rack up to 744 TB in six full racks.

In early April, SAS demonstrated the power of high performance analytics at its Global Forum meeting. In the first case, two racks (16 nodes) of Greenplum’s Data Computing Appliance (DCA) were used to run a logistic regression of bank loan defaults across a database with a billion records, applying just a few variables. The regression was able to complete in less than 80 seconds (as compared to 20 hours for an unspecified serial implementation). Another demonstration, this time on a 24-node Teradata platform, used 1,800 variables applied to 50 million observations. In this case, the analysis finished in 42 seconds.

Not everyone will require this integrated model for high performance, but every use case for advanced analytics is fair game. This includes everything from fraud detection, loan analysis, customer preference tracking, and financial risk scoring, to improving manufacturing yields. The San Antonio Spurs basketball team has even used the technology to “optimize player performance.”

Collins says the early adopters for its high performance analytics offerings will be in the insurance and financial sectors, where the value obtained is easily transferred to the bottom line. Although he wouldn’t name names, SAS already has some number of companies under trial with the technology. General availability for the product on both the Greenplum and Teradata platforms is scheduled for the fourth quarter of 2011.