Research at the Dell EMC and Intel HPC and AI Innovation Lab is demonstrating how organizations can build better, faster neural networks to drive recommendation engines.

Recommendation engines power the suggestive retail space. If you go to Netflix® to look for the next movie you want to watch, a recommendation engine will give you suggestions tailored to your interests and past viewing experiences. When you visit a website, you’re likely to see ads based on your browsing history and past purchases. If you shop on Amazon®, you will get all kinds of recommendations based on your purchasing history and the purchasing history of other customers with similar interests.

As Amazon explains, “We examine the items you’ve purchased, items you’ve told us you own, items you’ve rated, and items you’ve told us you like. Based on those interests, we make recommendations.” The company notes that its recommendations change regularly based on a number of factors, including “when you purchase, rate or like a new item, as well as changes in the interests of other customers like you.”[1]

And as good as today’s recommendation engines are, this is only the beginning. The retail world and its use cases for recommendation engines will be dramatically transformed by advances in artificial intelligence over the next five years. To this end, Intel® and the Dell EMC HPC and AI Innovation Lab are working actively to make the process of training AI models faster and more efficient on PowerEdge servers and solutions, in order to build better recommendations engines.

The path to a solution

The Lab team is exploring the use and optimization of neural networks to drive recommendation engines. The focus is on neural networks because they can outperform more traditional machine learning approaches in terms of accuracy of the recommendation made.

That’s the upside. The downside is that neural networks are very hard to train. And they are very sequential. That means you have to pick small samples, little snippets out of the data, and pass them through one at a time and then slowly, over the course of days or weeks, train your neural network to make recommendations for you.

Obviously, it is not very efficient to train a neural network if you have to train it one data point at a time, especially when you could have billions of data points that you need to pass through the network. So the team took on the task of making the process of training neural networks more efficient on Dell EMC PowerEdge™ servers with Intel® Xeon® processors in the Zenith cluster in the HPC and AI Innovation Lab.

In the first step in this process, the team developed a technique for using parallel processing to train this type of neural network, using parallel processing such as those used for image classification and language translation. Basically, the Lab wanted to create a neural net that can process more than one experience at a time, so it could do the task quickly and efficiently.

At this point, the team looked to the world of high performance computing to see what techniques had been used there to solve similar types of problems. This investigation led the lab team to an approach called Markov-Chain Monte Carlo (MCMC), which is basically a way of randomly sampling things at very high rates using parallel processing systems.

They then used that approach as the inspiration for parallelizing the training of Restricted Boltzmann Machines (RBMs), which is the type of neural network used to make recommendation engines that rely on collaborative filtering techniques.

For our experiments, they tested the parallelized training of an RBM model using the open source Horovod distributed deep learning framework and a publicly available MovieLens dataset with a billion movie recommendations. For this process, a Lab data scientist wrote a custom optimizer, or a custom set of software tools, then plugged it into the Horovod framework to parallelize the training of our RBM neural network.

The results

With the MovieLens dataset, the model went from two full days of training, or a little over 51 hours, to train our model on a single CPU node to 28 minutes to train our model on 140 nodes. This means that it can now process a billion rows of data in less than a half hour and impart all of that knowledge to a neural network so that it can make recommendations.

In our test case, the recommendations are for movies — for example, if you liked this last movie, you will probably also like this next movie. But this engine isn’t specific to movies. You could use these methodologies for any kind of rating system. You could do it for consumer products, books, ads or whatever the particular business case might be.

Even better, in experiments using this technique to train an RBM neural network with a MovieLens dataset, the team showed that both strong and weak scaling could be maintained out to 64 compute nodes while producing quality models in accordance with the scale of the dataset used. In future work, the Lab team will focus on training the model at an even greater scale using even larger datasets.

Key takeaways

The retail world and its use cases for recommendation engines will be dramatically transformed by advances in AI in the coming years. To further these advances, the Dell EMC HPC and AI Innovation Lab is working actively to make the process of training AI algorithms faster and more efficient on Dell EMC infrastructure in order to build better recommendations engines.

The Dell EMC HPC and AI Innovation Lab team demonstrated that organizations can parallelize the training of the type of neural networks on the Intel® 2^nd Gen Xeon Scalable Processors used to make recommendation engines. They further showed that with these approaches, organizations can achieve very good recommendations very quickly, even with large datasets.

One of the advantages of this approach with training the recommendation engine on the Intel-based PowerEdge servers is that it allows multiple workloads on a single system. These same systems can be used for AI, modeling and simulation and high performance data analytics (HPDA) – and that can translate into savings with a competitive edge. With the ability to capture large amounts of consumer data, analyze it and act on it cost effectively, retailers can differentiate themselves in an ever-growing competitive environment.

Retailers, content providers and others can capitalize on recommendation engines as a foundation for applications that can be developed and deployed today to drive customers to consumer products, music, video, books and countless other offerings. And a recommendation engine that is easily trained and retrained, that produces high-quality suggestions and scales to huge datasets, can be a significant driver of sales revenue.

To learn more

• Read “Putting Recommendation Engines to the Test in the Dell EMC HPC and AI Innovation Lab.”
• For deeper and more technical dive into the lab’s research focused on recommendation engines, see the paper “Parallelized Training of Restricted Boltzmann Machines using Markov-Chain Monte Carlo Methods.”
• To learn more about the resources available through the Dell EMC HPC and AI Innovation Lab, visit com/innovationlab.
• To learn more about Intel software optimizations, visit intel.com/ai

[1] Amazon, “About Recommendations,” accessed September 29, 2019.