Andrew Ng
VP & Chief Scientist, Baidu
Chairman and Co-founder, Coursera
Adjunct Professor, Stanford University

In 2011, Andrew Ng led the development of Stanford University’s main MOOC (Massive Open Online Courses) platform and also taught an online Machine Learning class to over 100,000 students, leading to the founding of Coursera. Ng’s goal is to give everyone in the world access to a great education, for free. Today, Coursera partners with some of the top universities in the world to offer high quality online courses, and is the largest MOOC platform in the world.

Ng also works on machine learning with an emphasis on deep learning. He founded and led the “Google Brain” project which developed massive-scale deep learning algorithms. This resulted in the famous “Google cat” result, in which a massive neural network with 1 billion parameters learned from unlabeled YouTube videos to detect cats. More recently, he continues to work on deep learning and its applications to computer vision and speech, including such applications as autonomous driving.

HPCwire: Hi Andrew. Congratulations on being selected as an HPCwire 2017 Person to Watch! While deep learning strategies are seemingly becoming critical for technology suppliers, the “hyperscalers” such as Baidu and Facebook are both deep learning developers as well as its biggest users. What are the key enabling DL technologies you see emerging and which seems the most promising?

Andrew Ng: Neural networks trained using supervised learning are already creating tremendous economic value, but there’s still ample room for growth of such models. Looking further out, I think transfer learning (and its sibling multi-task learning) will play a growing role. Even further out, unsupervised learning (including GANs) will also become important.

For example, at Baidu, deep learning trained with supervised learning is critical to web search, speech recognition, face recognition, anti-fraud, consumer loan approvals, even estimating the ETA of your food delivery order. There are still numerous supervised learning applications where our ability to acquire data surpasses our ability to efficiently process that data, so advances in HPC for deep learning offer a clear path to improving performance.

For problems where we’re running short on data, transfer learning is also becoming more important. For example, we might learn a speech recognition model for mandarin Chinese—for which we have ample data–and “transfer” what we’ve learned to a less commonly spoken dialect for which we have less data. In computer vision, speech, machine translation, and other areas, transfer learning is becoming more important.

Finally, we have a lot of unlabeled data, but not yet great algorithms for learning from such data. Research on unsupervised learning—which can learn from this unlabeled data—will go faster if we can build HPC systems that enable researchers to experiment efficiently on large amounts of data.

In summary, there are still many deep learning tasks that are bottlenecked on computational speed. We are also still not yet able to use existing large systems to their full potential for deep learning. Advances in HPC will both make existing algorithms work better, and help researchers invent new algorithms.

HPCwire: At ISC 2016, you said AI is the new electricity. Could you elaborate on what you meant by that and the vision you have for the future of AI?

100 years ago, the electrification of the US transformed industry after industry—manufacturing, transportation, communications, agriculture, and more. Today, AI is similarly poised to transform numerous industries. Just supervised learning trained on labeled data is enabling computers to do tasks that they couldn’t before, and this will be enough to change many industries. If new algorithms, perhaps in unsupervised learning, are invented, the potential of this transformation grows even further.

I find it hard to think of a major industry that I don’t think AI will transform.

HPCwire: As a founder and chairman of Coursera, you must have strong views on the role of online education. What’s your sense of the role online education can play and do you expect to see technical degree programs become more widely accepted by employers searching for new talent?

AI is very good at optimizing routine, repetitive tasks. Over the next few years, I think AI create tremendous value, but also displace numerous jobs. Similar to the earlier waves of technological job disruption, AI will also create new jobs, including many that we can’t yet imagine today—say, drug customization specialist, or 3D printed clothing designer. But we need a way to help those whose jobs were displaced gain the needed skills to tackle the new jobs. Scalable forms of education—such as MOOCs—will be part of the solution. But we need even more. For example, I favor a form of Basic Income which provides a social safety net, but rather than paying the unemployed to “do nothing”, we instead pay them to study, thus increasing the odds of them gaining the skills needed to re-enter the workforce and contribute back to the tax base that is paying for this income. Politicians will have to work with corporations and with universities to devise a new incentives and standards for education, to make sure every person has a path to meaningful work.

HPCwire: Outside of the professional sphere, what can you tell us about yourself – personal life, family, background, hobbies, etc.? Is there anything about you your colleagues might be surprised to learn?

My favorite way to spend a Saturday is staying at home reading on my Kindle, which currently has ~1300 books. I try to finish at least a book a week. I’ve recently been reading a lot about healthcare. (Example: The Digital Doctor, but Robert Wachter.) I think there’s a huge opportunity for AI to help, but many technologists also underestimate how hard it is to design something that the healthcare ecosystem will accept.

Ian Buck NVIDIA	Adolfy Hoisie PNNL	John Martinis Google
Satoshi Matsuoka AIST AIRC	Paul Messina ECP	Bernd Mohr SC17
Andrew Ng Baidu	Kirk Skaugen Lenovo	Avinash Sodani Cavium
Lisa Su AMD	Jeffrey Vetter ORNL	Guangwen Yang NSCC Wuxi