An AI Primer

Zaremba describes OpenAI’s aim as developing AI for humanity’s benefit and mentions work in robotics and game learning. He breaks robotics into locomotion, navigation, and manipulation, describing manipulation as the hardest because robots cannot reliably grasp arbitrary objects. A factory robot can be programmed for a specific item, but that solution may not generalize.

Zaremba says learning across many games could help AI transfer knowledge, as people do when they quickly pick up a new sport. He notes that Atari systems can require about three years of real-time play to learn simple games, though parallel computing can shorten training to days. In robotics, rewards are harder to define: for a robot scrambling eggs, measuring whether the result is good is difficult, and the environment cannot simply be reset.

Zaremba defines AI broadly as software that solves problems through intelligent behavior, including rule-based systems such as early Google Search using PageRank. Machine learning builds programs from data, with supervised learning mapping examples such as emails to labels such as spam or not spam. Deep learning uses multiple computation steps, and neural networks implement these through layers of matrix operations and nonlinear functions such as ReLU.

Zaremba says neural networks became useful because researchers could train them effectively, aided by stochastic gradient descent, more computing power, and techniques such as convolutional networks. Convolutions apply the same local computation across an image, and related methods can process speech represented by time and frequency. He points to ImageNet, with one million images across 1,000 classes, where reported error rates fell from about 25% to roughly 3% over several years; similar neural network approaches also advanced speech recognition and translation.

Zaremba distinguishes narrow AI, built for a specific problem, from general AI, which could handle many problems, and superintelligence, which exceeds human intelligence. He says supervised learning is the machine learning approach most ready for business, when enough input and output examples are available, while robotics remains difficult because tasks like grasping are hard to specify and train. He recommends Coursera, TensorFlow tutorials, and Andrej Karpathy’s Stanford class for learners, and argues that automation may require basic income.