Deep Learning’s Accuracy

Deep learning has knocked down one record after another on benchmark dataset after benchmark dataset since 2006. In many competitions, the only algorithm deep learning is up against is itself. Below are a few reports and resources indicating the power and persistent advance of deep neural nets:

• MNIST: Regularization of Neural Networks using DropConnect; 0.21% error using DropConnect
• DeepFace: Closing the Gap to Human-Level Performance in Face Verification; Reaches an accuracy of 97.35% on the Labeled Faces in the Wild (LFW) dataset, reducing the error of the current state of the art by more than 27%, closely approaching human-level performance (~97.53 percent).
• Andrew Ng, in his deep-learning talks circa 2011, compiled a great list of quantum leaps in accuracy deep learning has made in many domains.
• More recently, he has led his team at Baidu to several breakthroughs in speech and image recognition:
• Deep Speech outperformed, by about 9 percent, top academic speech-recognition models on a popular dataset called Hub5’00, and proved accurate 81 percent of the time in noisy environments, beating the leading commercial algorithms.
• Google won the 2014 ImageNet competition with a 6.66 percent error rate using deep networks.
• Table of results for CIFAR-10 dataset
• Rodrigo Benenson’s blog, which includes CIFAR-10, CIFAR-100, STL-10, SVHN, ILSVRC2012 task 1, Pascal VOC 2011 comp3, Caltech Pedestrians USA, INRIA Persons, MSRC-21, Leeds Sport Poses and the Salient Object Detection benchmark, among others.
• When Google adopted deep-learning-based speech recognition in its Android smartphone operating system, it achieved a 25% reduction in word errors.
• Kaggle President Jeremy Howard, now with Enlitic, said in 2013 that most winners used either Ensembles of decision trees (random forests) or Deep Learning
• Yann LeCun’s former student Pierre Sermanet won the Dogs vs Cats competition on Kaggle using a DL library.
• George Dahl et al improved pharma company Merck’s ability to predict useful drug candidates by 15% using deep learning in 2012.
• In 2013, all entrants to the ImageNet competition used deep learning.

Here’s a list of the competitions that Juergen Schmidhuber and his team have won with their deep recurrent nets:

• IJCNN 2011 Traffic Sign Recognition Competition
• ICPR 2012 Contest on “Mitosis Detection in Breast Cancer Histological Images.” This is important for breast cancer prognosis. Humans tend to find it very difficult to distinguish mitosis from other tissue.
• ISBI 2012 challenge on segmentation of neuronal structures. Given electron microscopy images of stacks of thin slices of animal brains, the goal is to build a detailed 3D model of the brain’s neurons and dendrites.
• ICDAR 2011 Offline Chinese Handwriting Competition. His team won the competition although none of its members speaks a word of Chinese.
• Online German Traffic Sign Recognition Contest (2011, first and second rank).
• ICDAR 2009 Arabic Connected Handwriting Competition
• ICDAR 2009 Handwritten Farsi/Arabic Character Recognition Competition (idem).
• ICDAR 2009 French Connected Handwriting Competition.

We could go on, but we’d start to sound like a broken record. :)