The field of deep learning is picking up steam to the point that it’s now inspiring a growing list of startups in areas such as natural language processing and image recognition. It’s also commanding a growing percentage of research and acquisition budgets at companies such as Google, Microsoft, Facebook and Yahoo. This post highlights some of the companies involved in this space and the type of products or projects they’re working on.
What deep learning is
First, though, a little primer: Despite its cognitive moniker, deep learning isn’t really about teaching machines to mimic the human brain a la the BRAIN Initiative President Obama announced in 2012. Rather, it’s about teaching machines to think more hierarchically or more contextually — to see a picture of a mole, for example, and work down from recognizing the features that comprise an animal to recognizing the specific features that make it a mole. With text, the process might be teaching machines to recognize how words are related to one another and how they fit together to form phrases or express ideas.
And while deep learning might hold huge promises in fields such medicine and astronomy, the best we can probably hope for in the near term are more-accurate text messages, search engines, language translation and targeted content. Why? Because that’s where the money is today and also where the data is.
In that sense, deep learning as a field is a lot like its bigger brother, big data: The best techniques might start off in (or be adopted from universities by) Silicon Valley offices solving arguably menial problems, but the big payoff will come when they make their way into new fields such as science and medicine. What’s more, deep learning models are quite computationally intensive, but the increasingly inexpensive, increasingly powerful resources available via cloud computing mean a lot more people without data centers at their disposal can get in on the action.
With that in mind, here’s a brief look at some of the companies, people and places I’ve come across in researching the field. I know this isn’t exhaustive, so please note any other notable people, companies or projects in the comments. We will try to update these lists periodically, as well.
The startups
AlchemyAPI: Denver, Colo.-based AlchemyAPI has been providing deep-learning-based text analysis via API for a while now, and recently showed off its work on image recognition, as well. A whole lot of companies you’ve heard of already use the service for tasks such as sentiment analysis, categorization and tagging. AlchemyAPI is working on new capabilities that should make it much more useful, even for folks who have no interest in consuming services via API.
Cortica: Cortica is an image-recognition specialist whose technology is modeled after the way the human brain processes images in the cortical neural network. In fact, Co-founder and CEO Igal Raichelgauz told me, its technology actually comes from laboratory research he did at the Israel Institute of Technology using real brain tissue. The company is already selling its technology as a product to publishers and advertisers that want to display ads or content related to on-screen images.
Ersatz: Ersatz is a deep learning platform developed by a San Francisco-based consulting firm called Blackcloud BSG. Rather than providing pre-determined capabilites such as image recognition or sentiment analysis, Ersatz provides the tools, kind of like an Amazon Web Services for deep learning. It gives users a web interface, API, GPU-based cloud resources and neural network implementations, and lets users go to town training and running their own models as they please. Alternatively, customers can just pay parent company Blackcloud to do everything for them.
Semantria: Amherst, Mass.-based Semantria is a spinoff of text-analysis veteran Lexalytics, only it’s delivered via API — or Excel plugin — rather than installed software. According to Founder and CEO Oleg Rogynskyy, Semantria is also improving its accuracy by incorporating more deep learning methods and expanding its data sources beyond Wikipedia (which is where the Lexalytics engine derives its semantic knowledge).
I suspect these companies are just a small percentage of what we’ll see pop in the next couple of years, though. Some well-known deep learning scholars in fields such as natural language processing and image recognition already have stealth-mode startups in the works, and it wouldn’t be surprising to see other vision-based startups such as Dropcam and Sight Machine adopt these techniques as they mature their products. There are other startups, too — such as Palm creator Jeff Hawkins’s new startup called Grok that straddle the line between deep learning concepts and actual brain-like computing architectures — that could push the field into new areas.
The big companies
Facebook: Facebook is a relative newcomer to the field, but certainly has plenty of text and images to analyze. The company is reportedly hoping deep learning will help it better optimize its NewsFeed application and allow for a more compelling photo-sharing experience. However, Facebook’s biggest contribution to the deep learning field might be its penchant for building and open sourcing specialized hardware — something that could come in handy for others trying to build infrastructure that can handle these workloads.
Google: Google is probably the most well-known company in the deep learning field, thanks in part to its highly publicized image-recognition research (its models were able to identify cats and human faces without any training) and its recent decision to open source some new tools for text analysis. It’s able to do so much in part because it has lots of computing resources, lots of smart people and lots of data. Deep learning is already powering speech recognition on Android phones and you can search your Google+ photos by content without ever having tagged them.
[protected-iframe id=”cfff1d55949ce206bba86475c7742b7d-14960843-6578147″ info=”http://new.livestream.com/accounts/74987/events/2117818/videos/21957034/player?autoPlay=false&height=360&mute=false&width=640″ width=”640″ height=”360″ frameborder=”0″ scrolling=”no”]
IBM: You’ve heard of Watson, right? The IBM system that defeated Jeopardy! champions and is now being applied to fields such as health care is an amalgmation of numerous data-analysis techniques, including deep learning. Beyond that, IBM is also pushing a number of efforts around what it calls “cognitive computing,” including a partnership with four leading universities that includes deep learning as a core research area.
[youtube=http://www.youtube.com/watch?v=07XPEqkHJ6U&w=640&h=360]
Microsoft: Microsoft Research has been at the big data game for a long time, and last November then-director Rich Rashid made headlines when he showed off its deep learning prowess by demonstrating a live English-to-Mandarin translation tool during an event in China. Like Google, Microsoft is collecting all sorts of data from its various web and mobile applications, and hopes deep learning can help it provide more compelling experiences on its web, mobile, gaming and even business-software platforms — although its ace in the hole might be Kinect.
[youtube=http://www.youtube.com/watch?v=Nu-nlQqFCKg&w=640&h=360]
Yahoo: Yahoo doesn’t get the attention that Google and Microsoft do, but it has snapped up two deep-learning-based image-recognition startups — IQ Engines and LookFlow — in the past few months. Yahoo has all sorts of ways it could utilize these sorts of technologies and people, but the obvious tactic is making Flickr more appealing by making it smarter and perhaps setting it loose on smartphone picture galleries.
The researchers
The Toronto crew: In many ways, the University of Toronto — specifically a research group led by Geoffrey Hinton — is the reason there’s so much talk about deep learning today. The team made some big breakthroughs several years ago and ultimately launched a startup that Google acquired in 2013. Hinton now splits his time between the University of Toronto and Google. Marc’Aurelio Ranzato was a member of Hinton’s lab (and earned his Ph.D. from NYU) and recently left Google to be part of Facebook’s deep learning team.
The Stanford crew: Stanford is one of the hotbeds of deep learning research, with professor and Coursera founder Andrew Ng among the biggest names in the space and Christopher Manning heading up the school’s NLP program. We recently covered research by a member of his team, Ph.D. candidate Richard Socher, that focuses on understanding whole sentences rather than single words and is 85 percent accurate in analyzing the sentiment of movie reviews from a popular dataset.
The NYU crew: Yann LeCun and Rob Fergus of NYU are both respected deep learning researchers and experts in image recognition, with LeCun having done some seminal work on handwriting analysis in the ’90s and Fergus currently doing work on making neural networks less like black boxes. NYU is also the university heading up deep learning research as part of the aforementioned IBM cognitive computing partnership.
The Montreal crew: The LISA Lab at the University of Montreal, led by Yoshua Bengio, is another major center for deep learning research, and created a popular open source library called Theano (you can download it here) that makes it easier to complex mathematical algorithms in Python and run them on GPUs (which are ideal for neural networks because of their parallelism). And, bonus, one of Theano’s creators, Joseph Turian, is also a Gigaom Research analyst.
The Switzerland crew: Juergen Schmidhuber and his team at the Dalle Molle Institute of Artificial Intelligence have been doing deep learning since 1991, and now that processing power is catching up with the models they’ve developed, their work has been taken to the next level. The team has won numerous competitions, including one on recognizing features of human brains.
Feature image courtesy of Shutterstock user Willy Deganello.
