Facebook’s latest AI experiments: generating captions and recognizing faces in videos

At its F8 developer conference in San Francisco today, Facebook demonstrated its latest artificial intelligence (AI) research efforts.

Joaquin Quiñonero Candela, Facebook's director of applied machine learning, talks about Facebook's latest AI research efforts around video at the company's F8 developer conference in San Francisco on April 13, 2016.

Above: Joaquin Quiñonero Candela, Facebook's director of applied machine learning, talks about Facebook's latest AI research efforts around video at the company's F8 developer conference in San Francisco on April 13, 2016.

Image Credit: Jordan Novet/VentureBeat

At its F8 developer conference in San Francisco today, Facebook demonstrated its latest artificial intelligence (AI) research efforts. Not surprisingly, they are about video.

Video implies a whole bunch of individual images put together. So it logically flows from Facebook’s progress around object recognition and image caption generationusing AI.

“You can imagine us building image search on steroids,” Joaquin Quiñonero Candela, Facebook’s director of Applied Machine Learning, said onstage today. Think of what Google Photoslets you do, but think of what Facebook’s implementation would look like — being able to search through all the pictures your Facebook friends have shared. But video is where scalability makes things more challenging and more interesting.

1) Generating captions for the things people say in videos.

2) Identifying the people who appear in videos so that they can be tagged, and even associated with specific times in the video, so that users can get right to the moment when a person first appears in the frame.

The company has been quite busy with artificial intelligence research in the two and a half years since tapping Yann LeCunto head up the Facebook Artificial Intelligence Research (FAIR) division.

Facebook is one of several Web companies engaged in the advancement of deep learning, a type of AI that generally involves training artificial neural networks on large sets of data and then getting them to make inferences about new data.

Facebook AI researchers have published many academic papers documenting their discoveries, and the company has open-sourced AI software and even shared designs for its specialty AI hardware. Earlier this year Facebook saidit would send AI-oriented servers to academic labs around the world.

In the specific domain of video, Snapchat, among others, is already engaged in AI research. But Facebook’s AI research operation is more mature than Snapchat’s, and that company’s user base is not as large as Facebook’s.


More information:

Facebook is the world’s largest social network, with over 1.39 billion monthly active users. Facebook was founded by Mark Zuckerberg in February 2004, initially as an exclusive network for Harvard students. It was a huge hit: in 2 we... read more »

New! Track Facebook's Landscape to stay on top of the industry in 3 minutes a day. Understand the entire ecosystem, monitor innovation, and track deal flows. Learn more.

Get more stories like this on Twitter& Facebook