Researchers have developed an artificial intelligence (AI)-based system that, given a still image of a dish, can recommend ingredients and recipes based on that food photo.

Researchers from Massachusett's Institute of Technology's (MIT) Computer Science and Artificial Intelligence Laboratory (CSAIL) believe that analysing photos like these could help us learn recipes and better understand people's eating habits."In computer vision, food is mostly neglected because we don't have the large-scale datasets needed to make predictions," says Yusuf Aytar, an MIT post-doc who co-wrote the paper with MIT Professor Antonio Torralba.
But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences.In a new paper with the Qatar Computing Research Institute (QCRI), the team trained an AI system called "Pic2Recipe" to look at a photo of food and be able to predict the ingredients and suggest similar recipes.

Given a photo of a food item, 'Pic2Recipe' could identify ingredients like flour, eggs and butter, and then suggest several recipes that it determined to be similar to images from the database."You can imagine people using this to track their daily nutrition, or to photograph their meal at a restaurant and know what's needed to cook it at home later," said Christoph Trattner, assistant professor at MODUL University Vienna who was not involved in the paper.The system did particularly well with desserts like cookies or muffins, since that was a main theme in the database. However, it had difficulty determining ingredients for more ambiguous foods, like sushi rolls and smoothies.In the future, the team hopes to be able to improve the system so that it can understand food in even more detail. The paper is set to be presented at the 'Computer Vision and Pattern Recognition' conference in Honolulu later this month.