Researchers from the University of Cambridge have developed a robotic “chef” that can learn and recreate recipes by watching cooking videos. The team trained the robot using a set of eight simple salad recipes. After observing a human demonstrator prepare a salad in a video, the robot successfully identified the recipe being made and replicated it.
In the study published in the journal IEEE Access, the researchers utilized video content as a valuable source of data for automated food production. They demonstrated that robots can learn incrementally from videos, enabling easier and more cost-effective deployment of robot chefs.
While the concept of robotic chefs has long been a staple in science fiction, cooking has proven to be a complex challenge for robots in reality. Although several companies have developed prototype robot chefs, none are currently available on the market and are far from matching the culinary skills of human chefs.
The team at Cambridge aimed to train their robot to learn recipes in a similar incremental manner to humans. To achieve this, they devised eight salad recipes and filmed themselves preparing them. They then used a publicly available neural network to train the robot, which had already been programmed to identify various objects, including the fruits and vegetables used in the salad recipes.
Using computer vision techniques, the robot analyzed each frame of the videos to identify different objects, such as ingredients, a knife, and the human demonstrator’s arms and hands. By correctly identifying the ingredients and actions of the human chef, the robot could determine the recipe being prepared. It also recognized slight variations within a recipe, such as a double portion or human error.
The robot achieved an impressive success rate, correctly identifying the recipe in 93% of the 16 videos it watched, even though it only detected 83% of the human chef’s actions. Additionally, the robot successfully recognized and incorporated a new, ninth salad recipe into its cookbook.
The researchers highlighted the robot’s ability to detect subtle nuances, such as recognizing that different quantities of chopped apples and carrots constitute the same recipe. They also noted that the robot’s training videos were not similar to fast-paced, visually flashy food videos found on social media platforms. As the technology advances, future iterations of robot chefs could potentially learn a wide range of recipes from platforms like YouTube.
This research, partly supported by Beko plc and the Engineering and Physical Sciences Research Council, showcases the potential for using video content to train robots in food preparation. It brings us closer to a future where robot chefs can assist in kitchens and facilitate the production of various dishes.
