Cal AI, MyFitnessPal, Lose It!: four photo calorie apps looked at 102 meals. All four came in low.

If you are building anything that turns a photo of food into a number, this one is worth a minute. Researchers in the intramural program of the US National Institute of Diabetes and Digestive and Kidney Diseases (NIH) cooked 102 meals in a metabolic kitchen, where the content of every plate is known. They analysed photos of each meal with the photo features of four apps and compared every estimate with the known value. Every app came in low. Mean energy underestimate per meal: App Underestimate 95 % CI Appediet 252 kcal 210–295 kcal MyFitnessPal 327 kcal 269–385 kcal Lose It! 333 kcal 282–383 kcal Cal AI 345 kcal 296–392 kcal That is roughly a third of the meal missing. All four also underestimated fat, by about 30 g; Lose It! and Cal AI underestimated carbohydrate too, by about 14 g. The fat line is the interesting one Thirty grams of fat is about 270 kcal on its own (9 kcal per gram, the standard Atwater factor). That is close to the whole gap. Our reading, not the study's: fat is the part of a plate a camera sees worst — oil in the pan, butter melted into the rice, the dressing under the leaves. A vision model can name the salad and still miss the tablespoon of oil. You can see how much of a food's energy sits in fat with any open nutrient table. Here is one, the FatTonic Food Index, where every row points at the USDA record it was read from: import csv, io, urllib.request url = "https://fattonic.com/data/export/foods.csv" foods = list(csv.DictReader(io.TextIOWrapper(urllib.request.urlopen(url), "utf-8"))) def kcal(f): # Atwater factors: 4 kcal/g carbohydrate and protein, 9 kcal/g fat return 4 * float(f["carbs_g"]) + 4 * float(f["protein_g"]) + 9 * float(f["fat_g"]) for f in sorted(foods, key=lambda f: 9 * float(f["fat_g"]) / max(kcal(f), 1), reverse=True)[:5]: share = 9 * float(f["fat_g"]) / kcal(f) print(f'{f["name"]: