source: Google Research: Seeing beyond BMI: Estimating cardiometabolic risk with smartphone imagery
level: research
Google Research introduced PhotoScan, a deep learning framework that estimates body composition metrics from standard 2D smartphone photos. The model predicts body fat percentage, Android-to-Gynoid fat ratio, and Visceral-to-Subcutaneous fat ratio. These metrics help assess insulin resistance, a key driver of metabolic disease. PhotoScan was pre-trained on over 35,000 UK Biobank records and fine-tuned on 677 adults. It aims to offer a scalable, non-invasive alternative to DXA scans, which are expensive and require specialized equipment.
In validation, PhotoScan achieved a mean absolute error of 2.15 for body fat percentage, outperforming smartwatch bioelectrical impedance analysis at 2.91. For insulin resistance classification, adding PhotoScan features to demographics improved AUROC from 0.692 to 0.760, close to DXA's 0.773. The Net Reclassification Index was 0.593 for PhotoScan versus 0.748 for DXA. The independent validation cohort had 132 participants, with 67% female, which slightly reduced error for regional fat ratios due to lower variance in those metrics.
PhotoScan uses a ResNet-50 backbone initialized with ImageNet weights, fine-tuned on real smartphone photos paired with DXA ground truth. The approach addresses limitations of BMI, which misses body fat distribution. While promising, the study notes body composition is only one aspect of cardiometabolic health. Future work may integrate wearable data, glucose dynamics, and blood biomarkers. The research is still a prototype, but it suggests a path toward accessible screening for insulin resistance risk using everyday devices.
why it matters: This method could enable low-cost, widespread screening for insulin resistance using only smartphone photos, potentially catching metabolic disease earlier than BMI alone.
source: Google Research: Seeing beyond BMI: Estimating cardiometabolic risk with smartphone imagery