Purpose: To enhance the accuracy of surface-guided RT (SGRT) for abdominal SBRT by designing an artificial intelligence (AI) enhanced computer-vision (CV) patient setup technique that predicts skeletal anatomy from surface imaging.
Methods: We have designed a modified SGRT technique, 'avatar guided-RT' (AgRT), that employs patient-specific "avatars" based on the recently published Sparse Trained Articulated Human Body Regressor (STAR) model. STAR is a realistic 3D model of human surface anatomy learned from >10,000 3D body scans that considers gender and BMI for pose-dependent surface variation and can be fitted to CT-based surface contours or surface meshes acquired from 2D video/depth images. We utilize a pre-existing neural network trained on 2,400 softtissue/skeleton training pairs obtained from dual-energy X-ray absorptiometry (DXA) scans to predict the skeletal anatomy from the body surface of a patient in treatment position.
Results: AgRT was tested using a calibrated multiple camera system. Real-time 3D pose extraction from multiple 2D images was tested in a virtual treatment room to optimize camera numbers and positions. Testing was then conducted on a healthy volunteer to track various treatment poses. The patient's 3D pose was mapped to an avatar with matching gender and BMI. The skeletal alignment technique was assessed on XCAT phantom data and retrospective patient CTs. Skeletal anatomy was predicted from surface imaging with sub-cm accuracy.
Conclusion: Real-time acquisition of 3D human pose and shape is feasible using video inputs and CT data. Inferring the skeletal anatomy can enable alignment to a patient's X-ray imaging and improve the correspondence between surface imaging and internal anatomy. Realistic body modelling in SGRT can potentially address issues caused by insufficient surface anatomic variation that can lead to poor correlation and large random errors in current SGRT techniques. Considering patient gender, BMI, pose, and body type can enhance SGRT's accuracy and reliability.