Ph.D Thesis

Ph.D StudentGroisser Benjamin
SubjectIntermodal Human Body Modeling: Combining Radiography
with Topographic Imaging for Non Ionizing
Assessment of Adolescent Idiopathic
DepartmentDepartment of Mechanical Engineering
Supervisors PROF. Alon Wolf
PROF. Ron Kimmel


Scoliosis is defined as abnormal curvature of the spine, diagnosed on planar radiographs. However, the disease is eminently three-dimensional, comprising translational and rotational deviations from normal vertebral positioning, as well as abnormalities in the rib cage and shoulders leading to characteristic deformity of the torso envelope. 

This thesis brings together techniques from biomechanical modeling, computer vision, machine learning, and orthopedic practice to generate a unique, inter-modal dataset of adolescent idiopathic scoliosis patients and age-matched controls. A novel scan protocol is described for full-body topographic scanning of subjects in a range of clinically important postures, as well as an automated processing pipeline for dense anatomical labeling of the body surface. Further tools are also developed for objective analysis of patient structure and postural alignment.

In the first chapter, the research objective is to enable 3D reconstruction of skeletal anatomy using only a plain X-ray system. To solve this problem, low-cost depth cameras are added to the radiographic environment. When the patient is repositioned (e.g. frontal and then lateral images) the body surface itself can be used to perform the stereo alignment, enabling 3D reconstruction of skeletal anatomy.

The second chapter deals with the specialized EOS(R) xray scanner. The slot-scanning design enables low-dose bi-plane imaging, but scans are slow leading to motion-induced radiographic artifacts. Once again, the radiographic system is augmented with fast depth scanning cameras to record patient movement, which is then used to correct motion artifact post-hoc. Tests are performed on synthetic radiographs as well as on a radiographic phantom. 

The third chapter addresses an important problem in computer science: finding anatomical correspondence between nonrigid surfaces, specifically for the clinically relevant case of human bodies. A new deep learning architecture is proposed that adapts the convolution operator to work directly on triangulated mesh structures.

Combining deep learning predictions with articulated body models enables state-of-the-art correspondence matching for real scans of humans.

The clinical portion of the thesis begins with a chapter describing a large-scale data collection collaboration with the Hospital for Special Surgery in New York.  Spine deformity patients and age-matched controls undergo clinical exams, fill out questionnaires, have EOS scans, and are recorded in a full-body optical scan system.

The fifth chapter applies the surface registration method from Chapter 3 and shows how surface measurements can be automatically and reliably extracted from the topographic scans. It is shown that automated measurements are extremely reliable; the main problem is patients’ difficulty adopting the exact same pose repeatedly. 

In the sixth and final chapter it is shown how topographic and radiographic reconstructions can be aligned in a shared coordinate space, and how this combined dataset can be used to train statistical geometric models that capture the correspondence between surface and skeletal anatomy. These models enable non-ionizing early detection of scoliosis, monitoring disease progression, and objective assessment of postural alignment.

The ultimate aim is to establish this multi-modal analysis as a useful tool for orthopedic assessment. Several new tools are described and evaluated using surface information with and without radiography.