Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera - Archive ouverte HAL
Conference Papers Year : 2016

Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera

Abstract

This paper presents a mirror-like augmented reality (AR) system to display the internal anatomy of a user. Using a single Microsoft V2.0 Kinect, we animate in real-time a user-specific internal anatomy according to the user’s motion and we superimpose it onto the user’s color map. The user can visualize his anatomy moving as if he was able to look inside his own body in real-time. A new calibration procedure to set up and attach a user-specific anatomy to the Kinect body tracking skeleton is introduced. At calibration time, the bone lengths are estimated using a set of poses. By using Kinect data as input, the practical limitation of skin correspondance in prior work is overcome. The generic 3D anatomical model is attached to the internal anatomy registration skeleton, and warped on the depth image using a novel elastic deformer, subject to a closest-point registration force and anatomical constraints. The noise in Kinect outputs precludes any realistic human display. Therefore, a novel filter to reconstruct plausible motions based on fixed length bones as well as realistic angular degrees of freedom (DOFs) and limits is introduced to enforce anatomical plausibility. Anatomical constraints applied to the Kinect body tracking skeleton joints are used to maximize the physical plausibility of the anatomy motion, while minimizing the distance to the raw data. At run-time, a simulation loop is used to attract the bones towards the raw data, and skinning shaders efficiently drag the resulting anatomy to the user’s tracked motion. Our user-specific internal anatomy model is validated by comparing the skeleton with segmented MRI images. A user study is established to evaluate the believability of the animated anatomy.
Fichier principal
Vignette du fichier
mig_2016.pdf (24.08 Mo) Télécharger le fichier
mig_submission_id_01.mp4 (27.19 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01366704 , version 1 (16-09-2016)

Identifiers

Cite

Armelle Bauer, Ali-Hamadi Dicko, François Faure, Olivier Palombi, Jocelyne Troccaz. Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera. ACM SIGGRAPH Conference on Motion in Games, Oct 2016, San Francisco, United States. pp.113-122, ⟨10.1145/2994258.2994259⟩. ⟨hal-01366704⟩

Relations

1449 View
588 Download

Altmetric

Share

More