Introduction: Surgical planning in neurosurgery requires use of accurate and powerful visualization. Traditional visualization of neurosurgical pathology is performed utilizing two-dimensional (2D) DICOM format, MRI and CT imaging. There have been few recent reports on the use of virtual reality and 3D printing in the visualization of intracranial pathology. These modalities have significant limitations in regards to practical use. Mixed reality visualization is based on the advancement of augmented reality, allowing for holographic display of any three-dimensional (3D) rendering in real-world space. We describe the creation of a novel, mixed reality platform which allows for visualization of neurosurgical pathology in holographic space.

Methods: A platform was created in a C# environment using a 3D rendering engine which allowed for placement of all 2D DICOM based voxel values in a 3D volumetric grid. Ray-marching based rendering was then utilized to generate stereoscopic imaging. This was then transferred to a head-mounted light field display (Microsoft Hololens, Seattle, Washington). DICOM based metadata was used to recreate the volume at the exact physical size of the original imaging. The volume rendering was created using a Windows 10 PC with an Core i7 processer and NVIDIA GTX1070 GPU with subsequent transfer to the stereoscopic display.

Results: The mixed reality platform was successful in generating 3D holographic renderings of all neurosurgical imaging, including MRI, CT and ultrasound data. The renderings were processed in under 30 seconds and continuously generated at 30-40 frames per second allowing for comfortable three dimensional analysis and observation of the relevant lesion. Coregistration of MRI, CT, and tractography data revealed potential for multi-modality holographic visualization.

Conclusions: Successful development and implementation of a holographic, mixed reality platform for neurosurgical visualization is possible. Augmented reality technology has the potential to transform medical visualization. Further development and research is necessary.

Patient Care: Through our research we aim to use augmented reality to create powerful tolls for visualization of neurosurgical anatomy. This can enable more accurate surgical planning, better education of students and residents, and enhanced patient communication.

Learning Objectives: By the conclusion of this session, participants should be able to: 1) Describe the role of visualization in neurosurgical planning 2) Identify new technologies which can aid in neurosurgical planning, including mixed reality, virtual reality, and 3D printing

References: Kockro, Ralf A., et al. "Planning and simulation of neurosurgery in a virtual reality environment." Neurosurgery 46.1 (2000): 118-137. Waran, Vicknes, et al. "Utility of multimaterial 3D printers in creating models with pathological entities to enhance the training experience of neurosurgeons: Technical note." Journal of neurosurgery 120.2 (2014): 489-492.