SHANDA BLACKMON: Three-Dimensional Modeling: How We Do It at Mayo Clinic in Rochester, Minnesota. We have no relevant disclosures. By the end of this presentation, the viewer should be able to identify what 3D modeling is and review the steps to create a 3D model and eventually print.

3D modeling is the process of developing a mathematical representation of any three-dimensional surface of an object by a specialized software program. The product is called a 3D model. It can be displayed as a two-dimensional image through a process called 3D rendering, as a computer simulation of a physical phenomena, or an actual physical model or hologram.

We have broken three-dimensional modeling down to nine separate steps. The first step is collaboration between the surgeon or the interventionalist or the educator and the 3D modeling team. The radiology team at Mayo Clinic is typically the team that we collaborate with from a Department of Surgery standpoint. Collaboration between the radiologist and the surgeon is the essential first step of making the model.

The second step is importing the DICOM images for segmentation, followed by segmentation and color coding, conversion of the segmented data into a virtual three-dimensional image, conversion of data into an STL for printing, exportation of the STL file into the printer, and selecting material and colors, printing the model and the support material, post-processing of the model, and then reviewing the model for accuracy. Each of these nine steps will be reviewed in specific detail.

Collaboration is the key to creating relevant models. Surgeons planning complex multidisciplinary interventions may have specific organs or structures they want to include or exclude from a model. It is during this time that the radiology team and intervention team begin the process of creating the 3D model. Materialize is the software we use to import the DICOM images from a scan. Thinner cuts on a CT scan will make a higher resolution model. Contrast allows structures like arteries and veins to be mapped with greater precision.

It is through this process that we take the images from the DICOM and import them into the software system. Using the Materialize software system, we are able to select different DICOM images, some from MRI and some from CT, to create our final model.

Segmentation is the separation and color coding of important anatomy for the surgeon. We may select factors like the tumor, the aorta, the superior vena cava, tracheal or bronchial tree, or both, pulmonary vessels, and the upper chest wall, including the spine, the ribs, clavicle, and sternum. The brachial plexus was segmented from MR imaging data and co-registered with the CT using bony landmarks.

This initial process of segmentation involves selecting the things that you want to segment and print. After each organ or specific item of interest has been selected, it is then used to create a separate STL file, and those STL files are then fused together. We convert the images imported into three planes. This first step is very similar to creating a 3D image from existing software without formal preparation or printing that many people already have in their hospital.

The steps that follow are required to fuse images from different types of scans, enhancing regions of interest and refining structures. This is the process by which we take an MRI image and a CT image to combine structures and give a more relevant image with every structure the surgeon may be concerned with.

The process of segmentation begins with windowing. With windowing, we select bone, tumor, arteries, veins, or airway structures each as a separate image. Each separate image file allows the fusion of a formal virtual 3D mask. We can segment from multiple sources. The best way to import images from the brachial plexus is using a sagittal T2 weighted MRI.

Not all of the cuts have to be imported, meaning you do not have to import the axial or coronal images of the T2 weighted MRI. Simply importing the DICOM images from a T2 weighted MRI would be adequate for mapping the brachial plexus.

We then line up these images with bony landmarks, allowing us to fuse the brachial plexus in with those images from our CT scan. We color code based on Hounsfield units initially. You can select dense bone for chest, aortic contrast, air, or certain soft tissue density. Thresholding is when you actually pick a certain Hounsfield value and then clean that image up.

There are a dozen different tools you can use to segment different structures. Thresholding is one. Another is region grow. Region grow highlights contiguous structures and allows you to select something such as air, and everywhere that air connects is highlighted, as seen in the image on the far bottom left. The team works together to select different regions of interest and exclude areas that are not of interest, carefully selecting and creating the model, integrating the different images, and carefully using the thresholding to select those Hounsfield units of the properties-- for example, the tumor-- that you would like to present. Once that is complete, cleaning up of these images is still required.

Utilizing software that allows automatic reconstruction of three-dimensional images, we can select areas like the tumor, lung, bone, and vascular structures. Masks are created by cropping or outlining the area of interest, making a three-dimensional structure of finer structures within the chest. Separate three dimensional structures can be fused based on these bony landmarks.

Each STL file is a separate color. We utilize certain software programs called blocking or multiple slice edit. Multiple slice edit looks at three planes to get the best view of the tumor and create the final 3D image. It is important to always go back and check the original images and ensure that the editing matches where the original parameters of the tumor. With some of this editing, creative license is taken, and one should be careful. And one should be sure that those images are in context with the boundaries of the tumor that were seen from the other imaging available.

We then use a program called Fix Wizard. Fix Wizard is like an automatic cleanup of the image. It shows you all of the triangles that will give you trouble with the printer because the printer prints surfaces, and these surfaces are made up of triangles. If we take a step too far and want to go back and erase the last step, Control Z is the option that allows us to go back to where we were before.

It is during this process that we use programs that allow for smoothing, which does constitute losing some detail that makes the image appear more like an organ, essentially reducing the pixelation of the 3D image. We can also use wrapping, which refines the image. When we have tiny, small, fine structures, you may not want to use too much smoothing or wrapping. However, when creating large structures like a diaphragm or the surface of a lung, one may want to use more smoothing to create an image that looks more like an organ.

Once we export the STL file to the printer, we then select material and colors that we would like to use. The picture on the bottom right shows the multiple different color palettes that can be selected. Only one color palette can be used, but all of the colors on that triangular palette can be selected. We can select material that is soft and movable, or we can select material that is hard and plastic-like. The different types of material that we select will be custom made and chosen by the surgeon based on the need or the intervention that's planned.

Sometimes, when creating teaching models, it's better to create flexible material that the residents and learners can move to expose other structures of interest. Color coding the models is typically done specifically using red for the aorta, blue for the vein, and different colors that are customized to specifically display things like a Pancoast tumor, with the brachial plexus yellow as most of us are typically used to seeing.

We then begin the process of printing the 3D model. The particular printer that we use is a Stratasys color printer. This allows us to lay down different polymers and expose the polymer to UV light, which hardens the polymer before the next successive layer is laid down. The printer will print just like a printer prints on a piece of paper, only each time it lays down either support material or the polymer that will become the 3D image. The 3D printed image, with the support material, will successively be laid down until the print is complete.

Once the printer has completed the 3D model, it can then be washed so that the support material is taken away. Large models, like a Pancoast tumor, may take up to 50 hours to print. And the majority of the cost of 3D printing is associated with the polymer and the time.

This video depicts a time elapsed printing of a Pancoast tumor. As the spine and the tumor, in black, can be seen, the printing material initially was not a color model, and this was our black and white and gray model the different shades of gray represent structures such as the bronchial tree, the aorta, the subclavian artery, the subclavian vein, and different structures. The tumor can be seen in black at the apex of the chest invading the first, second, and third ribs.

Each of these rib structures closely associated with the tumor can be seen in greater detail than with a typical CT scan. The T1 and T2 nerve roots can also be seen adjacent to the tumor. Once the model is printed, again, we can wash away the support material. A high pressure washer is used to create the final image of the model, washing the support material away.

Hands are inserted through gloves inside the box to prevent damage of the user. The gloved hands can then be used to manipulate the spray and stream of water under high pressure that is utilized to wash away the support material. Once the model has been printed, we review the model in a multidisciplinary fashion. If this is a complex surgery where orthopedic surgeons, neurosurgeons, vascular surgeons, and thoracic surgeons are all working together, we meet in a room to review the model and plan who will do what parts of the surgery.

We have found that 3D printed models are incredibly useful when presenting the specimen back to the pathology lab and properly orienting the specimen. We've also found that many of our learners find that it is incredibly easy to acquire the imaging and process the complex anatomy and learn the relationships of these structures in a way that they have never before been able to do.

When teaching the complex anatomy and teaching what you're doing during the surgery, having the model nearby allows an easy and straightforward explanation to take place, whereas before, we were often drawing on the drapes or showing different CT images. And often, it is much easier to just point to the model to show the particular part of the surgery that is happening.

We have also found that the 3D printed models enhance our ability to gain informed consent and show patients what we plan to do and how they may look after. One of the greatest opportunities I've had is to teach Pancoast anatomy to different patients and residents.

We have now shown you all of the processes that are required to create a 3D printed model. The process of 3D modeling is complex and requires many different specialties working together. Alternatives to 3D printing include holographic projection, large monitor display of 3D rotational images of a segmented model, simulation, or the use of an interactive hologram.

One good example of a hologram is that program created by Microsoft called HoloLens. HoloLens allows a user to put on a set of goggles that allow them to visualize a 3D image and integrate it into their environment. The typical use of a CT scan has traditionally not allowed us to integrate an image into our own three-dimensional world. By visualizing the body and the structures of the body in front of us, we can interact with it much easier. Visualizing the parts of the human body in a way that it was meant to be, the brain processes three-dimensional images in a much more meaningful way than it would with a 2D view.

Another integrated 3D modeling program that allows the use of glasses to be worn and visualization of an interactive hologram with the surrounding environment is that of Magic Leap. The Magic Leap program has received new funding and endeavors to find new ways to apply their new technology. I suspect that the future of this will certainly be in the field of medicine.

And in the words of Michelangelo, where he saw the angel in the marble and carved until he set it free, many surgeons, many other scientists, seek to find the true meaning of how to offer patients the safest surgery, how to see the tumor inside the lung, and whether or not it's adherent to the ribs of the chest wall, and how to carve out just enough, leaving the vital structures behind and doing the least amount of harm.

Like Michelangelo, many of us thoracic surgeons strive to give our patients the best care we can. And it is through this new technology that I believe we will find new ways to do that.

We'd like to thank you for the opportunity to present how to do 3D modeling. And we'd like to take a moment to recognize the work of Jane Matsumoto and Jay Morris, who work in the 3D modeling lab and spend countless hours dedicated to giving us these 3D models and allowing us to give our patients the best surgery possible. All of the teams that work together to create these 3D models make the difference in how we'll go forward in the future.