PreviewLabs

The Challenge

New methods are being developed to train robotic surgeons. Tony Reina of SkySurgery LLC approached us with a unique and interesting challenge: to develop a prototype for a simulation used to assist surgeons performing live robotic laparoscopic surgery.

Traditionally, a mentoring surgeon uses a telestrator to instruct the trainee. Telestration allows the mentor to draw on top of the operative video image and explain the surgical maneuvers through lines, circles, and arrows.

SkySurgery’s goal was to come up with the next generation of this technology by adding virtual surgical instruments and physics simulation that allows the mentor to demonstrate complex maneuvers of the surgical tools. Mentors would then be able to use these virtual tools to cut, retract, and even suture the operative video as if it were real human tissue.

The Algorithms & Prototype

Since this project needed a custom physics simulation to be implemented, we devised several algorithms:

• Creating the initial skin mesh (a set of triangles)
• Adding an additional point to the skin mesh (additional triangles) to allow grabbing and dragging an arbitrary point.
• The actual movement behavior of the skin after being pulled – treating the triangle’s edges as springs using Hooke’s Law.
• Cutting the skin when using the scalpel tool.
• Tearing the skin when it’s pulled hard.
• Removing redundant points.

We documented these to allow SkySurgery to formalize a patent application.

We then implemented the algorithms in the prototype, which was done in Unity3D. This work was broken up into four iterations, each time refining the algorithms and adding new features.

In the last iteration, we added support for the Razer Hydra – a pair of controllers from which an accurate three dimensional position and rotation can be obtained – resulting in a very intuitive way of controlling the simulator, allowing advising surgeons to literally pick it up and use without training.

The Result

SkySurgery partnered with Foresight Imaging LLC to patent this technology and incorporate it into Foresight’s TIMS Consultant, an FDA-approved telemedicine device that allows the mentor and trainee collaborate in realtime over a standard internet connection. They firmly believe that the new device greatly expands and enhances the capabilities of telepresence for both medical and non-medical applications.

In April, SkySurgery and Foresight Imaging presented the prototype at the Innovation, Design, and Emerging Alliances in Surgery conference. This poster summarizes the concept in medical terms.