Deep learning–based autonomous retinal vein cannulation in ex vivo porcine eyes

Peiyao Zhang, Peter Gehlbach, Russell H.Taylor, Iulian Iordachita, Marin Kobilarov
Johns Hopkins University

Science Robotics 2025

Paper Data & Code

We believe that expert surgical knowledge such as the principles required for successful retinal vein cannulation can be embedded into deep learning models, enabling clinicians without specialized training to perform robot-assisted autonomous procedures and achieve outcomes comparable to those of experienced surgeons.



Abstract

Retinal vein cannulation (RVC) is an emerging method for treating retinal vein occlusion (RVO). The success of this procedure depends on surgeon expertise and, recently, robotic assistance. This paper proposes an autonomous RVC workflow leveraging deep learning and computer vision. Two Steady-Hand Eye Robots (SHERs) controlled a 100-micrometer metal needle and a medical spatula to execute precise tasks. Three convolutional neural networks were trained to predict needle movement direction and identify contact and puncture events. A surgical microscope with an intraoperative optical coherence tomography (iOCT) system captured the surgical field through a microscope and cross-sectional images (B-scans). The goal was to enable the robot to autonomously carry out the critical steps of the RVC procedure, especially those that are challenging and require expert knowledge. The less technically demanding tasks were assigned to the user, who also supervised the robot during these steps. Our method was tested on 20 ex vivo porcine eyes, achieving a success rate of 90%. In addition, we simulated eye movements caused by breathing on six other ex vivo porcine eyes. With the eyes moving in a sinusoidal pattern, we achieved a success rate of 83%, demonstrating the robustness and stability of the proposed workflow. Our results demonstrate that the autonomous RVC workflow, incorporating deep learning and robotic assistance, achieves high success rates in both static and dynamic conditions, indicating its potential to enhance the precision and reliability of RVO treatment.

Workflow

Graphic Abstract

In this work, we propose a six step workflow for autonomous retinal vein cannulation that includes initial preparation, horizontal and vertical needle navigation, puncture, retraction, infusion, and needle removal. After the initial setup, the user was only required to select a target and perform the final infusion step. The target selection was accomplished by clicking on a pixel in the top-down microscope image using a mouse. At the same time, the infusion step required pressing the foot pedal of the vitrectomy machine. All robot movements were driven autonomously on the basis of visual feedback from the microscope images and iOCT B-scans.

Network Architecture

Three deep learning models were trained to provide essential information for robot control. The first one is an orientation network that uses ResNet18 as the backbone to predict the direction of needle motion toward a user-defined target. The second one is a contact network based on a YOLOv8 classification model that determines whether the needle tip has contacted the upper outer wall of the blood vessel. The third one is a puncture network based on a YOLOv8 object detection model that confirms whether the needle tip is inside the vein after puncture.

Resutls

A success rate of 18 of 20 was achieved in the fixed-eye experiments, and a success rate of 5 of 6 was achieved in the breathing simulation experiments. These results demonstrate the feasibility of autonomous RVC using a 100-μm metal needle.

Autonomous RVC on a fixed ex vivo porcine eye

The video visualizes both the microscope and B-scan views of the region of interest. The user selects a target with a mouse click, and the needle navigates horizontally toward the target. Upon reaching the target, the needle moves along the negative Z-axis until it contacts the outer wall of the retinal vein. The robot then initiates vein puncture along the tool's axis, retracting slightly afterward. These two steps are repeated until a successful puncture is confirmed. The user then begins the infusion process, which continues until either a successful or failed infusion is determined. In this case, the blood is flushed away by the water, as seen on the left. After infusion, the robot retracts the needle to a safe position. A blood reflux is observed in this eye.



Autonomous RVC on an ex vivo porcine eye with breathing compensation

The only difference between this video and previous one is the sinusoidal motion applied along the Z-axis of the eye. An additional control algorithm is implemented to ensure that the needle follows the motion pattern of the retina, based on the distance differences between B-scans calculated using the optical flow method. During the puncture process, this breathing compensation helps keep the needle tip inside the vein, as seen in the video. During the infusion step, the blood is flushed away as shown on the left. After retracting the needle, a blood reflux is also observed in this eye.

BibTeX

@article{zhang2025deep,
  title={Deep learning--based autonomous retinal vein cannulation in ex vivo porcine eyes},
  author={Zhang, Peiyao and Gehlbach, Peter and Taylor, Russell H and Iordachita, Iulian and Kobilarov, Marin},
  journal={Science Robotics},
  volume={10},
  number={109},
  pages={eadw2969},
  year={2025},
  publisher={American Association for the Advancement of Science}
}

Acknowledgments

We thank Prof. Nassir Navab and Prof. Mohammad Ali Nasseri for discussion and feedback on the experiments.