Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos

Ajay Kumar Tanwani, Pierre Sermanet, Andy Yan, Raghav Anand, Mariano Phielipp, Ken Y. Goldberg

Proceedings - IEEE International Conference on Robotics and Automation 2020 conference-paper cited by 43

Source Open-access PDF More in Robotics & Autonomy

importance 5/10 ai assisted Surgical Simulation and Training surgical robotics stage: research open-source model

Problem

Learning to segment and understand surgical actions from video demonstrations is hard because labeled surgical video data is scarce.

Why it matters

Action understanding from surgical video underpins skill training and robot imitation for surgery.

What the AI did

Motion2Vec learns a motion-centric embedding from surgical videos via a Siamese metric-learning network, then segments actions with a recurrent neural network using only a small labeled set.

Results

not stated in source

Impact

Evidence supports a semi-supervised method for surgical video segmentation; no measured outcome is stated.

Record

Venue	Proceedings - IEEE International Conference on Robotics and Automation
Published	2020
Work type	conference-paper
Authors	Ajay Kumar Tanwani, Pierre Sermanet, Andy Yan, Raghav Anand, Mariano Phielipp, Ken Y. Goldberg
Citations	43 (OpenAlex, at import time)
References	94
Model or method	Motion2Vec (Siamese network with recurrent neural network)
Organization	not stated in source
Primary source	https://doi.org/10.1109/icra40945.2020.9197324
Open access	green
Keywords	surgical video, representation learning, semi-supervised, action segmentation, imitation learning

Importance rating

5 / 10

real but narrower advance

Surgical video action understanding underpins skill training and robot imitation; abstract shows suturing segmentation/imitation on one public dataset, importance inferred as significant but research-level.

Basis: inferred. Judged from the source text, which gives no explicit scale. Stated scale: not stated in source. Method and limits.

More in Robotics & Autonomy

#	Paper	Year	Citations	Importance
1	Autonomous robotic laparoscopic surgery for intestinal anastomosis Hamed Saeidi, Justin David Opfermann, Michael Kam, Shuwen Wei, Simon L...	2022	384	7
2	Solving Rubik's Cube with a Robot Hand OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litw...	2019	569	6
3	PaLM-E: An Embodied Multimodal Language Model Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Cho...	2023	346	6
4	Assistive Navigation Using Deep Reinforcement Learning Guiding Robot With UWB/Voice Beacons and Semantic Feedbacks for Blind and Visually Impaired People Chen-Lung Lu, Ziyan Liu, Jui-Te Huang, Ching-I Huang, Bo-Hui Wang, Yi...	2021	48	6
5	Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges Yi Zhou, Lulu Liu, Haocheng Zhao, Miguel López‐Benítez, Limin Yu, Yuta...	2022	154	6