Capturing Motion,
Crafting Stories

Femoral shaft fractures are a common high-energy injury, and minimally invasive internal fixation is the preferred treatment method. However, issues such as invisibility, inaccuracy, and instability arise during the implantation of plates. A research team from Shandong University has proposed a dual-mode robot-assisted method for plate implantation. This method uses a neural network model to plan the trajectory of plate implantation, leveraging the high stability and precision of robotic arms for the procedure. This method is highly feasible and holds promise for addressing the challenge of achieving high-precision plate implantation in minimally invasive surgeries.

NOKOV motion capture system collects real-time positional data during plate implantation, aiding in the training of a neural network model to predict the implant trajectory.

Citation

Guo, Yanchao et al. “A Dual-Mode Robot-Assisted Plate Implantation Method for Femoral Shaft Fracture.” The international journal of medical robotics + computer assisted surgery : MRCAS 20 6 (2024): e70008 .

Experimental Methods

Determination of Plate Implantation Starting Point

Algorithm-based Automatic Mode: After determining the endpoint of the plate implantation in CT reconstruction software, a 3D model of the femur is anatomically sectioned along a plane perpendicular to the plate and passing through its centerline. Points A and B are selected on the femoral condyle within the section, and point C is selected on the surface of the soft tissue of the affected limb. The algorithm then determines the implantation starting point and the endpoint of the plate implantation based on the coordinates of these three points.

Preoperative Planning Process

Manual Mode Based on Physician Judgment: For complex clinical situations, physicians can manually select the starting point for plate implantation through a robotic arm control system.

Trajectory Acquisition Experiment

Researchers use the NOKOV motion capture system to collect trajectory data for plate implantation as a database for training neural network models. During the implantation process, each motion capture camera captures two-dimensional positional information of marker points. Through data fusion and kinematic analysis, the position, posture, velocity, and acceleration of the object of interest can be obtained. The trajectory acquisition experiment is conducted under the guidance of orthopedic surgeons to ensure the trajectories collected align with clinical scenarios.

Trajectory Acquisition Device

Neural Network Model Construction

Data Processing: The collected trajectory data is preprocessed by referencing the starting point coordinates. Each trajectory is divided into two parts, comprising a total of 10 posture points. The posture of all points is adjusted by subtracting the starting point posture, converting them into relative posture points with respect to the starting point. This reduces the complexity of data acquisition and model training and enables the model to be applicable in different environments. Hyperparameters are selected for two neural network models, M and A, using orthogonal testing and the AutoKeras automated machine learning framework.

Plate Implantation Experiment

Researchers developed a surgical robotic system with three robotic arms, each completing different steps: bone resection, plate implantation, and screw implantation (for securing the plate).

Surgical Robotic System Schematic

The affected limb and calibration sphere are scanned using a C-arm, followed by three-dimensional reconstruction using CT software. The calibration sphere coordinates in the CT reconstruction software are modified according to the robotic arm's base coordinate system to unify the coordinate systems. Bone fracture reduction is virtually performed in the CT software and then physically achieved and maintained by a reduction robotic arm and traction device. The plate model used in this process is imported into the software and adjusted to the ideal posture to determine the plate implantation endpoint. Finally, the screw implantation robotic arm completes the screw insertion according to the plate's position.

Experiments testing the effectiveness of this method showed that the neural network model's predicted plate implantation trajectory aligns with clinical practice. The robotic arm accurately implants the plate at the endpoint without colliding with the femoral model.

NOKOV motion capture system digitizes the surgical operation by capturing the plate implantation trajectory, facilitating the automation of the surgical process and providing a reference for the development of other robot-assisted surgical systems.

Author Introduction

Yanchao Guo, Ph.D. candidate at the School of Mechanical Engineering, Shandong University. Research focus: robotassisted plate implantation systems, digitized surgical operations.

Yimiao Chen, Master's candidate at the School of Mechanical Engineering, Shandong University. Research focus: robotassisted plate fixation systems, biomechanical modeling, and simulation.

Xianzheng Zhou, Ph.D. candidate at the School of Mechanical Engineering, Shandong University. Research focus: ultrasoundassisted screw implantation devices, analysis of bone drilling surface quality, and screw stability research.

Jianping Bi, Associate Chief Physician, Department of Orthopedics, Affiliated Hospital of Shandong University of Traditional Chinese Medicine. Research focus: highprecision surgical robotic systems for limb fractures, visualization systems for traditional Chinese bone setting, robotic-assisted traction cooperation for fracture reduction systems.

Jason Z. Moore is an Associate Professor in the Department of Mechanical Engineering at Pennsylvania State University, USA, and the Director of the Precision Medical Instrument Design Laboratory. His primary research focuses on the interaction between medical devices and human tissues, utilizing mechatronics to enhance medical training and therapeutic equipment.

Qinhe Zhang (Corresponding Author) is a Professor at the School of Mechanical Engineering, Shandong University, and a Ph.D. supervisor. His main research areas include non-traditional processing technologies and equipment, digital design and manufacturing of products, virtual prototyping technology, and advanced medical devices and medical robotics technology.