Capturing Motion,
Crafting Stories

The research team led by Prof. Yaoyao Hao from Zhejiang University published a paper in Nature Communications titled "Cortical representation of multidimensional handwriting movement and implications for neuroprostheses." This article systematically reveals, for the first time, the multidimensional encoding mechanisms of handwriting movements in the motor cortex, establishing a theoretical foundation for next-generation high-performance handwriting BCIs (Brain-Computer Interfaces). Beyond advancing our understanding of motor control, the research introduces a novel decoding paradigm.

Citation:

Wang, Z., Xu, G., Yu, B. et al. Cortical representation of multidimensional handwriting movement and implications for neuroprostheses. Nat Commun (2026). https://doi.org/10.1038/s41467-026-70536-7

Research Background and Limitations 

Traditional Perspective: Handwriting has traditionally been modeled solely as two-dimensional (2D) trajectories on the writing plane.

Key Blind Spot: Conventional models overlook critical multidimensional information, including kinetic parameters and 3D spatial dynamics.

Research Approach: Utilizing multidimensional movement data from healthy subjects as an external stereotyped template, the team mapped these features onto the cortical neural signals of a paralyzed subject through reverse mapping and semantic decoding.

Core Technical Methods 

Quantitative Neural Encoding Analysis: Linear-Nonlinear Poisson (LNP) model.

Linear Decoding (Comparative Validation): Kalman Filter (KF).

Non-linear Decoding (Final Output): Long Short-Term Memory (LSTM) network.

Character Recognition and Matching: Dynamic Time Warping (DTW).

Statistical Testing: Paired Wilcoxon signed-rank test (applied throughout).

Multidimensional Data Acquisition Platform 

NOKOV Optical Motion Capture System: Real-time output of 3D pen-tip velocity ($V_x$, $V_y$, $V_z$).

Thin-film Pressure Sensors: Measurement of three-finger grip force and writing pressure.

Myo Surface EMG Armband: Measurement of 8-channel electromyographic (EMG) envelopes from the forearm.

Key Comparative Experiments and Results 

• Single-model vs. Dual-model: The dual-model approach significantly outperformed the single-model in decoding similarity for both strokes (0.63 → 0.69) and pen lifts (0.72 → 0.86) ($p < 0.0001$), confirming the necessity of separate modeling.

• 2D vs. Multidimensional Encoding (Incremental Fitting):

  Stroke Phase: Encoding performance improved significantly ($p < 0.0001$) after incorporating grip force, pressure, and EMG, while $V_z$ provided no contribution.

  Pen-lift Phase: Incorporating $V_z$ and all additional dimensions led to significant improvements; the full model achieved the best bits/spike (0.07 ± 0.02).

• Ablation article (Full Model): Removing EMG or 3D velocity caused a significant drop in encoding performance for both strokes and pen lifts; removing $V_z$ specifically degraded pen-lift encoding, confirming its unique contribution to in-air movements.

• 2D vs. Multidimensional Decoding (DTW Recognition):

  2D Decoding Accuracy: 29.22% ± 19.87%.

  Multidimensional Gain: Recognition accuracy increased significantly with the addition of $V_z$, grip force, or pressure. Notably, decoding 3D velocity and pressure for recognition yielded an accuracy of 49.96% ± 19.48%, significantly outperforming models based only on 3D velocity.

Research Contributions

1. Reveals, for the first time, the fundamental differences in neural encoding between strokes and pen lifts, proving that the brain encodes multidimensional motor parameters during handwriting.

2. Proposes a new BCI paradigm that maps neural activity to templates from healthy individuals, which is generalizable to other motor-imagery BCIs.

3. Demonstrates that multidimensional decoding improves character recognition rates by over 70%, laying the groundwork for clinical handwriting BCI applications.

NOKOV Contributions

NOKOV motion capture system was used to acquire three-dimensional handwriting motion data, enabling the construction of multidimensional movement templates that serve as ground truth for neural decoding, thereby significantly improving the modeling and recognition of handwriting in brain–computer interfaces.