MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition


Handwritten text recognition is a well-known problem in computer vision. It is a difficult task due to complex shapes and the diversity of handwriting styles. Previous approaches tried to solve these problems by increasing the amount of training data. However, the model still struggles when encountered with unseen styles. Therefore, a recent study on suggests asking a user to write a specific sentence for adaption to individual style.

Image credit: Free-Photos via Pixabay, CC0 Public Domain

The proposed meta-learning framework uses character-specific weights instead of treating all characters equally. The model learns to learn instance-specific weights to prioritize learning from more discrepant characters. The meta-learning design can be coupled with any handwritten text recognition model. Tests on existing models show that the approach improves performance consistently.

Handwritten Text Recognition (HTR) remains a challenging problem to date, largely due to the varying writing styles that exist amongst us. Prior works however generally operate with the assumption that there is a limited number of styles, most of which have already been captured by existing datasets. In this paper, we take a completely different perspective — we work on the assumption that there is always a new style that is drastically different, and that we will only have very limited data during testing to perform adaptation. This results in a commercially viable solution — the model has the best shot at adaptation being exposed to the new style, and the few samples nature makes it practical to implement. We achieve this via a novel meta-learning framework which exploits additional new-writer data through a support set, and outputs a writer-adapted model via single gradient step update, all during inference. We discover and leverage on the important insight that there exists few key characters per writer that exhibit relatively larger style discrepancies. For that, we additionally propose to meta-learn instance specific weights for a character-wise cross-entropy loss, which is specifically designed to work with the sequential nature of text data. Our writer-adaptive MetaHTR framework can be easily implemented on the top of most state-of-the-art HTR models. Experiments show an average performance gain of 5-7% can be obtained by observing very few new style data. We further demonstrate via a set of ablative studies the advantage of our meta design when compared with alternative adaption mechanisms.

Research paper: Bhunia, A. K., Ghose, S., Kumar, A., Nath Chowdhury, P., Sain, A., and Song, Y.-Z., “MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition”, 2021. Link:

More to track …
Bulletin Observer Science and Technology

[Advertisement Plan]

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

As you found this post useful...

Follow us on social media!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Pin It on Pinterest

Translate »