Deskewing: How Our AI Mathematically Straightens Crooked Text

Take a photo of a street sign, a product label, or a book cover. Chances are it's not perfectly straight. The camera was tilted, the object was at an angle, or you were in a hurry. Even a 3-degree rotation can degrade font recognition accuracy by 15-20%.

FontFinder's deskewing step corrects rotation automatically before the image reaches our AI model. Here's exactly how it works.

Why Rotation Matters So Much

Our MobileNetV2 model was trained on rendered font samples — clean, horizontal text. When you feed it rotated text, the spatial relationships between features shift. A character that looks like an 'H' when horizontal might look like an unusual 'A' when rotated 45 degrees. Even small angles change the gradient orientations that the model uses to identify letterforms.

Method 1: Hough Line Transform

The Hough Line Transform is a classical computer vision technique for detecting straight lines in images. Applied to binarised text, it finds the dominant line angles — because text on a page creates many parallel horizontal lines (baselines, x-heights, cap heights).

We accumulate votes in angle-space: each edge pixel votes for every line that could pass through it. The angle with the most votes is the dominant text direction. We subtract 90 degrees to get the rotation correction angle.

Method 2: PCA Fallback

Hough transforms struggle on images with sparse text — a single word, a logo, or a stylised headline. For these cases, we fall back to PCA (Principal Component Analysis) of the foreground pixel coordinates.

We collect the (x, y) coordinates of all dark pixels in the binarised image and compute the principal components — the axes of maximum variance. The primary axis is aligned with the dominant text direction. We rotate to align this axis with horizontal.

Applying the Correction

Once we have the rotation angle, we apply an affine transformation to the image. We rotate around the image centre to avoid translation artifacts. Areas outside the rotated image boundary are filled with white (matching our text-on-white convention).

We limit corrections to ±45 degrees — beyond that, the text is likely intentionally angled (a stylistic choice) rather than an accidental rotation.

Results

On our test set of 5,000 real-world font identification images, adding the deskewing step improved top-1 accuracy from 71% to 84%. The largest gains came from photos of physical objects — packaging, signage, and printed materials — where small rotations are nearly universal.