AutoScan AI: Reading License Plates in the Real World, Not Just Clean Test Images

Automatic Number Plate Recognition sounds simple until you try to make it work on a real photo taken from a phone at an angle, in glare, at dusk. AutoScan AI is my attempt at building the full pipeline properly — from image capture to a verified, logged plate number — not just a demo that only works on the one sample image everyone uses in tutorials.

The pipeline, end to end

The system accepts input two ways: drag-and-drop upload (JPEG/PNG/WEBP) or a live webcam capture with a real-time viewfinder alignment guide, so the user knows the plate is actually framed before the shot is taken.

From there:

1. Detection — a YOLOv8 plate detector finds the plate region, with an OpenCV adaptive edge-and-contour detector as a fallback for when YOLO's confidence is low.
2. Preprocessing — bilateral filtering, grayscaling, adaptive Otsu thresholding, and contrast normalization clean the crop up before OCR ever sees it. This step matters more than people expect; skip it and OCR accuracy falls off a cliff under harsh shadows or glare.
3. OCR — EasyOCR extracts the characters, backed by heuristic regex correction for the classic OCR confusions (O↔0, I↔1, Z↔2, B↔8).
4. Reasoning — Google Gemini is used as a multimodal reasoning layer on top of the raw OCR output, to catch and correct plate formats that don't match regional conventions.

Why not just "detect and OCR" and call it done

The naive version of this project is two models chained together. The real-world version needs a correction layer, because OCR on a low-res, angled, partially-lit crop is never going to be 100% clean — the goal isn't a perfect model, it's a pipeline that's honest about uncertainty and self-corrects using format rules a human would apply instinctively (a plate doesn't start with a digit that OCR misread as a letter, for instance).

The rest of the stack

Recognized plates, timestamps, and vehicle metadata get logged to a serverless Postgres database (Neon), with captured images stored in Cloudinary. The dashboard is a Next.js app, so fleet logs are queryable and viewable without touching the backend directly.

The pipeline, stage by stage

Stage	Technology	Job
Detection	YOLOv8 + OpenCV fallback	Locates the plate region in the frame
Preprocessing	OpenCV (bilateral filter, Otsu threshold)	Cleans the crop before OCR
OCR	EasyOCR + heuristic correction	Extracts and corrects characters
Reasoning	Google Gemini	Validates against regional plate formats
Storage	Neon Postgres + Cloudinary	Logs plates, images, and metadata

OCR on a real-world photo is never perfectly clean — the pipeline that wins is the one that assumes that and corrects for it, not the one that hopes for a clean image.

Tech stack: TypeScript, Python, YOLOv8, OpenCV, EasyOCR, Gemini AI, Neon Postgres, Cloudinary. Code's on GitHub.