Minimal Character Recognition
Goal
The goal of this project is to create a JavaScript and wasm library that can identify shapes like characters in the browser from pen/touch/mouse input. In addition to being accurate, it should be fast (on the order of 10ms per character on most devices), lightweight (< 100KB zipped, all-inclusive), and easy to train.
Initially the priority is the simplest case where exactly one character is inputted into a box, but eventually fast and accurate algorithms to divide a collection of paths into characters and words will (hopefully) be included.
Algorithm
My plan is to follow a slightly modified form of shape contexts. I have currently only implemented a linear assignment of points to points, so there is still a lot of room to improve the accuracy.
Most of the code for finding the histograms and transforming the shapes has been written but I have yet to package it all together. Once I have more training data, it will be easier to optimize the algorithm.
Narrowing down the options with the current algorithm and then performing multiple transformations with shape contexts should result in an algorithm that is still very fast but also much more accurate. Considering the accuracy so far with a very basic algorithm I am hopeful about the ultimate accuracy even with reasonably sized training sets.
Applications
The current algorithm is good enough for checking sudoku and crossword puzzle answers. Visit digitizer.fun and either upload a .puz crossword or complete a sudoku.
The page to download saved data requires inputting a 4 character code. Limiting to a subset of 20 characters that are not similar helps make the accuracy quite good. The error rate is probably better than the error rate for on-screen keyboard input for most people.
If you have a limited set of words/phrases to check, then the results can be fairly good. Click here to search through all of the almost 2000 MLB all-stars, or create a custom dictionary of words. However, there is still significant work to do to better identify which paths correspond to different characters.
How to Use
The library is in a testing phase and you can check it out here. I wouldn't recommend using it in production and everything is likely to change in breaking ways, but if you want to play around please do so.
The home page has lots of info to get you started if you just want to check it out. You can train the characters, set the number of points, optionally add a list of words, test everything, and then download a zipped folder with an example HTML file.
By default your changes are saved to your browser, but you can upload them and then download to a different browser or device.
Future
I am working on a system to collect training data from users like you so check back in early December if you want to provide some characters. Once I have more data I will focus on improving the algorithm. Along with bug fixes and lots of other things, current priorities for the immediate future include:
- Better algorithm. There are several options that will hopefully make the algorithm better with minimal tradeoffs for speed and size.
- Better training set. The current default training set only uses paths from one person. I hope to gather more paths and then prune the set in a way that balances accuracy and size/speed.
- Easier to use. Make everything more user friendly from training to implementation.
- Open sourcing. I hope to release all of the underlying code, write some blog posts with in-depth explanations, and even make a video or two.
- More Features. Search a subset of characters. Better algorithm for dividing a bunch of paths into characters.
Test
Draw an upper case letter in the box below.
Clear
