Recognizing digits with OpenCV and Python

Today’s tutorial is inspired by a post I saw a few weeks back on /r/computervision asking how to recognize digits in an image containing a thermostat identical to the one at the top of this post.

As Reddit users were quick to point out, utilizing computer vision to recognize digits on a thermostat tends to overcomplicate the problem ― a simple data logging thermometer would give much more reliable results with a fraction of the effort.

On the other hand, applying computer vision to projects such as these are really good practice.

Whether you are just getting started with computer vision/OpenCV, or you’re already writing computer vision code on a daily basis, taking the time to hone your skills on mini-projects are paramount to mastering your trade ― in fact, I find it so important that I do exercises like this one twice a month.

Every other Friday afternoon I block off two hours on my calendar and practice my basic image processing and computer vision skills on computer vision/OpenCV questions I’ve found on Reddit or StackOverflow.

Doing this exercise helps me keep my skills sharp ― it also has the added benefit of making great blog post content.

In the remainder of today’s blog post, I’ll demonstrate how to recognize digits in images using OpenCV and python.

Looking for the source code to this post?

Jump right to the downloads section. Recognizing digits with OpenCV and Python

In the first part of this tutorial, we’ll discuss what a seven-segment display is and how we can apply computer vision and image processing operations to recognize these types of digits ( no machine learning required!)

From there I’ll provide actual Python and OpenCV code that can be used to recognize these digits in images.

The seven-segment display

You’re likely already familiar with a seven-segment display, even if youdon’t recognize the particular term.

A great example of such a display is your classic digital alarm clock:

Figure 1:A classic digital alarm clock that contains four seven-segment displays to represent the time of day.

Each digit on the alarm clock is represented by a seven-segment component just like the one below:

Figure 2:An example of a single seven-segment display. Each segment can be turned “on” or “off” to represent a particular digit (source: Wikipedia ).

Sevent-segmentdisplays can take on a total of 128 possible states:

Figure 3:A seven-segment display is capable of 128 possible states (source: Wikipedia ).

Luckily for us, we are only interested in ten of them ― the digits zero to nine:

Figure 4:For the task of digit recognition we only need to recognize ten of these states.

Our goal is to write OpenCV and Python code to recognize each of these ten digit states in an image.

Planning the OpenCV digit recognizer

Just like in the original post on /r/computervision , we’ll be using the thermostat image as input:

Figure 5:Our example input image. Our goal is to recognize the digits on the thermostat using OpenCV and Python.

Whenever I am trying to recognize/identify object(s) in an image I first take a few minutes to assess the problem. Given that my end goal is to recognize the digits on the LCD display I know I need to:

Step #1: Localize the LCD on the thermostat. This can be done using edge detection since there is enough contrast between the plastic shell and the LCD. Step #2: Extract the LCD. Given an input edge map I can find contours and look for outlines with a rectangular shape ― the largest rectangular region should correspond to the LCD. A perspective transform will give me a nice extraction of the LCD. Step #3: Extract the digit regions. Once I have the LCD itself I can focus on extracting the digits. Since there seems to be contrast between the digit regions and the background of the LCD I’m confident that thresholding and morphological operations can accomplish this. Step #4: Identify the digits. Recognizing the actual digits with OpenCV will involve dividing the digit ROI into seven segments. From there I can apply pixel counting on the thresholded image to determine if a given segment is “on” or “off”.

So see how we can accomplish this four-step process to digit recognition with OpenCV and Python, keep reading.

Recognizing digits with computer vision and OpenCV

Let’s go ahead and get this example started.

Open up a new file, name it recognize_digits . py , and insert the following code:

# import the necessary packages from imutils.perspectiveimport four_point_transform from imutilsimport contours import imutils import cv2 # define the dictionary of digit segments so we can identify # each digit on the thermostat DIGITS_LOOKUP = { (1, 1, 1, 0, 1, 1, 1): 0, (0, 0, 1, 0, 0, 1, 0): 1, (1, 0, 1, 1, 1, 1, 0): 2, (1, 0, 1, 1, 0, 1, 1): 3, (0, 1, 1, 1, 0, 1, 0): 4, (1, 1, 0, 1, 0, 1, 1): 5, (1, 1, 0, 1, 1, 1, 1): 6, (1, 0, 1, 0, 0, 1, 0): 7, (1, 1, 1, 1, 1, 1, 1): 8, (1, 1, 1, 1, 0, 1, 1): 9 }

Lines 2-5import our required Python packages. We’ll be using imutils , my series of convenience functions to make working with OpenCV + Python easier. If you don’t already have imutils installed, you should take a second now to install the package on your system using pip :

$ pipinstallimutils

Lines 9-20define a Python dictionary named DIGITS_LOOKUP . Inspired by the approach of /u/Jonno_FTW in the Reddit thread, we can easily define this lookup table where:

They key to the table is the seven-segment array. A one in the array indicates that the given segment is on and a zero indicates that the segment is off . The value is the actual numerical digit itself:0-9.

Once we identify the segments in the thermostat display we can pass the array into our DIGITS_LOOKUP table and obtain the digit value.

For reference, this dictionary uses the same segment ordering as in Figur