Faster video file FPS with cv2.VideoCapture and OpenCV

Have you ever worked with a video file via OpenCV’s cv2 . VideoCapture function and found that reading frames just felt slow and sluggish?

I’ve been there ― and I know exactly how it feels.

Your entire video processing pipeline crawls along, unable to process more than one or two frames per second ― eventhough you aren’t doing any type of computationally expensive image processing operations.

Why is that?

Why, at times, does it seem like an eternity for cv2 . VideoCapture and the associated . read method to poll another frame from your video file?

The answer is almost always video compressionand frame decoding.

Depending on your video file type, the codecs you have installed, and not to mention, the physical hardware of your machine, much of your video processing pipeline can actually be consumed by reading and decoding the next frame in the video file.

That’s just computationally wasteful ― and there is a better way.

In the remainder of today’s blog post, I’ll demonstrate how to use threading and a queue data structure to improve your video file FPS rate by over 52%!

Looking for the source code to this post?

Jump right to the downloads section. Faster video file FPS with cv2.VideoCapture and OpenCV

When working with video files and OpenCV you are likely using the cv2 . VideoCapture function.

First, you instantiate your cv2 . VideoCapture object by passing in the path to your input video file.

Then you start a loop, calling the . read method of cv2 . VideoCapture to poll the next frame from the video file so you can process it in your pipeline.

The problem (and the reason why this method can feel slow and sluggish) is that you’re both reading and decoding the frame in your main processing thread!

As I’ve mentioned inprevious posts, the . read method is a blocking operation ― themain thread of your python + OpenCV application is entirely blocked (i.e., stalled) until the frame is read from the video file, decoded, and returned to the calling function.

By moving these blocking I/O operations to a separate thread and maintaining a queue of decoded frames we can actually improve our FPS processing rate by over 52%!

This increase in frame processing rate (and therefore our overall video processing pipeline) comes from dramatically reducing latency ― we don’t have to wait for the . read method to finish reading and decoding a frame; instead, there is always a pre-decoded frame ready for us to process.

To accomplish this latency decrease our goal will be to move the reading and decoding of video file frames to an entirely separate thread of the program, feeing up our main thread to handle the actual image processing.

But before we can appreciate the faster, threaded method to video frame processing, we first need to set a benchmark/baseline with the slower, non-threaded version.

The slow, naive method to reading video frames with OpenCV

The goal of this section is to obtain a baseline on our video frame processing throughput rate using OpenCV and Python.

To start, open up a new file, name it read_frames_slow . py , and insert the following code:

# import the necessary packages from imutils.videoimport FPS import numpyas np import argparse import imutils import cv2 # construct the argument parse and parse the arguments ap = argparse.ArgumentParser() ap.add_argument("-v", "--video", required=True, help="path to input video file") args = vars(ap.parse_args()) # open a pointer to the video stream and start the FPS timer stream = cv2.VideoCapture(args["video"]) fps = FPS().start()

Lines 2-6import our required Python packages. We’ll be using my imutils library , a series of convenience functions to make image and video processing operations easier with OpenCV and Python.

If you don’t already have imutils installed or if you are using a previous version, you can install/upgrade imutils by using the following command:

$ pipinstall --upgradeimutils

Lines 9-12then parse our command line arguments. We only need a single switch for this script, -- video , which is the path to our input video file.

Line 15opens a pointer to the -- video file using the cv2 . VideoCapture class while Line 16 starts a timer that we can use to measure FPS, or more specifically, the throughput rate of our video processing pipeline.

With cv2 . VideoCapture instantiated, we can start reading frames from the video file and processing them one-by-one:

# loop over frames from the video file stream while True: # grab the frame from the threaded video file stream (grabbed, frame) = stream.read() # if the frame was not grabbed, then we have reached the end # of the stream if not grabbed: break # resize the frame and convert it to grayscale (while still # retaining 3 channels) frame = imutils.resize(frame, width=450) frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) frame = np.dstack([frame, frame, frame]) # display a piece of text to the frame (so we can benchmark # fairly against the fast method) cv2.putText(frame, "Slow Method", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2) # show the frame and update the FPS counter cv2.imshow("Frame", frame) cv2.waitKey(1) fps.update()

On Line 19 we start looping over the frames of our video file.

A call to the . read method on Line 21 returns a 2-tuple containing:

grabbed : A boolean indicating if the frame was successfully read or not. frame : The actual video frame itself.

If grabbed is False then we know we have reached the end of the video file and can break from the loop ( Lines 25 and 26 ).

Otherwise, we perform some basic image processing tasks, including:

Resizing the frame to have a width of 450 pixels. Converting the frame to grayscale. Drawing the text on the frame via the cv2 . putText