How to uplevel your Security Cam by detecting Objects with a Streamlit Data App?

Webcams and door cams, such as Google Nest Cam or any other cheap network camera, are great to watch your property from remote.

It’s always a good feeling to see that everything is alright at home and that your property is safely guarded by automatic eyes.

One negative aspect that I encountered during my tests with my own cheap network cameras was its unreliable alerting on any kind of motion.

Once activated, no matter how sensitive or insensitive I defined the motion detection thresholds, it’s always resulting in either an alert spam or in no alert at all.

Raise your hands if you encountered the same behaviour for your own camera’s built-in motion detection feature. 😠

Why does plain motion detection fail?

There are many reasons why the motion detection fails for cheap network cameras, where light and shadow situations on a cloudy day, or wind causing movement of bushes are just some of them.

Real world scenarios are tricky and there are an unlimited amount of scenarios where standard motion detection completely fails to deliver value to users.

Another aspect is that simple motion detection never catches the true meaning of a given camera observed situation.

Of course it’s tremendously difficult for an automatic software routine to evaluate the true meaning of a given situation.

One improvement that modern security cameras, such as Google Nest Cam already offer on top of plain motion detection is to evaluate a given scene by giving you the information about what objects the camera detects within its recorded view.

So in case a cat is triggering the motion detection in front of your door camera, you can automatically decide if it’s worth an alert or rather not.

This of course depends on what you would like to be notified about, so in case a dog poops at your front door your attention level could be different, who knows. 💩

How to detect objects by using OpenCV and AI?

One of the most prominent and widely adopted frameworks for computer vision and object detection is OpenCV.

So let’s stick to OpenCV and try to implement a detection model that is capable of identifying individual objects within any given picture, preferably within our security cameras still image.

The first step is to initialise a standard OpenCV classification model.

We will use a Deep Neural Network model that is capable of using an already pre trained model state.

Download and use a pre trained model

By using a pre trained model we do save the time of collecting all the example images for each individual class of objects that would be necessary for training a detection model.

Let’s download a pre trained model from the TensorFlow Object Detection API page. Download both, the weights file (.pb) as well as the configuration file (.pbtxt) and put it into your local project folder.

Now, as we successfully downloaded a pre trained model, we can initialise OpenCV to use those pre trained weights by referencing both downloaded files:

config_file = ‘model_config.pbtxt’
frozen_model = ‘graph.pb’
model = cv2.dnn_DetectionModel(frozen_model, config_file)

The model still needs some configuration, so that it fits to our pre trained weights config file, which means we have to specify the input width and heights that you can find within the downloaded .pbtxt file.

Then use those values to further initialise the model as shown below:

model.setInputSize(320, 320)
model.setInputScale(1.0/127.5)
model.setInputMean((127.5,127.5,127.5))
model.setInputSwapRB(True)
model.setInputCrop(False)

After initialising the model with the pre trained model, it is ready to classify any given image object. A call to the model’s detect routine with a given image will return an array of detected classes (given as class indexes) and a confidence value between 0.0 and 1.0 for each detected object. In order to find the detected object within the given image, the detect call also returns an array of bounding boxes that can be used to mark the detected objects within your given image after the detection run.

ClassIndex, confidence, bbox = model.detect(img, confThreshold=0.2)

As the detection result just reveals the class index of an object without any human readable names, we also have to put a file with all the human readable class names into our project (labels.txt). Within that file you find all the 80 human readable class names that our pre trained model is able to identify.

Wrap everything into a Streamlit Data App

Now that we have the model ready, we will wrap the detection process as well as the picture selection process into a convenient Streamlit app.

The Streamlit app will allow users to either specify an URL for a given image, or to upload a local image.

A slider controls the detection confidence threshold and finally the original image is shown along with the bouncing boxes of all the detected objects.

I also added a table view at the bottom of the data app to show the details of each detected object along with the model’s confidence value.

Let’s look at the individual parts of our Streamlit app, starting with the simplest part, which is the title of our application:

st.title(‘Camera Object Detection’)

Now add an input field for specifying any URL that leads to a valid image. Whenever the value of the input field is changed by the user, my download image helper function is called to load the given URL and create an image out of it:

url = st.text_input(‘Specify a Picture URL’)
if url is not None and url.startswith(‘http’):
img = download_cam_image(url)
img_file_buffer = None
Now, alternatively offer an image upload widget for users to directly upload their images:
img_file_buffer = st.file_uploader(“Upload an image”, type=[“png”, “jpg”, “jpeg”])
if img_file_buffer is not None:
image = Image.open(img_file_buffer)
img_array = np.asarray(image)
img = cv2.cvtColor(img_array, cv2.COLOR_RGB2BGR)

Now adding a slider that allows users to control the detection sensitivity:

confidence_threshold = st.slider(
“Confidence threshold”, 0.0, 1.0, DEFAULT_CONFIDENCE_THRESHOLD, 0.05
)

Finally, we will display the resulting image, as well as the table of detected objects with the models detection confidence:

st.image(img)
df = pd.DataFrame(results, columns=([‘Class’, ‘Label’, ‘Confidence’]))
st.dataframe(df)

The resulting Streamlit user interface will look like the screenshot below:

Build and deploy with Docker

As you can read in my previous blog about ‘Streamlit — How to build and share your Data Apps on Steroids?’, I came to the conclusion that building Streamlit apps in Docker right from the beginning is the much better choice.

So, all we have to do is to add a ‘Dockerfile’, along with a Python ‘requirements.txt’ file, as shown below:

FROM python:3.7
LABEL maintainer=”Wolfgang Beer @wolfgangB33r”
EXPOSE 8501
WORKDIR /app
COPY requirements.txt ./requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
CMD streamlit run src/app.py
streamlit
opencv-python-headless
numpy
requests

Now you can easily build the Streamlit app by using following Docker build command on the console:

docker build -f Dockerfile -t cam:latest .

Run the app on your local machine with the Docker command below:

docker run -p 8501:8501 cam:latest

See a recorded demo of the resulting app and how it can be used to detect objects in any given picture.

Try it out

In case that my article made you curious about how good the trained model is able to detect objects within your own images, you can easily try out the app at Streamlit.io.

Streamlit allows users to deploy and share their data apps within their own runtime platform. If you want to host the app on your local machine, or you would like to run the app in any other Docker deployment environment, feel free to clone my GitHub repository below.

Get the source code

In case you want to play around with the Streamlit data app presented in this blog, just head over to my GitHub repository and feel free to clone and modify it.

Summary

Network cameras are everywhere and they are a convenient way to keep a remote eye on your property. Automating those checks mostly fails as plain motion detection does not work well with real world, outdoor scenarios.

Using AI and deep learning models to automatically identify objects within your security camera feed is a great possibility to up-level your currently deployed camera.

By using out of the box trained detection models, OpenCV as a detection framework along with Streamlit for the user interface, it’s possible to build and deploy your own camera detector app with just 80 lines of Python code. 🚀

Thanks a lot for reading, I hope my article was of some use for you.

Follow me on Medium to keep up to date with any future articles!