Improved machine learning model for towel change monitoring system

Overview

I have created a system that monitors hanging towels with a WEB camera and notifies when they are replaced.

"Whether or not there is a towel" is judged by machine learning the image of the WEB camera, but after all there were many false positives at the beginning. I have devised various ways to reduce false positives, so I will introduce the method.

1　Do not judge when the lights in the room are off

The first common false positives were when the lights were off and the room was dark.

Since all the learning was done using the image when it was bright, the difference in brightness in the image changed when it was dark, and it was probably a false detection.

As a countermeasure, I decided not to judge when the room was dark.

Of course, you can add an image when it is dark to the learning model, but due to the nature of the application, it is unlikely that towel replacement will occur when the lights in the room are off, so we decided not to judge. This is less costly for re-learning and less processing load.

Specifically, after acquiring an image, the "brightness" of that image is calculated, and only when the threshold is exceeded, the image judgment process is started. (Refer to the previous section for the calculation method of "brightness")

2　Finely classify "yes"

The next most common false positives were when the body was covered and part of the towel was hidden, or when the towel was wiped.

As a countermeasure, I decided to classify the condition of towels in more detail.

Until now, there were three types of towels: "Yes", "No", and "Undecidable" (for example, when the towel is completely hidden by the body).

We have increased the classification to 5 types, "Yes", "In use", "Body covered", "No", and "Undecidable". (Until now, towel in use or covered by body is classified as "Yes")

From the upper left, "Yes", "In use", "Body is covered", "Undecidable", "Not" Originally, the top three were all classified as "Yes".

Then, in the actual judgment process, after guessing one of the above five types, if any of "In use", "Body is covered", and "Yes" changes to "Not", I judge that "towel exchange has been done"

この修正の前後で「あるなし」の正解率（「使用中」「体が被っている」は「ある」に再変換し、「ある」「なし」「判定不能」の3種類に分類した場合の正解率）を計算しました。

Before and after this correction, the correct answer rate of "Yes No" ("In use" and "Body covered" is converted back to "Yes" and classified into three types: "Yes", "No", and "Undecidable") was calculated.

As a result, it improved from 0.82 to 0.94.

You might have thought, "You return to the 3 categories after all?", but considering the actual application, it is okay to mistake "in use" for "covered by the body". The reason for increasing the classification is to improve the efficiency of learning.

Lastly

I read the second method "Detailing the classification" in a book, but I realized that it would be useful in such a place.

Please try it if you find it helpful.

However, although the number has decreased considerably, there are still false positives, so I would like to continue improving.