Updates:

Facial Recognition: (Ryan, Kurtis, Brad)

We received our camera and were able to begin with the facial recognition implementation. First, we installed the our dependencies onto the Raspberry Pi 3. Here are the dependencies needed:

  • python-shell - Needed to run Python scripts from Node.js. Our Smart Mirror OS uses Node.js so we use it here to run our OpenCV Python based scripts.
npm install
  • OpenCV - The computer vision library we will use for face recognition.
sudo apt-get install libopencv-dev python-opencv

OpenCV has three main algorithms for face recognition.

  • Local Binary Patterns Histograms (LBPH)
  • Eigenfaces
  • Fisherfaces

Eigenfaces and Fisherfaces view the entire dataset to find a recognized face, while LBPH compares individual sets with each other until a match is found. Since LBPH compares individually, we hypothesized that this would be the fastest algorithm for our purposes.

To recognize a face using these methods, you first must provide a set of images that can be analyzed to create a training set. We took about 20 pictures of Ryan’s face and then used those images for creating our training set. With the training set compiled, we then ran a test to compare the video stream from the camera to the saved data set. On our first test, we were able to get a strong match!!!

Next we added Brad’s profile to the data set. Here is where we ran into trouble. The method did not match Brad with his profile, but instead kept matching with Ryan’s profile. We tried using different algorithms with varying thresholds, and still had the same issue. Currently we are investigating why this may be the cause, but our suspicions lie with the camera used. Our camera is a 1 Megapixel camera. We chose a this camera with a lower resolution because we worried that the Raspberry Pi would not be able to process images of high resolution fast enough. We think that this resolution may actually be too low to differentiate faces from one another, especially with someone standing a feet or two away from the camera lens. From doing some research, our problems may also stem from the fact that we did not “train” our facial detection system enough. For our next trials, we will experiment with taking a high volume of high quality pictures (100+ pictures at good angles). This will require messing around with the given capture script, as each picture taken takes approximately 2 seconds each.

Mobile Application: Updates (Brandon)

An open source web application that acts as a Remote Control for the SmartMirror was available online. The menu of the controller can be seen in the picture below. We were able to download the contents in order to begin understanding the code. The current code does not provide a user-friendly way of adding devices that can access the Remote Control webpage and instead must be manually added into a configuration file directly.

picture alt


Next Steps:

Facial Recognition: (Ryan, Kurtis, Brad)

We ordered the Pi Camera which has a much higher resolution of 8 Megapixels. It also supports 1080p video at 30fps or 720p at 60fps. With these specifications our results should be much better, and we can actually adjust the resolution of the images taken so it will help us isolate this issue. We have also ordered ribbon cables so that the camera will be able to be appropriately mounted on top of the SmartMirror.

This next week we want to get recognition working properly. With the new camera we will try the same tests and see if the results are better. This camera has the ability to adjust resolution so with some testing we should be able to come up with an optimal resolution that has good performance, as well as reliable recognition. Once this is setup correctly we can begin with the custom user profile options for our Smart Mirror.

Mobile Application: (Brandon)

There are many features to the open source SmartMirror Remote Control. However, the Remote Control does not include several key features we wish to have in our design. For example, currently it does not have the ability to add devices we wish to use easily. If we want to use our phone, we have to add our phone’s IP Address manually into a configuration file on the Raspberry Pi. Also, the default controller also only allows customization of the one Magic Mirror shared amongst all users. With our facial recognition, we will incorporate the ability to customize for each user by saving the configurations and reverting to the saved changes upon facial recognition. This next week Brandon will look into the developing a login information webpage to add a device which is trying to access the Remote Control.

Gesture Control: (Ryan, Kurtis)

Gesture control can definitely be done with the camera, but we have to consider if the Raspberry Pi 3 is up to the task. Since the Pi will already be doing facial recognition, using it at the same time for gesture recognition may be taxing on the processor. This next week Ryan will look into the option of using OpenCV with the camera for gesture control versus using a separate module for gesture sensing, such as the Hover, (shown below).

Smart Mirror ON/OFF Protocol: (Brad)

We are planning to use a proximity-sensor like protocol to control the ON/OFF state for our SmartMirror. Luckily, there are many resources online for this behavior. We have identified a couple of resources online that use OpenCV and Python in order to detect motion through the PiCamera, which we will be implementing.

1: https://forum.magicmirror.builders/topic/52/pir-sensor-put-your-mirror-to-sleep-if-not-used/9

2: http://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/