Kid's Eye and Energy Safe Smart TV - Part 2

Sometimes I feel, kids do this just to irritate us. Or may be, we always have a prioritized  list of problems in our subconscious mind. When we solve the top one, the next one pops up and gets highlighted. It is indeed never ending.

Few extra hours of running TV will hardly make any difference in monthly bill. But it bothers me a lot. Conservation of energy is really important to me. Especially when I know, I am not using renewable energy.

In my last project (Kid's Eye Safe Smart TV - Part 1), I was successful in addressing my topmost concern. Reinforcement learning worked very well for them. But nowadays, I am noticing that TV continues to run for hours without any viewers. No one bothers to switch off the TV before going outdoor or moving to another room to play. May be that's why they are called Kids.

Let them be kids.  Let's do something to trade off with this situation.

Project Concept

This is enhancement of my last project (Kid's Eye Safe Smart TV - Part 1).  I have added one more sensor called PIR motion sensor (HC-SR501).

Dual Technology Occupancy Sensor (Ultrasonic sensor & Motion sensor) yields a very responsive and reliable solution to detect occupancy. PIR sensor detects the change of infrared radiations emitted by the subject in motion in its field of view. Perfect for my project to detect audience in front of TV.

Circuit built using ESP32 will read Ultrasonic Distance Sensor (HC-SR04) and PIR Motion Sensor (HC-SR501). Based on that reading, Micropython logic will decide when to pause or resume or switch off the TV by calling smart TV API (e.g. in my case Roku TV API to toggle Play/Pause  http://$ROKU_DEV_TARGET:8060/keypress/play).

In this PoC, if viewer goes very close (within preconfigured threshold - 1 meter) to the TV or no viewer (no motion) detected for certain period of time (say 15 seconds), TV will be paused (switch off  may be the preferable option in real life)  automatically.

Tools,Technologies and Components used in this article

• Thonny, Python IDE for beginners
• MicroPython Firmware on the ESP32
• Roku TV External Control Protocol (ECP)
• MicroPython Libraries:
  • Ultrasonic sensor HC-SR04 driver
  • SSD1306 128x64 OLED display driver
  • WiFi Manager
• Hardware Components:
  • ESP-WROOM-32 Dev Kit
  • HC-SR04 Ultrasonic Sensor
  • HC-SR501 PIR Motion Sensor
  • Oled display 128x64 - Optional
  • Breadboard
  • 3.3v & 5v Breadboard Power Supply
  • Breadboard Jumper Wires
  • 1kΩ & 470Ω Resistors
  • Less Smart TV (Roku)

Build the Circuit

Connect all the components as shown in the diagram below.

Schematic Diagram:

Circuit Design:

Code

1. Connect ESP32 development board to your computer.
2. Open Thonny Python IDE.
3. Select Run --> Select interpreter...
4. Choose MicroPython (ESP32) as device and corresponding port.

   

5. Upload following python libraries
   • HC-SR04 driver - hcsr04.py
   • SSD1306 OLED driver - ssd1306.py
   • WiFi Manager - wifimgr.py
6. Upload the below "main.py" (pre-alpha version). For my Roku TV, Rest endpoint to simulate "Play/Pause" remote button is http://192.168.0.20:8060/keypress/play

import wifimgr
from hcsr04 import HCSR04
from machine import Pin,I2C
import ssd1306,time
import urequests

def init():
    i2c = I2C(scl=Pin(19), sda=Pin(18))
    oled = ssd1306.SSD1306_I2C(128, 64, i2c, 0x3c)
    hcsr04 = HCSR04(trigger_pin=32, echo_pin=35, echo_timeout_us=1000000)
    hcsr501Pin = Pin(26, Pin.IN)
    hcsr501Pin.irq(trigger=Pin.IRQ_FALLING, handler=interruptCallbackHandler)
    
    return oled, hcsr04


def interruptCallbackHandler(p):
    global motionDetected
    motionDetected = True
    global lastTimeMotionDetected
    lastTimeMotionDetected = time.time()
    print("$$$$Motion detected at %s" % lastTimeMotionDetected)


def log(msg, x, y):
    oled.text(msg, x, y)
    oled.show()


def clearDisplay():
    oled.fill(0)


def toggleTvPlayPause():
    try:
        response = urequests.post("http://192.168.0.20:8060/keypress/play")
        print("API Response %s" % response.status_code)
        return response.status_code == 200
    except:
        print("Error !!!")
        log("Error !!!", 0, 30)
        return False
    

def anyoneWatching():
    global motionDetected
    
    if motionDetected:
        # If no motion detected in next 15 secs
        if ((time.time() - lastTimeMotionDetected) > 15):
            motionDetected = False
            print("Not Watching at %s" % time.time())
            return False
        else:
            print("Watching at %s" % time.time())
            return True
    else:
        print(">>>Not Watching at %s" % time.time())
        return False


def play():
    global isPaused
    
    if isPaused and toggleTvPlayPause():
        isPaused = False


def pause():
    global isPaused
    
    if not isPaused and toggleTvPlayPause():
        isPaused = True


# Initialize OLED display, Distance & Motion sensor
oled, distanceSensor = init()

# Connect to WIfi
log("Connecting...", 0, 0)
wlan = wifimgr.get_connection()
if wlan is None:
    log("No wifi !!!", 0, 20)
    print("Unable to connect to Wifi")
else:
    log("Connected :-)", 0, 20)
    wifimgr.deactivate_ap()
    print("Deactivated AP mode.")

time.sleep_ms(1000)

prevDistance = -1
isPaused = False
motionDetected = True
lastTimeMotionDetected = time.time()


while True:
    currDistance = int(distanceSensor.distance_cm())
    time.sleep(1)
    
    if currDistance != prevDistance and motionDetected:
        print("Distance: %s cm" % currDistance)
        clearDisplay()
        log("Dist: %s cm" % currDistance,0, 0)
        
        prevDistance = currDistance
        
        if anyoneWatching():
            if currDistance < 100:
                pause()
            else:
                play()
        else:
            pause()
            
        print("TV -> %s" % ("Pause" if isPaused else "Play"))
        log("TV -> %s" % ("Pause" if isPaused else "Play"), 0, 30)
        print("Motion -> %s" % ("Y" if motionDetected else "N"))
        log("Motion -> %s" % ("Y" if motionDetected else "N"), 0, 40)
        print("***************************")

Note:
Use the following method to configure interrupt for a Pin Pin.irq(handler=None, trigger=(Pin.IRQ_FALLING | Pin.IRQ_RISING), \*, priority=1, wake=None, hard=False)

I have added lots of print statements to capture the flow in console log for better understanding of demo video. Try to minimize the use of print statements. Specifically for interruptCallbackHandler (External Interrupt Handler), DO NOT use anything extra. It has to be simple and short as much as possible. Please refer Tips and recommended practices for writing interrupt handlers.

Demo

Now, I am feeling responsible towards mother planet The Earth.

Circuit In Action

Video Demo:

Possible Extension

If you have pet, then you can include ESP32-CAM to detect the type of audience (pet or human) and use that input to drive the logic accordingly.

Download SrcCodes

All codes used in this post are available on Github: srccodes/smart-tv-occupancy-sensor.

References

• machine.Pin.irq
• Tips and recommended practices - Interrupt Handlers
• Cheap Pyroelectric Infrared PIR Motion Sensor on 3.3v
• ESP32 DevKit V1 Pinout
• PIR Motion Sensor HC-SR501 datasheet
• Ultrasonic sensor HC-SR04 datasheet
• Getting started with MicroPython on the ESP32