Google Coral TPU and Frigate Integration Guide

This guide walks you through setting up a Coral USB Accelerator with a Proxmox host running a Frigate container. It explains the prerequisites, device initialization, udev configurations, and integration steps to enhance object detection performance using the Coral TPU.

1. Prerequisites

  1. Coral USB Accelerator

    • Have your Coral USB Accelerator (e.g., Coral USB Accelerator v2) ready.

  2. Proxmox Host Details

    • A Proxmox node (v7.x or newer) where you can create and modify LXCs.

    • A working Frigate LXC/container already running Frigate with your camera feeds set up.

  3. Tools to Install on the Proxmox Host

    apt update
apt install -y usbutils udev python3 python3-venv git unzip

    • usbutils: for running lsusb

    • udev: to create custom rules for the Coral device

    • python3/python3-venv: to run PyCoral examples before LXC passthrough

    • git/unzip: to clone and extract repositories

2. Plug in the Coral and Verify on the Host

  1. Insert the Coral USB into a USB 3.0 port on your Proxmox host (avoid using a passive USB hub).

  2. Check lsusb Immediately

    lsusb | grep -i coral

    • If it shows something like:

      Bus 003 Device 006: ID 1a6e:089a Global Unichip Corp.

      that means the Coral is still in “Global Unichip” mode (1a6e:089a) and hasn’t been initialized yet.

  3. Install the Edge TPU Runtime on the Host

    wget https://dl.google.com/coral/edgetpu_api/edgetpu_runtime_20210620.zip
unzip edgetpu_runtime_20210620.zip
# Choose whichever matches your hardware:
dpkg -i libedgetpu1-max_*.deb    # or libedgetpu1-std_<version>.deb
dpkg -i libedgetpu-dev_<version>.deb  # (optional, for headers)

  4. Install Pyenv on the Host (to get Python 3.9)

    # Follow the Real Python tutorial:
curl https://pyenv.run | bash
# Then add these lines to ~/.bashrc (or ~/.zshrc):
export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init --path)"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
source ~/.bashrc

# Install Python 3.9 via pyenv:
pyenv install 3.9.16
pyenv virtualenv 3.9.16 coral-tpu
pyenv activate coral-tpu

  5. Clone the PyCoral and Test Data Repos

    git clone https://github.com/google-coral/pycoral.git
git clone https://github.com/google-coral/test_data.git

    • Why? pycoral contains example scripts like classify_image.py; test_data provides models and images those examples use.

  6. Install PyCoral via pip

    pip install numpy
pip install pycoral

    • Why? The Ubuntu package python3-pycoral can be outdated or missing. Installing via pip ensures you have the latest library.

    • Link: (User note) “I couldn’t apt install pycoral, so I did pip install pycoral.”

3. Wake Up the Coral Device (Host)

  1. Run the Classification Example From the directory one level above both pycoral/ and test_data/:

    cd ~/           # or wherever you cloned both repos
python3 pycoral/examples/classify_image.py \
  --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
  --labels test_data/inat_bird_labels.txt \
  --input test_data/parrot.jpg

    • What Happens:

      • PyCoral loads the quantized MobileNet V2 Bird model onto the Coral and runs inference on parrot.jpg.

      • In the process, the Coral’s USB interface switches from 1a6e:089a (Global Unichip) to 18d1:9302 (Google Inc.).

  2. Verify the Mode Switch

    lsusb | grep -E "18d1:9302"

    You should see something like:

    Bus 003 Device 005: ID 18d1:9302 Google Inc.

    • If it still shows 1a6e:089a, double-check you installed the correct libedgetpu1-* package.

  3. Record the Bus/Device Path

    • In this example, it’s /dev/bus/usb/003/005. You’ll use that path in the LXC config.

4. Create Udev Rules (Host)

  1. Create a New Rule File

    cat <<EOF > /etc/udev/rules.d/98-coral.rules
SUBSYSTEMS=="usb", ATTRS{idVendor}=="1a6e", ATTRS{idProduct}=="089a", OWNER="root", MODE="0666", GROUP="plugdev"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="9302", OWNER="root", MODE="0666", GROUP="plugdev"
EOF

    • Why? This ensures that when the Coral switches modes, it always gets 0666 permissions so any process (including the LXC) can open it.

  2. Reload and Trigger the Rules

    udevadm control --reload-rules && udevadm trigger

    • Confirm Permissions

      ls -l /dev/bus/usb/003/005

      You should see:

      crw-rw-rw- 1 root plugdev 189, 4 Jun  1 16:15 /dev/bus/usb/003/005

5. Passthrough Coral into Your Frigate LXC

  1. Locate the LXC Config File On the Proxmox host:

    /etc/pve/lxc/<CTID>.conf

    Replace <CTID> with your container ID (e.g., 113).

  2. Edit /etc/pve/lxc/113.conf and append:

    # Allow USB character devices
lxc.cgroup2.devices.allow: c 189:* rwm

# Bind-mount the Coral device into the container
dev0: /dev/bus/usb/003/005

    • Explanation:

      • c 189:* rwm grants the LXC permission to access all USB character devices (major 189).

      • dev0: /dev/bus/usb/003/005 ensures the Coral shows up inside the container at the same path.

  3. Restart the LXC

    pct stop 113
pct start 113

    • Verify Inside the LXC

      pct exec 113 -- lsusb | grep -i "Google Inc"

      You should see:

      Bus 003 Device 005: ID 18d1:9302 Google Inc.

    If it still appears as Global Unichip inside the container, run the classification example on the host again (so it’s already in 18d1:9302 mode) and then restart the LXC.

6. (Optional) Install PyCoral & Edge TPU Runtime Inside the LXC

If you’d like to run PyCoral scripts from within your Frigate LXC (or Frigate itself needs Python bindings), do this inside the container:

  1. Enter the LXC

    pct exec 113 -- bash

  2. Install the Edge TPU Runtime

    wget https://dl.google.com/coral/edgetpu_api/edgetpu_runtime_20210620.zip
unzip edgetpu_runtime_20210620.zip
dpkg -i libedgetpu1-max_*.deb  # or libedgetpu1-std_<version>.deb

    • Why? The container needs the same kernel driver to open /dev/bus/usb/003/005.

  3. Install PyCoral via pip

    apt update
apt install -y python3-pip python3-venv
python3 -m venv /opt/py39
source /opt/py39/bin/activate
pip install numpy
pip install pycoral

    • Why? You can’t rely on apt install python3-pycoral. Installing via pip ensures compatibility.

  4. Test Inside the Container

    python3 - <<EOF
from pycoral.utils.edgetpu import list_edge_tpus
print(list_edge_tpus())
EOF

    • Expected output:

      ['/dev/apex_0']   # or ['/dev/bus/usb/003/005'], depending on your setup

7. Update Frigate’s Configuration to Use the Coral

  1. Edit Frigate’s config.yml (e.g., /etc/frigate/config.yml inside the LXC):

    • Before (CPU/OpenVINO):

      detectors:
  ov:
    type: openvino
    device: CPU
    model:
      path: /openvino-model/FP16/ssdlite_mobilenet_v2.xml
model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

    • After (Edge TPU):

      detectors:
  coral:
    type: edgetpu
    device: usb

    • Why? This switches Frigate’s object detector from CPU/OpenVINO to the Coral TPU. You can remove or ignore the previous model: block because Coral will load its built-in COCO model by default (or you can supply your own .tflite and .txt if desired).

  2. Restart Frigate inside the LXC:

    pct exec 113 -- supervisorctl restart frigate

    • Verify the logs show something like:

      [frigate.detectors.utils] Starting EdgeTPU detector on usb:0

8. Verify Performance Gains

  1. Check Frigate Logs

    • Look for “EdgeTPU” messages rather than “OpenVINO.”

    • Typical log line on startup:

      Detected Edge TPU at /dev/bus/usb/003/005

  2. Monitor CPU Usage

    • In the Proxmox UI or by running htop inside the LXC, confirm that CPU usage for object detection has dropped significantly.

    • Expected improvements:

      • On a USB 2.0 port, inference might be ~ 40 ms/frame.

      • On a USB 3.0 port, inference should be < 10 ms/frame.

    • Overall, you’ll often see a 50–70 % CPU reduction compared to CPU/OpenVINO.

9. Troubleshooting Checklist

  • Coral Still Shows “Global Unichip Corp.” Inside LXC

    1. Make sure you ran the PyCoral classify_image.py example on the host before starting the container.

    2. Verify that udevadm control --reload-rules && udevadm trigger was executed after creating /etc/udev/rules.d/98-coral.rules.

    3. Confirm that the host’s lsusb shows ID 18d1:9302 Google Inc. before you start the LXC.

  • Permission Denied When Accessing /dev/bus/usb/003/005

    1. Check ls -l /dev/bus/usb/003/005 on both host and inside LXC—permissions should be crw-rw-rw- root plugdev.

    2. Ensure Frigate runs as root (or a user in the plugdev group).

  • LXC Fails to Start with USB Errors

    1. Double-check /etc/pve/lxc/113.conf—the dev0: path must exactly match the host’s /dev/bus/usb/XXX/YYY.

    2. Confirm lxc.cgroup2.devices.allow: c 189:* rwm appears above the dev0: line.

    3. If you plug the Coral into a different port (so the bus/device changes), update that path and restart the LXC.

  • Edge TPU Runtime Version Issues

    • Some Coral hardware requires libedgetpu1-std instead of -max. If you see “Cannot open device” errors, uninstall any existing libedgetpu1-* packages and install the alternate variant.

  • Frigate Doesn’t Detect the Coral

    1. Inside LXC, run dmesg | grep -i edgetpu—you should see the Edge TPU driver loading.

    2. Verify lsusb inside LXC shows 18d1:9302 Google Inc.

    3. Check Frigate logs for any “Failed to open device” messages.

  • USB 2.0 vs. 3.0 Performance

    • If you plug into a USB 2.0 port, inference times may hover around ~ 40 ms. On a USB 3.0 port, expect < 10 ms. Move the Coral to a USB 3.0 port for best performance.

TODO

  • Verify that the dev0: /dev/bus/usb/003/005 entry works consistently—if you move the Coral to a different port, update this path accordingly and restart the LXC.

  • Also whether lxc.mount.entry: /dev/bus/usb/003/005 dev/bus/usb/003/005 none bind,optional,create=file which was stopping from the container starting due to

safe_mount: 1425 No such file or directory - Failed to mount "/dev/serial/by-id" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/serial/by-id"
safe_mount: 1425 No such file or directory - Failed to mount "/dev/ttyUSB0" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/ttyUSB0"
safe_mount: 1425 No such file or directory - Failed to mount "/dev/ttyUSB1" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/ttyUSB1"
safe_mount: 1425 No such file or directory - Failed to mount "/dev/ttyACM0" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/ttyACM0"
safe_mount: 1425 No such file or directory - Failed to mount "/dev/ttyACM1" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/ttyACM1"
safe_mount: 1425 No such file or directory - Failed to mount "/dev/fb0" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/fb0"
run_buffer: 571 Script exited with status 17
lxc_setup: 3948 Failed to run autodev hooks
do_start: 1273 Failed to setup container "113"
sync_wait: 34 An error occurred in another process (expected sequence number 4)
TASK ERROR: startup for container '113' failed

can be added back instead of dev0: which seems to dedicate the TPU to the container instead of sharing it.