pool-watch

YOLO26 pose tracking from a webcam: detects and tracks people, classifies activity (standing in pool / swimming / walking / sitting), counts unique people, and saves one body avatar and one face crop per person. Draws boxes, activity labels, and the pool zone overlay on a live window.

Inference always runs on CPU (hardcoded in main.py); device in config is the camera device only.

Install

uv venv .venv --python 3.12
uv pip install --python .venv torch torchvision --index-url https://download.pytorch.org/whl/cpu
uv pip install --python .venv "ultralytics==8.4.171" pyyaml lap openvino

openvino backs the face-profile embedder (profiles.py); without it, profiles are silently disabled. lap is required by the built-in tracker.

Windows notes

• Use .venv\Scripts\python instead of .venv/bin/python (or ignore mise.toml), and run with an explicit camera index: --device 0 (OpenCV DSHOW backend is selected automatically). /dev/videoN paths in config.yaml are Linux-only.
• Face-profile models: download face-reidentification-retail-0095.{xml,bin} (FP32) from the Open Model Zoo into the project root, else profiles are disabled: https://storage.openvinotoolkit.org/repositories/open_model_zoo/2023.0/models_bin/1/face-reidentification-retail-0095/FP32/

Run

mise.toml in the repo plus a one-time eval "$(mise activate bash)" in ~/.bashrc puts .venv/bin on the path, so plain python main.py works from the project dir in new shells. Otherwise:

.venv/bin/python main.py   # or: source .venv/bin/activate

Flags:

Flag	Meaning
--config PATH	config file (default config.yaml)
--device DEV	override camera device (e.g. /dev/video0)
--no-show	headless: process and log only, no window
--seconds N	auto-quit after N seconds (default: run forever)
--auto-zone N	auto-learn the pool zone from N minutes of footage (default: off)

Quit with Ctrl-C or q/Esc in the window. On exit it prints seen N unique people. Avatars go to avatar_dir and faces to face_dir (default runs/faces/) as person_<ID>.jpg.

Pool zone drawing (window only, not with --no-show): press z, click the vertices of the pool, u undoes the last vertex, Enter saves the normalized polygon to config.yaml (applied live), Esc cancels. Pool mode itself can be toggled any time with p (overlay + in-pool labels off/on, nothing written to config). Needs at least 3 points. As with auto-learn, saving rewrites the YAML, so comments in config.yaml are lost.

Auto zone learning (--auto-zone N): run with --auto-zone N to learn the pool zone automatically from the first N minutes of footage (works headless, unlike z). It collects centroid positions of confirmed people moving slower than pool_speed_max_px_s — the pool is where people linger. At the window end it needs at least 20 such points; with fewer it keeps the current zone and prints a message. Otherwise the convex hull of the points is simplified to ~6–8 vertices, normalized to 0–1, saved back into config.yaml, and applied to the live overlay. It runs once per invocation; z-mode drawing (above) remains the manual alternative. As with z-mode, saving rewrites the YAML, so comments in config.yaml are lost.

Headless smoke tests (6 checks: model load + predict, face-crop math + aspect gate, 9 activity threshold cases, bad-device exit, live run, avatars written):

.venv/bin/python test_smoke.py

Tuning (config.yaml)

Key	Default	Effect
device	/dev/video2	Camera device. On this machine /dev/video2 is the Microsoft LifeCam.
capture_w / capture_h	1280 / 720	Capture resolution. Set via MJPG (needed for 720p over USB2); lower to 640x480 if your camera/driver misbehaves.
model	yolo26n-pose.pt	Weights file. Pose model enables face crops. With openvino: true the matching exported <model>_openvino_model/ dir is preferred when it exists; otherwise falls back to yolo11n-pose.pt, then yolo26n.pt (see below).
conf	0.4	Detection confidence threshold. Lower = more detections, more false positives.
imgsz	320	Inference image size. Bigger = more accurate, slower. Default dropped from 480 after measuring ~1 FPS at 480 on this CPU.
min_confirm_frames	10	An ID counts toward the unique-people total only after being seen this many frames (debounces flicker IDs).
walk_speed_px_s	120	Centroid speed (px/s) above this = "walking".
pool_speed_max_px_s	30	Inside pool zone below this speed = in-pool label ("standing in pool" / "swimming", fallback "using the pool").
standing_verticality	1.0	Torso verticality ratio (height/width of shoulder-to-hip line) at or above this = "standing in pool", below = "swimming".
sit_stand_ratio	0.3	Min hip-to-ankle vertical distance as a fraction of person height to label a slow mover "standing" instead of "sitting"; without confident leg keypoints the label stays "sitting".
pool_zone	bottom band	Normalized [x, y] polygon (0–1) scaled to frame size; checked against the person's centroid.
avatar_dir	runs/avatars	Directory for avatar crops (created if missing).
face_dir	runs/faces	Directory for face crops (created if missing).
blur_faces	false	Gaussian-blur the face region inside saved body avatar crops. Best-effort: avatars saved in frames without a valid face box stay unblurred.
openvino	true	Prefer the exported <model>_openvino_model/ dir at load time when present; silently falls back to the .pt chain (see below).
show_hud	true	The People: N here, M total line in the window's top-left corner.
pool_mode	true	Pool-zone overlay + in-pool labels + z drawing + --auto-zone. false (or press p live) = plain detection with walking/sitting only; unique count unaffected.
show_faces	true	Filmstrip of the last 5 saved faces as ID-labeled thumbnails along the window's bottom edge (seeded from face_dir at startup).
face_match_threshold	0.45	Cosine similarity a face crop needs against a profile centroid to join that profile instead of creating a new one (see Profiles).
profiles_dir	runs/profiles	Directory for per-person profile images (profile_<id>.jpg) and the profiles.json store.

Camera captures at capture_wxcapture_h (default 1280x720) via MJPG over V4L2.

How counting works

Each track ID keeps a frame counter and a 30-point trail. An ID is only counted as a unique person once it has been tracked for at least min_confirm_frames (default 10) frames — brief false IDs never reach the total. The count is recomputed every frame; lost IDs stay counted.

Activity: centroid speed is measured over the trail window. If the centroid is inside pool_zone and speed < pool_speed_max_px_s, the in-pool label is decided from torso keypoints (COCO shoulders 5,6 / hips 11,12, conf ≥ 0.5): a vertical torso (verticality ≥ standing_verticality) → "standing in pool", horizontal → "swimming". When those keypoints are missing → "using the pool" (fallback). Outside the pool: speed > walk_speed_px_s → "walking"; else → "sitting".

Avatars

Whenever a track gets its highest-confidence detection so far, the frame crop of that box (with 10% margin, clamped to frame bounds) is written to avatar_dir/person_<ID>.jpg. Each ID ends with its single best crop. With blur_faces: true, the face region is gaussian-blurred inside the saved crop (best-effort — avatars saved in frames without a valid face box stay unblurred).

Faces

When the loaded model is a pose model, each face crop is computed from the COCO keypoints 0–4 (nose, eyes, ears) with confidence ≥ 0.5, bounding those points with a 25% margin, and written to face_dir (default runs/faces) as person_<ID>.jpg — same policy as body avatars: a crop is only overwritten when the current detection beats the ID's best confidence so far. If a non-pose fallback model loads, faces are disabled (one-time warning: no keypoints: faces disabled).

Profiles

Saved face crops are matched into per-person profiles. Each crop is embedded with the Intel Open Model Zoo face-reidentification-retail-0095 descriptor model (256-d, BGR 128×128 input, L2-normalized so dot product = cosine). The model ships as OpenVINO IR in the repo root (face-reidentification-retail-0095.xml + .bin, gitignored — download once from storage.openvinotoolkit.org) and runs on the openvino package that ultralytics already requires.

• Grouping: at startup every face_dir/person_*.jpg not yet listed in any profile's files is embedded and clustered: cosine ≥ face_match_threshold against a profile centroid joins it (centroid becomes the running mean, re-normalized), otherwise a new profile is created — printed as grouped K face files into N profiles. During runs, each newly saved best face is matched live the same way; track IDs that belong to a profile are listed in the exit summary (profiles: p1 [tids ...], ..., or profiles: none) and shown as p<id> <tid> in the filmstrip. Because track IDs restart every run, membership across runs is tracked by face files, not IDs.
• Persistence: everything lives in profiles_dir/profiles.json ({id, tids, best_conf, face_file, files, centroid} per profile), written atomically (tmp + os.replace) after every live change and after startup grouping; centroids are 256-float vectors. The best face of each profile is kept as profiles_dir/profile_<id>.jpg.
• Disable behavior: if openvino is not importable or the model files are missing, startup prints face profiles disabled (<reason>) and the rest of the program runs unchanged.

Troubleshooting

• Wrong camera (error: could not open camera): find the right node with lsusb and v4l2-ctl --list-devices, then pass --device /dev/videoN or edit device in config.yaml.
• Low FPS on CPU: lower imgsz (e.g. 320) in config.yaml.

OpenVINO

With openvino: true (the default in the repo), main.py prefers yolo26n-pose_openvino_model/ (gitignored, so it must be exported once per checkout) at load time, falling back to the .pt chain when the dir is missing. Export it with:

.venv/bin/yolo export model=yolo26n-pose.pt format=openvino imgsz=320

Measured on this CPU: 3.97 → 5.49 FPS (~+38%) at imgsz: 320.

Model fallback

config.yaml loads yolo26n-pose.pt (or its OpenVINO export, above). If that fails (missing weights, older ultralytics, etc.), main.py warns and falls back to yolo11n-pose.pt, then yolo26n.pt. If all fail it exits with an error.