PATROL SIGHT — Product Fact Sheet (source of truth for this kit)

Compiled from files under /home/humanityaid/Desktop/code. Every claim below was traced to a file path. Marked INFERRED where it is a reading rather than a literal statement.

1. Identity

• Deployed as one Docker Swarm stack named patrol-sight — deploy.sh L5, L31.
• Stack project dir HumanityAid-OnPrem-FaceRecognition, stack file docker-stack.yml, env .env — deploy.sh L32-34.
• "A single-host, GPU-accelerated, on-premise face-recognition platform" — ARCHITECTURE.md L3.
• GPU inference container image: humanityaidai/patrol-sight-ai:tether-udp-be-webrtc-v3.2.0 — docker-stack.yml L802, L882.
• Underlying engine: HumanityAid Face Recognition API v3.3.1 "Scalable Edition", supports 13M+ face vectors — HumanityAid-Face_API/docs/README.md.
• 34 services in the stack (minio-init 0/1 is normal) — DEPLOYED_APPS_LINKS.md.
• PatrolSightLogo component exists in the shared UI kit — HumanityAid-UIKit/src/components/atoms/Logo/PatrolSightLogo.tsx.

2. Architecture — 5 Swarm tiers (docker-stack.yml)

Tier	Services
1 — Stateful (L55)	postgres 15, minio, minio-init, backend-redis, recognition-redis, node-redis, custom-recognition-redis
2 — CPU stateless (L272)	orm (Alembic gate), pgbouncer, auth-service, backend, recognition-server, nats-recognition, ar-postgres, ar-settings-api, ar-settings-frontend, node-api, frontend, frontend-custom-recognition, db-manager
3 — GPU (L792)	app (patrol-sight-ai), worker (FAISS shard rebuild + hot reload), ai-quality-engine (FQA v4)
4 — Infrastructure (L988)	coturn, nginx (mode: global), nats, cloudflared
5 — Museum AR (L1178)	museum-postgres, -redis, -api, -frontend, -nginx, -prometheus, -grafana on isolated museum-network

• Networks: app-network, museum-network (both overlay, attachable) — L1452-1460.
• 22 external volumes — "External prevents docker stack rm from deleting data" — L1466-1539.
• GPU model: 1 GPU shared via CUDA time-slicing, placement constraint node.labels.gpu == true, no generic_resources; multi-GPU path documented and pre-written but commented out — L18-29, L875-985. RAM caps: app 6 GB, worker 4 GB, FQA 4 GB.
• Bounded contexts (ARCHITECTURE.md): nginx sole intended public surface (80/8080/443/9000); auth-service JWT (shares JWT_SECRET, AES_KEK, AES_DEK_SALT); backend FastAPI 8000 → FACE_RECOGNITION_URL=http://app:5555; recognition-server consumes NATS v1.vision.frame.* and publishes to v1.vision.alert (default 2 replicas); node-api NodeHub device WS hub; app exposes 5555 TCP, UDP 5556 (JPEG), UDP 5557 (H.264).
• Two DBs on one engine: business humanity_aid_db (pooled via pgbouncer) + AI facedb (direct); ar_postgres isolated.

3. Recognition pipeline

Component	Value	Source
Detector	SCRFD det_10g.onnx (16.9 MB)	.env L448
Recognizer	ArcFace ResNet-50 w600k_r50.onnx, 512-d	.env L449-453
TensorRT engine	arcface_w600k_r50_fp16.engine, USE_TENSORRT=true	.env L459; docker-stack.yml L822
Thresholds	RECOGNITION_THRESHOLD=0.50, DET_CONF_THRESHOLD=0.65	.env L460-462
FAISS	FAISS_USE_GPU=true, IVF-PQ, FAISS_NPROBE=64	docker-stack.yml L816-818; config.py L111-113

Pipeline V3: Decode → SCRFD detect → 5-pt affine align (112×112) → ArcFace embed (512-d) → FAISS search → hybrid cosine fallback → resolve person (L1/L2/L3 cache) → format. ONNX provider priority TRT → CUDA → CPU; rotation fallback 90/180/270°; det_size 640 default / 1280 long-range.

Two-layer tracker: L1 IoU bbox → stable tracking_id; L2 embedding+identity cache that skips FAISS entirely — reuse ratio 0.976 (search skipped on 97.6% of frames).

Production FAISS snapshot (2026-07-09): 93,728 persons / 93,751 embeddings, 1 shard, IVF4096,PQ64x8, 95,667 vectors on RTX 4060 Ti — FAISS-ARCHITECTURE.md L1-13. ADC calibration: "IVF-PQ ADC similarities read ~0.86x of exact cosine."

Cache layers: L1 in-memory <1 ms · L2 Redis 5-30 ms · L3 FAISS GPU 0.5-5 ms · L4 Postgres cosine (disabled in the glasses path — "caused false positives + 3-12 ms latency per miss" — E2E_Pipeline_Review.md Stage 8).

4. Glasses / edge app (HumanityAid-UdpTetheredApp-)

• Android package com.example.androidcamerastream; README title "RayNeo X3 Pro Camera Stream".
• Low-latency path: Camera2 → MediaCodec H.264 HW encoder Surface → RTP/H.264 packetization → UDP — deliberately avoids ImageReader/JPEG/NV21 conversion.
• Config hardcoded in StreamConfigUtils.java: UDP_PORT=5557, RESULT_UDP_PORT=17391, 2432×1368 @ 30 fps, 20 Mbps, CAMERA_ZOOM=1.25f ("brings distant faces closer for better detection"). WebRTC variant 2560×1440 @ 30 fps, WEBRTC_DEVICE_ID=rayneo-x3-webrtc.
• deploy-glasses.sh: auto-detects ADB device, builds :app:assembleRelease (R8-optimized "so the StreamConfigUtils host IP / ports are baked in"), installs -r, launches.
• Client-side label stability: LabelSmoother EMA alpha=0.45 + spatial proximity, 3 s stale eviction; IoU>0.15 coasted-track suppression; TRACKER_TTL_FRAMES=15; ICE DISCONNECTED no longer fatal.
• Server transports: udp_stream_service.py (JPEG/UDP 5556, opt-in) and h264_stream_service.py (RTP/H.264 UDP 5557; NVDEC preferred ~1-2 ms, ffmpeg cuvid fallback; "newest-frame-wins" reader + FPS-capped worker "so the GPU never builds an unbounded backlog"). H.264 chosen for "~14-30x less bandwidth… which matters for battery/thermal-limited head-worn devices". Both run simultaneously; the app chooses by port.
• H.264 defaults: H264_DET_SIZE=1280, H264_RECOGNITION_FPS=15, H264_FRAME_ROTATE=cw90, H264_ALLOWED_IPS allowlist, H264_USE_HWACCEL=true.
• Label wire format name@@targetLabel@@#color@@, decoded client-side by LabelCodec.
• Transport decision doc (2026-07-07) compared custom UDP/RTP vs WebRTC/RTSP/SRT/QUIC on an isolated LAN (RTT <2 ms): keep custom UDP/RTP. "USB tethering investigated but not supported on Rokid firmware."

5. Hardware / OS

• Deployment host GPU: NVIDIA RTX 4060 Ti (8 GB, compute 8.9) — RESPIN_RUNBOOK.md L4.
• Swarm prereqs: nvidia default runtime in daemon.json; docker swarm init; docker node update --label-add gpu=true; external volumes — docker-stack.yml L7-11.
• Driver: NVIDIA 525+ (550+ recommended), CUDA 11.8 min / 12.4+ recommended, cuDNN 8.6/8.9+, NVIDIA Container Toolkit. Supported GPUs GTX 1080 Ti → RTX 5090, V100/A100/H100/L4/L40 — docs/10-performance/gpu-support.md.
• RTX 50-series (sm_120) needs FAISS compiled with explicit CMAKE_CUDA_ARCHITECTURES — pip faiss-gpu lacks sm_120.
• System RAM budget ≈ 12 GB (Postgres 4 GB shared_buffers, Redis 2 GB, FastAPI+workers 2-4 GB, FAISS CPU 1-2 GB, OS 2 GB). VRAM ≈ 1 GB per 1M faces, ≈ 2 GB at 13M — docs/10-performance/benchmarks.md.
• Glasses: RayNeo X3 Pro (5 GHz only — no WiFi 6E/6 GHz), Rokid AR glasses also referenced.
• Network: Cudy 5 GHz router; host LAN 192.168.10.199; glasses stream to 192.168.10.199:5557.

6. Deployment & ops

deploy.sh (337 lines) commands: up/deploy, down, restart, update <svc…>, status, logs, health, pull, config, cleanup, help. - require_swarm() aborts unless Swarm active. - Hand-parses and exports .env rather than using docker compose config, "because that renderer normalizes swarm fields… into forms docker stack deploy rejects". - Deploy flags --with-registry-auth --resolve-image=never --prune --detach=true — "single-node cluster, images pre-pulled locally… a registry hiccup otherwise wedges GPU services in 'preparing' forever". - down polls ~60 s until 0 services/networks carry the stack namespace. - health probes nginx :8080/health, backend :8000/api/, custom-recognition :8080/custom-recognition/health, recognition-server :8001/, FQA :8002/health, MinIO :9000/minio/health/live, both tunnel domains, then unhealthy containers.

Boot autostart: systemd humanityaid-infra.service (Type=oneshot, RemainAfterExit=yes, ExecStart=deploy.sh up, ExecStop=deploy.sh down, TimeoutStartSec=300, After=docker.service network-online.target). Swarm stack "auto-restores from Swarm state on boot". gpu-startup-wrapper.sh waits up to 5×5 s for nvidia-smi, +3 s driver settle, up to 15 app-start attempts, specifically retrying Faiss assertion … CUDA error 999.

.swarm-state/ holds pre/post-deploy image snapshots back to 2026-03-19. Release snapshots in releases/docker-stack-v1.1.7.yml — "byte-for-byte copy of the docker-stack.yml that produced this deployment".

Expected non-200s: :8000/api/ → 401, :8001/ → 404, minio-init 0/1 by design. Harmless: No module named 'faiss.swigfaiss_avx2' (base swigfaiss fallback).

v1.1.7 incident: login 504s from pgbouncer pool_mode=session pool exhaustion at size 10 shared by auth-service + backend; fix pool 10→40, documented as "headroom, not a cure" — real fix is transaction mode, blocked on asyncpg prepared_statement_cache_size=0.

7. Security / data handling

Posture: on-prem single host; nginx "the only intended public surface". ARCHITECTURE.md L15 flags that Postgres, Redis, NATS, backend, recognition, custom-recognition and AR services also publish host ports directly — "in production these should be collapsed behind nginx or bound to the private LAN IP (commented-out LOCAL_IP_ADDRESS bindings exist for exactly this)".

Glasses LAN "physically isolated — no untrusted devices, no internet uplink during operation"; consequently custom UDP/RTP is plaintext ("Security: None (plaintext)").

Auth/crypto: auth-service JWT login/refresh; NATS Nkey zero-trust auth per service (per-service nkey dirs mounted RO); AES-256-GCM payload encryption between nodehub and c2-frontend (NODEHUB_PAYLOAD_ENC_MODE=required — raw curl returns 400 by design); Face API key middleware + admin session; H264_ALLOWED_IPS allowlist; audit middleware + audit_router.py + AuditLogs UI; USE_SSL=True with self-signed certs for LAN IPs; ENABLE_CORS=false, DEVELOPMENT_MODE=false.

Credential findings present in the tree (findings, not recommendations). Literal secret values are deliberately NOT reproduced here — this document is distributed. Each item cites where to find the value in the tree; treat every one as compromised and rotate. - docker-stack.yml L810-835 hardcodes defaults for: the AI database password, DEFAULT_ADMIN_USERNAME/DEFAULT_ADMIN_PASSWORD, DEFAULT_API_KEY, and ADMIN_SESSION_SECRET. - Museum tier (docker-stack.yml L1199-1201): database user/password, Grafana admin password, and a development API key, all hardcoded. - .env L425 COTURN_USERS carries two hardcoded TURN credentials; L476 leaves the Grafana admin password at a placeholder value. - Production master API key committed in Android source — StreamConfigUtils.java L22 (WEBRTC_API_KEY), shipped inside the APK. The same key prefix also appears in an ops runbook. Highest-priority rotation. - Legacy Django C2 backend has its own CRITICAL committed-secrets findings (weak Django secret key and database password, DEBUG=True) — c2-backend-build/README.md. - .env L511-524 itself carries an operator checklist to rotate everything and generate NATS nkeys on a secure admin machine.

Privilege surfaces flagged in ARCHITECTURE.md L101-103: db-manager mounts the host Docker socket (RO) to run migrations via docker exec; privileged gpu-init; coturn — "the highest-risk nodes in the topology".

Data stores: MinIO buckets media/static/recognition/logs (public download policies, served via nginx /s3); Redis 1 GB/2 GB LRU + 512 MB AOF; log retention/archival services server-side.

8. Measured performance (all with file paths)

End-to-end glasses→label (RayNeo X3 Pro, RTX 4060 Laptop) — E2E_Pipeline_Review.md

Stage	Min	Typical	Max
Camera capture + H.264 encode	10	12	15 ms
Network glasses→server	2	3	5 ms
H.264 decode (ffmpeg CPU)	5	7	10 ms
Frame rotation	2	2	3 ms
SCRFD detection (det 1920)	15	20	30 ms
Embedding (TRT FP16)	5	8	12 ms
Tracker	1	1	2 ms
FAISS search	1	5	10 ms
JSON + UDP send	0.5	0.5	1 ms
Network server→glasses	2	3	5 ms
JSON parse + render	5	7	10 ms
TOTAL	~49 ms	~68 ms	~103 ms

Also: effective recognition 14-20 fps; RTT 3.5-5.1 ms avg 4.1, 0% packet loss; ~83 KB/frame ≈ 57-60 RTP packets. FAISS L3 over 1,931 searches: min 1.2 / avg 5.4 / median 5.8 / P95 8.5 / P99 11.7 / max 25.4 ms. GPU warmup dummy search removes ~170 ms cold start. Recognition distance: reliable ~5 m, max 6-7 m in good light; 7 m+ rejected by the 40×40 px min-face filter. DET_CONF_THRESHOLD 0.35→0.60 removed 2-11 false detections/frame at det 1920.

WebRTC path — LATENCY_OPTIMIZATION.md

Phase 1: NVDEC decode 1.3-2.7 ms (was 15-25 ms CPU); encoder min-bitrate floor = bitrate/2 to stop GCC collapse; DataChannel ordered=false, maxRetransmits=0. Phase 2 measured E2E with clock sync: p50 26-32 ms, p95 63-85 ms, p99 95-169 ms. DSCP marking reverted (p95 tail ~2× worse). Detection size kept at 1024 ("Need long-range detection (~8 m)"); nprobe kept at 64 ("Scaling to 100K+ faces — keeping nprobe=64 for recall"). BBOX_SMOOTHING=0.55.

Radio/network gains — CHANGELOG-2026-07-09-webrtc-nvdec-5ghz.md

2.4 GHz hotspot (72 Mbps, ping ~100 ms with power save) → 5 GHz WiFi 6 (480 Mbps, ping ~4 ms); 2432×1368 → 2560×1440; 6 → 20 Mbps; decode 1.2-2.3 ms steady.

Model accuracy MBF vs R50 — 0-90-team-benchmark-results.md (24 images, 3 people, yaw 0-90°)

R50 wins 19/21 comparisons (90%); avg similarity 0.6528 → 0.7072 (+0.0544). By bracket: 45-59° 0.834→0.857; 60-74° 0.776→0.812 (3/3); 75-90° 0.620→0.680 (15/17, 88%). MBF 3.4M params / 6.0 ms CPU vs R50 43.6M / 43.9 ms CPU (GPU TRT ~1-2 ms vs ~4-6 ms). "Switching to R50 reduces false negatives at extreme pose by ~60%."

API/service benchmarks — docs/10-performance/benchmarks.md

Single-face warm-cache total 50-80 ms. Batch FAISS: 1 face 85→65 ms; 10 faces 850→180 ms (4.7×); 100 faces 2070→106 ms (19.5×). Latency by dataset: 10K → 40-60 ms; 100K → 50-70; 1M → 60-90; 5M → 70-100; 13M → 80-130 ms. WebSocket JPEG 640×480 25-30 fps (p95 85 ms); 1280×720 15-20 fps (p95 120 ms). Tracking cuts FAISS searches/frame to 10-20%, avg latency 65→30 ms, GPU util 85%→40%. Throughput: /recognize 15-25 rps (8 workers), /detect 30-50, /search 20-30, /register 10-15. Bulk registration 8-12 → 40-60 img/s (1000 images in 15-25 s). FAISS memory at 13M: 26 GB FlatIP vs 962 MB IVF-PQ (27× compression).

Smart-glasses WebSocket JPEG — SMARTGLASSES_EXECUTIVE_SUMMARY.md (RTX 5060, 720p)

14-22 FPS at 25-57 ms/frame; long-range 8-10 m ~12 FPS at det 1280; L2 reuse 0.976; test suites "streaming 27/27, API 175/175, behavioral 45/45 passing".

Single-image API regression — RECOGNITION_LATENCY_ANALYSIS.md

1254 ms for a 9.4 MB 5712×4284 upload; ML pipeline only ~8 ms, Cloudflare upload leg ~937 ms. Fix = client resize to ≤1280 px → expected 150-250 ms.

9. Features

Engine/API routers: faces (+enhanced), groups, search, recognize-optimized, bulk register/upload/delete, jobs, admin, analytics, audit, logs + archived logs, cache management, evaluation, export + system export, GPU, maintenance, metrics/system-metrics/frontend-metrics, statistics, system dashboard, magface, stream, udp_stream, webrtc.

Operator UI pages: Dashboard, Faces, Search, Verify, LiveCamera, StreamMonitor, Analytics, Batch, BulkUpload, Evaluation, PerformanceLab, AuditLogs, Logs, ApiKeyManager, AdminJobs, AdminLogin, Settings, SystemReset. /stream-monitor shows per-device FPS/latency/face counts with annotated MJPEG preview; /live-camera allows browser-webcam testing.

Alerting: encoded colour-coded labels via /api/v1/faces/annotate-encoded-label, format "name @@label@@#hexcolor@@", show_name toggle, 10 colours verified — DEPLOYMENT_SUMMARY.md. NATS alert channel live at v1.vision.alert.

Multi-device: each pair of glasses is an independent session keyed by device_id; on-device-embedding privacy option sends the 512-float faceprint (~2 KB) instead of the image (~100 KB); ?long_range=true for 8-10 m.

Also: adaptive nprobe tuning, circuit breaker, index integrity service, cache preloader, startup cache, MagFace alternative recognizer, MTCNN alternative detector; ar-settings-api + ar_postgres + ar-settings-frontend as a dedicated glasses-config context (UI :8085); FQA face-quality engine; coturn STUN/TURN relay.

Watchlist domain = C2 "Targets": C2 frontend pages Targets, Labels, TrackingHistory, DeviceTracking, RealTime, AgentLevel, AgentReport, Actions, AssociatedActions, ActivityLogs, Administrator, Role, Devices, Workspace, StreamMonitor. Django C2 apps: target, target_label, surveillant, surveillant_level, surveillant_action, tracking_history, activity_log, role, user. The word "watchlist" appears only as a future item — "External watchlist integration — sync identities from third-party watchlists into the enrolled index", alongside "Audio / alert channel — push spoken or haptic alerts back to the wearer" and "Behavior / anomaly analytics" (SMARTGLASSES_EXECUTIVE_SUMMARY.md §7). INFERRED: the shipped watchlist equivalent is the C2 Targets + target_label model plus per-person enrolment in facedb; no third-party watchlist connector exists.

10. Inconsistencies to disclose honestly

1. TRT engine default disagrees with .env. docker-stack.yml L823/L894 default to arcface_rtx4060ti_fp16.engine, which .env L454-458 documents as a directory in the v3.2.0 image → load fails "Is a directory" → silent fallback to ONNX. Only the .env override is correct; engines must be paired (r50 engine ↔ r50 model).
2. Threshold drift across docs: recognition 0.35 / 0.45 / 0.50; detection 0.5 → 0.60 → 0.65.
3. Glasses host IP differs per document: 10.204.4.213, 192.168.10.199, 192.168.10.60, 192.168.100.238, 10.42.0.1. INFERRED: different test sites over time.
4. Two parallel deployments coexist: the Swarm patrol-sight stack and a separate recapi compose stack (own cloudflared, network_mode: host), tracked independently with older tags.
5. HumanityAid-OnPrem-FaceRecognition/README.md is stale (describes start.sh/compose/192.168.50.1 hotspot, has corrupted opening text).
6. ARCHITECTURE.md describes the compose topology, not the Swarm one — use docker-stack.yml as today's source of truth.