AI Site Monitoring on Existing CCTV
Vision analytics on a coal-handling site's 15 existing CCTV cameras, with nothing installed on the cameras: excavator productivity, PPE safety, conveyor state and fleet numbers, delivered as alerts and shift reports the client can count in money.
- excavator dumps counted on held-out footage (16/18 on a second clip)
- 21/21
- excavator dumps counted on held-out footage (16/18 on a second clip)
- no-helmet alert precision after retraining on operator reviews, up from 18/66
- 15/15
- no-helmet alert precision after retraining on operator reviews, up from 18/66
- fleet-number reads with voting, against 166/300 for a single read
- 300/300
- fleet-number reads with voting, against 166/300 for a single read
- existing cameras, nothing installed on them
- 15
- existing cameras, nothing installed on them
Context
Nuralindo AI · Project Technical Delivery Lead and Software Architect · Aug 2026 – present. The client is anonymised, and every figure below was measured on the site's own held-out footage.
Problem
The client wanted productivity and safety numbers it could count in money, from the 15 cameras it already had, with nothing installed on the cameras.
System
- On-site RTSP relay with GPU re-encoding, pushing to a cloud video gateway (MediaMTX)
- A GPU AI engine on an NVIDIA L4, one process per camera, holding no database of its own
- A FastAPI backend with dashboard, alerts and automated PDF and WhatsApp shift reports
What it measures
- Excavator productivity: a frozen SigLIP vision-transformer embedding of two frames, with a linear probe trained on 15,367 hand-labelled frames, plus hysteresis tracking and state rules. It counted 16/18 and 21/21 dumps on held-out footage and replaced a third-party tool (AUC 0.50–0.83). A second probe recovers about 9 in 10 trips hidden behind other machines.
- Safety: hard-hat and hi-vis probes trained on site data. No-helmet alert precision rose from 18/66 to 15/15 after retraining on operator reviews, and alerts are re-checked by a vision-language model after each shift.
- Operations: optical-flow conveyor state, PatchCore-style anomaly detection, and OCR fleet-number reading with voting across a pass (300/300 synthetic frames, against 166/300 for a single read).
Delivery
I lead technical delivery with the client and with vendors in Indonesia and China, and scoped Phase 2: GPS fleet tracking, vibration and temperature motor-health sensors, and electric-fleet planning.
Reflection
Cameras miss events under glare, dust and occlusion, and every output still ends in an alert for a person. The next phase adds sensors on the motors, and the question behind it is how machines and robots can act on such signals reliably.