Skip to content
All projects
AI & ML · Cloud & DevOpsAug 2026

AI Site Monitoring on Existing CCTV

Vision analytics on a coal-handling site's 15 existing CCTV cameras, with nothing installed on the cameras: excavator productivity, PPE safety, conveyor state and fleet numbers, delivered as alerts and shift reports the client can count in money.

No public repository
excavator dumps counted on held-out footage (16/18 on a second clip)
21/21
excavator dumps counted on held-out footage (16/18 on a second clip)
no-helmet alert precision after retraining on operator reviews, up from 18/66
15/15
no-helmet alert precision after retraining on operator reviews, up from 18/66
fleet-number reads with voting, against 166/300 for a single read
300/300
fleet-number reads with voting, against 166/300 for a single read
existing cameras, nothing installed on them
15
existing cameras, nothing installed on them

Context

Nuralindo AI · Project Technical Delivery Lead and Software Architect · Aug 2026 – present. The client is anonymised, and every figure below was measured on the site's own held-out footage.

Problem

The client wanted productivity and safety numbers it could count in money, from the 15 cameras it already had, with nothing installed on the cameras.

System

  • On-site RTSP relay with GPU re-encoding, pushing to a cloud video gateway (MediaMTX)
  • A GPU AI engine on an NVIDIA L4, one process per camera, holding no database of its own
  • A FastAPI backend with dashboard, alerts and automated PDF and WhatsApp shift reports

What it measures

  • Excavator productivity: a frozen SigLIP vision-transformer embedding of two frames, with a linear probe trained on 15,367 hand-labelled frames, plus hysteresis tracking and state rules. It counted 16/18 and 21/21 dumps on held-out footage and replaced a third-party tool (AUC 0.50–0.83). A second probe recovers about 9 in 10 trips hidden behind other machines.
  • Safety: hard-hat and hi-vis probes trained on site data. No-helmet alert precision rose from 18/66 to 15/15 after retraining on operator reviews, and alerts are re-checked by a vision-language model after each shift.
  • Operations: optical-flow conveyor state, PatchCore-style anomaly detection, and OCR fleet-number reading with voting across a pass (300/300 synthetic frames, against 166/300 for a single read).

Delivery

I lead technical delivery with the client and with vendors in Indonesia and China, and scoped Phase 2: GPS fleet tracking, vibration and temperature motor-health sensors, and electric-fleet planning.

Reflection

Cameras miss events under glare, dust and occlusion, and every output still ends in an alert for a person. The next phase adds sensors on the motors, and the question behind it is how machines and robots can act on such signals reliably.