Anomaly Detection with Machine Learning: How AI Is Replacing Reactive Surveillance in Physical Security

For years, physical security worked like this: something bad happened, the operator saw it on video, and only then did the response begin. Reactive by definition. The camera was a witness, not a detector.

Machine learning anomaly detection is changing that at the root. Instead of waiting for a human to notice something odd, the system learns what's "normal" at each site and triggers an alert when something deviates from that pattern. Before the incident, not after.

But as with almost everything in AI applied to real operations, the gap between what marketing promises and what works in production is enormous. In this article we explain how anomaly detection in security really works, what makes a system useful versus one that only generates noise, and why most solutions on the market fail at the same point.

What Is Machine Learning Anomaly Detection?

Machine learning anomaly detection is the process by which a statistical model learns the usual patterns of an environment and generates alerts when it observes behavior that deviates from that baseline.

In the context of video surveillance, that means:

• A person standing in front of a display case longer than average
• A vehicle parked in a restricted zone after hours
• A door opening with unusual frequency in the early morning
• A crowd exceeding the historical threshold in a given area
• An object appearing where there normally are no objects

The system doesn't need anyone to have told it "alert if there's an abandoned object." It learns that in that zone, between 2 am and 6 am, there's normally nothing. If something appears, that's an anomaly.

That conceptual difference, between fixed rules and pattern learning, is what separates real video anomaly detection from traditional video analytics based on static rules.

→ To go deeper into how a model builds that idea of "normal," read Anomaly Detection and Machine Learning: How AI Learns What Normal Looks Like.

Why Traditional Systems Fail

Before machine learning for security, video surveillance systems ran on basic motion detection or manually programmed rules. The problems were predictable:

Too many false alarms. A tree swaying in the wind, a lighting change from passing clouds, a small animal crossing the frame: everything triggered an alert. The operator ended up ignoring 95% of notifications because 95% were noise.

Rules that don't scale. Every site required manual configuration. If you have 50 clients with 10 cameras each, programming and maintaining rules for 500 cameras is unworkable.

No temporal context. A rules-based system doesn't know whether something is anomalous. It only knows whether a condition is met or not. It can't tell a normal visitor at 3 pm from the same behavior at 3 am.

No learning. If the environment changes, you have to reprogram it. The system doesn't adapt on its own.

Machine learning for anomaly detection solves these problems because the model learns continuously. It builds a dynamic profile of normality per camera, per zone, and per time window. What's normal on a Monday at noon isn't the same as what's normal on a Sunday at 2 am, and the model knows it.

How Anomaly Detection in Security Works in Practice

A well-built machine learning anomaly detection system has several layers working in sequence.

1. Video Ingestion and Normalization

The system receives the video stream from each camera. For machine learning anomaly detection to work at scale, it needs to connect to any existing camera without requiring proprietary hardware. The RTSP and ONVIF standard protocols make that possible with any IP camera brand on the market.

2. Object Detection and Segmentation

Before detecting anomalies, the model needs to know what's in the frame. It classifies objects: person, vehicle, animal, static object. That classification is the foundation the patterns are built on.

3. Building the Normality Baseline

This is where machine learning for security truly comes in. The model analyzes weeks of video to build a statistical profile of what's normal: how many people pass through that zone, at what times, at what speed, how often they stop, and for how long.

4. Real-Time Anomaly Scoring

With the baseline established, every event receives an anomaly score. A high-scoring event doesn't automatically mean an alert: it goes through a contextual validation process to determine whether it's relevant.

5. Context Filters

This layer is critical, and most systems underestimate it. The anomaly detection system must distinguish between:

• A person walking fast (normal) versus a person running while looking back over their shoulder (anomalous)
• A vehicle stopped for 30 seconds (normal in a parking lot) versus the same vehicle stopped for 20 minutes in front of an emergency exit (anomalous)
• A door that opens often during the day (normal at an access point) versus the same door opening three times between 1 am and 3 am (anomalous)

Context includes zone, time of day, historical frequency, and correlation with other events on adjacent cameras.

6. Alerting and Documentation

Only events that pass the filters generate an alert. The operator receives the notification with the video clip, the anomaly score, the historical context of that zone, and a suggested action. They don't have to go looking for the footage: it's already there, with full context.

The False Alarm Problem in Anomaly Detection

No anomaly detection system has zero false alarms. The real goal of anomaly detection in security is to reduce them to a level that makes the system operationally useful.

Here's the core problem: most systems that claim to use machine learning for anomaly detection actually have the model as a superficial layer on top of traditional motion detection. The result is a system that detects motion, calls it AI, and keeps generating the same level of noise as before.

The systems that really work have multiple independent validation layers before generating an alert. Each layer removes a different category of false positives:

• Environmental layer: removes alerts caused by lighting changes, rain, wind, and reflections
• Classification layer: distinguishes between people, animals, vehicles, and objects
• Behavior layer: evaluates whether the behavior is anomalous in that context
• Correlation layer: cross-checks the event with other cameras to verify consistency
• Learning layer: continuously updates what's normal in that specific environment

With this multi-layer approach, the best systems cut false alarms by up to 88% compared with conventional systems. That's no small number: it means an operator who used to receive 100 alerts per shift now receives fewer than 15, and most of them are real events.

Real Use Cases for Video Anomaly Detection by Industry

Machine learning anomaly detection doesn't have a single use case. Patterns of "normality" are completely different depending on the type of site.

Retail and Shopping Centers

In retail, the most relevant anomalies aren't just "someone is in the store after closing." They're pre-theft behaviors: loitering in an area longer than average, repeatedly handling a product without buying it, presence at the checkout with specific behaviors. The model learns the normal flow of customers per hour and detects when someone isn't behaving like a typical customer.

Warehouses and Logistics Infrastructure

In logistics, the critical anomalies are after-hours access to restricted zones, vehicles stopping at unusual points along their route, and people in loading areas during unauthorized time windows. The anomaly detection system learns the movement flows of each shift and detects deviations precisely.

Residential Buildings and Condominiums

In residential settings, the model learns residents' usual patterns: what times they come and go, which vehicles are authorized. Someone loitering in the parking lot for 40 minutes without a clear access pattern is an anomaly. An unknown vehicle parked in a restricted zone at 3 am is an anomaly. Machine learning for security builds that context site by site.

Hospitals and Clinics

In healthcare, the most critical anomalies are behaviors in restricted-access areas (pharmacy, ICU, medication rooms), early detection of confrontations in the emergency department before they escalate into physical violence, and behavior suggesting an at-risk patient intends to leave. The system learns the traffic flow of each hospital zone by shift.

Airports and Critical Infrastructure

In critical infrastructure, machine learning anomaly detection works across multiple dimensions at once: abandoned objects in waiting areas, people in restricted-access areas, vehicles that don't follow authorized traffic patterns, and crowding at boarding gates that exceeds historical capacity.

What Sets Closely Apart in Anomaly Detection

Closely was designed from the start to operate at real sites with real operators, not in lab demos. The fundamental difference comes down to three points.

First, full compatibility with existing infrastructure. Closely connects via RTSP to any IP camera, regardless of brand: Hikvision, Dahua, Axis, Bosch, Reolink, or any camera that uses the standard protocol. That means a physical security company can add Closely's machine learning anomaly detection layer on top of all the hardware it already has installed, with no investment in new cameras or proprietary hardware.

Second, detection rules configurable by industry. Closely's five agents (Watcher, Gatekeeper, Investigator, Supervisor, and Reporter) can be configured with specific rules for each type of site. The anomaly detection system has no fixed catalog: rules are combinable, updatable, and adapted to each client's real operation.

Third, a verifiable reduction in false alarms. Closely's Watcher agent cuts false alarms by up to 88% compared with conventional systems. That's what lets a single operator monitor up to 100 sites at once, something impossible with systems that generate constant noise.

The complete flow in Closely works like this:

• Watcher analyzes video in real time with the anomaly detection model configured for that site
• Gatekeeper manages visitors and access autonomously
• Investigator lets you search historical recordings with natural language descriptions
• Supervisor tracks security staff patrols
• Reporter automatically documents every event with full traceability

Closely offers a free two-week demo on the cameras you already have installed, with no infrastructure changes and no contract commitments.

The Current Limits of Machine Learning for Anomaly Detection

Being honest about limitations is part of understanding the real state of the technology:

The learning period. Every machine learning security system needs time to build its normality baseline. During the first weeks, the system may generate more false positives because it's still learning what's normal at that site.

Accuracy depends on video quality. A machine learning anomaly detection model is only as good as the video it receives. Cameras with insufficient resolution, blind spots, or poor lighting limit the accuracy of the analysis.

Unprecedented events. By definition, the model detects anomalies based on what it has learned is normal. A completely new type of event may go undetected if there isn't enough historical context.

The need for continuous adjustment. When the environment changes (renovations, changes in foot traffic, new access areas), the model needs to relearn. The best systems do this automatically, but there's a transition period.

These limits don't make machine learning anomaly detection useless. They make it manageable with the right architecture and realistic expectations about what the system can and can't do on its own.

How to Evaluate an Anomaly Detection System for Security

If you're evaluating video anomaly detection platforms for your operation, these are the questions to ask any vendor:

• How many filtering layers does the system have before generating an alert?
• How long does the model take to build the normality baseline?
• What is the real false alarm rate at production sites, not in demos?
• Does it require proprietary hardware or connect to existing cameras?
• Are the detection rules configurable or a fixed catalog?
• Does the system learn and adjust automatically when the environment changes?
• How does it document and justify each alert it generates?
• How fast is the response time from detection to operator notification?
• Is there a trial period on existing infrastructure?
• How many sites can one operator monitor at once with the system?

The answers to these questions separate a real machine learning security system from motion detection with AI branding.

---

Frequently Asked Questions About Machine Learning Anomaly Detection

1. What exactly is machine learning anomaly detection in physical security?

It's a system that learns the usual patterns of an environment (who passes through, when, and how) and generates alerts when it detects behavior that deviates from that baseline. Unlike fixed-rule systems, it doesn't need anyone to manually program what's dangerous: it learns what's normal and alerts on what isn't.

2. How long does an anomaly detection system take to start working well?

It depends on the complexity of the environment, but most systems need two to four weeks to build a reliable normality baseline. During that period, the false alarm rate is higher. Well-designed systems like Closely shorten that period with accelerated learning and initial configuration by type of site.

3. Can machine learning anomaly detection work with my current cameras?

Yes, as long as your cameras support RTSP or ONVIF, which are standard on virtually every IP camera on the market. You don't need to change your existing infrastructure. Closely connects to Hikvision, Dahua, Axis, Bosch, Reolink, and any other brand that uses those protocols.

4. What's the difference between traditional motion detection and AI anomaly detection?

Motion detection triggers an alert every time there's movement. Video anomaly detection learns whether, in that zone and at that time, the movement is normal or anomalous. The result is a drastic reduction in false alarms: a system with good filtering layers can remove up to 88% of the noise compared with conventional motion detection.

5. Does anomaly detection work in real time or only on recordings?

Modern machine learning security systems operate in real time: they analyze the video stream as it happens and generate alerts within seconds. They also allow searching historical recordings, but the main value is in live detection.

6. What types of anomalies can a well-configured system detect?

People in restricted zones, abandoned objects, vehicles in unauthorized places, pre-theft behaviors, crowds that exceed historical thresholds, doors open after hours, prolonged loitering in sensitive areas, and any significant deviation from the normality pattern learned for that specific site.

7. Does an anomaly detection system replace security operators?

It doesn't replace them: it multiplies them. A human operator can effectively watch 4 to 6 cameras with real attention. With a machine learning anomaly detection system like Closely, the same operator can supervise up to 100 sites, because they only receive notifications when there's something real to review.

8. How does the system handle seasonal or environmental changes, such as construction or renovations?

Good systems learn continuously and adapt automatically when the environment changes. There's a transition period during which the false positive rate may rise, but the model recalibrates its normality baseline within weeks.

9. Which industries benefit most from anomaly detection in security?

Retail and shopping centers (pre-theft behaviors), hospitals and clinics (access to restricted zones, confrontations in the emergency department), airports and critical infrastructure (abandoned objects, unauthorized access), logistics and warehouses (after-hours access, route deviations), and residential buildings (loitering, unauthorized vehicles).

10. How can I try a machine learning anomaly detection system with no commitment?

Closely offers a free two-week demo on the cameras you already have installed. No infrastructure changes, no new hardware, no contract commitments. It's the most direct way to see whether the system works in your real operation before making any decision.

Want to see how machine learning anomaly detection works in your operation? Request your free demo at closely.dev.