Brent Boekestein, CEO & Co-Founder, Vintra discusses whether we are ready for a second wave of video analytics.

What began as simple motion and object detection, video analytics have now evolved into advanced, accessible security tools capable of meeting the challenges of today and tomorrow. But this evolution did not occur overnight. Instead, it was driven by advancements in AI technology, the growth of digitization and the demand for more secure and safer environments.

These drivers compound year over year and result in new waves of video analytic capabilities that are changing the way security end users mitigate risk. Today, innovative providers are already offering video analytics that are poised to benefit all market sectors in terms of greater speed, efficiencies and security – known as the second wave.

The first wave of video analytics

To understand where we are going, we first must understand from where we have come. One of the earliest forms of video analytics was the identification of changes in individual pixels to detect motion.

Introduced in the early 1990s, such early video analytic iterations left much to be desired and were marred by performance issues and customer complaints. The first true wave of widely adopted video analytics focused on fixing these issues, getting deep learning-based object recognition to work on fixed and mobile video for both descriptive (men in blue shirts near white trucks) and definitive objects (face, person, vehicle). It was also important that this technology worked across challenging scene conditions, ever smaller objects and when objects are not fully visible.

Unfortunately, many video analytic solutions available in the market have not yet fully embraced such features characteristic of Wave 1. These solutions do not offer lightning-fast searches, accurate tracking or robust re-identification capabilities. In spite of this, we are at a point where the flourishing players already offering such ‘Wave 1-like’ solutions are now using their existing capabilities to create even more capabilities. If the first wave of video analytics was all information and analytics, then this second wave is about getting recommendations and answers, enabling human operators to make the right decision faster and with greater efficiency.

The second wave of video analytics

Wave 2 video analytics apply machine learning to the ever-growing body of machine learning-produced metadata captured from video to create a system that works alongside security operators. This new wave delegates more responsibility to the system and forces the system to control its accuracy, learn from its own outputs and distinguish specific priorities. In this way, the burden of labor on operators is dramatically decreased while their situational awareness is increased.

The benefits and capabilities offered by this second wave can take on a variety of forms, including:

• Pattern of life discoveries – humans have always used pattern recognition to make logical decisions based on known experiences. A great example of applying this kind of learning to video analytics is known as correlation link analysis. Using machine learning, correlation link analysis quickly establishes distinct patterns, matches events to those patterns and finds anomalies where known patterns are violated. When used in a public safety application, this kind of technology identifies persons of interest and can answer critical security questions such as: Who are the potential associates of this person? Where do they meet? On which days? At what time? It is these “pattern of life” discoveries that help security teams quickly answer these key questions and paint a more holistic security picture.

• Greater contextual awareness – in Wave 2, AI-powered video analytic solutions will be able to know exactly what each camera is looking at, providing a level of detail and context never seen. For example, the difference between a parking garage, a lobby, or an elevator. The system then takes such context into account before returning search results and issuing alerts, meaning users get ever-smarter search and alert results with less false positives. This contextual awareness also helps frame a particular issue more clearly and takes situational awareness to an entirely new level.

• Improving machine learning models – new, real-life security challenges highlight the need for machine learning training data and models to be constantly pushed and expanded. Wave 2 solutions utilize approaches like synthetic training data and transfer learning to push their platform to perform at state-of-the-art levels. To extract more information from videos such as new detection categories, Wave 2 also demands flexible model architectures that make it easy to add new detection categories quickly and accurately. In the early days of the COVID-19 pandemic, it was this kind of technology that allowed mask detection analytics to be built in days, not months.

• Intelligent monitoring – security operation centers often display hundreds of video feeds at one time, relying solely on security personnel to monitor all of these feeds at once to identify possible threats. In both enterprise and public safety sectors, this task is almost impossible to perform 24/7 and free from human error. As part of Wave 2, the system will control the monitor selection to show only the camera feeds on which personnel need to focus at that moment in time. AI-powered video analytics will therefore not need to define any rule or simplistic or empirical condition — the system will be able to present to the user only the things that truly matter.

• More imaging solutions – Wave 1 video analytic solutions work well on fixed and mobile cameras, but what about infrared cameras commonly used by first responders and healthcare professionals? Or multi-imager cameras that are growing in popularity due to their cost and security benefits? Look to see AI-powered video analytics available for more types of imaging solutions during Wave 2.

Implications of this second wave

A common misconception about all video analytics is that they were created as and are evolving into, a solution designed to replace humans in security applications. Quite the opposite: The second wave of video analytics allows security operations to redeploy human capital where it is most needed most, such as alarm response or crime deterrence.

Furthermore, these responses are better informed and can be performed faster under the guidance and capabilities provided by Wave 2 solutions. Security teams at industry-leading enterprises, critical infrastructure, healthcare and national security organizations are now able to build and operate the safest and smartest environments possible using the next generation of video analytics.

For more information, visit: vintra.io

Return to Security Journal Americas NEWS INDEX