Airports, casinos and correctional centers are but a few facilities that simply cannot afford to lose any camera recordings, reports BCD.

Unfortunately, recording servers are not infallible. Hardware fails and systems need to be maintained and upgraded and servers can be compromised by threats or human errors. As a result, most facilities experience blackout periods ranging from a couple of hours to weeks every year.

Even when using Milestone’s failover recording servers, you are at risk of losing critical recordings. This article explores various options that will further protect your video data.

The challenge

Traditional backup software is totally inefficient when it comes to safeguarding large amounts of changing data. That’s because it typically runs on a daily schedule and is optimized for incremental backups, where only a small percentage of files change.

Safeguarding camera recordings is akin to drinking from a firehose: new camera data continuously gets recorded while the oldest is deleted. The problem is compounded by the fact that safeguarding video files is just half the battle. These files are indexed in an SQL database. Without an up-to-date database, recordings cannot be loaded on the Milestone XProtect timeline.

Milestone already offers solutions that protect its management, event and recording servers. However, if databases cannot be properly protected, recordings are still at risk.

Milestone failover for recording servers

Milestone offers a failover option for its recording servers. The solution relies on having one (or more) stand-by recording servers that can be fired up to replace any failing recording server. When Milestone Management Server detects the failure of an active recording server, it wakes up a failover recording server and assigns all active cameras to it. Depending on the number and driver capabilities of the cameras, this may take up to a few minutes. Meanwhile, camera data may be lost.

When the failover server is finally able to start capturing new camera data, it does not have access to any of the past recordings (that are indexed in the failed server’s database). As a result, operators lose the ability to review recordings leading up to the failover event until the original server can be brought back to life. Of course, if it was the storage associated with the original recording server that failed, all associated recordings would be permanently lost.

Disaster recovery

Tiger Technology has developed flexible and easy-to-use disaster recovery and high availability plug-ins for Milestone XProtect. These plug-ins are designed to cover virtually all use cases, are easy to deploy and are fully integrated, thanks to Milestone’s flexible MIPS architecture. When this plug-in is paired with BCD’s trusted video recording servers, users can rest assured uptime will be maximized and video data will always be accessible. BCD’s servers are purpose-built with the latest Intel processors, designed to withstand the taxing demands of high availability and high-intensity workloads. Utilizing BCD’s Milestone XProtect – ready appliances in addition to Tiger Technology’s Surveillance Bridge plug-in offers peace of mind for users with mission-critical surveillance systems.

The Tiger Technology Surveillance Bridge plug-in makes it easy to add disaster recovery capability to any existing Milestone recording server without disruption of service. Disaster recovery can be added to a live recording drive or to an archive volume so recordings can be replicated to an on-site or off-site location – such as cloud storage, literally seconds after they were recorded locally.

This approach represents a cost-effective way to minimize the amount of data loss in the event of a disaster and is a great way to complement Milestone’s failover option by safely preserving a copy of the original recordings. However, the following issues remain unresolved: 1) during a failover, operators do not have access to previous recordings and 2) if the original recording server cannot be brought back to life, its associated recordings must still be recovered. While most facilities can happily live with these limitations, they can remain of concern for environments demanding nothing but flawless operations.

High availability

For such critical environments, Tiger Technology offers a High Availability (HA) plug-in for BCD’s Milestone XProtect-ready appliances that also works with all existing recording servers. However, with Tiger Technology’s solution, all previous recordings remain immediately available after a failover. This means operators may not even notice a failover took place. Tiger Technology supports three different High Availability configurations for maximum deployment flexibility.

Recording server HA – Active/Passive

In this configuration, two recording servers are clustered together. The failover detection mechanism ensures that only one of these servers is running at any time (hence the Active/Passive label). During normal operations, XProtect only “sees” and writes to the primary server. Every time a new camera file is written to disk, the HA plug-in replicates it to the local disk of the secondary recording server so both servers always maintain the same camera data folder structure.  

When the primary recording server fails, the HA plug-in automatically updates the configuration file of the secondary recording server, substitutes IP addresses and starts it. After a short discontinuity (the time it takes to start the recording server services), the secondary recording server starts assuming the function of the original server, with all data (except perhaps the very last blk files) available. To the management server, it looks as if the original server came back online.

When the primary recording server resumes operations, the secondary recording server recedes in the background and any missing data can be backfilled to the primary server.

This asynchronous configuration will introduce a short blackout period when switching between recoding servers. It remains, however, the best one to use when camera data cannot simultaneously be directed to two recording servers or if Milestone encryption must be enabled.

Recording server HA – Active/Active

In this configuration, two recording servers are also clustered together. However, unlike in the Active/Passive scenario where only one of them is active at a time, they are both recording in parallel. Still only one of them holds the representation token (the network identification). This means that management clients and smart clients will be directed to this “main” server (oblivious to the fact that there is a twin “shadow” server also running). Since both servers share the same server configuration, they are both connected to the same cameras and are independently recording the data to their own storage.

If the primary server fails, the network representation token is immediately transferred to the secondary server. Since this only takes a few seconds, smart clients will continue to be connected and preview data, not realizing that it is now coming from a different server.

When the primary recording server resumes operations, the secondary recording server recedes in the background and any missing data can be backfilled to the primary server. Unless both servers fail at the same time, smart clients always stay connected and will always see the entire recorded dataset.

Camera HA – Active/Active

In this configuration, multiple recording servers are running independently (i.e., not clustered). For each camera, the plug-in keeps track of whichever primary and secondary recording servers are connected to it. When the plug-in detects that a camera feed is no longer available from its primary recording server, it automatically re-routes the smart client, so it seamlessly reconnects to the secondary recording server where this camera is also being recorded.

When a particular camera data becomes available again from the primary recording server, the plug-in will re-route the smart client back after any missing data for that camera has been backfilled onto the primary server. Unless both servers fail at the same time, smart clients always stay connected and will always see the entire recorded dataset.

This configuration offers the most flexibility, as pairing for failover is done at the camera level between any two recording servers receiving its feed. The only drawback is that this configuration requires an extra step to pair the cameras (easy and highly automated!) and requires Milestone XProtect licenses for all recording servers.

There are various ways one can use to prevent the loss of camera data. Depending on the desired functionality and the available resources, one (or more) of the configurations below can be considered. Ultimately, it boils down to one simple question: how much camera data can you afford to lose?

For more information, visit: www.bcdvideo.com

This article was originally published in the bumper September edition of Security Journal Americas. To read your FREE digital edition, click here.

Return to Security Journal Americas NEWS INDEX