Download the document

Application Cloud Storage in Intelligent Video Surveillance
High-definition (HD) video surveillance is a megatrend of the society. HD video images have given huge
impact on traditional video surveillance systems, particularly explosive growth of collected, transmitted,
stored, and computed data. Storage is an indispensable component of video surveillance.
However, few security protection companies worldwide have the capabilities to research, develop, and
customize storage devices. Large IT companies with storage device R&D and customization capabilities rarely
set foot in the security protection industry. They provide only standard storage devices. This explains why
dedicated storage products have not been developed to meet video surveillance service features. Industryspecific storage devices are urgently needed for security protection companies.
Service Continuity
In the banking industry, services will be interrupted if a data fault occurs. Customers can make deposits or
withdrawals only after the data is restored accurately. In our daily life, we often see the "Temporarily Out of
Service" notice board at a bank. There are two situations: The bank has insufficient tellers or the bank's
system data is faulty. Of course, banks usually have several data copies. To banks, data reliability and
integrity come before service continuity. This is the "standard data storage" in the storage field, which,
however, differs from that in the video surveillance industry where customers cannot accept their entire
video surveillance system stops working because several hard disks are faulty. To apply to scenarios where
service continuity comes before data integrity (a small number of data errors are allowed), "application
storage" with video surveillance service features has been developed.
A traditional storage device requires several disks to form up a RAID group to enhance data reliability. The
RAID 5 algorithm, for example, prevents data loss when a disk is faulty. Customers can restore service data
through parity data. Generally speaking, only one RAID group is established on a 16-slot disk array. Then, 14
slots are available for data storage. If a mainstream 3 TB disk is used, the 16-slot disk array can store video
images from 70 standard-definition (SD) cameras for one month. In this case, the entire data of the RAID
group will be lost if two or more disks in the group are corrupted.
The cause to this issue is that traditional storage vendors use RAID technology to design storage devices for
scenarios where data reliability is most important. Standard data storage is designed to store structured data.
Take the above-mentioned RAID group as an example. One file may be divided into 14 data pieces, with each
piece stored in each disk. If several data pieces are lost, the entire file cannot be restored any longer.
This issue triggers great concern among customers in the video surveillance industry. Based on our survey,
customers can bear the loss of a slight number of video images to ensure service continuity. They expect data
to be read from uncorrupted disks to maximize data integrity.
To ensure service continuity, special designs are performed at the bottom layer of storage devices. We have
customized and reinvented the RAID algorithm. As long as one disk is working properly, video images in the
RAID group can be read properly. If a hard disk is faulty, customers feel that the video image is frozen for a
second and continues to be played. This technology is called SafeVideo, applied in our storage devices
dedicated for video surveillance scenarios. SafeVideo ensures service continuity and maximizes data integrity
when several hard disks are faulty. In addition, the RAID group is automatically reconstructed without any
configurations after maintenance personnel replace faulty disks with new disks.
Data Reliability
In video surveillance scenarios, a majority of customers consider service continuity as their top priority, but IT
vendors still pursue to maximize data security and integrity. The mentioned SafeVideo technology is such an
attempt to minimize data loss in case of hard disk faults.
The "application cloud storage" concept emerges after cloud storage systems are incorporated into video
surveillance. The primary feature of application cloud storage is high reliability.
First, all video surveillance services that can be abstracted – such as camera access, video storage, forwarding,
video on-demand (VoD), and intelligent analysis – are embedded into application cloud storage. Working as a
middleware between cameras and the service platform, a single-node device is able to provide customers
with efficient and reliable services.
Second, all application cloud storage node devices are deployed in a peer-to-peer manner. All node devices
can be virtualized as one giant device. Physically, what people see is that devices are placed in order in a
cabinet. Logically, tens of thousands of cameras and diverse background software are displayed as a single
device. The device features massive access, storage, and forwarding.
The "N+0" concept appears to ensure high reliability. Traditional disk arrays and innovative cloud storage
devices employ the "N+M" deployment mode. Here, N refers to active devices, and M refers to standby
devices. When an active device is faulty, services are automatically switched to the standby device.
However, standby devices are not required in the "N+0" deployment mode where all devices are active and
deployed in a peer-to-peer manner. When a device is faulty, other devices jointly share the workload of the
faulty device. Therefore, video surveillance services are not affected, ensuring service continuity. Such design
greatly reduces redundancy device investment and O&M costs.
In a traditional storage architecture based on disk arrays, one or two database servers are deployed to
identify Small Computer System Interface (SCSI) data blocks in disks. When the database is corrupted or the
database server is faulty, all data will be lost. This issue can be well resolved by application storage. An
application is embedded into storage devices to work as a small database. When an active device is faulty,
services are automatically switched to other active devices. When the network between a camera and the
application cloud storage device is disconnected, the camera will start its embedded SD storage module to
continue video recording. When the network is restored, the camera uploads live and historical recordings to
the application cloud storage device. Thanks to multiple reliability assurance approaches, the video cloud
storage solution improves video data integrity to an unprecedented 99.95%.
High-Performance Parallel Computing
The biggest headache for customers in the Safe City sector is low case clearance rate. One direct technical
cause is inefficient video search and location. Database servers are deployed in a traditional storage
architecture to store video indexes based on image frames or time.
Regardless of the database type (SQL Server, Oracle, MySQL, or PGSQL) or the index type, databases are
facing a common challenge: Search and location become increasingly inefficient as the data volume grows.
Generally, a video consists of 25 or 30 frames. If we understand one frame as one image, a video consists of
massive images. If these images are played quickly and in order, it becomes a video. This is also the
production mode of animated cartoons. If one index is specified for each image in the database, this is the
frame index. Here, take a Safe City project in China as an example. The number of indexes reaches 16,848
billion for 26,000 25-frame IP cameras to storage video images for one month. If these indexes are stored in
the SQL Server database, it takes about 110 minutes to locate an index by running a command. The time can
be shortened to approximately 30 minutes after a significant number of optimizations.
Such inefficient search and location may lead to horrible results. Here comes a true story. A car accident
occurred on a street. A witness reported the license plate number of the car to the police. Related personnel
searched the video surveillance system for the number, but the result still did not come out after thirty
minutes.
Then, why some vendors claim that they implement search and location within a matter of seconds? Yes,
they can, when the system is just deployed. But, it is really difficult to say so as the data volume grows after
six months.
High-performance parallel computing is urgently required when a major incident occurs or video data is open
to the public. To suit customer needs, we have rolled out the "distributed parallel search" based on the
application cloud storage architecture.
During data writing, the application cloud storage architecture uses a harsh algorithm (developed for the
video surveillance scenarios) to store a distributed index. Within a cloud storage device, it takes less than 10
microseconds to locate a frame in the disk. The 10 microseconds mean immediacy to personal feeling.
This is only the performance of a single-node device. It takes about 100 milliseconds to search all node
devices for an index.
Low Total Cost of Ownership (TCO)
We have mentioned the major difference between application storage and traditional standard data storage:
the former integrates industry-specific applications into storage devices. Such integration reduces a large
number of servers. A single cloud storage device is able to provide holistic video services. This also greatly
reduces the capital expenditure (CAPEX) and operating expenditure (OPEX). According to a survey based on
numerous projects, the cloud storage solution can reduce the OPEX by approximately 30%.
For a large-scale Safe City project, search and location based on cloud storage will not be inefficient as the
Safe City data volume grows. In combination with a variety of approaches (such as intelligent analysis, video
abstracting, case archiving, and case correlation analysis), the video surveillance efficiency can be
dramatically enhanced.
In the near future, the ever increasing TCO will bring us a better customer experience as efficient data
storage and support technologies are broadly applied to video surveillance systems.