DAVIS: Distributed, Automated Video Surveillance

In next generation video surveillance system, video streams are not watched by humans most of the time, but, instead, are processed by automated software employing computer vision algorithms. Computer vision has a much lower requirements on video quality than human vision. Exploiting this fact, surveillance video streams can be sent at a low quality without affecting the accuracy of the vision algorithms.

  • Rate-Accuracy Tradeoff in Automated, Distributed Video Surveillance Systems Pavel Korshunov,
    in proceedings of the 14th ACM International Conference on Multimedia (ACMMM'06), Santa Barbara, California, USA, October 23-27, 2006. [pdf] [slides]
    For the past years video analysis algorithms are attracting more research interest and becoming commonly used in various multimedia applications. Since video analysis algorithms operate over large amounts of data, such as videos and images, they impose high system requirements on storage and computational resources, network bandwidth, and power for battery--powered devices. Despite this fact, surprisingly little attention was paid to the resources minimization problem accompanying the use of video analysis algorithms. We found that video analysis algorithms can sustain a significant degradation in the input video quality without decrease in their accuracy. This finding leads to the problem of determining the tradeoff between video bitrate and the accuracy of a given algorithm. Our approach to the solution of the porblem is to focus on video features that affect performance of the algorithm. By studying how a video adaptation, which reduces video bitrate, degrades video features, we can estimate the rate--accuracy tradeoff.
  • Critical Video Quality for Distributed Automated Video Surveillance Pavel Korshunov and Wei Tsang Ooi,
    in proceedings of the 13th ACM International Conference on Multimedia (ACMMM'05), Singapore, November 8-10, 2005. [pdf] [slides]

    Large-scale distributed video surveillance systems pose new scalability challenges. Due to the large number of video sources in such systems, the amount of bandwidth required to transmit video streams for monitoring often strains the capability of the network. On the other hand, large-scale surveillance systems often rely on computer vision algorithms to automate surveillance tasks. We observe that these surveillance tasks present an opportunity for trade-off between the accuracy of the tasks and the bit rate of the video being sent. This paper shows that there exists a sweet spot, which we term critical video quality that can be used to reduce video bit rate without significantly affecting the accuracy of the surveillance tasks. We demonstrate this point by running extensive experiments on standard face detection and face tracking algorithms. Our experiments show that face detection works equally well even if the quality of compression is significantly reduced, and face tracking still works even if the frame rate is reduced to 6 frames per second. We further develop a prototype video surveillance system to demonstrate this idea. Our evaluation shows that we can achieve up to 29 times reduction in video bit rate when detecting faces and 16 times reduction when tracking faces. This paper also proposes a formal rate-accuracy optimization framework which can be used to determine appropriate encoding parameters in distributed video surveillance systems that are subjected to either bandwidth constraints or accuracy constraints.

    The following two images demonstrate the visual difference between compression quality acceptable for human visual system (left) and compression quality accpetable for face detection algorithm (right). JPEG compression algorithm by IJG was used and left image is compressed with quantizer 90, while right with quantizer 11. Viola-Jones face detection algorithm implemented in OpenCV was used as an example of video analysis algorithm. Test image is from MIT/CMU dataset.

    Audrey q90 Audrey q11
    JPEG compressed with quantizer 90. File size 22027 bytes. JPEG compressed with quantizer 11. File size 4708 bytes.