Bamba--Audio and video streaming over the Internet

IBM Journal of Research and Development, Mar 1998 by Willebeek-LeMair, M H, Kumar, K G, Snible, E C

A conventional H.263 encoder first partitions a video image into a set of blocks of 16 pixels by 16 pixels. The coding control function searches for the best match between each block in the current frame and blocks in the previous frame. If a sufficiently close match is found, the block is encoded as a P-block based on the difference between the block and the closest matching block in the previous frame. The closeness of the match is evaluated in terms of the number of bits needed to encode the difference between the block and the closest matching block in the previous frame. If the difference is too great, it is deemed more efficient to encode the block independently, without reference to previous data. Such a block is referred to as an intracoded I-block.

The resulting H.263 compressed stream consists primarily of P-blocks, with I-block insertions caused by scene changes or severe motion. To prevent error accumulation, the standard also requires that each block be encoded as an I-block at least once every 132 frames. Although the H.263 standard defines how I-blocks and P-blocks are encoded, it allows considerable flexibility in selecting when to encode a block as either an I- or P-block. We exploit this flexibility to improve the robustness of a standards-compliant stream by carefully choosing when and where to insert I-blocks during the encoding process.

I-block encoding exploits only the spatial redundancy within the block in the compression process, while P-block encoding exploits both the temporal and spatial redundancies of the video. Although interblock encoding generally achieves more compression gain, the encoding dependencies of P-blocks reduce their resilience to errors. If the region referenced by a P-block has been corrupted, the decoding of the P-block will generate incorrect pixel values. If all or a part of this corrupted P-block is again referenced by other P-blocks, the erroneous pixel values will cause errors to propagate from one frame to another. This is known as the error-propagation problem of motion-compensated video compression. The propagation stops when all corrupted regions are updated by I-blocks.

On the Internet, where loss is primarily attributed to network congestion, the loss of a UDP/IP packet that contains video data can result in the loss of several frames in a low-bit-rate video system. For example, with a target bit rate of 20 Kb/s, a typical QCIF compressed video frame may contain 165 bytes. Hence, a 500-byte IP packet contains roughly three frames of compressed video data. If the packet is lost, these frames cannot be recovered, and errors begin to propagate.

For non-real-time applications, knowledge about the interdependence among blocks in a sequence can be obtained from the dependencies reflected by the motion vectors. It is thus possible to assign a measure of importance to a pixel or block by counting the number of pixels or blocks that depend on it. This operation is anticausal, i.e., traversing backward in time. The higher the dependence on a block, the more critical it is that this block be correct and that it be encoded as an I-block. Furthermore, dependence chains may be broken by encoding intermediate blocks in the chain as I-blocks.