FIG. 3 is a block diagram illustrating the logical encoding flow of the digital video camera 200 of FIG. 2. In general, the video encoder component 218 applies a hybrid coding scheme of block-based prediction and transform coding to each of the 2D video sequences. In such a scheme, each input signal is split into rectangular blocks that are predicted from the previously decoded data by either motion compensated inter prediction or intra prediction. The resulting prediction error is coded by applying block transforms based on, for example, an integer approximation of the discrete cosine transform, which is followed by quantization and entropy coding of the transform coefficients.

As shown in FIG. 3, the left view frames from the left imaging component 208 are coded by a left view encoder 302 and the right view frames from the right imaging component 206 are coded by a right view encoder 304. Each of these view encoders 302, 304 processes the respective video stream in a similar fashion to generate a compressed bit stream. Prior to encoding of the left view frame and the right view frame, a depth map is generated 300 for the two frames. This depth map represents the depth of each pixel in the 3D frame corresponding to the two 2D frames. The depth map is generated by the image and depth processing component 204.