Referring to FIG. 9c, if a size of a macroblock is 32×32, it is able to use one prediction mode for each 8×8 block. In particular, one most suitable prediction mode is selected from the 9 kinds of prediction modes shown in FIG. 8 for each 8×8 block and the selected prediction mode can be coded by 4×4 block unit. For instance, if an encoder selects a vertical prediction mode for a 8×8 block represented as 0, the encoder encodes the selected prediction mode and then transmits the encoded prediction mode in FIG. 9c, a decoder decodes the 0th 4×4 block using the received vertical prediction mode. Subsequently, a reconstructed residual is added to a prediction sample. A 1st 4×4 block is then decoded using a vertical prediction mode equal to a prediction mode of the 0th 4×4 block and the reconstructed 0th 4×4 block. In this manner, all blocks in the 8×8 block represented as 0 are decoded. Subsequently, other 8×8 blocks are decoded in displayed order.

Referring to FIG. 9d, if a size of a macroblock is 32×32, it is able to use one prediction mode for each 4×4 block constructing the macroblock. In particular, one most suitable prediction mode is selected from the 9 kinds of prediction modes shown in FIG. 8 for each 4×4 block and the selected prediction mode can be coded by 4×4 block unit. All blocks in the 32×32 block are decoded in order shown in FIG. 9d using each prediction mode.