Then, the process proceeds to S1340 to determine whether to skip the vertical direction optical flow. It is determined whether the vertical complexity D5 is less than the threshold T (S1341). When the vertical complexity D5 is less than the threshold T, the vertical optical flow vy is set to 0 (S1342). This means that the vertical optical flow is not applied. When the vertical complexity D5 is greater than or equal to the threshold T, d2 and d6 are calculated (S1343), and s2, s5, and s6 are calculated using d2, d5, and d6 (S1344). When the vertical optical flow vy is calculated using Equation 7 or 9, only s2, s5, and s6 are required. Since d5 has already been calculated in S1320, d2 and d6 are calculated in S1343, and s2, s5, and s6 are calculated in S1344 using d2, d5, and d6. Then, the vertical optical flow vy is calculated using s2, s5, and s6 according to Equation 7 or 9 (S1345).
When the vertical optical flow vy is calculated using Equation 8, only s5 and s6 are required. In this case, therefore, calculation of d2 and s2 may be omitted in S1343 and S1344.