FIG. 7 shows an example of bilateral matching used in the Frame-Rate Up Conversion (FRUC) method. The bilateral matching is used to derive motion information of the current CU by finding the closest match between two blocks along the motion trajectory of the current CU (700) in two different reference pictures (710, 711). Under the assumption of continuous motion trajectory, the motion vectors MV0 (701) and MV1 (702) pointing to the two reference blocks are proportional to the temporal distances, e.g., TD0 (703) and TD1 (704), between the current picture and the two reference pictures. In some embodiments, when the current picture 700 is temporally between the two reference pictures (710, 711) and the temporal distance from the current picture to the two reference pictures is the same, the bilateral matching becomes mirror based bi-directional MV.