A description will now be given of a difference between an effect of a smaller calculation area and an effect of a large calculation area. In a case where the calculation area is made small, the XY coordinates (position) of the focus object in the next frame are predicted based on the object detection result of the previous frame, the calculation area is selected, and calculation is performed, the control has an effect in that objects located at different distances are not mixed in the calculation area. Therefore, it is less likely to erroneously focus on the background and foreground included in the calculation area. On the other hand, in a case where the calculation area is made large, the calculation area (focus detecting calculation area) for the same object can be widely secured. Therefore, the in-focus accuracy can be improved because the influence of factors can be reduced such as a high ISO speed, low illuminance, and low contrast, which prevent the focus detecting calculation.

The image shift amount calculating unit 110 calculates an image shift amount between the A-image (A-image signal) and the B-image (B-image signal) output from the capture unit 104, and outputs the calculated image shift amount to the lens driving control unit 111. The lens driving control unit 111 converts the image shift amount into a defocus amount, and drives the focus lens in the optical system unit 101 (performs focus control).