There are arranged a plurality of unit pixel cells P each having divided pixels A and B under one microlens 2. The pixels A and B placed under the microlens 2 are pixels that are divided using the microlens 2 as an exit pupil. During the focus detection, signals output from a plurality of pixels A (A-image pixel unit) and signals output from a plurality of pixels B (B-image pixel unit) are combined in the column direction (or in the row direction) to generate the A-image and the B-image as outputs of the same color unit pixel cell unit. The shift of each corresponding point is calculated by the SAD calculation. The result of the SAD calculation is obtained by the following equation (1).
C=Σ|YAn?YBn|??(1)

In equation (1), n is the number (number) of microlenses in a range for performing the SAD calculation (correlation calculation) in the horizontal direction. YAn and YBn are a luminance signal of the A-image and a luminance signal of the B-image, respectively, corresponding to an n-th microlens. A value obtained by shifting a corresponding pixel for the luminance signal YBn of the B-image is plotted, and a position that minimizes the shift amount is an in-focus position.

In the in-focus state illustrated in FIG. 3A, imaging positions of the imaging optical system are the pixels A and B under the microlens of the unit pixel cell P7 and thus the A-image pixel unit and the B-image pixel unit approximately coincide with each other. At this time, an image shift amount d(a) between the A-image pixel unit and the B-image pixel unit obtained by the correlation calculation is approximated to zero.