FIG. 2G is a perspective view of the FinFET 20 at one of various stages of the manufacturing method, and FIG. 3G is a cross-sectional view of the FinFET 20 taken along the line I-I′ of FIG. 2G. In Step 22, the gate stack 140 is formed, after removing the stack structure 120. In one embodiment, the polysilicon strips 122 and the hard mask strips 124 located on the polysilicon strips 122 are removed by anisotropic etching and the spacers 126 are remained. Then, a gate dielectric layer 142 is formed within the recesses between the spacers 126 and on the top surfaces 116a and the sidewalls 117b of the fins 116. In some embodiments, the material of the gate dielectric layer 142 comprises silicon oxide, silicon nitride or the combination thereof. In some embodiments, the gate dielectric layer 142 comprises a high-k dielectric material, and the high-k dielectric material has a k value greater than about 7.0 and includes a metal oxide or a silicate of Hf, Al, Zr, La, Mg, Ba, Ti, Pb and combinations thereof. In some embodiments, the gate dielectric layer 142 is formed by ALD, molecular beam deposition (MBD), physical vapor deposition (PVD) or thermal oxidation. Then, a gate electrode layer 144 is formed over the gate dielectric layer 142, over the covered portions 119 (the channel regions) and fills the remaining recesses between the spacers 130. In some embodiments, the gate electrode layer 144 comprises a metal-containing material, such as Al, Cu, W, Co, Ti, Ta, Ru, TiN, TiAl, TiAlN, TaN, TaC, NiSi, CoSi or a combination thereof. Depending on whether the FinFET 20 is a p-type FinFET or an n-type FinFET, the materials of the gate dielectric layer 142 and/or the gate electrode layer are chosen. Optionally, a CMP process is performed to remove the excess portions of gate dielectric layer 142 and the gate electrode layer 144. The spacers 126 are located on sidewalls of the gate dielectric layer 142 and the gate electrode layer 144. That is, the stack structures 120 are replaced and the replacement gate stacks 140 are formed. In some embodiments described herein, the gate stacks 140 are replacement gates, but the gate stack structures or the fabrication processes thereof are not limited by these embodiments.