Referring to FIG. 9B, a stack of layers including a blocking dielectric layer 52, a charge storage layer 54, a tunneling dielectric layer 56, and a semiconductor channel material layer 60L may be sequentially deposited in the memory openings 49. The blocking dielectric layer 52 may include a single dielectric material layer or a stack of a plurality of dielectric material layers. In one embodiment, the blocking dielectric layer may include a dielectric metal oxide layer consisting essentially of a dielectric metal oxide. As used herein, a dielectric metal oxide refers to a dielectric material that includes at least one metallic element and at least oxygen. The dielectric metal oxide may consist essentially of the at least one metallic element and oxygen, or may consist essentially of the at least one metallic element, oxygen, and at least one non-metallic element such as nitrogen. In one embodiment, the blocking dielectric layer 52 may include a dielectric metal oxide having a dielectric constant greater than 7.9, i.e., having a dielectric constant greater than the dielectric constant of silicon nitride. The thickness of the dielectric metal oxide layer may be in a range from 1 nm to 20 nm, although lesser and greater thicknesses may also be used. The dielectric metal oxide layer may subsequently function as a dielectric material portion that blocks leakage of stored electrical charges to control gate electrodes. In one embodiment, the blocking dielectric layer 52 includes aluminum oxide. Alternatively or additionally, the blocking dielectric layer 52 may include a dielectric semiconductor compound such as silicon oxide, silicon oxynitride, silicon nitride, or a combination thereof.