The choice of a chroma QP offset value is determined by the relative visual complexities or textures among the luma and chroma components. The quantization groups of the image 1600 are assigned to different categories 1611-1614 according to their texture levels. For example, areas that appear as flat across all color components belong to a first category 1611, flat luma areas with high texture color components belong to a second category 1612, high texture luma areas with flat color information belong to a third category 1613, while areas with high texture across all three components belong to a fourth category 1614. The encoder in some embodiments may create additional subcategories for each category based on motion and intensity. In some embodiments, additional categorizations are made by taking into account the differences between the two chroma components themselves.